langgenius/dify-official-plugins
1
0
Fork 0
mirror of https://github.com/langgenius/dify-official-plugins.git synced 2026-07-22 01:55:27 -04:00
Code Issues 54 Packages Projects Releases Wiki Activity

jina reader Crawl Subpages config does not work #787

New Issue
Closed
opened 2026-02-16 10:20:30 -05:00 by yindo · 1 comment
yindo commented 2026-02-16 10:20:30 -05:00
Owner
Copy Link
Copy Source

Originally created by @davidkhala on GitHub (Nov 7, 2025).

Self Checks

  • I have read the Contributing Guide and Language Policy.
  • This is only for bug report, if you would like to ask a question, please head to Discussions.
  • I have searched for existing issues search for existing issues, including closed ones.
  • I confirm that I am using English to submit this report, otherwise it will be closed.
  • 【中文用户 & Non English User】请使用英语提交，否则会被关闭 ：）
  • Please do not modify this template :) and fill in all the required fields.

Dify version

1.9.2

Cloud or Self Hosted

Cloud

Steps to reproduce

  1. Install JINA Reader plugin

  2. Open/Create a dataset with pipeline mode

  3. In pipleline canvas, add JINA Reader as data source

  4. Set Crawl Subpages config to False as below, and accordingly trying to configure this source only crawl provided page content alone.

  5. test run it, provides a url (e.g. https://thei.edu.hk/about/about-thei/)

  6. investiage the fetched page has same title with the url page

✔️ Expected Behavior

In Step 6, only one page that matching the provided url should appear as result

If we create a dataset by wizard and choose JINA Reader route

It can fetch properly

This 2 ways of using JINA Reader should behave the same

❌ Actual Behavior

In Step 6, 1 subpage that is a link in content of provided url appear as result

Originally created by @davidkhala on GitHub (Nov 7, 2025). ### Self Checks - [x] I have read the [Contributing Guide](https://github.com/langgenius/dify/blob/main/CONTRIBUTING.md) and [Language Policy](https://github.com/langgenius/dify/issues/1542). - [x] This is only for bug report, if you would like to ask a question, please head to [Discussions](https://github.com/langgenius/dify/discussions/categories/general). - [x] I have searched for existing issues [search for existing issues](https://github.com/langgenius/dify/issues), including closed ones. - [x] I confirm that I am using English to submit this report, otherwise it will be closed. - [x] 【中文用户 & Non English User】请使用英语提交，否则会被关闭 ：） - [x] Please do not modify this template :) and fill in all the required fields. ### Dify version 1.9.2 ### Cloud or Self Hosted Cloud ### Steps to reproduce 1. Install JINA Reader plugin 2. Open/Create a dataset with pipeline mode 3. In pipleline canvas, add JINA Reader as data source 4. Set `Crawl Subpages` config to False as below, and accordingly trying to configure this source only crawl provided page content alone. <img src="https://github.com/user-attachments/assets/0a92943c-c9fc-431d-ab93-74c5c792f70f" /> 5. test run it, provides a url (e.g. https://thei.edu.hk/about/about-thei/) 6. investiage the fetched page has same title with the url page ### ✔️ Expected Behavior In Step 6, only one page that matching the provided url should appear as result If we create a dataset by wizard and choose JINA Reader route <img src="https://github.com/user-attachments/assets/1396aaf5-491f-4627-8f88-888e2429fa64" /> It can fetch properly <img src="https://github.com/user-attachments/assets/9b691319-2df2-47e6-a14f-7272da76ba83" /> This 2 ways of using JINA Reader should behave the same ### ❌ Actual Behavior In Step 6, 1 subpage that is a link in content of provided url appear as result <img src="https://github.com/user-attachments/assets/3efd431a-b55e-4105-99d6-de8750c9af3a" />
yindo added the bug label 2026-02-16 10:20:30 -05:00
yindo closed this issue 2026-02-16 10:20:30 -05:00
yindo commented 2026-02-16 10:20:30 -05:00
Author
Owner
Copy Link
Copy Source

@dosubot[bot] commented on GitHub (Nov 27, 2025):

Hi, @davidkhala. I'm Dosu, and I'm helping the dify-official-plugins team manage their backlog and am marking this issue as stale.

Issue Summary:

  • You reported that in Dify version 1.9.2 Cloud, the JINA Reader plugin's "Crawl Subpages" setting does not work correctly in pipeline mode.
  • Specifically, even when "Crawl Subpages" is set to False, subpage content is still fetched in pipeline mode.
  • The setting works correctly when using the JINA Reader through the dataset wizard, causing inconsistent behavior.
  • No further comments or updates have been provided on this issue.

Next Steps:

  • Please let me know if this issue is still relevant with the latest version of the dify-official-plugins repository by commenting here.
  • If I do not hear back within 5 days, I will automatically close this issue.

Thank you for your understanding and contribution!

@dosubot[bot] commented on GitHub (Nov 27, 2025): Hi, @davidkhala. I'm [Dosu](https://dosu.dev), and I'm helping the dify-official-plugins team manage their backlog and am marking this issue as stale. **Issue Summary:** - You reported that in Dify version 1.9.2 Cloud, the JINA Reader plugin's "Crawl Subpages" setting does not work correctly in pipeline mode. - Specifically, even when "Crawl Subpages" is set to False, subpage content is still fetched in pipeline mode. - The setting works correctly when using the JINA Reader through the dataset wizard, causing inconsistent behavior. - No further comments or updates have been provided on this issue. **Next Steps:** - Please let me know if this issue is still relevant with the latest version of the dify-official-plugins repository by commenting here. - If I do not hear back within 5 days, I will automatically close this issue. Thank you for your understanding and contribution!
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
bug
bug
bug
dependencies
documentation
enhancement
good first issue
invalid
pull-request
Mirrored from GitHub Pull Request
 question
stale
No Label bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: langgenius/dify-official-plugins#787
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?