langgenius/dify
1
0
Fork 0
mirror of https://github.com/langgenius/dify.git synced 2026-07-21 20:15:31 -04:00
Code Issues 775 Packages Projects Releases 10 Wiki Activity

process_rule mode does not contemplate the hierarchical option #7601

New Issue
Closed
opened 2026-02-21 18:21:26 -05:00 by yindo · 3 comments
yindo commented 2026-02-21 18:21:26 -05:00
Owner
Copy Link
Copy Source

Originally created by @pr-maia on GitHub (Jan 8, 2025).

Self Checks

  • I have searched for existing issues search for existing issues, including closed ones.
  • I confirm that I am using English to submit report (我已阅读并同意 Language Policy).
  • [FOR CHINESE USERS] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:）
  • Please do not modify this template :) and fill in all the required fields.

Provide a description of requested docs changes

The documentation states that the mode parameter of process_rules can take on two values (automatic or custom) when in fact if we want to use the parent-child we should also change this to hierarchical.

If I use all the other parameters correctly, such as doc_form assuming hierarchical_model and parent_mode assuming paragraph, the document is still not processed as parent/child but as general.

image

Originally created by @pr-maia on GitHub (Jan 8, 2025). ### Self Checks - [X] I have searched for existing issues [search for existing issues](https://github.com/langgenius/dify/issues), including closed ones. - [X] I confirm that I am using English to submit report (我已阅读并同意 [Language Policy](https://github.com/langgenius/dify/issues/1542)). - [X] [FOR CHINESE USERS] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:） - [X] Please do not modify this template :) and fill in all the required fields. ### Provide a description of requested docs changes The documentation states that the `mode` parameter of `process_rules` can take on two values (**automatic or custom)** when in fact if we want to use the parent-child we should also change this to **`hierarchical`**. If I use all the other parameters correctly, such as `doc_form` assuming hierarchical_model and parent_mode assuming paragraph, the document is still not processed as parent/child but as general. ![image](https://github.com/user-attachments/assets/6a1893e4-3d5f-46c0-b2da-4eda6a644228)
yindo added the 📚 documentationstale labels 2026-02-21 18:21:26 -05:00
yindo closed this issue 2026-02-21 18:21:26 -05:00
yindo commented 2026-02-21 18:21:26 -05:00
Author
Owner
Copy Link
Copy Source

@dosubot[bot] commented on GitHub (Jan 8, 2025):

The process_rules function in the Dify codebase does indeed support a hierarchical mode. This mode is specifically configured within the getProcessRule function. When the hierarchical mode is selected, the function sets up a ProcessRule object that includes pre-processing rules, segmentation for parent chunks, and subchunk segmentation for child chunks. The configuration uses delimiters and maximum token lengths specified in parentChildConfig for both parent and child segments, and the parent_mode is determined by parentChildConfig.chunkForContext [1].

If you're experiencing issues with the hierarchical mode not functioning as expected, it might be worth checking the configuration of parentChildConfig and ensuring that all related parameters are correctly set. This includes verifying that doc_form is set to hierarchical_model and parent_mode is set to the desired value, such as paragraph. If these configurations are correct and the issue persists, it might be necessary to review the implementation details or any related modules that interact with process_rules.

To continue talking to Dosu, mention @dosu.

Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Other

@dosubot[bot] commented on GitHub (Jan 8, 2025): <!-- Answer --> The `process_rules` function in the Dify codebase does indeed support a `hierarchical` mode. This mode is specifically configured within the `getProcessRule` function. When the `hierarchical` mode is selected, the function sets up a `ProcessRule` object that includes pre-processing rules, segmentation for parent chunks, and subchunk segmentation for child chunks. The configuration uses delimiters and maximum token lengths specified in `parentChildConfig` for both parent and child segments, and the `parent_mode` is determined by `parentChildConfig.chunkForContext` <sup>[[1]](https://github.com/langgenius/dify/blob/main/web/app/components/datasets/create/step-two/index.tsx)</sup>. If you're experiencing issues with the `hierarchical` mode not functioning as expected, it might be worth checking the configuration of `parentChildConfig` and ensuring that all related parameters are correctly set. This includes verifying that `doc_form` is set to `hierarchical_model` and `parent_mode` is set to the desired value, such as `paragraph`. If these configurations are correct and the issue persists, it might be necessary to review the implementation details or any related modules that interact with `process_rules`. <!-- Dosu Comment Footer --> *To continue talking to [Dosu](https://dosu.dev), mention @dosu.* --- <sup>Help Dosu learn! Give it feedback: [Great Response](https://app.dosu.dev/response-feedback/b027a3d8-7f39-4457-97b0-67e9e20b3950?feedback_type=great_response) | [Irrelevant Answer](https://app.dosu.dev/response-feedback/b027a3d8-7f39-4457-97b0-67e9e20b3950?feedback_type=irrelevant_answer) | [Incorrect Sources](https://app.dosu.dev/response-feedback/b027a3d8-7f39-4457-97b0-67e9e20b3950?feedback_type=incorrect_sources) | [Too Verbose](https://app.dosu.dev/response-feedback/b027a3d8-7f39-4457-97b0-67e9e20b3950?feedback_type=too_verbose) | [Hallucination](https://app.dosu.dev/response-feedback/b027a3d8-7f39-4457-97b0-67e9e20b3950?feedback_type=hallucination) | [Other](https://app.dosu.dev/response-feedback/b027a3d8-7f39-4457-97b0-67e9e20b3950?feedback_type=other)</sup>
yindo commented 2026-02-21 18:21:26 -05:00
Author
Owner
Copy Link
Copy Source

@crazywoola commented on GitHub (Jan 8, 2025):

We had a fix https://github.com/langgenius/dify/pull/12495 here. And we will upgrade the docs soon.

@crazywoola commented on GitHub (Jan 8, 2025): We had a fix https://github.com/langgenius/dify/pull/12495 here. And we will upgrade the docs soon.
yindo commented 2026-02-21 18:21:26 -05:00
Author
Owner
Copy Link
Copy Source

@dosubot[bot] commented on GitHub (Feb 8, 2025):

Hi, @pr-maia. I'm Dosu, and I'm helping the Dify team manage their backlog. I'm marking this issue as stale.

Issue Summary:

  • You raised a concern about the missing "hierarchical" option in the process_rules function documentation.
  • I confirmed the support for the hierarchical mode and provided guidance.
  • @crazywoola mentioned a fix in a pull request to update the documentation.
  • You acknowledged the resolution with a thumbs-up reaction.

Next Steps:

  • Please confirm if this issue is still relevant to the latest version of the Dify repository. If so, feel free to comment to keep the discussion open.
  • If there are no further updates, this issue will be automatically closed in 15 days.

Thank you for your understanding and contribution!

@dosubot[bot] commented on GitHub (Feb 8, 2025): Hi, @pr-maia. I'm [Dosu](https://dosu.dev), and I'm helping the Dify team manage their backlog. I'm marking this issue as stale. **Issue Summary:** - You raised a concern about the missing "hierarchical" option in the `process_rules` function documentation. - I confirmed the support for the `hierarchical` mode and provided guidance. - @crazywoola mentioned a fix in a pull request to update the documentation. - You acknowledged the resolution with a thumbs-up reaction. **Next Steps:** - Please confirm if this issue is still relevant to the latest version of the Dify repository. If so, feel free to comment to keep the discussion open. - If there are no further updates, this issue will be automatically closed in 15 days. Thank you for your understanding and contribution!
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Knowledge Pipeline
Proposal
WIP
cloud
dependencies
feat:webapp
good first issue
outdated
performance
pull-request
Mirrored from GitHub Pull Request
 python:uv
refactor
size:L
stale
status: accepting prs
status: blocked
status: in discussion
stt & tts
vulnerablity
web
wont-fix
🌊 feat:workflow
🌚 invalid
🐞 bug
👻 feat:rag
💪 enhancement
📚 documentation
🔨 feat:tools
🙋‍♂️ question
🙏 help wanted
🤔 cant-reproduce
🤖 feat:agent
No Label stale 📚 documentation
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: langgenius/dify#7601
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?