langgenius/dify
1
0
Fork 0
mirror of https://github.com/langgenius/dify.git synced 2026-07-21 03:55:25 -04:00
Code Issues 775 Packages Projects Releases 10 Wiki Activity

Create from Knowledge Pipeline, parsing error: Index chunk variable is required #20396

New Issue
Closed
opened 2026-02-21 20:07:12 -05:00 by yindo · 6 comments
yindo commented 2026-02-21 20:07:12 -05:00
Owner
Copy Link
Copy Source

Originally created by @jasonleelover on GitHub (Nov 18, 2025).

Self Checks

  • I have read the Contributing Guide and Language Policy.
  • This is only for bug report, if you would like to ask a question, please head to Discussions.
  • I have searched for existing issues search for existing issues, including closed ones.
  • I confirm that I am using English to submit this report, otherwise it will be closed.
  • 【中文用户 & Non English User】请使用英语提交，否则会被关闭 ：）
  • Please do not modify this template :) and fill in all the required fields.

Dify version

1.10.0

Cloud or Self Hosted

Self Hosted (Source)

Steps to reproduce

  1. Create from Knowledge Pipeline
  2. Upload CSV document, parsing error: Index chunk variable is required

Knowledge Pipeline：
Image

file config：
Image

Q&A config:
Image

Q&A runing input:
Image

Q&A runing output:
Image

api error log:
Image

✔️ Expected Behavior

running successful

❌ Actual Behavior

running failed

Originally created by @jasonleelover on GitHub (Nov 18, 2025). ### Self Checks - [x] I have read the [Contributing Guide](https://github.com/langgenius/dify/blob/main/CONTRIBUTING.md) and [Language Policy](https://github.com/langgenius/dify/issues/1542). - [x] This is only for bug report, if you would like to ask a question, please head to [Discussions](https://github.com/langgenius/dify/discussions/categories/general). - [x] I have searched for existing issues [search for existing issues](https://github.com/langgenius/dify/issues), including closed ones. - [x] I confirm that I am using English to submit this report, otherwise it will be closed. - [x] 【中文用户 & Non English User】请使用英语提交，否则会被关闭 ：） - [x] Please do not modify this template :) and fill in all the required fields. ### Dify version 1.10.0 ### Cloud or Self Hosted Self Hosted (Source) ### Steps to reproduce 1. **Create from Knowledge Pipeline** 2. **Upload CSV document, parsing error: Index chunk variable is required** Knowledge Pipeline： <img width="1736" height="572" alt="Image" src="https://github.com/user-attachments/assets/e32ef846-7ec7-4fee-9c22-4272447f6624" /> file config： <img width="544" height="626" alt="Image" src="https://github.com/user-attachments/assets/21d28cf8-e760-4f28-b49b-8d7a35878703" /> Q&A config: <img width="539" height="610" alt="Image" src="https://github.com/user-attachments/assets/73139977-6220-4639-89ce-69f8ceb54c42" /> Q&A runing input: <img width="547" height="633" alt="Image" src="https://github.com/user-attachments/assets/a8319591-9657-4451-b3b1-ee01c6beefc0" /> Q&A runing output: <img width="523" height="578" alt="Image" src="https://github.com/user-attachments/assets/ad00f8df-1b14-405b-bde8-3d3243cbc664" /> api error log: <img width="1844" height="604" alt="Image" src="https://github.com/user-attachments/assets/61773829-6f57-444b-8e71-1081e7572004" /> ### ✔️ Expected Behavior running successful ### ❌ Actual Behavior running failed
yindo added the 🐞 bugKnowledge Pipeline labels 2026-02-21 20:07:12 -05:00
yindo closed this issue 2026-02-21 20:07:12 -05:00
yindo commented 2026-02-21 20:07:12 -05:00
Author
Owner
Copy Link
Copy Source

@dosubot[bot] commented on GitHub (Nov 18, 2025):

The error means the knowledge base node can't find a valid chunk variable for indexing. This usually happens if the chunk variable (the processed segments from your CSV) isn't produced or mapped correctly from upstream nodes, often due to a schema/type mismatch or a failed CSV retrieval step. The chunk variable must match the expected schemaType for your selected chunk structure (like qa_structure for Q&A chunking) and be registered as a shared variable in the pipeline. If the variable doesn't appear in the selector, it's likely a type or registration issue.

To resolve this, double-check that your chunking node (QA chunker or General Chunk tool) is installed and configured, and that its output variable matches the expected schema for the knowledge base node. Explicitly select the chunk variable in the knowledge base node, or recreate it with the correct schemaType if needed. If you're running Dify v1.10.0 or v1.10.1, be aware there's a known bug blocking pipeline creation and deployment—upgrading to a newer release where this validation bug is fixed is the only reliable solution. See issue details and code references and validation bug fix PR for more info.

If you can't upgrade immediately, make sure all required services are running and try installing the General Chunk tool, as this has resolved the issue for other users.

To reply, just mention @dosu.

How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other  Chat with Dosu Join Discord Share on X

@dosubot[bot] commented on GitHub (Nov 18, 2025): <!-- Answer --> The error means the knowledge base node can't find a valid chunk variable for indexing. This usually happens if the chunk variable (the processed segments from your CSV) isn't produced or mapped correctly from upstream nodes, often due to a schema/type mismatch or a failed CSV retrieval step. The chunk variable must match the expected schemaType for your selected chunk structure (like `qa_structure` for Q&A chunking) and be registered as a shared variable in the pipeline. If the variable doesn't appear in the selector, it's likely a type or registration issue. To resolve this, double-check that your chunking node (QA chunker or General Chunk tool) is installed and configured, and that its output variable matches the expected schema for the knowledge base node. Explicitly select the chunk variable in the knowledge base node, or recreate it with the correct schemaType if needed. If you're running Dify v1.10.0 or v1.10.1, be aware there's a known bug blocking pipeline creation and deployment—upgrading to a newer release where this validation bug is fixed is the only reliable solution. See [issue details and code references](https://github.com/langgenius/dify/issues/26298) and [validation bug fix PR](https://github.com/langgenius/dify/pull/28203) for more info. If you can't upgrade immediately, make sure all required services are running and try installing the General Chunk tool, as this has resolved the issue for other users. <!-- Dosu Comment Footer --> *To reply, just mention [@dosu](https://go.dosu.dev/dosubot).* --- <sup>How did I do? [Good](https://app.dosu.dev/response-feedback/1e7ccb7d-a64e-4d0c-bf95-9a6eec8baf62?feedback_type=great_response) | [Irrelevant](https://app.dosu.dev/response-feedback/1e7ccb7d-a64e-4d0c-bf95-9a6eec8baf62?feedback_type=irrelevant_answer) | [Incorrect](https://app.dosu.dev/response-feedback/1e7ccb7d-a64e-4d0c-bf95-9a6eec8baf62?feedback_type=incorrect_sources) | [Verbose](https://app.dosu.dev/response-feedback/1e7ccb7d-a64e-4d0c-bf95-9a6eec8baf62?feedback_type=too_verbose) | [Hallucination](https://app.dosu.dev/response-feedback/1e7ccb7d-a64e-4d0c-bf95-9a6eec8baf62?feedback_type=hallucination) | [Report 🐛](https://app.dosu.dev/response-feedback/1e7ccb7d-a64e-4d0c-bf95-9a6eec8baf62?feedback_type=bug_report) | [Other](https://app.dosu.dev/response-feedback/1e7ccb7d-a64e-4d0c-bf95-9a6eec8baf62?feedback_type=other)</sup>&nbsp;&nbsp;[![Chat with Dosu](https://dosu.dev/dosu-chat-badge.svg)](https://app.dosu.dev/6088a4f4-073a-43f3-b3d4-4f06fb0c05a0/ask?utm_source=github)&nbsp;[![Join Discord](https://img.shields.io/badge/join-5865F2?logo=discord&logoColor=white&label=)](https://go.dosu.dev/discord-bot)&nbsp;[![Share on X](https://img.shields.io/badge/X-share-black)](https://twitter.com/intent/tweet?text=%40dosu_ai%20helped%20me%20solve%20this%20issue!&url=https%3A//github.com/langgenius/dify/issues/28369)
yindo commented 2026-02-21 20:07:13 -05:00
Author
Owner
Copy Link
Copy Source

@water-in-stone commented on GitHub (Nov 18, 2025):

Could you please provide the simple DSL for reproduce this issue

@water-in-stone commented on GitHub (Nov 18, 2025): Could you please provide the simple DSL for reproduce this issue
yindo commented 2026-02-21 20:07:13 -05:00
Author
Owner
Copy Link
Copy Source

@jasonleelover commented on GitHub (Nov 18, 2025):

Could you please provide the simple DSL for reproduce this issue

Of course, GitHub does not support xx. pipeline. Please modify the file suffix before importing
QA-test.txt

@jasonleelover commented on GitHub (Nov 18, 2025): > Could you please provide the simple DSL for reproduce this issue Of course, GitHub does not support xx. pipeline. Please modify the file suffix before importing [QA-test.txt](https://github.com/user-attachments/files/23617613/QA-test.txt)
yindo commented 2026-02-21 20:07:13 -05:00
Author
Owner
Copy Link
Copy Source

@holenzzz commented on GitHub (Nov 19, 2025):

same problem.

@holenzzz commented on GitHub (Nov 19, 2025): same problem.
yindo commented 2026-02-21 20:07:13 -05:00
Author
Owner
Copy Link
Copy Source

@water-in-stone commented on GitHub (Nov 22, 2025):

Could you please provide the simple DSL for reproduce this issue

Of course, GitHub does not support xx. pipeline. Please modify the file suffix before importing QA-test.txt

The DSL you are using is outdated and cannot complete initialization. Could you please provide a more recent version (e.g., 0.0.4) ? So I can reproduce this issue

Image Image
@water-in-stone commented on GitHub (Nov 22, 2025): > > Could you please provide the simple DSL for reproduce this issue > > Of course, GitHub does not support xx. pipeline. Please modify the file suffix before importing [QA-test.txt](https://github.com/user-attachments/files/23617613/QA-test.txt) The DSL you are using is outdated and cannot complete initialization. Could you please provide a more recent version (e.g., 0.0.4) ? So I can reproduce this issue <img width="1128" height="1048" alt="Image" src="https://github.com/user-attachments/assets/9a5fe961-45c7-49c1-b69c-c036c4520fde" /> <img width="1214" height="56" alt="Image" src="https://github.com/user-attachments/assets/77e5c691-53d8-4e8a-8d17-5b6a83e9c0a7" />
yindo commented 2026-02-21 20:07:13 -05:00
Author
Owner
Copy Link
Copy Source

@dosubot[bot] commented on GitHub (Jan 12, 2026):

Hi, @jasonleelover. I'm Dosu, and I'm helping the Dify team manage their backlog and am marking this issue as stale.

Issue Summary:

  • You reported a parsing error "Index chunk variable is required" when creating a knowledge pipeline from a CSV upload in Dify v1.10.0 (self-hosted).
  • The error was linked to missing or mismatched chunk variables due to schema or registration issues.
  • The problem was commonly resolved by ensuring the chunking node outputs the correct variable and upgrading to a version later than v1.10.1.
  • Other users experienced the same issue, and an updated DSL file was requested to reproduce the problem.
  • The issue has been resolved by upgrading Dify past v1.10.1 and using the correct DSL configuration.

Next Steps:

  • Please confirm if this issue is still relevant with the latest version of Dify you are using by commenting here.
  • If no further updates are provided, I will automatically close this issue in 15 days.

Thank you for your understanding and contribution!

@dosubot[bot] commented on GitHub (Jan 12, 2026): Hi, @jasonleelover. I'm [Dosu](https://dosu.dev), and I'm helping the Dify team manage their backlog and am marking this issue as stale. **Issue Summary:** - You reported a parsing error "Index chunk variable is required" when creating a knowledge pipeline from a CSV upload in Dify v1.10.0 (self-hosted). - The error was linked to missing or mismatched chunk variables due to schema or registration issues. - The problem was commonly resolved by ensuring the chunking node outputs the correct variable and upgrading to a version later than v1.10.1. - Other users experienced the same issue, and an updated DSL file was requested to reproduce the problem. - The issue has been resolved by upgrading Dify past v1.10.1 and using the correct DSL configuration. **Next Steps:** - Please confirm if this issue is still relevant with the latest version of Dify you are using by commenting here. - If no further updates are provided, I will automatically close this issue in 15 days. Thank you for your understanding and contribution!
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Knowledge Pipeline
Proposal
WIP
cloud
dependencies
feat:webapp
good first issue
outdated
performance
pull-request
Mirrored from GitHub Pull Request
 python:uv
refactor
size:L
stale
status: accepting prs
status: blocked
status: in discussion
stt & tts
vulnerablity
web
wont-fix
🌊 feat:workflow
🌚 invalid
🐞 bug
👻 feat:rag
💪 enhancement
📚 documentation
🔨 feat:tools
🙋‍♂️ question
🙏 help wanted
🤔 cant-reproduce
🤖 feat:agent
No Label Knowledge Pipeline 🐞 bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: langgenius/dify#20396
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?