langgenius/dify
1
0
Fork 0
mirror of https://github.com/langgenius/dify.git synced 2026-07-22 04:25:29 -04:00
Code Issues 775 Packages Projects Releases 10 Wiki Activity

Knowledge Base serves old document version #21761

New Issue
Closed
opened 2026-02-21 20:14:11 -05:00 by yindo · 4 comments
yindo commented 2026-02-21 20:14:11 -05:00
Owner
Copy Link
Copy Source

Originally created by @DavideDelbianco on GitHub (Jan 19, 2026).

Self Checks

  • I have read the Contributing Guide and Language Policy.
  • This is only for bug report, if you would like to ask a question, please head to Discussions.
  • I have searched for existing issues search for existing issues, including closed ones.
  • I confirm that I am using English to submit this report, otherwise it will be closed.
  • 【中文用户 & Non English User】请使用英语提交，否则会被关闭 ：）
  • Please do not modify this template :) and fill in all the required fields.

Dify version

1.11.1

Cloud or Self Hosted

Self Hosted (Docker)

Steps to reproduce

I have two copies of a chatflow.
in the chatflow the agent node uses a workflow published as MCP to search the Knowledge Base.

I have update a document keeping the same name through the api call: /datasets/{{dataset_id}}/document/create-by-text

the document was graphically overridden and in the UI i see the new version.
But the Knowledge Search sometimes returns the old "version" and sometimes the new "version"
Inserting manually the GUID of the old documents returns it in dify UI

✔️ Expected Behavior

Then inserting a document with the same name, it should override the old document, if the UI does not show the old document, the previous version should no longer exists

❌ Actual Behavior

The old version still exists and is present in both Vector DB and Dify DB and can be navigated and retrieved by manually inputting it's old GUID

Originally created by @DavideDelbianco on GitHub (Jan 19, 2026). ### Self Checks - [x] I have read the [Contributing Guide](https://github.com/langgenius/dify/blob/main/CONTRIBUTING.md) and [Language Policy](https://github.com/langgenius/dify/issues/1542). - [x] This is only for bug report, if you would like to ask a question, please head to [Discussions](https://github.com/langgenius/dify/discussions/categories/general). - [x] I have searched for existing issues [search for existing issues](https://github.com/langgenius/dify/issues), including closed ones. - [x] I confirm that I am using English to submit this report, otherwise it will be closed. - [x] 【中文用户 & Non English User】请使用英语提交，否则会被关闭 ：） - [x] Please do not modify this template :) and fill in all the required fields. ### Dify version 1.11.1 ### Cloud or Self Hosted Self Hosted (Docker) ### Steps to reproduce I have two copies of a chatflow. in the chatflow the agent node uses a workflow published as MCP to search the Knowledge Base. I have update a document keeping the same name through the api call: `/datasets/{{dataset_id}}/document/create-by-text` the document was graphically overridden and in the UI i see the new version. But the Knowledge Search sometimes returns the old "version" and sometimes the new "version" Inserting manually the GUID of the old documents returns it in dify UI ### ✔️ Expected Behavior Then inserting a document with the same name, it should override the old document, if the UI does not show the old document, the previous version should no longer exists ### ❌ Actual Behavior The old version still exists and is present in both Vector DB and Dify DB and can be navigated and retrieved by manually inputting it's old GUID
yindo added the 🐞 bug label 2026-02-21 20:14:11 -05:00
yindo closed this issue 2026-02-21 20:14:11 -05:00
yindo commented 2026-02-21 20:14:11 -05:00
Author
Owner
Copy Link
Copy Source

@dosubot[bot] commented on GitHub (Jan 19, 2026):

This is a known issue in Dify v1.11.1: when you update a document using /datasets/{{dataset_id}}/document/create-by-text with the same name, Dify overwrites the existing document's content and triggers re-indexing, but it does not explicitly delete old versions. As a result, old document GUIDs can remain accessible in both the database and vector store, causing inconsistent search results—even if the UI only shows the new version. This overwrite-by-name behavior is intentional and managed in backend logic, but cleanup of old entries is incomplete, so manual retrieval by GUID can still surface previous versions reference.

The update and re-indexing process is asynchronous, which can also cause brief inconsistencies during processing reference. There is no API parameter to disable this deduplication/overwrite logic; to avoid overwriting, you must use unique document names or modify backend logic.

If you need to fully remove old versions, the current workaround is to manually delete outdated documents by their GUID using the delete API. For persistent indexing issues, check that your background worker is running and healthy, or consider deleting and recreating the document if stuck reference.

No recent updates or merged pull requests address this issue, so upgrading to a newer version may not resolve it yet.

To reply, just mention @dosu.

How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other

Chat with Dosu Open in Cursor Join Discord Share on X

@dosubot[bot] commented on GitHub (Jan 19, 2026): <!-- Answer --> This is a known issue in Dify v1.11.1: when you update a document using `/datasets/{{dataset_id}}/document/create-by-text` with the same name, Dify overwrites the existing document's content and triggers re-indexing, but it does not explicitly delete old versions. As a result, old document GUIDs can remain accessible in both the database and vector store, causing inconsistent search results—even if the UI only shows the new version. This overwrite-by-name behavior is intentional and managed in backend logic, but cleanup of old entries is incomplete, so manual retrieval by GUID can still surface previous versions [reference](https://github.com/langgenius/dify/discussions/23245). The update and re-indexing process is asynchronous, which can also cause brief inconsistencies during processing [reference](https://github.com/langgenius/dify/issues/27763). There is no API parameter to disable this deduplication/overwrite logic; to avoid overwriting, you must use unique document names or modify backend logic. If you need to fully remove old versions, the current workaround is to manually delete outdated documents by their GUID using the delete API. For persistent indexing issues, check that your background worker is running and healthy, or consider deleting and recreating the document if stuck [reference](https://github.com/langgenius/dify/discussions/22016). No recent updates or merged pull requests address this issue, so upgrading to a newer version may not resolve it yet. <!-- Dosu Comment Footer --> *To reply, just mention [@dosu](https://go.dosu.dev/dosubot).* --- <sup>How did I do? [Good](https://app.dosu.dev/response-feedback/daf81e9d-0397-4c0d-8d5e-8ea966b06b97?feedback_type=great_response) | [Irrelevant](https://app.dosu.dev/response-feedback/daf81e9d-0397-4c0d-8d5e-8ea966b06b97?feedback_type=irrelevant_answer) | [Incorrect](https://app.dosu.dev/response-feedback/daf81e9d-0397-4c0d-8d5e-8ea966b06b97?feedback_type=incorrect_sources) | [Verbose](https://app.dosu.dev/response-feedback/daf81e9d-0397-4c0d-8d5e-8ea966b06b97?feedback_type=too_verbose) | [Hallucination](https://app.dosu.dev/response-feedback/daf81e9d-0397-4c0d-8d5e-8ea966b06b97?feedback_type=hallucination) | [Report 🐛](https://app.dosu.dev/response-feedback/daf81e9d-0397-4c0d-8d5e-8ea966b06b97?feedback_type=bug_report) | [Other](https://app.dosu.dev/response-feedback/daf81e9d-0397-4c0d-8d5e-8ea966b06b97?feedback_type=other)</sup> [![Chat with Dosu](https://dosu.dev/dosu-chat-badge.svg)](https://app.dosu.dev/6088a4f4-073a-43f3-b3d4-4f06fb0c05a0/ask?utm_source=github)&nbsp;[![Open in Cursor](https://dosu.dev/dosu-in-cursor.svg)](https://cursor.com/link/prompt?text=This%20is%20a%20known%20issue%20in%20Dify%20v1.11.1%3A%20when%20you%20update%20a%20document%20using%20%60/datasets/%7B%7Bdataset_id%7D%7D/document/create-by-text%60%20with%20the%20same%20name%2C%20Dify%20overwrites%20the%20existing%20document%27s%20content%20and%20triggers%20re-indexing%2C%20but%20it%20does%20not%20explicitly%20delete%20old%20versions.%20As%20a%20result%2C%20old%20document%20GUIDs%20can%20remain%20accessible%20in%20both%20the%20database%20and%20vector%20store%2C%20causing%20inconsistent%20search%20results%E2%80%94even%20if%20the%20UI%20only%20shows%20the%20new%20version.%20This%20overwrite-by-name%20behavior%20is%20intentional%20and%20managed%20in%20backend%20logic%2C%20but%20cleanup%20of%20old%20entries%20is%20incomplete%2C%20so%20manual%20retrieval%20by%20GUID%20can%20still%20surface%20previous%20versions%20%5Breference%5D%28https%3A//github.com/langgenius/dify/discussions/23245%29.%0A%0AThe%20update%20and%20re-indexing%20process%20is%20asynchronous%2C%20which%20can%20also%20cause%20brief%20inconsistencies%20during%20processing%20%5Breference%5D%28https%3A//github.com/langgenius/dify/issues/27763%29.%20There%20is%20no%20API%20parameter%20to%20disable%20this%20deduplication/overwrite%20logic%3B%20to%20avoid%20overwriting%2C%20you%20must%20use%20unique%20document%20names%20or%20modify%20backend%20logic.%0A%0AIf%20you%20need%20to%20fully%20remove%20old%20versions%2C%20the%20current%20workaround%20is%20to%20manually%20delete%20outdated%20documents%20by%20their%20GUID%20using%20the%20delete%20API.%20For%20persistent%20indexing%20issues%2C%20check%20that%20your%20background%20worker%20is%20running%20and%20healthy%2C%20or%20consider%20deleting%20and%20recreating%20the%20document%20if%20stuck%20%5Breference%5D%28https%3A//github.com/langgenius/dify/discussions/22016%29.%0A%0ANo%20recent%20updates%20or%20merged%20pull%20requests%20address%20this%20issue%2C%20so%20upgrading%20to%20a%20newer%20version%20may%20not%20resolve%20it%20yet.)&nbsp;[![Join Discord](https://img.shields.io/badge/join-5865F2?logo=discord&logoColor=white&label=)](https://go.dosu.dev/discord-bot)&nbsp;[![Share on X](https://img.shields.io/badge/X-share-black)](https://twitter.com/intent/tweet?text=%40dosu_ai%20helped%20me%20solve%20this%20issue!&url=https%3A//github.com/langgenius/dify/issues/31225)
yindo commented 2026-02-21 20:14:12 -05:00
Author
Owner
Copy Link
Copy Source

@DavideDelbianco commented on GitHub (Jan 19, 2026):

Using the UI an adding a file with same name, replaces the old one keeping same GUID.
The same thing using API call generates a new GUID

@DavideDelbianco commented on GitHub (Jan 19, 2026): Using the UI an adding a file with same name, replaces the old one keeping same GUID. The same thing using API call generates a new GUID
yindo commented 2026-02-21 20:14:12 -05:00
Author
Owner
Copy Link
Copy Source

@dosubot[bot] commented on GitHub (Jan 19, 2026):

This difference comes from how the backend handles document updates in the UI vs. the API. When you upload a file with the same name through the UI, it uses an update path that sets the original document's ID, so the document is updated in place and keeps the same GUID. In contrast, the API's create endpoints (like /datasets/{dataset_id}/document/create-by-text) do not set the original document ID, so they always create a new document record with a new GUID—even if the name matches an existing document. That's why you see the GUID retained in the UI but a new one generated via the API.

If you want to update a document via API and keep the same GUID, use the update endpoints (such as /datasets/{dataset_id}/documents/{document_id}/update-by-text or /update-by-file) and provide the document's ID. This will update the existing document in place, just like the UI does. More details and code references are available in the backend logic here and here.

To reply, just mention @dosu.

How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other

Chat with Dosu Open in Cursor Join Discord Share on X

@dosubot[bot] commented on GitHub (Jan 19, 2026): <!-- Answer --> This difference comes from how the backend handles document updates in the UI vs. the API. When you upload a file with the same name through the UI, it uses an update path that sets the original document's ID, so the document is updated in place and keeps the same GUID. In contrast, the API's create endpoints (like `/datasets/{dataset_id}/document/create-by-text`) do not set the original document ID, so they always create a new document record with a new GUID—even if the name matches an existing document. That's why you see the GUID retained in the UI but a new one generated via the API. If you want to update a document via API and keep the same GUID, use the update endpoints (such as `/datasets/{dataset_id}/documents/{document_id}/update-by-text` or `/update-by-file`) and provide the document's ID. This will update the existing document in place, just like the UI does. More details and code references are available in the backend logic [here](https://github.com/langgenius/dify/blob/2d4289a92588390f92f192bb130bfb23c79392eb/api/controllers/service_api/dataset/document.py#L77-L444) and [here](https://github.com/langgenius/dify/blob/2d4289a92588390f92f192bb130bfb23c79392eb/api/services/dataset_service.py#L1516-L2301). <!-- Dosu Comment Footer --> *To reply, just mention [@dosu](https://go.dosu.dev/dosubot).* --- <sup>How did I do? [Good](https://app.dosu.dev/response-feedback/a755a757-4758-4e9f-b61a-0e8147d8f4e2?feedback_type=great_response) | [Irrelevant](https://app.dosu.dev/response-feedback/a755a757-4758-4e9f-b61a-0e8147d8f4e2?feedback_type=irrelevant_answer) | [Incorrect](https://app.dosu.dev/response-feedback/a755a757-4758-4e9f-b61a-0e8147d8f4e2?feedback_type=incorrect_sources) | [Verbose](https://app.dosu.dev/response-feedback/a755a757-4758-4e9f-b61a-0e8147d8f4e2?feedback_type=too_verbose) | [Hallucination](https://app.dosu.dev/response-feedback/a755a757-4758-4e9f-b61a-0e8147d8f4e2?feedback_type=hallucination) | [Report 🐛](https://app.dosu.dev/response-feedback/a755a757-4758-4e9f-b61a-0e8147d8f4e2?feedback_type=bug_report) | [Other](https://app.dosu.dev/response-feedback/a755a757-4758-4e9f-b61a-0e8147d8f4e2?feedback_type=other)</sup> [![Chat with Dosu](https://dosu.dev/dosu-chat-badge.svg)](https://app.dosu.dev/6088a4f4-073a-43f3-b3d4-4f06fb0c05a0/ask?utm_source=github)&nbsp;[![Open in Cursor](https://dosu.dev/dosu-in-cursor.svg)](https://cursor.com/link/prompt?text=This%20difference%20comes%20from%20how%20the%20backend%20handles%20document%20updates%20in%20the%20UI%20vs.%20the%20API.%20When%20you%20upload%20a%20file%20with%20the%20same%20name%20through%20the%20UI%2C%20it%20uses%20an%20update%20path%20that%20sets%20the%20original%20document%27s%20ID%2C%20so%20the%20document%20is%20updated%20in%20place%20and%20keeps%20the%20same%20GUID.%20In%20contrast%2C%20the%20API%27s%20create%20endpoints%20%28like%20%60/datasets/%7Bdataset_id%7D/document/create-by-text%60%29%20do%20not%20set%20the%20original%20document%20ID%2C%20so%20they%20always%20create%20a%20new%20document%20record%20with%20a%20new%20GUID%E2%80%94even%20if%20the%20name%20matches%20an%20existing%20document.%20That%27s%20why%20you%20see%20the%20GUID%20retained%20in%20the%20UI%20but%20a%20new%20one%20generated%20via%20the%20API.%0A%0AIf%20you%20want%20to%20update%20a%20document%20via%20API%20and%20keep%20the%20same%20GUID%2C%20use%20the%20update%20endpoints%20%28such%20as%20%60/datasets/%7Bdataset_id%7D/documents/%7Bdocument_id%7D/update-by-text%60%20or%20%60/update-by-file%60%29%20and%20provide%20the%20document%27s%20ID.%20This%20will%20update%20the%20existing%20document%20in%20place%2C%20just%20like%20the%20UI%20does.%20More%20details%20and%20code%20references%20are%20available%20in%20the%20backend%20logic%20%5Bhere%5D%28https%3A//github.com/langgenius/dify/blob/2d4289a92588390f92f192bb130bfb23c79392eb/api/controllers/service_api/dataset/document.py%23L77-L444%29%20and%20%5Bhere%5D%28https%3A//github.com/langgenius/dify/blob/2d4289a92588390f92f192bb130bfb23c79392eb/api/services/dataset_service.py%23L1516-L2301%29.)&nbsp;[![Join Discord](https://img.shields.io/badge/join-5865F2?logo=discord&logoColor=white&label=)](https://go.dosu.dev/discord-bot)&nbsp;[![Share on X](https://img.shields.io/badge/X-share-black)](https://twitter.com/intent/tweet?text=%40dosu_ai%20helped%20me%20solve%20this%20issue!&url=https%3A//github.com/langgenius/dify/issues/31225)
yindo commented 2026-02-21 20:14:12 -05:00
Author
Owner
Copy Link
Copy Source

@DavideDelbianco commented on GitHub (Jan 19, 2026):

turns out it was a badly updated version with screwed up Weaviate, not a dify issue

@DavideDelbianco commented on GitHub (Jan 19, 2026): turns out it was a badly updated version with screwed up Weaviate, not a dify issue
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Knowledge Pipeline
Proposal
WIP
cloud
dependencies
feat:webapp
good first issue
outdated
performance
pull-request
Mirrored from GitHub Pull Request
 python:uv
refactor
size:L
stale
status: accepting prs
status: blocked
status: in discussion
stt & tts
vulnerablity
web
wont-fix
🌊 feat:workflow
🌚 invalid
🐞 bug
👻 feat:rag
💪 enhancement
📚 documentation
🔨 feat:tools
🙋‍♂️ question
🙏 help wanted
🤔 cant-reproduce
🤖 feat:agent
No Label 🐞 bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: langgenius/dify#21761
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?