langgenius/dify
1
0
Fork 0
mirror of https://github.com/langgenius/dify.git synced 2026-07-21 20:15:31 -04:00
Code Issues 775 Packages Projects Releases 10 Wiki Activity

Error: Response output is missing or does not contain embeddings. #6434

New Issue
Closed
opened 2026-02-21 18:15:43 -05:00 by yindo · 9 comments
yindo commented 2026-02-21 18:15:43 -05:00
Owner
Copy Link
Copy Source

Originally created by @yinzih on GitHub (Oct 30, 2024).

Self Checks

  • This is only for bug report, if you would like to ask a question, please head to Discussions.
  • I have searched for existing issues search for existing issues, including closed ones.
  • I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
  • [FOR CHINESE USERS] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:）
  • Please do not modify this template :) and fill in all the required fields.

Dify version

0.10.2

Cloud or Self Hosted

Cloud

Steps to reproduce

When I create a knowledge base, I get an error when embedding processing.
[tongyi] Error: Response output is missing or does not contain embeddings.
QQ20241030-095722
How can I solve this problem?

✔️ Expected Behavior

No response

❌ Actual Behavior

No response

Originally created by @yinzih on GitHub (Oct 30, 2024). ### Self Checks - [X] This is only for bug report, if you would like to ask a question, please head to [Discussions](https://github.com/langgenius/dify/discussions/categories/general). - [X] I have searched for existing issues [search for existing issues](https://github.com/langgenius/dify/issues), including closed ones. - [X] I confirm that I am using English to submit this report (我已阅读并同意 [Language Policy](https://github.com/langgenius/dify/issues/1542)). - [X] [FOR CHINESE USERS] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:） - [X] Please do not modify this template :) and fill in all the required fields. ### Dify version 0.10.2 ### Cloud or Self Hosted Cloud ### Steps to reproduce When I create a knowledge base, I get an error when embedding processing. [tongyi] Error: Response output is missing or does not contain embeddings. ![QQ20241030-095722](https://github.com/user-attachments/assets/de83b52b-6719-4efa-8d0f-22e57160b768) How can I solve this problem? ### ✔️ Expected Behavior _No response_ ### ❌ Actual Behavior _No response_
yindo added the 👻 feat:rag label 2026-02-21 18:15:43 -05:00
yindo closed this issue 2026-02-21 18:15:43 -05:00
yindo commented 2026-02-21 18:15:44 -05:00
Author
Owner
Copy Link
Copy Source

@dosubot[bot] commented on GitHub (Oct 30, 2024):

I found a similar unsolved discussion regarding the error "Response output is missing or does not contain embeddings" [1]. Additionally, there are related issues that might provide some insights:

  1. An issue titled "upload document embedding error & After deleting the failed document, the knowledge base will still retain the content of the failed document" suggests ensuring that the document size does not exceed the limit and verifying the ETL configuration. It also provides a method to manually clean up database entries if automatic deletion fails [2].

  2. Another issue, "psycopg2.errors.UndefinedTable: relation 'embedding_vector_index_0799a0c6_ca9a_418e_9e33_900236a74af4_nod' does not exist," suggests checking the Docker Compose configuration and ensuring that migrations are enabled and properly executed [3].

For your specific case, ensure that the document size is within the allowed limits and check the configuration settings related to the embedding process. If the problem persists, reviewing the logs for any additional error messages might provide more clues.

To continue talking to Dosu, mention @dosu.

@dosubot[bot] commented on GitHub (Oct 30, 2024): <!-- Answer --> I found a similar unsolved discussion regarding the error "Response output is missing or does not contain embeddings" <sup>[[1]](https://github.com/langgenius/dify/discussions/10032)</sup>. Additionally, there are related issues that might provide some insights: 1. An issue titled "[upload document embedding error & After deleting the failed document, the knowledge base will still retain the content of the failed document](https://github.com/langgenius/dify/issues/5355)" suggests ensuring that the document size does not exceed the limit and verifying the ETL configuration. It also provides a method to manually clean up database entries if automatic deletion fails <sup>[[2]](https://github.com/langgenius/dify/issues/5355)</sup>. 2. Another issue, "[psycopg2.errors.UndefinedTable: relation 'embedding_vector_index_0799a0c6_ca9a_418e_9e33_900236a74af4_nod' does not exist](https://github.com/langgenius/dify/issues/4696)," suggests checking the Docker Compose configuration and ensuring that migrations are enabled and properly executed <sup>[[3]](https://github.com/langgenius/dify/issues/4696)</sup>. For your specific case, ensure that the document size is within the allowed limits and check the configuration settings related to the embedding process. If the problem persists, reviewing the logs for any additional error messages might provide more clues. <!-- Dosu Comment Footer --> *To continue talking to [Dosu](https://dosu.dev), mention @dosu.*
yindo commented 2026-02-21 18:15:44 -05:00
Author
Owner
Copy Link
Copy Source

@zyy13579 commented on GitHub (Oct 30, 2024):

image
I I'm running into this problem as well，on dify 0.9.1

@zyy13579 commented on GitHub (Oct 30, 2024): ![image](https://github.com/user-attachments/assets/bf5405e8-6ee7-46eb-939a-f84a5a87d643) I I'm running into this problem as well，on dify 0.9.1
yindo commented 2026-02-21 18:15:44 -05:00
Author
Owner
Copy Link
Copy Source

@shisaru292 commented on GitHub (Oct 30, 2024):

I got the same problem with the version 0.10.1. In the sentry log, it shows that the api key is invalid.

image

This issue might be related to API key encryption and storage?

image

This appears to be caused by using encrypted credentials directly as API key

@shisaru292 commented on GitHub (Oct 30, 2024): I got the same problem with the version 0.10.1. In the sentry log, it shows that the api key is invalid. <img width="1012" alt="image" src="https://github.com/user-attachments/assets/ae97dab9-eb4e-49b2-a2bc-883396121e49"> This issue might be related to API key encryption and storage? <img width="639" alt="image" src="https://github.com/user-attachments/assets/e8ac4862-6b75-4b72-be7e-dec7d2d7f582"> This appears to be caused by using encrypted credentials directly as API key
yindo commented 2026-02-21 18:15:44 -05:00
Author
Owner
Copy Link
Copy Source

@shisaru292 commented on GitHub (Oct 31, 2024):

I got the same problem with the version 0.10.1. In the sentry log, it shows that the api key is invalid.
...
This appears to be caused by using encrypted credentials directly as API key

I reviewed the source code, and during the process of obtaining the API key, it first retrieves historical decrypted results from the Redis cache. The stored data in Redis consists of the encrypted results.

image

It seems that this piece of code did not properly modify the credentials, resulting in the direct use of the encrypted values from the database:

https://github.com/langgenius/dify/blob/0.10.1/api/core/provider_manager.py#L588-L595

@shisaru292 commented on GitHub (Oct 31, 2024): > I got the same problem with the version 0.10.1. In the sentry log, it shows that the api key is invalid. > ... > This appears to be caused by using encrypted credentials directly as API key I reviewed the source code, and during the process of obtaining the API key, it first retrieves historical decrypted results from the Redis cache. The stored data in Redis consists of the encrypted results. <img width="602" alt="image" src="https://github.com/user-attachments/assets/a8a659db-57ef-44f2-97d3-6103478deed9"> It seems that this piece of code did not properly modify the credentials, resulting in the direct use of the encrypted values from the database: https://github.com/langgenius/dify/blob/0.10.1/api/core/provider_manager.py#L588-L595
yindo commented 2026-02-21 18:15:44 -05:00
Author
Owner
Copy Link
Copy Source

@shisaru292 commented on GitHub (Nov 1, 2024):

I obtained the encrypted credentials data and private key from Redis in the environment which has the error and simulated the decryption process locally. An exception was thrown on this line of code:

https://github.com/langgenius/dify/blob/main/api/libs/rsa.py#L76

Error message is: Incorrect decryption.

I conducted further debugging and found that the failure was caused by the invalid is 255 on this line of code:

https://github.com/langgenius/dify/blob/main/api/libs/gmpy2_pkcs10aep_cipher.py#L200

I further examined the encryption and decryption process and ultimately discovered that the error was due to the RSA public key stored in the db being inconsistent with the public key retrieved from the private key file. However, I am currently unsure why this inconsistency exists.

image

Perhaps there was an issue with updating the public key when generating the new private key.

image
@shisaru292 commented on GitHub (Nov 1, 2024): I obtained the encrypted credentials data and private key from Redis in the environment which has the error and simulated the decryption process locally. An exception was thrown on this line of code: https://github.com/langgenius/dify/blob/main/api/libs/rsa.py#L76 Error message is: `Incorrect decryption.` I conducted further debugging and found that the failure was caused by the `invalid` is 255 on this line of code: https://github.com/langgenius/dify/blob/main/api/libs/gmpy2_pkcs10aep_cipher.py#L200 I further examined the encryption and decryption process and ultimately discovered that the error was due to the RSA public key stored in the db being inconsistent with the public key retrieved from the private key file. However, I am currently unsure why this inconsistency exists. <img width="693" alt="image" src="https://github.com/user-attachments/assets/e6a206e8-bdd5-4c2c-84d1-180e37b376f9"> Perhaps there was an issue with updating the public key when generating the new private key. <img width="929" alt="image" src="https://github.com/user-attachments/assets/74c99bcd-43aa-4fb5-b35e-eb3c81fc70c4">
yindo commented 2026-02-21 18:15:44 -05:00
Author
Owner
Copy Link
Copy Source

@shisaru292 commented on GitHub (Nov 1, 2024):

From the code, it appears that the public key in the db is only written once during tenant creation. If the private key for the same tenant is regenerated afterward, it will lead to an inconsistency. It seems there are no other methods in the code to update the public key.

One SOLUTION is to use the command provided by Dify to reset the key pair, but be aware that this will invalidate the existing model authentication and require reconfiguration of all providers.

https://github.com/langgenius/dify/blob/main/api/commands.py#L98-L115

In api container, flask reset-encrypt-key-pair

@shisaru292 commented on GitHub (Nov 1, 2024): From the code, it appears that the public key in the db is only written once during tenant creation. If the private key for the same tenant is regenerated afterward, it will lead to an inconsistency. It seems there are no other methods in the code to update the public key. One SOLUTION is to use the command provided by Dify to reset the key pair, but be aware that this will invalidate the existing model authentication and require reconfiguration of all providers. https://github.com/langgenius/dify/blob/main/api/commands.py#L98-L115 In api container, `flask reset-encrypt-key-pair`
yindo commented 2026-02-21 18:15:44 -05:00
Author
Owner
Copy Link
Copy Source

@shisaru292 commented on GitHub (Nov 3, 2024):

I have submitted a Pull Request(#10210) for your review. Thank you! @crazywoola

@shisaru292 commented on GitHub (Nov 3, 2024): I have submitted a Pull Request(#10210) for your review. Thank you! @crazywoola
yindo commented 2026-02-21 18:15:44 -05:00
Author
Owner
Copy Link
Copy Source

@dosubot[bot] commented on GitHub (Dec 4, 2024):

Hi, @yinzih. I'm Dosu, and I'm helping the Dify team manage their backlog. I'm marking this issue as stale.

Issue Summary

  • You reported an error during the embedding process in Dify version 0.10.2 on the cloud platform.
  • The error was related to missing or incomplete embeddings in the response output.
  • User shisaru292 identified a potential issue with API key encryption and proposed a solution.
  • A Pull Request (#10210) was submitted to address the public key inconsistency.

Next Steps

  • Please confirm if this issue is still relevant to the latest version of Dify.
  • If the issue is resolved or no longer relevant, it will be automatically closed in 15 days unless further comments are made.

Thank you for your understanding and contribution!

@dosubot[bot] commented on GitHub (Dec 4, 2024): Hi, @yinzih. I'm [Dosu](https://dosu.dev), and I'm helping the Dify team manage their backlog. I'm marking this issue as stale. **Issue Summary** - You reported an error during the embedding process in Dify version 0.10.2 on the cloud platform. - The error was related to missing or incomplete embeddings in the response output. - User shisaru292 identified a potential issue with API key encryption and proposed a solution. - A Pull Request (#10210) was submitted to address the public key inconsistency. **Next Steps** - Please confirm if this issue is still relevant to the latest version of Dify. - If the issue is resolved or no longer relevant, it will be automatically closed in 15 days unless further comments are made. Thank you for your understanding and contribution!
yindo commented 2026-02-21 18:15:44 -05:00
Author
Owner
Copy Link
Copy Source

@v-49 commented on GitHub (Jan 6, 2025):

still happend at v0.12.1
image

@v-49 commented on GitHub (Jan 6, 2025): still happend at v0.12.1 ![image](https://github.com/user-attachments/assets/4a566716-ff77-4a95-97e1-b8655581b86d)
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Knowledge Pipeline
Proposal
WIP
cloud
dependencies
feat:webapp
good first issue
outdated
performance
pull-request
Mirrored from GitHub Pull Request
 python:uv
refactor
size:L
stale
status: accepting prs
status: blocked
status: in discussion
stt & tts
vulnerablity
web
wont-fix
🌊 feat:workflow
🌚 invalid
🐞 bug
👻 feat:rag
💪 enhancement
📚 documentation
🔨 feat:tools
🙋‍♂️ question
🙏 help wanted
🤔 cant-reproduce
🤖 feat:agent
No Label 👻 feat:rag
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: langgenius/dify#6434
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?