langgenius/dify
1
0
Fork 0
mirror of https://github.com/langgenius/dify.git synced 2026-07-21 20:15:31 -04:00
Code Issues 775 Packages Projects Releases 10 Wiki Activity

The Qwen2-72B-Instruct model cannot return results using the OpenAI-API-compatible configuration #4170

New Issue
Closed
opened 2026-02-21 18:05:02 -05:00 by yindo · 4 comments
yindo commented 2026-02-21 18:05:02 -05:00
Owner
Copy Link
Copy Source

Originally created by @zengyunda on GitHub (Jun 19, 2024).

Self Checks

  • This is only for bug report, if you would like to ask a question, please head to Discussions.
  • I have searched for existing issues search for existing issues, including closed ones.
  • I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
  • Please do not modify this template :) and fill in all the required fields.

Dify version

0.6.8

Cloud or Self Hosted

Self Hosted (Docker)

Steps to reproduce

I deployed the Qwen2-72B-Instruct model using one-api, and deployed the model using OpenAI-API-compatible on dify. When setting one of the fields Completion mode, I cannot select "Dialogue" (an error will be reported), and can only select "Complete" by pulling down. After successful deployment, I select this model in the chat assistant, and there is no content output in the window dialogue. Is there something wrong when the model is deployed?
I can call this model normally using Python code, confirming that this model is deployed successfully.
But the chat assistant using dify cannot return content.
I checked the logs, and this model returns an empty string when requested on dify

✔️ Expected Behavior

Return normal model output results

❌ Actual Behavior

ecf30001647e7af7f64453a158b17fb
Workflow logs returned by using chatglm3 and using the Qwen2-72B-Instruct model respectively

Originally created by @zengyunda on GitHub (Jun 19, 2024). ### Self Checks - [X] This is only for bug report, if you would like to ask a question, please head to [Discussions](https://github.com/langgenius/dify/discussions/categories/general). - [X] I have searched for existing issues [search for existing issues](https://github.com/langgenius/dify/issues), including closed ones. - [X] I confirm that I am using English to submit this report (我已阅读并同意 [Language Policy](https://github.com/langgenius/dify/issues/1542)). - [X] Please do not modify this template :) and fill in all the required fields. ### Dify version 0.6.8 ### Cloud or Self Hosted Self Hosted (Docker) ### Steps to reproduce I deployed the Qwen2-72B-Instruct model using one-api, and deployed the model using OpenAI-API-compatible on dify. When setting one of the fields Completion mode, I cannot select "Dialogue" (an error will be reported), and can only select "Complete" by pulling down. After successful deployment, I select this model in the chat assistant, and there is no content output in the window dialogue. Is there something wrong when the model is deployed? I can call this model normally using Python code, confirming that this model is deployed successfully. But the chat assistant using dify cannot return content. I checked the logs, and this model returns an empty string when requested on dify ### ✔️ Expected Behavior Return normal model output results ### ❌ Actual Behavior ![ecf30001647e7af7f64453a158b17fb](https://github.com/langgenius/dify/assets/38974755/49e16523-d178-408d-9bf7-499fd4e30802) Workflow logs returned by using chatglm3 and using the Qwen2-72B-Instruct model respectively
yindo added the 🐞 bug label 2026-02-21 18:05:02 -05:00
yindo closed this issue 2026-02-21 18:05:02 -05:00
yindo commented 2026-02-21 18:05:03 -05:00
Author
Owner
Copy Link
Copy Source

@dosubot[bot] commented on GitHub (Jun 19, 2024):

@zengyunda Hello! I'm here to help you with any bugs, questions, or contributions you have for the repository. Let's tackle this issue together!

To address the issue with the Qwen2-72B-Instruct model not returning results in Dify's chat assistant, you can follow these troubleshooting steps:

  1. Check API Key and Quota:
    Ensure your OpenAI key is valid and your account has sufficient quota. If your account has run out of funds, you need to recharge it.

  2. Verify Model Deployment:
    Confirm that the Qwen2-72B-Instruct model is successfully deployed. Since you mentioned it works with Python code, this step might already be verified.

  3. Check for Rate Limits:
    Ensure you have not exceeded the official API call rate limits. Refer to the OpenAI official documentation for details.

  4. Update to Latest Version:
    Make sure both the frontend and backend versions of your Dify deployment are up to date and consistent.

  5. Adjust Model Parameters:
    Ensure the parameter values are within the acceptable range for the current model. For example, the temperature should be between 0 and 1.

  6. Reduce Token Limits:
    If the query or prefix prompt is too long, reduce the "max tokens" value in the parameter settings.

  7. Clear Cache:
    Clear the browser cache or the corresponding app cache on mobile devices. Alternatively, regenerate the app URL and access it again.

  8. Check Embedding Model Rate Limits:
    If the Q&A segmentation mode shows "queuing," check if the embedding model API key has reached its rate limit.

Additionally, the "Completion mode" field in Dify affects how the model generates responses. The Qwen2-72B-Instruct model is configured for chat interactions with mode: chat. If "Dialogue" mode in Dify differs from the "chat" mode expected by the model, it could lead to compatibility issues or unexpected behavior [1].

By following these steps, you should be able to diagnose and resolve the issue with the Qwen2-72B-Instruct model in Dify's chat assistant [2].

To continue talking to Dosu, mention @dosu.

@dosubot[bot] commented on GitHub (Jun 19, 2024): <!-- Greeting --> @zengyunda Hello! I'm here to help you with any bugs, questions, or contributions you have for the repository. Let's tackle this issue together! <!-- Answer --> To address the issue with the Qwen2-72B-Instruct model not returning results in Dify's chat assistant, you can follow these troubleshooting steps: 1. **Check API Key and Quota**: Ensure your OpenAI key is valid and your account has sufficient quota. If your account has run out of funds, you need to recharge it. 2. **Verify Model Deployment**: Confirm that the Qwen2-72B-Instruct model is successfully deployed. Since you mentioned it works with Python code, this step might already be verified. 3. **Check for Rate Limits**: Ensure you have not exceeded the official API call rate limits. Refer to the [OpenAI official documentation](https://platform.openai.com/docs/guides/rate-limits) for details. 4. **Update to Latest Version**: Make sure both the frontend and backend versions of your Dify deployment are up to date and consistent. 5. **Adjust Model Parameters**: Ensure the parameter values are within the acceptable range for the current model. For example, the `temperature` should be between 0 and 1. 6. **Reduce Token Limits**: If the query or prefix prompt is too long, reduce the "max tokens" value in the parameter settings. 7. **Clear Cache**: Clear the browser cache or the corresponding app cache on mobile devices. Alternatively, regenerate the app URL and access it again. 8. **Check Embedding Model Rate Limits**: If the Q&A segmentation mode shows "queuing," check if the embedding model API key has reached its rate limit. Additionally, the "Completion mode" field in Dify affects how the model generates responses. The Qwen2-72B-Instruct model is configured for chat interactions with `mode: chat`. If "Dialogue" mode in Dify differs from the "chat" mode expected by the model, it could lead to compatibility issues or unexpected behavior <sup>[[1]](https://github.com/langgenius/dify/blob/main/api/core/model_runtime/model_providers/siliconflow/llm/qwen2-72b-instruct.yaml)</sup>. By following these steps, you should be able to diagnose and resolve the issue with the Qwen2-72B-Instruct model in Dify's chat assistant <sup>[[2]](https://github.com/langgenius/dify-docs/blob/main/zh_CN/learn-more/faq/llms-use-faq.md)</sup>. <!-- Dosu Comment Footer --> *To continue talking to [Dosu](https://dosu.dev), mention @dosu.*
yindo commented 2026-02-21 18:05:03 -05:00
Author
Owner
Copy Link
Copy Source

@jay-jjwu commented on GitHub (Jul 6, 2024):

@zengyunda , is this issue resloved? i use OpenAI-API-compatible on dify to integrate in one-api models, but only can set to chat, instead of completion (always error...see the screenshots), how you resovled this?
image
1720317288120_d

@jay-jjwu commented on GitHub (Jul 6, 2024): @zengyunda , is this issue resloved? i use OpenAI-API-compatible on dify to integrate in one-api models, but only can set to chat, instead of completion (always error...see the screenshots), how you resovled this? ![image](https://github.com/langgenius/dify/assets/78014320/89d10f35-2537-4be2-abf2-bacecc6d3ff4) ![1720317288120_d](https://github.com/langgenius/dify/assets/78014320/ca91631f-e298-49ea-b88f-818133692549)
yindo commented 2026-02-21 18:05:03 -05:00
Author
Owner
Copy Link
Copy Source

@zengyunda commented on GitHub (Jul 6, 2024):

@zengyunda , is this issue resloved? i use OpenAI-API-compatible on dify to integrate in one-api models, but only can set to chat, instead of completion (always error...see the screenshots), how you resovled this? image 1720317288120_d

没有解决，官方没有给出解决方案，根据在官方群的回复，这个应该是底层代码不支持

@zengyunda commented on GitHub (Jul 6, 2024): > @zengyunda , is this issue resloved? i use OpenAI-API-compatible on dify to integrate in one-api models, but only can set to chat, instead of completion (always error...see the screenshots), how you resovled this? ![image](https://private-user-images.githubusercontent.com/78014320/346307041-89d10f35-2537-4be2-abf2-bacecc6d3ff4.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjAzMTc3NjEsIm5iZiI6MTcyMDMxNzQ2MSwicGF0aCI6Ii83ODAxNDMyMC8zNDYzMDcwNDEtODlkMTBmMzUtMjUzNy00YmUyLWFiZjItYmFjZWNjNmQzZmY0LnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNDA3MDclMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjQwNzA3VDAxNTc0MVomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPWM3OTA2ZjQ4YjkyNTljYTJiYmFkMzVmNDA5OGJmZDlkZmI3Y2NmY2YzOGZmN2RkZjEwODcwMzJiMjU4MjBjZWUmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0JmFjdG9yX2lkPTAma2V5X2lkPTAmcmVwb19pZD0wIn0.h0ZpHP13H95WtBlljdsNjguh78QljMnkXXm95jxXO5w) ![1720317288120_d](https://private-user-images.githubusercontent.com/78014320/346307049-ca91631f-e298-49ea-b88f-818133692549.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjAzMTc3NjEsIm5iZiI6MTcyMDMxNzQ2MSwicGF0aCI6Ii83ODAxNDMyMC8zNDYzMDcwNDktY2E5MTYzMWYtZTI5OC00OWVhLWI4OGYtODE4MTMzNjkyNTQ5LnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNDA3MDclMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjQwNzA3VDAxNTc0MVomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPTBmZThiZmEzY2YyZjZjZjNmNWE3M2U4MmI4YjNlOTUzY2VmNTg0NzkyMjA0ODI1ZWNlZmYwOGNiODdhZTg1NTcmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0JmFjdG9yX2lkPTAma2V5X2lkPTAmcmVwb19pZD0wIn0.JAW8af1d3VkbYgrCfMcB9nyZlleEwyySBVmm9nmmXWk) 没有解决，官方没有给出解决方案，根据在官方群的回复，这个应该是底层代码不支持
yindo commented 2026-02-21 18:05:03 -05:00
Author
Owner
Copy Link
Copy Source

@jay-jjwu commented on GitHub (Jul 6, 2024):

好的，谢谢

@jay-jjwu commented on GitHub (Jul 6, 2024): 好的，谢谢
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Knowledge Pipeline
Proposal
WIP
cloud
dependencies
feat:webapp
good first issue
outdated
performance
pull-request
Mirrored from GitHub Pull Request
 python:uv
refactor
size:L
stale
status: accepting prs
status: blocked
status: in discussion
stt & tts
vulnerablity
web
wont-fix
🌊 feat:workflow
🌚 invalid
🐞 bug
👻 feat:rag
💪 enhancement
📚 documentation
🔨 feat:tools
🙋‍♂️ question
🙏 help wanted
🤔 cant-reproduce
🤖 feat:agent
No Label 🐞 bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: langgenius/dify#4170
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?