langgenius/dify
1
0
Fork 0
mirror of https://github.com/langgenius/dify.git synced 2026-07-21 20:15:31 -04:00
Code Issues 775 Packages Projects Releases 10 Wiki Activity

vllm support #7031

New Issue
Closed
opened 2026-02-21 18:18:31 -05:00 by yindo · 4 comments
yindo commented 2026-02-21 18:18:31 -05:00
Owner
Copy Link
Copy Source

Originally created by @OKC13 on GitHub (Dec 3, 2024).

Self Checks

  • This is only for bug report, if you would like to ask a question, please head to Discussions.
  • I have searched for existing issues search for existing issues, including closed ones.
  • I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
  • [FOR CHINESE USERS] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:）
  • Please do not modify this template :) and fill in all the required fields.

Dify version

latest

Cloud or Self Hosted

Self Hosted (Docker)

Steps to reproduce

I have successfully deployed and started the service using vllm, and the API calls work fine. However, when I try to add it in dify, I get the following error message: 'base model was not found.
image

✔️ Expected Behavior

No response

❌ Actual Behavior

No response

Originally created by @OKC13 on GitHub (Dec 3, 2024). ### Self Checks - [X] This is only for bug report, if you would like to ask a question, please head to [Discussions](https://github.com/langgenius/dify/discussions/categories/general). - [X] I have searched for existing issues [search for existing issues](https://github.com/langgenius/dify/issues), including closed ones. - [X] I confirm that I am using English to submit this report (我已阅读并同意 [Language Policy](https://github.com/langgenius/dify/issues/1542)). - [X] [FOR CHINESE USERS] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:） - [X] Please do not modify this template :) and fill in all the required fields. ### Dify version latest ### Cloud or Self Hosted Self Hosted (Docker) ### Steps to reproduce I have successfully deployed and started the service using vllm, and the API calls work fine. However, when I try to add it in dify, I get the following error message: 'base model was not found. ![image](https://github.com/user-attachments/assets/65c8f2fc-91d0-442a-8027-5a227d634f02) ### ✔️ Expected Behavior _No response_ ### ❌ Actual Behavior _No response_
yindo added the 🐞 bug label 2026-02-21 18:18:31 -05:00
yindo closed this issue 2026-02-21 18:18:31 -05:00
yindo commented 2026-02-21 18:18:31 -05:00
Author
Owner
Copy Link
Copy Source

@dosubot[bot] commented on GitHub (Dec 3, 2024):

The error message 'base model was not found' when integrating a custom model like vllm in Dify can be caused by several issues:

  1. Model Credentials Not Initialized: Ensure that the model credentials are properly set up. If they are not initialized, a ProviderTokenNotInitError will be raised [1].

  2. Model Does Not Exist: Verify that the model is correctly defined in your configuration. If the provider model is not found, a ValueError will be raised indicating that the model does not exist [1].

  3. Model Schema Not Found: Check that the model schema is correctly specified. If the schema is missing, a ValueError will be raised [1].

Make sure that your configuration file (litellm_config.yaml) is correctly set up and that the model name, API key, and endpoint URL are accurate. You can refer to the steps for integrating a custom LLM model via the LiteLLM Proxy for guidance [2].

To continue talking to Dosu, mention @dosu.

@dosubot[bot] commented on GitHub (Dec 3, 2024): <!-- Answer --> The error message 'base model was not found' when integrating a custom model like vllm in Dify can be caused by several issues: 1. **Model Credentials Not Initialized**: Ensure that the model credentials are properly set up. If they are not initialized, a `ProviderTokenNotInitError` will be raised <sup>[[1]](https://github.com/langgenius/dify/blob/main/api/core/app/app_config/easy_ui_based_app/model_config/converter.py)</sup>. 2. **Model Does Not Exist**: Verify that the model is correctly defined in your configuration. If the provider model is not found, a `ValueError` will be raised indicating that the model does not exist <sup>[[1]](https://github.com/langgenius/dify/blob/main/api/core/app/app_config/easy_ui_based_app/model_config/converter.py)</sup>. 3. **Model Schema Not Found**: Check that the model schema is correctly specified. If the schema is missing, a `ValueError` will be raised <sup>[[1]](https://github.com/langgenius/dify/blob/main/api/core/app/app_config/easy_ui_based_app/model_config/converter.py)</sup>. Make sure that your configuration file (`litellm_config.yaml`) is correctly set up and that the model name, API key, and endpoint URL are accurate. You can refer to the steps for integrating a custom LLM model via the LiteLLM Proxy for guidance <sup>[[2]](https://github.com/langgenius/dify-docs/blob/main/zh_CN/development/models-integration/litellm.md)</sup>. <!-- Dosu Comment Footer --> *To continue talking to [Dosu](https://dosu.dev), mention @dosu.*
yindo commented 2026-02-21 18:18:31 -05:00
Author
Owner
Copy Link
Copy Source

@OKC13 commented on GitHub (Dec 3, 2024):

did not solved

@OKC13 commented on GitHub (Dec 3, 2024): did not solved
yindo commented 2026-02-21 18:18:32 -05:00
Author
Owner
Copy Link
Copy Source

@wsbao commented on GitHub (Dec 3, 2024):

Consider OpenAI-API-compatible instead of Open AI.

Your model name is not in OpenAI's model list. Your can either restart vllm service with a spoofing OpenAI model name (e.g., gpt-4o) or use OpenAI-API-compatible interface.

@wsbao commented on GitHub (Dec 3, 2024): Consider **OpenAI-API-compatible** instead of Open AI. Your model name is not in OpenAI's model list. Your can either restart vllm service with a spoofing OpenAI model name (e.g., gpt-4o) or use **OpenAI-API-compatible** interface.
yindo commented 2026-02-21 18:18:32 -05:00
Author
Owner
Copy Link
Copy Source

@crazywoola commented on GitHub (Dec 3, 2024):

This provider is intended for the official OpenAI keys.

@crazywoola commented on GitHub (Dec 3, 2024): This provider is intended for the official OpenAI keys.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Knowledge Pipeline
Proposal
WIP
cloud
dependencies
feat:webapp
good first issue
outdated
performance
pull-request
Mirrored from GitHub Pull Request
 python:uv
refactor
size:L
stale
status: accepting prs
status: blocked
status: in discussion
stt & tts
vulnerablity
web
wont-fix
🌊 feat:workflow
🌚 invalid
🐞 bug
👻 feat:rag
💪 enhancement
📚 documentation
🔨 feat:tools
🙋‍♂️ question
🙏 help wanted
🤔 cant-reproduce
🤖 feat:agent
No Label 🐞 bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: langgenius/dify#7031
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?