langgenius/dify
1
0
Fork 0
mirror of https://github.com/langgenius/dify.git synced 2026-07-23 21:15:32 -04:00
Code Issues 775 Packages Projects Releases 10 Wiki Activity

how to use Xinference's rerank models in Dify ? #1055

New Issue
Closed
opened 2026-02-21 17:29:44 -05:00 by yindo · 5 comments
yindo commented 2026-02-21 17:29:44 -05:00
Owner
Copy Link
Copy Source

Originally created by @whoamiWo on GitHub (Feb 16, 2024).

Originally assigned to: @takatost, @Yeuoly on GitHub.

Self Checks

Provide a description of the help you need

I use Xinference's rerank models in Dify ,It not working
"[xinference] Error: Failed to get the model description, detail: Not Found"
BFC57F13-53E1-428e-85C1-5C629ACBFEF9
but in console it working! why?
9CB4F2A4-D21C-4a53-A82B-1B876BEEA3F4

Originally created by @whoamiWo on GitHub (Feb 16, 2024). Originally assigned to: @takatost, @Yeuoly on GitHub. ### Self Checks - [X] I have searched for existing issues [search for existing issues](https://github.com/langgenius/dify/issues), including closed ones. - [X] I confirm that I am using English to file this report (我已阅读并同意 [Language Policy](https://github.com/langgenius/dify/issues/1542)). ### Provide a description of the help you need I use Xinference's rerank models in Dify ,It not working "[xinference] Error: Failed to get the model description, detail: Not Found" ![BFC57F13-53E1-428e-85C1-5C629ACBFEF9](https://github.com/langgenius/dify/assets/160295799/f8d1be13-8f93-406b-a4bb-9b57fb9f1557) but in console it working! why? ![9CB4F2A4-D21C-4a53-A82B-1B876BEEA3F4](https://github.com/langgenius/dify/assets/160295799/c00084bf-63d0-4acb-9aad-971fc4c9942e)
yindo added the 📚 documentation label 2026-02-21 17:29:44 -05:00
yindo closed this issue 2026-02-21 17:29:44 -05:00
yindo commented 2026-02-21 17:29:44 -05:00
Author
Owner
Copy Link
Copy Source

@dosubot[bot] commented on GitHub (Feb 16, 2024):

🤖

Hey there @whoamiWo! Nice to see you diving into the world of Dify again. How's everything going on your end?

I'm sorry, but I'm unable to view the images you've attached. However, based on your description, it seems like you're encountering an error when trying to use Xinference's rerank models in Dify. The error message "Failed to get the model description, detail: Not Found" typically indicates that the model you're trying to access does not exist or cannot be found.

Here are a few things you could check:

  1. Incorrect model_uid: The model_uid you're using in the credentials dictionary might be incorrect or does not exist. This model_uid is used to get the model from the Xinference client. If the model_uid is incorrect, the get_model method will not be able to find the model, resulting in a "Not Found" error.

  2. Incorrect server_url: The server_url in the credentials dictionary might be incorrect. This URL is used to initialize the Xinference client. If the URL is incorrect, the client will not be able to connect to the server, and thus, will not be able to find the model.

  3. The model is not a rerank model: The model you're trying to invoke might not be a rerank model. If the model is not a rerank model, the isinstance check will fail, and an InvokeBadRequestError will be raised with the message "please check model type, the model you want to invoke is not a rerank model".

You can find more details about this in the rerank.py file in the Dify repository.

If you've checked these and are still encountering the issue, it would be helpful if you could provide more information about the steps you're taking to use the rerank models, as well as any relevant code snippets. This will help in identifying the cause of the issue.

Sources

About Dosu

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

@dosubot[bot] commented on GitHub (Feb 16, 2024): _🤖_ <!-- Greeting --> Hey there @whoamiWo! Nice to see you diving into the world of Dify again. How's everything going on your end? <!-- Answer --> I'm sorry, but I'm unable to view the images you've attached. However, based on your description, it seems like you're encountering an error when trying to use Xinference's rerank models in Dify. The error message "Failed to get the model description, detail: Not Found" typically indicates that the model you're trying to access does not exist or cannot be found. Here are a few things you could check: 1. **Incorrect `model_uid`**: The `model_uid` you're using in the `credentials` dictionary might be incorrect or does not exist. This `model_uid` is used to get the model from the Xinference client. If the `model_uid` is incorrect, the `get_model` method will not be able to find the model, resulting in a "Not Found" error. 2. **Incorrect `server_url`**: The `server_url` in the `credentials` dictionary might be incorrect. This URL is used to initialize the Xinference client. If the URL is incorrect, the client will not be able to connect to the server, and thus, will not be able to find the model. 3. **The model is not a rerank model**: The model you're trying to invoke might not be a rerank model. If the model is not a rerank model, the `isinstance` check will fail, and an `InvokeBadRequestError` will be raised with the message "please check model type, the model you want to invoke is not a rerank model". You can find more details about this in the [rerank.py](https://github.com/langgenius/dify/blob/main/api/core/model_runtime/model_providers/xinference/rerank/rerank.py) file in the Dify repository. If you've checked these and are still encountering the issue, it would be helpful if you could provide more information about the steps you're taking to use the rerank models, as well as any relevant code snippets. This will help in identifying the cause of the issue. ## Sources - [使用xinference接入模型时报错](https://github.com/langgenius/dify/issues/1837) - [0.3.23版本，新增Xinference类型的chatglm2模型的embedding能力，报错，但是text-generation正常](https://github.com/langgenius/dify/issues/1207) - [api/core/model_runtime/model_providers/xinference/rerank/rerank.py](https://github.com/langgenius/dify/blob/main/api/core/model_runtime/model_providers/xinference/rerank/rerank.py) <!-- Dosu Comment Footer --> <details open> <summary> #### About Dosu </summary> This response is meant to be useful and save you time. It is **not meant to be a precise solution**, but rather a starting point for your own research. Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant. If you want to continue the conversation, start your reply with @dosu-bot. </details>
yindo commented 2026-02-21 17:29:45 -05:00
Author
Owner
Copy Link
Copy Source

@crazywoola commented on GitHub (Feb 17, 2024):

Is the masked ip address a public ip address?

@crazywoola commented on GitHub (Feb 17, 2024): Is the masked ip address a public ip address?
yindo commented 2026-02-21 17:29:45 -05:00
Author
Owner
Copy Link
Copy Source

@whoamiWo commented on GitHub (Feb 17, 2024):

Is the masked ip address a public ip address?

yes , a public ip address

@whoamiWo commented on GitHub (Feb 17, 2024): > Is the masked ip address a public ip address? yes , a public ip address
yindo commented 2026-02-21 17:29:45 -05:00
Author
Owner
Copy Link
Copy Source

@whoamiWo commented on GitHub (Feb 17, 2024):

XINFERENCE_MODEL_SRC=modelscope /usr/local/bin/python38/bin/xinference-local -H 0.0.0.0

@whoamiWo commented on GitHub (Feb 17, 2024): XINFERENCE_MODEL_SRC=modelscope /usr/local/bin/python38/bin/xinference-local -H 0.0.0.0
yindo commented 2026-02-21 17:29:45 -05:00
Author
Owner
Copy Link
Copy Source

@Yeuoly commented on GitHub (Feb 18, 2024):

model_uid should not contains "/". Xinference use /v1/models/{model_uid} to get the detailed information of a model, however, it leads to an incorrect path for retrieving the model, such as /v1/models/chatglm becoming /v1/models/chat/glm, removing "/" from your model_uid is a way to solve this, we will fix it in next release also.

@Yeuoly commented on GitHub (Feb 18, 2024): `model_uid` should not contains "/". Xinference use `/v1/models/{model_uid}` to get the detailed information of a model, however, it leads to an incorrect path for retrieving the model, such as `/v1/models/chatglm` becoming `/v1/models/chat/glm`, removing "/" from your `model_uid` is a way to solve this, we will fix it in next release also.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Knowledge Pipeline
Proposal
WIP
cloud
dependencies
feat:webapp
good first issue
outdated
performance
pull-request
Mirrored from GitHub Pull Request
 python:uv
refactor
size:L
stale
status: accepting prs
status: blocked
status: in discussion
stt & tts
vulnerablity
web
wont-fix
🌊 feat:workflow
🌚 invalid
🐞 bug
👻 feat:rag
💪 enhancement
📚 documentation
🔨 feat:tools
🙋‍♂️ question
🙏 help wanted
🤔 cant-reproduce
🤖 feat:agent
No Label 📚 documentation
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: langgenius/dify#1055
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?