langgenius/dify
1
0
Fork 0
mirror of https://github.com/langgenius/dify.git synced 2026-07-21 12:05:30 -04:00
Code Issues 775 Packages Projects Releases 10 Wiki Activity

ERROR while importing the reranker model using xinference #5180

New Issue
Closed
opened 2026-02-21 18:09:46 -05:00 by yindo · 2 comments
yindo commented 2026-02-21 18:09:46 -05:00
Owner
Copy Link
Copy Source

Originally created by @AnomeZ on GitHub (Aug 22, 2024).

Self Checks

  • This is only for bug report, if you would like to ask a question, please head to Discussions.
  • I have searched for existing issues search for existing issues, including closed ones.
  • I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
  • [FOR CHINESE USERS] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:）
  • Please do not modify this template :) and fill in all the required fields.

Dify version

Version 0.7.0

Cloud or Self Hosted

Self Hosted (Docker)

Steps to reproduce

When I try to use the reranker model provided in xinference, I get the following error:
image
image
HTTPConnectionPool(host='192.168.0.118', port=9997): Max retries exceeded with url: /v1/cluster/auth (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7ff8713da9b0>: Failed to establish a new connection: [Errno 111] Connection refused'))

My xinference is configured directly using pip, while dify is configured using docker. I guess the error is caused by docker not being able to access the port of xinference.

Unfortunately, I don't know what went wrong in the configuration (I don't know much about docker).

How can I solve this problem, or try to solve this problem?

✔️ Expected Behavior

The bce-reranker-base_v1 model is correctly imported

❌ Actual Behavior

Failed to import correctly, error: HTTPConnectionPool(host='192.168.0.118', port=9997): Max retries exceeded with url: /v1/cluster/auth (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7ff8713da9b0>: Failed to establish a new connection: [Errno 111] Connection refused'))

Originally created by @AnomeZ on GitHub (Aug 22, 2024). ### Self Checks - [X] This is only for bug report, if you would like to ask a question, please head to [Discussions](https://github.com/langgenius/dify/discussions/categories/general). - [X] I have searched for existing issues [search for existing issues](https://github.com/langgenius/dify/issues), including closed ones. - [X] I confirm that I am using English to submit this report (我已阅读并同意 [Language Policy](https://github.com/langgenius/dify/issues/1542)). - [X] [FOR CHINESE USERS] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:） - [X] Please do not modify this template :) and fill in all the required fields. ### Dify version Version 0.7.0 ### Cloud or Self Hosted Self Hosted (Docker) ### Steps to reproduce When I try to use the reranker model provided in xinference, I get the following error: ![image](https://github.com/user-attachments/assets/7b5802fc-c0a0-4f65-aa7c-8a4b262456f3) ![image](https://github.com/user-attachments/assets/39845ab0-d40f-4de1-ac4a-621d62386d2a) HTTPConnectionPool(host='192.168.0.118', port=9997): Max retries exceeded with url: /v1/cluster/auth (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7ff8713da9b0>: Failed to establish a new connection: [Errno 111] Connection refused')) My xinference is configured directly using pip, while dify is configured using docker. I guess the error is caused by docker not being able to access the port of xinference. Unfortunately, I don't know what went wrong in the configuration (I don't know much about docker). How can I solve this problem, or try to solve this problem? ### ✔️ Expected Behavior The bce-reranker-base_v1 model is correctly imported ### ❌ Actual Behavior Failed to import correctly, error: HTTPConnectionPool(host='192.168.0.118', port=9997): Max retries exceeded with url: /v1/cluster/auth (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7ff8713da9b0>: Failed to establish a new connection: [Errno 111] Connection refused'))
yindo added the 🌚 invalid label 2026-02-21 18:09:46 -05:00
yindo closed this issue 2026-02-21 18:09:46 -05:00
yindo commented 2026-02-21 18:09:46 -05:00
Author
Owner
Copy Link
Copy Source

@dosubot[bot] commented on GitHub (Aug 22, 2024):

Hello, @AnomeZ! I'm here to help you with any bugs, questions, or contributions. Let's work together to solve the issue you're facing.

I found a similar issue and discussion that might help you resolve the HTTPConnectionPool error when importing the bce-reranker-base_v1 model using xinference with Dify configured via Docker:

  1. Similar Discussions and Issues:

  2. Solution:
    To solve the HTTPConnectionPool error, ensure that the Dify container can access the Xinference endpoint. The Dify container cannot access localhost inside the container, so you need to use the host IP address. Here are the steps to follow:

    • Install Xinference:

      $ pip install "xinference[all]"

    • Start Xinference with the correct host configuration:

      $ xinference-local -H 0.0.0.0

      This command allows access from any non-local client.

    • Ensure the Dify container can access the Xinference endpoint:
      In the Dify settings, use the host machine's IP address instead of localhost or 127.0.0.1. For example, if your host machine's IP address is 192.168.1.100, configure the server URL in Dify as follows:

      Server URL: http://192.168.1.100:9997

    • Deploy the model in Xinference:
      Access http://192.168.1.100:9997 in your browser, select the model you want to deploy, and note the model UID.

    • Configure the model in Dify:
      In Settings > Model Providers > Xinference, enter the following details:

      • Model name: bce-reranker-base_v1
      • Server URL: http://192.168.1.100:9997 (Replace with your machine IP address)
      • Model UID: (copy from the Running Models page)

      Click "Save" to use the model in the Dify application.

By following these steps, you should be able to resolve the connection timeout error and successfully import the bce-reranker-base_v1 model using xinference with Dify configured via Docker [4][5][6].

To continue talking to Dosu, mention @dosu.

@dosubot[bot] commented on GitHub (Aug 22, 2024): <!-- Greeting --> Hello, @AnomeZ! I'm here to help you with any bugs, questions, or contributions. Let's work together to solve the issue you're facing. <!-- Answer --> I found a similar issue and discussion that might help you resolve the `HTTPConnectionPool` error when importing the `bce-reranker-base_v1` model using `xinference` with Dify configured via Docker: 1. **Similar Discussions and Issues**: - [Adding a local ollama model error](https://github.com/langgenius/dify/discussions/7362) <sup>[[1]](https://github.com/langgenius/dify/discussions/7362)</sup> - [Xinference failed to access the local model](https://github.com/langgenius/dify/discussions/7205) <sup>[[2]](https://github.com/langgenius/dify/discussions/7205)</sup> - [Xinference failed to access the local model](https://github.com/langgenius/dify/issues/7203) <sup>[[3]](https://github.com/langgenius/dify/issues/7203)</sup> 2. **Solution**: To solve the `HTTPConnectionPool` error, ensure that the Dify container can access the Xinference endpoint. The Dify container cannot access `localhost` inside the container, so you need to use the host IP address. Here are the steps to follow: - **Install Xinference**: ```bash $ pip install "xinference[all]" ``` - **Start Xinference with the correct host configuration**: ```bash $ xinference-local -H 0.0.0.0 ``` This command allows access from any non-local client. - **Ensure the Dify container can access the Xinference endpoint**: In the Dify settings, use the host machine's IP address instead of `localhost` or `127.0.0.1`. For example, if your host machine's IP address is `192.168.1.100`, configure the server URL in Dify as follows: ```plaintext Server URL: http://192.168.1.100:9997 ``` - **Deploy the model in Xinference**: Access `http://192.168.1.100:9997` in your browser, select the model you want to deploy, and note the model UID. - **Configure the model in Dify**: In `Settings > Model Providers > Xinference`, enter the following details: - Model name: `bce-reranker-base_v1` - Server URL: `http://192.168.1.100:9997` (Replace with your machine IP address) - Model UID: (copy from the `Running Models` page) Click "Save" to use the model in the Dify application. By following these steps, you should be able to resolve the connection timeout error and successfully import the `bce-reranker-base_v1` model using `xinference` with Dify configured via Docker <sup>[[4]](https://github.com/langgenius/dify-docs/blob/main/en/guides/model-configuration/ollama.md)</sup><sup>[[5]](https://github.com/langgenius/dify-docs/blob/main/jp/guides/model-configuration/xinference.md)</sup><sup>[[6]](https://github.com/langgenius/dify-docs/blob/main/en/guides/model-configuration/xinference.md)</sup>. <!-- Dosu Comment Footer --> *To continue talking to [Dosu](https://dosu.dev), mention @dosu.*
yindo commented 2026-02-21 18:09:46 -05:00
Author
Owner
Copy Link
Copy Source

@AnomeZ commented on GitHub (Aug 22, 2024):

This problem has been solved, the reference https://github.com/langgenius/dify/discussions/7362, only need to modify the url to http://host.docker.internal:9997.@dosu

@AnomeZ commented on GitHub (Aug 22, 2024): This problem has been solved, the reference https://github.com/langgenius/dify/discussions/7362, only need to modify the url to http://host.docker.internal:9997.@dosu
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Knowledge Pipeline
Proposal
WIP
cloud
dependencies
feat:webapp
good first issue
outdated
performance
pull-request
Mirrored from GitHub Pull Request
 python:uv
refactor
size:L
stale
status: accepting prs
status: blocked
status: in discussion
stt & tts
vulnerablity
web
wont-fix
🌊 feat:workflow
🌚 invalid
🐞 bug
👻 feat:rag
💪 enhancement
📚 documentation
🔨 feat:tools
🙋‍♂️ question
🙏 help wanted
🤔 cant-reproduce
🤖 feat:agent
No Label 🌚 invalid
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: langgenius/dify#5180
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?