langgenius/dify-official-plugins
1
0
Fork 0
mirror of https://github.com/langgenius/dify-official-plugins.git synced 2026-07-22 01:55:27 -04:00
Code Issues 54 Packages Projects Releases Wiki Activity

Add rerank mode at localAi Plugin failed #263

New Issue
Closed
opened 2026-02-16 10:18:39 -05:00 by yindo · 6 comments
yindo commented 2026-02-16 10:18:39 -05:00
Owner
Copy Link
Copy Source

Originally created by @crazychengmm on GitHub (May 8, 2025).

Self Checks

  • This is only for bug report, if you would like to ask a question, please head to Discussions.
  • I have searched for existing issues search for existing issues, including closed ones.
  • I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
  • [FOR CHINESE USERS] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:)
  • Please do not modify this template :) and fill in all the required fields.

Dify version

1.1.3

Cloud or Self Hosted

Self Hosted (Docker)

Steps to reproduce

localAi : 0.0.3

Dify api pod has error

2025-05-09 02:16:56,720.720 ERROR [Dummy-263] [models.py:162] - Failed to save model credentials, tenant_id: 4c382abd-8332-4f6d-91f9-61dad9ffd99a, model: /models/bge-reranker-v2-m3, model_type: rerank

Fri, May 9 2025 10:16:56 am | Traceback (most recent call last):
Fri, May 9 2025 10:16:56 am | File "/app/api/controllers/console/workspace/models.py", line 154, in post
Fri, May 9 2025 10:16:56 am | model_provider_service.save_model_credentials(
Fri, May 9 2025 10:16:56 am | File "/app/api/services/model_provider_service.py", line 237, in save_model_credentials
Fri, May 9 2025 10:16:56 am | provider_configuration.add_or_update_custom_model_credentials(
Fri, May 9 2025 10:16:56 am | File "/app/api/core/entities/provider_configuration.py", line 422, in add_or_update_custom_model_credentials
Fri, May 9 2025 10:16:56 am | provider_model_record, credentials = self.custom_model_credentials_validate(model_type, model, credentials)
Fri, May 9 2025 10:16:56 am | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Fri, May 9 2025 10:16:56 am | File "/app/api/core/entities/provider_configuration.py", line 402, in custom_model_credentials_validate
Fri, May 9 2025 10:16:56 am | credentials = model_provider_factory.model_credentials_validate(
Fri, May 9 2025 10:16:56 am | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Fri, May 9 2025 10:16:56 am | File "/app/api/core/model_runtime/model_providers/model_provider_factory.py", line 191, in model_credentials_validate
Fri, May 9 2025 10:16:56 am | self.plugin_model_manager.validate_model_credentials(
Fri, May 9 2025 10:16:56 am | File "/app/api/core/plugin/manager/model.py", line 137, in validate_model_credentials
Fri, May 9 2025 10:16:56 am | for resp in response:
Fri, May 9 2025 10:16:56 am | ^^^^^^^^
Fri, May 9 2025 10:16:56 am | File "/app/api/core/plugin/manager/base.py", line 189, in _request_with_plugin_daemon_response_stream
Fri, May 9 2025 10:16:56 am | self._handle_plugin_daemon_error(error.error_type, error.message)
Fri, May 9 2025 10:16:56 am | File "/app/api/core/plugin/manager/base.py", line 219, in _handle_plugin_daemon_error
Fri, May 9 2025 10:16:56 am | raise CredentialsValidateFailedError(error_object.get("message"))
Fri, May 9 2025 10:16:56 am | core.model_runtime.errors.validate.CredentialsValidateFailedError: 400 Client Error: Bad Request for url: http://bge-reranker-clusterip-service.default.svc.cluster.local/v2/rerank

Has any modification been made to the parameters? What should the correct ones be? Could it be a version compatibility issue? After testing, the rerank model appears to be functioning normally.

✔️ Expected Behavior

No response

❌ Actual Behavior

No response

Originally created by @crazychengmm on GitHub (May 8, 2025). ### Self Checks - [x] This is only for bug report, if you would like to ask a question, please head to [Discussions](https://github.com/langgenius/dify/discussions/categories/general). - [x] I have searched for existing issues [search for existing issues](https://github.com/langgenius/dify/issues), including closed ones. - [x] I confirm that I am using English to submit this report (我已阅读并同意 [Language Policy](https://github.com/langgenius/dify/issues/1542)). - [x] [FOR CHINESE USERS] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:) - [x] Please do not modify this template :) and fill in all the required fields. ### Dify version 1.1.3 ### Cloud or Self Hosted Self Hosted (Docker) ### Steps to reproduce localAi : 0.0.3 Dify api pod has error 2025-05-09 02:16:56,720.720 ERROR [Dummy-263] [models.py:162] - Failed to save model credentials, tenant_id: 4c382abd-8332-4f6d-91f9-61dad9ffd99a, model: /models/bge-reranker-v2-m3, model_type: rerank -- Fri, May 9 2025 10:16:56 am | Traceback (most recent call last): Fri, May 9 2025 10:16:56 am | File "/app/api/controllers/console/workspace/models.py", line 154, in post Fri, May 9 2025 10:16:56 am | model_provider_service.save_model_credentials( Fri, May 9 2025 10:16:56 am | File "/app/api/services/model_provider_service.py", line 237, in save_model_credentials Fri, May 9 2025 10:16:56 am | provider_configuration.add_or_update_custom_model_credentials( Fri, May 9 2025 10:16:56 am | File "/app/api/core/entities/provider_configuration.py", line 422, in add_or_update_custom_model_credentials Fri, May 9 2025 10:16:56 am | provider_model_record, credentials = self.custom_model_credentials_validate(model_type, model, credentials) Fri, May 9 2025 10:16:56 am | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Fri, May 9 2025 10:16:56 am | File "/app/api/core/entities/provider_configuration.py", line 402, in custom_model_credentials_validate Fri, May 9 2025 10:16:56 am | credentials = model_provider_factory.model_credentials_validate( Fri, May 9 2025 10:16:56 am | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Fri, May 9 2025 10:16:56 am | File "/app/api/core/model_runtime/model_providers/model_provider_factory.py", line 191, in model_credentials_validate Fri, May 9 2025 10:16:56 am | self.plugin_model_manager.validate_model_credentials( Fri, May 9 2025 10:16:56 am | File "/app/api/core/plugin/manager/model.py", line 137, in validate_model_credentials Fri, May 9 2025 10:16:56 am | for resp in response: Fri, May 9 2025 10:16:56 am | ^^^^^^^^ Fri, May 9 2025 10:16:56 am | File "/app/api/core/plugin/manager/base.py", line 189, in _request_with_plugin_daemon_response_stream Fri, May 9 2025 10:16:56 am | self._handle_plugin_daemon_error(error.error_type, error.message) Fri, May 9 2025 10:16:56 am | File "/app/api/core/plugin/manager/base.py", line 219, in _handle_plugin_daemon_error Fri, May 9 2025 10:16:56 am | raise CredentialsValidateFailedError(error_object.get("message")) Fri, May 9 2025 10:16:56 am | core.model_runtime.errors.validate.CredentialsValidateFailedError: 400 Client Error: Bad Request for url: http://bge-reranker-clusterip-service.default.svc.cluster.local/v2/rerank Has any modification been made to the parameters? What should the correct ones be? Could it be a version compatibility issue? After testing, the rerank model appears to be functioning normally. ### ✔️ Expected Behavior _No response_ ### ❌ Actual Behavior _No response_
yindo closed this issue 2026-02-16 10:18:39 -05:00
yindo commented 2026-02-16 10:18:39 -05:00
Author
Owner
Copy Link
Copy Source

@dosubot[bot] commented on GitHub (May 8, 2025):

I found a similar discussion thread titled "How to add local huggingface text-embeddings-inference rerank model to dify" which is currently unsolved [1].

Regarding the error you're encountering, the CredentialsValidateFailedError typically occurs due to issues with service connectivity or incorrect configuration parameters. In your case, the error message indicates a "400 Client Error: Bad Request" for the URL related to the rerank model. Here are some steps you can take to troubleshoot and resolve the issue:

  1. Verify Endpoint URL: Ensure that the URL http://bge-reranker-clusterip-service.default.svc.cluster.local/v2/rerank is correct and accessible from your Dify setup. Check if the service is running and reachable.

  2. Check Model Credentials: Make sure that the credentials provided match the provider_credential_schema or model_credential_schema defined in the provider's YAML configuration file. This includes parameters like api_key, server_url, and model_uid if applicable [2].

  3. Configuration Parameters: Double-check the configuration parameters used for the rerank model. Ensure that they are correctly defined and used according to the model's requirements.

  4. Service Stability: If the issue persists, consider checking the stability of the service or using alternative providers if available [2].

These steps should help you identify and resolve the issue with adding the rerank model to the localAi plugin.

To continue talking to Dosu, mention @dosu.

Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Bug Report | Other

@dosubot[bot] commented on GitHub (May 8, 2025): <!-- Answer --> I found a similar discussion thread titled "How to add local huggingface text-embeddings-inference rerank model to dify" which is currently unsolved <sup>[[1]](https://github.com/langgenius/dify/discussions/18817)</sup>. Regarding the error you're encountering, the `CredentialsValidateFailedError` typically occurs due to issues with service connectivity or incorrect configuration parameters. In your case, the error message indicates a "400 Client Error: Bad Request" for the URL related to the rerank model. Here are some steps you can take to troubleshoot and resolve the issue: 1. **Verify Endpoint URL**: Ensure that the URL `http://bge-reranker-clusterip-service.default.svc.cluster.local/v2/rerank` is correct and accessible from your Dify setup. Check if the service is running and reachable. 2. **Check Model Credentials**: Make sure that the credentials provided match the `provider_credential_schema` or `model_credential_schema` defined in the provider's YAML configuration file. This includes parameters like `api_key`, `server_url`, and `model_uid` if applicable <sup>[[2]](https://github.com/langgenius/dify/issues/16050)</sup>. 3. **Configuration Parameters**: Double-check the configuration parameters used for the rerank model. Ensure that they are correctly defined and used according to the model's requirements. 4. **Service Stability**: If the issue persists, consider checking the stability of the service or using alternative providers if available <sup>[[2]](https://github.com/langgenius/dify/issues/16050)</sup>. These steps should help you identify and resolve the issue with adding the rerank model to the localAi plugin. <!-- Dosu Comment Footer --> *To continue talking to [Dosu](https://dosu.dev), mention @dosu.* --- <sup>Help Dosu learn! Give it feedback: [Great Response](https://app.dosu.dev/response-feedback/e527b372-88fc-406d-8a6b-a6f35d96b6ab?feedback_type=great_response) | [Irrelevant Answer](https://app.dosu.dev/response-feedback/e527b372-88fc-406d-8a6b-a6f35d96b6ab?feedback_type=irrelevant_answer) | [Incorrect Sources](https://app.dosu.dev/response-feedback/e527b372-88fc-406d-8a6b-a6f35d96b6ab?feedback_type=incorrect_sources) | [Too Verbose](https://app.dosu.dev/response-feedback/e527b372-88fc-406d-8a6b-a6f35d96b6ab?feedback_type=too_verbose) | [Hallucination](https://app.dosu.dev/response-feedback/e527b372-88fc-406d-8a6b-a6f35d96b6ab?feedback_type=hallucination) | [Bug Report](https://app.dosu.dev/response-feedback/e527b372-88fc-406d-8a6b-a6f35d96b6ab?feedback_type=bug_report) | [Other](https://app.dosu.dev/response-feedback/e527b372-88fc-406d-8a6b-a6f35d96b6ab?feedback_type=other)</sup>
yindo commented 2026-02-16 10:18:39 -05:00
Author
Owner
Copy Link
Copy Source

@crazychengmm commented on GitHub (May 8, 2025):

localAI rerank model


def validate_credentials(self, model: str, credentials: dict) -> None:
        """
        Validate model credentials

        :param model: model nameCredentialsValidateFailedError
        :param credentials: model credentials
        :return:
        """
        try:
            self._invoke(
                model=model,
                credentials=credentials,
                query="What is the capital of the United States?",
                docs=[
                    "Carson City is the capital city of the American state of Nevada. At the 2010 United States Census, Carson City had a population of 55,274.",
                    "The Commonwealth of the Northern Mariana Islands is a group of islands in the Pacific Ocean that are a political division controlled by the United States. Its capital is Saipan.",
                ],
                score_threshold=0.8,
            )
        except Exception as ex:
            raise CredentialsValidateFailedError(str(ex))
@crazychengmm commented on GitHub (May 8, 2025): localAI rerank model ``` def validate_credentials(self, model: str, credentials: dict) -> None: """ Validate model credentials :param model: model nameCredentialsValidateFailedError :param credentials: model credentials :return: """ try: self._invoke( model=model, credentials=credentials, query="What is the capital of the United States?", docs=[ "Carson City is the capital city of the American state of Nevada. At the 2010 United States Census, Carson City had a population of 55,274.", "The Commonwealth of the Northern Mariana Islands is a group of islands in the Pacific Ocean that are a political division controlled by the United States. Its capital is Saipan.", ], score_threshold=0.8, ) except Exception as ex: raise CredentialsValidateFailedError(str(ex)) ```
yindo commented 2026-02-16 10:18:39 -05:00
Author
Owner
Copy Link
Copy Source

@gakugaku commented on GitHub (Jun 22, 2025):

@crazychengmm
Potentially related: #1142

@gakugaku commented on GitHub (Jun 22, 2025): @crazychengmm Potentially related: #1142
yindo commented 2026-02-16 10:18:39 -05:00
Author
Owner
Copy Link
Copy Source

@nhha1602 commented on GitHub (Jul 21, 2025):

I have same issue here:

Dify-api: 1.6.0
Dify-Plugin-deamon: 0.1.3

2025-07-21 13:42:43,647.647 DEBUG [Thread-411 (process_request_thread)] [connectionpool.py:241] - Starting new HTTP connection (1): dify-plugin.chatbot-llm-dev:80
2025-07-21 13:42:43,863.863 DEBUG [Thread-411 (process_request_thread)] [connectionpool.py:544] - http://dify-plugin.chatbot-llm-dev:80 "POST /plugin/290190fe-b2ad-4f3f-a9c8-b61e5d2076f2/dispatch/model/validate_model_credentials HTTP/1.1" 200 None
2025-07-21 13:42:43,865.865 ERROR [Thread-411 (process_request_thread)] [models.py:162] - Failed to save model credentials, tenant_id: 290190fe-b2ad-4f3f-a9c8-b61e5d2076f2, model: Qwen3-Reranker-4B, model_type: rerank
Traceback (most recent call last):
File "/app/api/controllers/console/workspace/models.py", line 154, in post
model_provider_service.save_model_credentials(
File "/app/api/services/model_provider_service.py", line 237, in save_model_credentials
provider_configuration.add_or_update_custom_model_credentials(
File "/app/api/core/entities/provider_configuration.py", line 422, in add_or_update_custom_model_credentials
provider_model_record, credentials = self.custom_model_credentials_validate(model_type, model, credentials)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/api/core/entities/provider_configuration.py", line 402, in custom_model_credentials_validate
credentials = model_provider_factory.model_credentials_validate(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/api/core/model_runtime/model_providers/model_provider_factory.py", line 191, in model_credentials_validate
self.plugin_model_manager.validate_model_credentials(
File "/app/api/core/plugin/impl/model.py", line 137, in validate_model_credentials
for resp in response:
^^^^^^^^
File "/app/api/core/plugin/impl/base.py", line 214, in _request_with_plugin_daemon_response_stream
self._handle_plugin_daemon_error(error.error_type, error.message)
File "/app/api/core/plugin/impl/base.py", line 244, in _handle_plugin_daemon_error
raise CredentialsValidateFailedError(error_object.get("message"))
core.model_runtime.errors.validate.CredentialsValidateFailedError: 'results'
2025-07-21 13:42:43,906.906 INFO [Thread-411 (process_request_thread)] [_internal.py:97] - 127.0.0.6 - - [21/Jul/2025 06:42:43] "POST /console/api/workspaces/current/model-providers/langgenius/openai_api_compatible/openai_api_compatible/models HTTP/1.1" 400 -

@nhha1602 commented on GitHub (Jul 21, 2025): I have same issue here: Dify-api: 1.6.0 Dify-Plugin-deamon: 0.1.3 2025-07-21 13:42:43,647.647 DEBUG [Thread-411 (process_request_thread)] [connectionpool.py:241] - Starting new HTTP connection (1): dify-plugin.chatbot-llm-dev:80 2025-07-21 13:42:43,863.863 DEBUG [Thread-411 (process_request_thread)] [connectionpool.py:544] - http://dify-plugin.chatbot-llm-dev:80 "POST /plugin/290190fe-b2ad-4f3f-a9c8-b61e5d2076f2/dispatch/model/validate_model_credentials HTTP/1.1" 200 None 2025-07-21 13:42:43,865.865 ERROR [Thread-411 (process_request_thread)] [models.py:162] - Failed to save model credentials, tenant_id: 290190fe-b2ad-4f3f-a9c8-b61e5d2076f2, model: Qwen3-Reranker-4B, model_type: rerank Traceback (most recent call last): File "/app/api/controllers/console/workspace/models.py", line 154, in post model_provider_service.save_model_credentials( File "/app/api/services/model_provider_service.py", line 237, in save_model_credentials provider_configuration.add_or_update_custom_model_credentials( File "/app/api/core/entities/provider_configuration.py", line 422, in add_or_update_custom_model_credentials provider_model_record, credentials = self.custom_model_credentials_validate(model_type, model, credentials) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/app/api/core/entities/provider_configuration.py", line 402, in custom_model_credentials_validate credentials = model_provider_factory.model_credentials_validate( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/app/api/core/model_runtime/model_providers/model_provider_factory.py", line 191, in model_credentials_validate self.plugin_model_manager.validate_model_credentials( File "/app/api/core/plugin/impl/model.py", line 137, in validate_model_credentials for resp in response: ^^^^^^^^ File "/app/api/core/plugin/impl/base.py", line 214, in _request_with_plugin_daemon_response_stream self._handle_plugin_daemon_error(error.error_type, error.message) File "/app/api/core/plugin/impl/base.py", line 244, in _handle_plugin_daemon_error raise CredentialsValidateFailedError(error_object.get("message")) core.model_runtime.errors.validate.CredentialsValidateFailedError: 'results' 2025-07-21 13:42:43,906.906 INFO [Thread-411 (process_request_thread)] [_internal.py:97] - 127.0.0.6 - - [21/Jul/2025 06:42:43] "POST /console/api/workspaces/current/model-providers/langgenius/openai_api_compatible/openai_api_compatible/models HTTP/1.1" 400 -
yindo commented 2026-02-16 10:18:40 -05:00
Author
Owner
Copy Link
Copy Source

@nhha1602 commented on GitHub (Jul 23, 2025):

I upgraded vllm to 0.9.2 with following instruction - https://github.com/vllm-project/vllm/pull/19260
And it worked.

@nhha1602 commented on GitHub (Jul 23, 2025): I upgraded vllm to 0.9.2 with following instruction - https://github.com/vllm-project/vllm/pull/19260 And it worked.
yindo commented 2026-02-16 10:18:40 -05:00
Author
Owner
Copy Link
Copy Source

@dosubot[bot] commented on GitHub (Aug 21, 2025):

Hi, @crazychengmm. I'm Dosu, and I'm helping the dify-official-plugins team manage their backlog and am marking this issue as stale.

Issue Summary:

  • You reported a CredentialsValidateFailedError when adding a rerank model with the localAi plugin on Dify v1.1.3.
  • The error was due to failure to save model credentials for "/models/bge-reranker-v2-m3," resulting in a 400 Bad Request.
  • Suggestions included verifying the service URL, credentials schema, and configuration parameters.
  • Another user confirmed a similar issue on a newer Dify version and resolved it by upgrading the vllm package to 0.9.2.
  • The issue was resolved by upgrading vllm to version 0.9.2.

Next Steps:

  • Please confirm if this issue is still relevant with the latest version of dify-official-plugins.
  • If it is, feel free to keep the discussion open by commenting; otherwise, I will automatically close this issue in 5 days.

Thank you for your understanding and contribution!

@dosubot[bot] commented on GitHub (Aug 21, 2025): Hi, @crazychengmm. I'm [Dosu](https://dosu.dev), and I'm helping the dify-official-plugins team manage their backlog and am marking this issue as stale. Issue Summary: - You reported a CredentialsValidateFailedError when adding a rerank model with the localAi plugin on Dify v1.1.3. - The error was due to failure to save model credentials for "/models/bge-reranker-v2-m3," resulting in a 400 Bad Request. - Suggestions included verifying the service URL, credentials schema, and configuration parameters. - Another user confirmed a similar issue on a newer Dify version and resolved it by upgrading the vllm package to 0.9.2. - The issue was resolved by upgrading vllm to version 0.9.2. Next Steps: - Please confirm if this issue is still relevant with the latest version of dify-official-plugins. - If it is, feel free to keep the discussion open by commenting; otherwise, I will automatically close this issue in 5 days. Thank you for your understanding and contribution!
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
bug
bug
bug
dependencies
documentation
enhancement
good first issue
invalid
pull-request
Mirrored from GitHub Pull Request
 question
stale
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: langgenius/dify-official-plugins#263
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?