langgenius/dify-official-plugins
1
0
Fork 0
mirror of https://github.com/langgenius/dify-official-plugins.git synced 2026-07-21 17:45:23 -04:00
Code Issues 54 Packages Projects Releases Wiki Activity

Reranker model Qwen3-Reranker-8B add failed with hint 'NoneType' object is not iterable #353

New Issue
Closed
opened 2026-02-16 10:19:06 -05:00 by yindo · 8 comments
yindo commented 2026-02-16 10:19:06 -05:00
Owner
Copy Link
Copy Source

Originally created by @elvizlai on GitHub (Jun 9, 2025).

Self Checks

  • This is only for bug report, if you would like to ask a question, please head to Discussions.
  • I have searched for existing issues search for existing issues, including closed ones.
  • I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
  • [FOR CHINESE USERS] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:)
  • Please do not modify this template :) and fill in all the required fields.

Dify version

1.4.1

Cloud or Self Hosted

Self Hosted (Docker)

Steps to reproduce

Using OpenAI OpenAI-API-compatible to add Reranked model served by vLLM

The screencap of dify:

Image

Returns error: 'NoneType' object is not iterable

Here is vLLM running log:

Image

And here is log from docker-api-1:

2025-06-09 10:16:29.830 ERROR [Dummy-26] [models.py:162] - Failed to save model credentials, tenant_id: 23e8e3e0-7893-4b48-9edf-0409a22a5b01, model: Qwen3-Reranker-8B, model_type: rerank
Traceback (most recent call last):
  File "/app/api/controllers/console/workspace/models.py", line 154, in post
    model_provider_service.save_model_credentials(
  File "/app/api/services/model_provider_service.py", line 237, in save_model_credentials
    provider_configuration.add_or_update_custom_model_credentials(
  File "/app/api/core/entities/provider_configuration.py", line 422, in add_or_update_custom_model_credentials
    provider_model_record, credentials = self.custom_model_credentials_validate(model_type, model, credentials)
                                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/api/core/entities/provider_configuration.py", line 402, in custom_model_credentials_validate
    credentials = model_provider_factory.model_credentials_validate(
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/api/core/model_runtime/model_providers/model_provider_factory.py", line 191, in model_credentials_validate
    self.plugin_model_manager.validate_model_credentials(
  File "/app/api/core/plugin/impl/model.py", line 137, in validate_model_credentials
    for resp in response:
                ^^^^^^^^
  File "/app/api/core/plugin/impl/base.py", line 192, in _request_with_plugin_daemon_response_stream
    self._handle_plugin_daemon_error(error.error_type, error.message)
  File "/app/api/core/plugin/impl/base.py", line 222, in _handle_plugin_daemon_error
    raise CredentialsValidateFailedError(error_object.get("message"))
core.model_runtime.errors.validate.CredentialsValidateFailedError: 'NoneType' object is not iterable

✔️ Expected Behavior

Rerank model added

❌ Actual Behavior

'NoneType' object is not iterable

Originally created by @elvizlai on GitHub (Jun 9, 2025). ### Self Checks - [x] This is only for bug report, if you would like to ask a question, please head to [Discussions](https://github.com/langgenius/dify/discussions/categories/general). - [x] I have searched for existing issues [search for existing issues](https://github.com/langgenius/dify/issues), including closed ones. - [x] I confirm that I am using English to submit this report (我已阅读并同意 [Language Policy](https://github.com/langgenius/dify/issues/1542)). - [x] [FOR CHINESE USERS] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:) - [x] Please do not modify this template :) and fill in all the required fields. ### Dify version 1.4.1 ### Cloud or Self Hosted Self Hosted (Docker) ### Steps to reproduce Using OpenAI OpenAI-API-compatible to add Reranked model served by vLLM The screencap of dify: <img width="603" alt="Image" src="https://github.com/user-attachments/assets/458b7651-2a4c-463b-8aad-22ac21d6f08c" /> Returns error: 'NoneType' object is not iterable Here is vLLM running log: ![Image](https://github.com/user-attachments/assets/3ad65444-5be2-47a2-b75f-85775abf6d7b) And here is log from `docker-api-1`: ``` 2025-06-09 10:16:29.830 ERROR [Dummy-26] [models.py:162] - Failed to save model credentials, tenant_id: 23e8e3e0-7893-4b48-9edf-0409a22a5b01, model: Qwen3-Reranker-8B, model_type: rerank Traceback (most recent call last): File "/app/api/controllers/console/workspace/models.py", line 154, in post model_provider_service.save_model_credentials( File "/app/api/services/model_provider_service.py", line 237, in save_model_credentials provider_configuration.add_or_update_custom_model_credentials( File "/app/api/core/entities/provider_configuration.py", line 422, in add_or_update_custom_model_credentials provider_model_record, credentials = self.custom_model_credentials_validate(model_type, model, credentials) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/app/api/core/entities/provider_configuration.py", line 402, in custom_model_credentials_validate credentials = model_provider_factory.model_credentials_validate( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/app/api/core/model_runtime/model_providers/model_provider_factory.py", line 191, in model_credentials_validate self.plugin_model_manager.validate_model_credentials( File "/app/api/core/plugin/impl/model.py", line 137, in validate_model_credentials for resp in response: ^^^^^^^^ File "/app/api/core/plugin/impl/base.py", line 192, in _request_with_plugin_daemon_response_stream self._handle_plugin_daemon_error(error.error_type, error.message) File "/app/api/core/plugin/impl/base.py", line 222, in _handle_plugin_daemon_error raise CredentialsValidateFailedError(error_object.get("message")) core.model_runtime.errors.validate.CredentialsValidateFailedError: 'NoneType' object is not iterable ``` ### ✔️ Expected Behavior Rerank model added ### ❌ Actual Behavior 'NoneType' object is not iterable
yindo closed this issue 2026-02-16 10:19:06 -05:00
yindo commented 2026-02-16 10:19:06 -05:00
Author
Owner
Copy Link
Copy Source

@dosubot[bot] commented on GitHub (Jun 9, 2025):

The error "'NoneType' object is not iterable" likely happens because the rerank model instance or its response is None at some point in the pipeline. This can occur if the reranker model configuration is missing required details (like provider name or credentials), or if the model provider/plugin is not properly registered or initialized in Dify. The code expects a list of reranked documents, but if the model instance or its response is None, this error will be raised when the code tries to iterate over it (source, source).

Double-check that:

  • The reranker model (Qwen3-Reranker-8B) is correctly registered as a provider in Dify.
  • All required fields (provider name, model name, credentials) are set in the model configuration.
  • The plugin/provider for Qwen3-Reranker-8B is available and enabled in your Dify instance.

If any of these are missing or misconfigured, Dify will not be able to instantiate the reranker model, resulting in the observed error. There is no hardcoded support for Qwen3-Reranker-8B or vLLM in the reranker pipeline; everything is routed through the provider/plugin abstraction, so proper registration and configuration are essential.

If you have already checked these and the issue persists, please share the exact reranker model configuration (with sensitive info redacted) for further troubleshooting.

To reply, just mention @dosu.

How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other  Join Discord Share on X

@dosubot[bot] commented on GitHub (Jun 9, 2025): <!-- Answer --> The error "'NoneType' object is not iterable" likely happens because the rerank model instance or its response is None at some point in the pipeline. This can occur if the reranker model configuration is missing required details (like provider name or credentials), or if the model provider/plugin is not properly registered or initialized in Dify. The code expects a list of reranked documents, but if the model instance or its response is None, this error will be raised when the code tries to iterate over it ([source](https://github.com/langgenius/dify/blob/0720bc74083ec12174c38d34399968693e94f7b4/api/core/rag/rerank/rerank_model.py), [source](https://github.com/langgenius/dify/blob/0720bc74083ec12174c38d34399968693e94f7b4/api/core/rag/data_post_processor/data_post_processor.py)). Double-check that: - The reranker model (Qwen3-Reranker-8B) is correctly registered as a provider in Dify. - All required fields (provider name, model name, credentials) are set in the model configuration. - The plugin/provider for Qwen3-Reranker-8B is available and enabled in your Dify instance. If any of these are missing or misconfigured, Dify will not be able to instantiate the reranker model, resulting in the observed error. There is no hardcoded support for Qwen3-Reranker-8B or vLLM in the reranker pipeline; everything is routed through the provider/plugin abstraction, so proper registration and configuration are essential. If you have already checked these and the issue persists, please share the exact reranker model configuration (with sensitive info redacted) for further troubleshooting. <!-- Dosu Comment Footer --> *To reply, just mention [@dosu](https://go.dosu.dev/dosubot).* --- <sup>How did I do? [Good](https://app.dosu.dev/response-feedback/6659981d-b0ee-4533-a082-74db044bd381?feedback_type=great_response) | [Irrelevant](https://app.dosu.dev/response-feedback/6659981d-b0ee-4533-a082-74db044bd381?feedback_type=irrelevant_answer) | [Incorrect](https://app.dosu.dev/response-feedback/6659981d-b0ee-4533-a082-74db044bd381?feedback_type=incorrect_sources) | [Verbose](https://app.dosu.dev/response-feedback/6659981d-b0ee-4533-a082-74db044bd381?feedback_type=too_verbose) | [Hallucination](https://app.dosu.dev/response-feedback/6659981d-b0ee-4533-a082-74db044bd381?feedback_type=hallucination) | [Report 🐛](https://app.dosu.dev/response-feedback/6659981d-b0ee-4533-a082-74db044bd381?feedback_type=bug_report) | [Other](https://app.dosu.dev/response-feedback/6659981d-b0ee-4533-a082-74db044bd381?feedback_type=other)</sup>&nbsp;&nbsp;[![Join Discord](https://img.shields.io/badge/join-5865F2?logo=discord&logoColor=white&label=)](https://go.dosu.dev/discord-bot)&nbsp;[![Share on X](https://img.shields.io/badge/X-share-black)](https://twitter.com/intent/tweet?text=%40dosu_ai%20helped%20me%20solve%20this%20issue!&url=https%3A//github.com/langgenius/dify/issues/20827)
yindo commented 2026-02-16 10:19:06 -05:00
Author
Owner
Copy Link
Copy Source

@elvizlai commented on GitHub (Jun 9, 2025):

maybe same as https://github.com/langgenius/dify/issues/14140, looks it still not solved. @dosu

@elvizlai commented on GitHub (Jun 9, 2025): maybe same as https://github.com/langgenius/dify/issues/14140, looks it still not solved. @dosu
yindo commented 2026-02-16 10:19:06 -05:00
Author
Owner
Copy Link
Copy Source

@dosubot[bot] commented on GitHub (Jun 9, 2025):

Yes, your issue does appear closely related to langgenius/dify#14140. Both involve rerank models served by vLLM and errors triggered when the top_n parameter is None. In the code, if top_n is not set, it can propagate as None and cause failures when the rerank model or its response is missing or malformed, leading to the 'NoneType' object is not iterable error (source). The workaround in langgenius/dify#14140—setting a default value for top_n—addresses the symptom but not the root cause, and there’s no permanent fix merged yet.

To avoid this error, make sure your rerank model config always provides a valid top_n value and that all required fields (provider name, model name, credentials) are set. If you’re using vLLM’s rerank API, you may need to patch your deployment to ensure top_n is never None in requests, or adjust your config/UI to always set it explicitly. The underlying issue is still open and not fully resolved upstream.

To reply, just mention @dosu.

How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other  Join Discord Share on X

@dosubot[bot] commented on GitHub (Jun 9, 2025): <!-- Answer --> Yes, your issue does appear closely related to langgenius/dify#14140. Both involve rerank models served by vLLM and errors triggered when the top_n parameter is None. In the code, if top_n is not set, it can propagate as None and cause failures when the rerank model or its response is missing or malformed, leading to the 'NoneType' object is not iterable error ([source](https://github.com/langgenius/dify/blob/0720bc74083ec12174c38d34399968693e94f7b4/api/core/rag/rerank/rerank_model.py)). The workaround in langgenius/dify#14140—setting a default value for top_n—addresses the symptom but not the root cause, and there’s no permanent fix merged yet. To avoid this error, make sure your rerank model config always provides a valid top_n value and that all required fields (provider name, model name, credentials) are set. If you’re using vLLM’s rerank API, you may need to patch your deployment to ensure top_n is never None in requests, or adjust your config/UI to always set it explicitly. The underlying issue is still open and not fully resolved upstream. <!-- Dosu Comment Footer --> *To reply, just mention [@dosu](https://go.dosu.dev/dosubot).* --- <sup>How did I do? [Good](https://app.dosu.dev/response-feedback/51f58055-7ffd-4e59-a53e-b2ac72589810?feedback_type=great_response) | [Irrelevant](https://app.dosu.dev/response-feedback/51f58055-7ffd-4e59-a53e-b2ac72589810?feedback_type=irrelevant_answer) | [Incorrect](https://app.dosu.dev/response-feedback/51f58055-7ffd-4e59-a53e-b2ac72589810?feedback_type=incorrect_sources) | [Verbose](https://app.dosu.dev/response-feedback/51f58055-7ffd-4e59-a53e-b2ac72589810?feedback_type=too_verbose) | [Hallucination](https://app.dosu.dev/response-feedback/51f58055-7ffd-4e59-a53e-b2ac72589810?feedback_type=hallucination) | [Report 🐛](https://app.dosu.dev/response-feedback/51f58055-7ffd-4e59-a53e-b2ac72589810?feedback_type=bug_report) | [Other](https://app.dosu.dev/response-feedback/51f58055-7ffd-4e59-a53e-b2ac72589810?feedback_type=other)</sup>&nbsp;&nbsp;[![Join Discord](https://img.shields.io/badge/join-5865F2?logo=discord&logoColor=white&label=)](https://go.dosu.dev/discord-bot)&nbsp;[![Share on X](https://img.shields.io/badge/X-share-black)](https://twitter.com/intent/tweet?text=%40dosu_ai%20helped%20me%20solve%20this%20issue!&url=https%3A//github.com/langgenius/dify/issues/20827)
yindo commented 2026-02-16 10:19:06 -05:00
Author
Owner
Copy Link
Copy Source

@Taimin commented on GitHub (Jul 22, 2025):

I have similar issue. But instead of the error mentioned above, the error I have is:

Image I am running the main branch in the docker hub of dify so dify is updated.
@Taimin commented on GitHub (Jul 22, 2025): I have similar issue. But instead of the error mentioned above, the error I have is: <img width="910" height="463" alt="Image" src="https://github.com/user-attachments/assets/3be3ca77-4ae0-41f2-aaec-e6373a4e6603" /> I am running the main branch in the docker hub of dify so dify is updated.
yindo commented 2026-02-16 10:19:06 -05:00
Author
Owner
Copy Link
Copy Source

@harrygleb commented on GitHub (Jul 29, 2025):

I have similar issue. But instead of the error mentioned above, the error I have is:
Image I am running the main branch in the docker hub of dify so dify is updated.

A similar problem

@harrygleb commented on GitHub (Jul 29, 2025): > I have similar issue. But instead of the error mentioned above, the error I have is: > <img alt="Image" width="910" height="463" src="https://private-user-images.githubusercontent.com/6102931/469401781-3be3ca77-4ae0-41f2-aaec-e6373a4e6603.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3NTM3NzU5NjMsIm5iZiI6MTc1Mzc3NTY2MywicGF0aCI6Ii82MTAyOTMxLzQ2OTQwMTc4MS0zYmUzY2E3Ny00YWUwLTQxZjItYWFlYy1lNjM3M2E0ZTY2MDMucG5nP1gtQW16LUFsZ29yaXRobT1BV1M0LUhNQUMtU0hBMjU2JlgtQW16LUNyZWRlbnRpYWw9QUtJQVZDT0RZTFNBNTNQUUs0WkElMkYyMDI1MDcyOSUyRnVzLWVhc3QtMSUyRnMzJTJGYXdzNF9yZXF1ZXN0JlgtQW16LURhdGU9MjAyNTA3MjlUMDc1NDIzWiZYLUFtei1FeHBpcmVzPTMwMCZYLUFtei1TaWduYXR1cmU9Njc3MzE3NTlmNzM1OWI5YmU5MjBkMjljMDIxYzU1YmY5YTcwMDVlNDU0YzBiZjY2OWZlMzQyNGI0MmRmOWI2MyZYLUFtei1TaWduZWRIZWFkZXJzPWhvc3QifQ.mSe34IhMraQmZaJbB3RoxxsMVmypy0IXOiDbhk9znsc"> I am running the main branch in the docker hub of dify so dify is updated. A similar problem
yindo commented 2026-02-16 10:19:06 -05:00
Author
Owner
Copy Link
Copy Source

@dosubot[bot] commented on GitHub (Aug 21, 2025):

Hi, @elvizlai. I'm Dosu, and I'm helping the dify-official-plugins team manage their backlog and am marking this issue as stale.

Issue Summary:

  • You reported a "'NoneType' object is not iterable" error when adding the Qwen3-Reranker-8B model with OpenAI API compatibility on Dify v1.4.1 (Docker).
  • The error appears related to missing or misconfigured reranker model details such as provider name, credentials, or the rerank model's top_n parameter being None.
  • A related unresolved issue (#14140) was referenced, indicating this is a known problem.
  • Another user, Taimin, reported a similar but distinct error on the main branch, suggesting ongoing instability with reranker integration.
  • Workarounds include ensuring proper model registration and explicitly setting the top_n parameter.

Next Steps:

  • Please let me know if this issue is still relevant with the latest version of dify-official-plugins by commenting here to keep the discussion open.
  • If I don’t hear back within 5 days, I will automatically close this issue to help us focus on active problems.

Thank you for your understanding and contribution!

@dosubot[bot] commented on GitHub (Aug 21, 2025): Hi, @elvizlai. I'm [Dosu](https://dosu.dev), and I'm helping the dify-official-plugins team manage their backlog and am marking this issue as stale. **Issue Summary:** - You reported a "'NoneType' object is not iterable" error when adding the Qwen3-Reranker-8B model with OpenAI API compatibility on Dify v1.4.1 (Docker). - The error appears related to missing or misconfigured reranker model details such as provider name, credentials, or the rerank model's top_n parameter being None. - A related unresolved issue (#14140) was referenced, indicating this is a known problem. - Another user, Taimin, reported a similar but distinct error on the main branch, suggesting ongoing instability with reranker integration. - Workarounds include ensuring proper model registration and explicitly setting the top_n parameter. **Next Steps:** - Please let me know if this issue is still relevant with the latest version of dify-official-plugins by commenting here to keep the discussion open. - If I don’t hear back within 5 days, I will automatically close this issue to help us focus on active problems. Thank you for your understanding and contribution!
yindo commented 2026-02-16 10:19:06 -05:00
Author
Owner
Copy Link
Copy Source

@elvizlai commented on GitHub (Aug 21, 2025):

I solved this by changing vllm version and using --task score.

@elvizlai commented on GitHub (Aug 21, 2025): I solved this by changing vllm version and using --task score.
yindo commented 2026-02-16 10:19:06 -05:00
Author
Owner
Copy Link
Copy Source

@dosubot[bot] commented on GitHub (Aug 21, 2025):

Thank you for closing the issue, elvizlai! We appreciate your help.

@dosubot[bot] commented on GitHub (Aug 21, 2025): Thank you for closing the issue, elvizlai! We appreciate your help.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
bug
bug
bug
dependencies
documentation
enhancement
good first issue
invalid
pull-request
Mirrored from GitHub Pull Request
 question
stale
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: langgenius/dify-official-plugins#353
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?