langgenius/dify
1
0
Fork 0
mirror of https://github.com/langgenius/dify.git synced 2026-07-20 09:10:05 -04:00
Code Issues 775 Packages Projects Releases 10 Wiki Activity

Slow rerank model causes timeout #21589

New Issue
Open
opened 2026-02-21 20:13:20 -05:00 by yindo · 9 comments
yindo commented 2026-02-21 20:13:20 -05:00
Owner
Copy Link
Copy Source

Originally created by @DavideDelbianco on GitHub (Jan 12, 2026).

Self Checks

  • I have read the Contributing Guide and Language Policy.
  • This is only for bug report, if you would like to ask a question, please head to Discussions.
  • I have searched for existing issues search for existing issues, including closed ones.
  • I confirm that I am using English to submit this report, otherwise it will be closed.
  • 【中文用户 & Non English User】请使用英语提交，否则会被关闭 ：）
  • Please do not modify this template :) and fill in all the required fields.

Dify version

1.11.1

Cloud or Self Hosted

Self Hosted (Docker)

Steps to reproduce

Have many documents to rerank
use a model that is experiencing some latency.

for example i am trying with jina rerank m0

✔️ Expected Behavior

should be able to set timeout for rerank model

❌ Actual Behavior

after 10 sec i get a timeout error that is swallowed and 0 documents are returned

Originally created by @DavideDelbianco on GitHub (Jan 12, 2026). ### Self Checks - [x] I have read the [Contributing Guide](https://github.com/langgenius/dify/blob/main/CONTRIBUTING.md) and [Language Policy](https://github.com/langgenius/dify/issues/1542). - [x] This is only for bug report, if you would like to ask a question, please head to [Discussions](https://github.com/langgenius/dify/discussions/categories/general). - [x] I have searched for existing issues [search for existing issues](https://github.com/langgenius/dify/issues), including closed ones. - [x] I confirm that I am using English to submit this report, otherwise it will be closed. - [x] 【中文用户 & Non English User】请使用英语提交，否则会被关闭 ：） - [x] Please do not modify this template :) and fill in all the required fields. ### Dify version 1.11.1 ### Cloud or Self Hosted Self Hosted (Docker) ### Steps to reproduce Have many documents to rerank use a model that is experiencing some latency. for example i am trying with jina rerank m0 ### ✔️ Expected Behavior should be able to set timeout for rerank model ### ❌ Actual Behavior after 10 sec i get a timeout error that is swallowed and 0 documents are returned
yindo added the 👻 feat:ragstale labels 2026-02-21 20:13:20 -05:00
yindo commented 2026-02-21 20:13:20 -05:00
Author
Owner
Copy Link
Copy Source

@dosubot[bot] commented on GitHub (Jan 12, 2026):

In Dify v1.11.1, rerank model timeouts (like with Jina rerank m0) are not configurable—timeouts are hardcoded in the plugin or HTTP request layers. When a slow rerank model hits the timeout (usually after 10 seconds), the error is caught in a retrieval thread and stored, which results in zero documents returned and no clear error surfaced to the user. This is a known limitation in this version and matches what others have reported in similar setups source.

Recent Dify versions (after v1.9.1) added support for configurable plugin/model timeouts via environment variables such as PLUGIN_DAEMON_TIMEOUT, HTTP_REQUEST_MAX_CONNECT_TIMEOUT, HTTP_REQUEST_MAX_READ_TIMEOUT, and HTTP_REQUEST_MAX_WRITE_TIMEOUT source. If you need to control rerank timeouts, upgrading to a newer version and setting these variables is the recommended solution.

For v1.11.1, practical workarounds are limited: you can try optimizing queries, increasing Docker/container resources, or patching the code to increase hardcoded timeouts, but there is no built-in way to set a rerank timeout. Monitoring logs for InvokeBadRequestError with "read operation timed out" can help you spot when this happens source.

Let me know if you want details on patching the code or more info about upgrading!

To reply, just mention @dosu.

How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other

Chat with Dosu Open in Cursor Join Discord Share on X

@dosubot[bot] commented on GitHub (Jan 12, 2026): <!-- Answer --> In Dify v1.11.1, rerank model timeouts (like with Jina rerank m0) are not configurable—timeouts are hardcoded in the plugin or HTTP request layers. When a slow rerank model hits the timeout (usually after 10 seconds), the error is caught in a retrieval thread and stored, which results in zero documents returned and no clear error surfaced to the user. This is a known limitation in this version and matches what others have reported in similar setups [source](https://github.com/langgenius/dify/issues/27075). Recent Dify versions (after v1.9.1) added support for configurable plugin/model timeouts via environment variables such as `PLUGIN_DAEMON_TIMEOUT`, `HTTP_REQUEST_MAX_CONNECT_TIMEOUT`, `HTTP_REQUEST_MAX_READ_TIMEOUT`, and `HTTP_REQUEST_MAX_WRITE_TIMEOUT` [source](https://github.com/langgenius/dify/issues/27075). If you need to control rerank timeouts, upgrading to a newer version and setting these variables is the recommended solution. For v1.11.1, practical workarounds are limited: you can try optimizing queries, increasing Docker/container resources, or patching the code to increase hardcoded timeouts, but there is no built-in way to set a rerank timeout. Monitoring logs for `InvokeBadRequestError` with "read operation timed out" can help you spot when this happens [source](https://github.com/langgenius/dify/issues/22921). Let me know if you want details on patching the code or more info about upgrading! <!-- Dosu Comment Footer --> *To reply, just mention [@dosu](https://go.dosu.dev/dosubot).* --- <sup>How did I do? [Good](https://app.dosu.dev/response-feedback/ba628ba9-283b-4135-b2c0-72b61145f59c?feedback_type=great_response) | [Irrelevant](https://app.dosu.dev/response-feedback/ba628ba9-283b-4135-b2c0-72b61145f59c?feedback_type=irrelevant_answer) | [Incorrect](https://app.dosu.dev/response-feedback/ba628ba9-283b-4135-b2c0-72b61145f59c?feedback_type=incorrect_sources) | [Verbose](https://app.dosu.dev/response-feedback/ba628ba9-283b-4135-b2c0-72b61145f59c?feedback_type=too_verbose) | [Hallucination](https://app.dosu.dev/response-feedback/ba628ba9-283b-4135-b2c0-72b61145f59c?feedback_type=hallucination) | [Report 🐛](https://app.dosu.dev/response-feedback/ba628ba9-283b-4135-b2c0-72b61145f59c?feedback_type=bug_report) | [Other](https://app.dosu.dev/response-feedback/ba628ba9-283b-4135-b2c0-72b61145f59c?feedback_type=other)</sup> [![Chat with Dosu](https://dosu.dev/dosu-chat-badge.svg)](https://app.dosu.dev/6088a4f4-073a-43f3-b3d4-4f06fb0c05a0/ask?utm_source=github)&nbsp;[![Open in Cursor](https://dosu.dev/dosu-in-cursor.svg)](https://cursor.com/link/prompt?text=In%20Dify%20v1.11.1%2C%20rerank%20model%20timeouts%20%28like%20with%20Jina%20rerank%20m0%29%20are%20not%20configurable%E2%80%94timeouts%20are%20hardcoded%20in%20the%20plugin%20or%20HTTP%20request%20layers.%20When%20a%20slow%20rerank%20model%20hits%20the%20timeout%20%28usually%20after%2010%20seconds%29%2C%20the%20error%20is%20caught%20in%20a%20retrieval%20thread%20and%20stored%2C%20which%20results%20in%20zero%20documents%20returned%20and%20no%20clear%20error%20surfaced%20to%20the%20user.%20This%20is%20a%20known%20limitation%20in%20this%20version%20and%20matches%20what%20others%20have%20reported%20in%20similar%20setups%20%5Bsource%5D%28https%3A//github.com/langgenius/dify/issues/27075%29.%0A%0ARecent%20Dify%20versions%20%28after%20v1.9.1%29%20added%20support%20for%20configurable%20plugin/model%20timeouts%20via%20environment%20variables%20such%20as%20%60PLUGIN_DAEMON_TIMEOUT%60%2C%20%60HTTP_REQUEST_MAX_CONNECT_TIMEOUT%60%2C%20%60HTTP_REQUEST_MAX_READ_TIMEOUT%60%2C%20and%20%60HTTP_REQUEST_MAX_WRITE_TIMEOUT%60%20%5Bsource%5D%28https%3A//github.com/langgenius/dify/issues/27075%29.%20If%20you%20need%20to%20control%20rerank%20timeouts%2C%20upgrading%20to%20a%20newer%20version%20and%20setting%20these%20variables%20is%20the%20recommended%20solution.%0A%0AFor%20v1.11.1%2C%20practical%20workarounds%20are%20limited%3A%20you%20can%20try%20optimizing%20queries%2C%20increasing%20Docker/container%20resources%2C%20or%20patching%20the%20code%20to%20increase%20hardcoded%20timeouts%2C%20but%20there%20is%20no%20built-in%20way%20to%20set%20a%20rerank%20timeout.%20Monitoring%20logs%20for%20%60InvokeBadRequestError%60%20with%20%22read%20operation%20timed%20out%22%20can%20help%20you%20spot%20when%20this%20happens%20%5Bsource%5D%28https%3A//github.com/langgenius/dify/issues/22921%29.%0A%0ALet%20me%20know%20if%20you%20want%20details%20on%20patching%20the%20code%20or%20more%20info%20about%20upgrading%21)&nbsp;[![Join Discord](https://img.shields.io/badge/join-5865F2?logo=discord&logoColor=white&label=)](https://go.dosu.dev/discord-bot)&nbsp;[![Share on X](https://img.shields.io/badge/X-share-black)](https://twitter.com/intent/tweet?text=%40dosu_ai%20helped%20me%20solve%20this%20issue!&url=https%3A//github.com/langgenius/dify/issues/30871)
yindo commented 2026-02-21 20:13:20 -05:00
Author
Owner
Copy Link
Copy Source

@fatelei commented on GitHub (Jan 12, 2026):

Self Checks

  • I have read the Contributing Guide and Language Policy.[x] This is only for bug report, if you would like to ask a question, please head to Discussions.[x] I have searched for existing issues search for existing issues, including closed ones.[x] I confirm that I am using English to submit this report, otherwise it will be closed.[x] 【中文用户 & Non English User】请使用英语提交，否则会被关闭 ：）[x] Please do not modify this template :) and fill in all the required fields.

Dify version

1.11.1

Cloud or Self Hosted

Self Hosted (Docker)

Steps to reproduce

Have many documents to rerank use a model that is experiencing some latency.

for example i am trying with jina rerank m0

✔️ Expected Behavior

should be able to set timeout for rerank model

❌ Actual Behavior

after 10 sec i get a timeout error that is swallowed and 0 documents are returned

have you try call jina rerank m0 directly ?

@fatelei commented on GitHub (Jan 12, 2026): > ### Self Checks > * [x] I have read the [Contributing Guide](https://github.com/langgenius/dify/blob/main/CONTRIBUTING.md) and [Language Policy](https://github.com/langgenius/dify/issues/1542).[x] This is only for bug report, if you would like to ask a question, please head to [Discussions](https://github.com/langgenius/dify/discussions/categories/general).[x] I have searched for existing issues [search for existing issues](https://github.com/langgenius/dify/issues), including closed ones.[x] I confirm that I am using English to submit this report, otherwise it will be closed.[x] 【中文用户 & Non English User】请使用英语提交，否则会被关闭 ：）[x] Please do not modify this template :) and fill in all the required fields. > > ### Dify version > 1.11.1 > > ### Cloud or Self Hosted > Self Hosted (Docker) > > ### Steps to reproduce > Have many documents to rerank use a model that is experiencing some latency. > > for example i am trying with jina rerank m0 > > ### ✔️ Expected Behavior > should be able to set timeout for rerank model > > ### ❌ Actual Behavior > after 10 sec i get a timeout error that is swallowed and 0 documents are returned have you try call jina rerank m0 directly ?
yindo commented 2026-02-21 20:13:20 -05:00
Author
Owner
Copy Link
Copy Source

@DavideDelbianco commented on GitHub (Jan 13, 2026):

Yes, it has latency issues and replies in 10-40 sec even for simple frank tasks.

But does not fail with error.

This means the error comes from a timeout setting in Dify httpx client

@DavideDelbianco commented on GitHub (Jan 13, 2026): Yes, it has latency issues and replies in 10-40 sec even for simple frank tasks. But does not fail with error. This means the error comes from a timeout setting in Dify httpx client
yindo commented 2026-02-21 20:13:20 -05:00
Author
Owner
Copy Link
Copy Source

@fatelei commented on GitHub (Jan 13, 2026):

Yes, it has latency issues and replies in 10-40 sec even for simple frank tasks.

But does not fail with error.

This means the error comes from a timeout setting in Dify httpx client

you can adjust PLUGIN_DAEMON_TIMEOUT and HTTP_REQUEST_MAX_CONNECT_TIMEOUT

@fatelei commented on GitHub (Jan 13, 2026): > Yes, it has latency issues and replies in 10-40 sec even for simple frank tasks. > > But does not fail with error. > > This means the error comes from a timeout setting in Dify httpx client you can adjust PLUGIN_DAEMON_TIMEOUT and HTTP_REQUEST_MAX_CONNECT_TIMEOUT
yindo commented 2026-02-21 20:13:20 -05:00
Author
Owner
Copy Link
Copy Source

@DavideDelbianco commented on GitHub (Jan 13, 2026):

The error is:

Error in stream response for plugin {'code': -500, 'message': '{"message":"{\\"args\\":{\\"description\\":\\"[models] Bad Request Error, The read operation timed out\\"},\\"error_type\\":\\"InvokeBadRequestError\\",\\"message\\":\\"[models] Bad Request Error, The read operation timed out\\"}","error_type":"PluginInvokeError","args":null}', 'data': None}

I had set:

GUNICORN_TIMEOUT=1200
PLUGIN_MAX_EXECUTION_TIMEOUT=1200
WORKFLOW_MAX_EXECUTION_STEPS=5000
WORKFLOW_MAX_EXECUTION_TIME=1200
HTTP_REQUEST_MAX_READ_TIMEOUT=1200

Will try again with the ENV variables you suggested, especially PLUGIN_DAEMON_TIMEOUT that was not set to match the daemon on api side.
I will not set HTTP_REQUEST_MAX_CONNECT_TIMEOUT because the error is about a "read timeout" not a connect timeout and the ENV variable suggested seems to be used just in the HTTP Node of workflows, not daemon requests

Anyway, looking at the most recent docker compose, I don't get why there are 2 ENV variables:
PLUGIN_DAEMON_TIMEOUT (api side)
PLUGIN_MAX_EXECUTION_TIMEOUT (daemon side)

Image

If their value should be kept in sync... just use a single ENV variable and pass it to both services!
In addition: PLUGIN_DAEMON_TIMEOUT is explicitly set in api service in docker-compose
Workflows executed by Triggers, for example the Schedule, are executed by the worker service.
it is just by coincidence that also the worker gets the correct value because of <<: *shared-api-worker-env

I don't understand why some values are explicitly set and some values are "inherited" the docker-compose could use a refactor to pass just the correct values to | api + worker +beat | daemon

@DavideDelbianco commented on GitHub (Jan 13, 2026): The error is: ```bash Error in stream response for plugin {'code': -500, 'message': '{"message":"{\\"args\\":{\\"description\\":\\"[models] Bad Request Error, The read operation timed out\\"},\\"error_type\\":\\"InvokeBadRequestError\\",\\"message\\":\\"[models] Bad Request Error, The read operation timed out\\"}","error_type":"PluginInvokeError","args":null}', 'data': None} ``` I had set: ```yaml GUNICORN_TIMEOUT=1200 PLUGIN_MAX_EXECUTION_TIMEOUT=1200 WORKFLOW_MAX_EXECUTION_STEPS=5000 WORKFLOW_MAX_EXECUTION_TIME=1200 HTTP_REQUEST_MAX_READ_TIMEOUT=1200 ``` Will try again with the ENV variables you suggested, especially PLUGIN_DAEMON_TIMEOUT that was not set to match the daemon on api side. I will not set HTTP_REQUEST_MAX_CONNECT_TIMEOUT because the error is about a "read timeout" not a connect timeout and the ENV variable suggested seems to be used just in the HTTP Node of workflows, not daemon requests Anyway, looking at the most recent docker compose, I don't get why there are 2 ENV variables: PLUGIN_DAEMON_TIMEOUT (api side) PLUGIN_MAX_EXECUTION_TIMEOUT (daemon side) <img width="840" height="145" alt="Image" src="https://github.com/user-attachments/assets/c9170a0f-9973-4869-b3ec-3c69fde27d1b" /> If their value should be kept in sync... just use a single ENV variable and pass it to both services! In addition: PLUGIN_DAEMON_TIMEOUT is explicitly set in api service in docker-compose Workflows executed by Triggers, for example the Schedule, are executed by the worker service. it is just by coincidence that also the worker gets the correct value because of `<<: *shared-api-worker-env` I don't understand why some values are explicitly set and some values are "inherited" the docker-compose could use a refactor to pass just the correct values to | api + worker +beat | daemon
yindo commented 2026-02-21 20:13:21 -05:00
Author
Owner
Copy Link
Copy Source

@fatelei commented on GitHub (Jan 13, 2026):

PLUGIN_DAEMON_TIMEOUT mean api call dify plugin timeout
PLUGIN_MAX_EXECUTION_TIMEOUT mean dify plugin call large model service timeout

@fatelei commented on GitHub (Jan 13, 2026): PLUGIN_DAEMON_TIMEOUT mean api call dify plugin timeout PLUGIN_MAX_EXECUTION_TIMEOUT mean dify plugin call large model service timeout
yindo commented 2026-02-21 20:13:21 -05:00
Author
Owner
Copy Link
Copy Source

@fatelei commented on GitHub (Jan 13, 2026):

the error is dify plugin daemon call large model service timeout

@fatelei commented on GitHub (Jan 13, 2026): the error is dify plugin daemon call large model service timeout
yindo commented 2026-02-21 20:13:21 -05:00
Author
Owner
Copy Link
Copy Source

@DavideDelbianco commented on GitHub (Jan 13, 2026):

Which was already set to 1200 sec, today jina model is back to normal timings and I can't reproduce the timeout issue. I would need to host something that mocks the remote API and make it slow.

I've added the other ENV, but the default values would have been 5 min, and the issue happened with just 10 to 40 sec timings.
Maybe the variables are there, but are not used...

@DavideDelbianco commented on GitHub (Jan 13, 2026): Which was already set to 1200 sec, today jina model is back to normal timings and I can't reproduce the timeout issue. I would need to host something that mocks the remote API and make it slow. I've added the other ENV, but the default values would have been 5 min, and the issue happened with just 10 to 40 sec timings. Maybe the variables are there, but are not used...
yindo commented 2026-02-21 20:13:21 -05:00
Author
Owner
Copy Link
Copy Source

@dosubot[bot] commented on GitHub (Feb 13, 2026):

Hi, @DavideDelbianco. I'm Dosu, and I'm helping the Dify team manage their backlog and am marking this issue as stale.

Issue Summary:

  • You reported a timeout problem in Dify v1.11.1 when using slow rerank models, causing no documents to return after 10 seconds.
  • The rerank timeout was hardcoded in that version and not configurable.
  • It was recommended to upgrade to newer versions where timeouts can be adjusted via environment variables like PLUGIN_DAEMON_TIMEOUT.
  • You expressed confusion about multiple timeout environment variables in the Docker setup and noted that setting high timeout values might not fully resolve the issue.
  • The issue remains open without a definitive fix, pending better timeout handling or code patches.

Next Steps:

  • Please let me know if this issue is still relevant with the latest version of Dify by commenting here.
  • Otherwise, this issue will be automatically closed in 15 days.

Thank you for your understanding and contribution!

@dosubot[bot] commented on GitHub (Feb 13, 2026): Hi, @DavideDelbianco. I'm [Dosu](https://dosu.dev), and I'm helping the Dify team manage their backlog and am marking this issue as stale. **Issue Summary:** - You reported a timeout problem in Dify v1.11.1 when using slow rerank models, causing no documents to return after 10 seconds. - The rerank timeout was hardcoded in that version and not configurable. - It was recommended to upgrade to newer versions where timeouts can be adjusted via environment variables like `PLUGIN_DAEMON_TIMEOUT`. - You expressed confusion about multiple timeout environment variables in the Docker setup and noted that setting high timeout values might not fully resolve the issue. - The issue remains open without a definitive fix, pending better timeout handling or code patches. **Next Steps:** - Please let me know if this issue is still relevant with the latest version of Dify by commenting here. - Otherwise, this issue will be automatically closed in 15 days. Thank you for your understanding and contribution!
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Knowledge Pipeline
Proposal
WIP
cloud
dependencies
feat:webapp
good first issue
outdated
performance
pull-request
Mirrored from GitHub Pull Request
 python:uv
refactor
size:L
stale
status: accepting prs
status: blocked
status: in discussion
stt & tts
vulnerablity
web
wont-fix
🌊 feat:workflow
🌚 invalid
🐞 bug
👻 feat:rag
💪 enhancement
📚 documentation
🔨 feat:tools
🙋‍♂️ question
🙏 help wanted
🤔 cant-reproduce
🤖 feat:agent
No Label stale 👻 feat:rag
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: langgenius/dify#21589
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?