Mintplex-Labs/anything-llm
1
0
Fork 0
mirror of https://github.com/Mintplex-Labs/anything-llm.git synced 2026-07-21 01:25:28 -04:00
Code Issues 328 Packages Projects Releases 10 Wiki Activity

[GH-ISSUE #4700] [FEAT]: Allow configuring time-out #2979

New Issue
Open
opened 2026-02-22 18:32:07 -05:00 by yindo · 4 comments
yindo commented 2026-02-22 18:32:07 -05:00
Owner
Copy Link
Copy Source

Originally created by @mgraffam on GitHub (Dec 1, 2025).
Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/4700

What would you like to see?

It's crucial for mobile, less so for desktop: allow configuring the time-out on the model response.

When I have bad service on mobile, models time out just because of connectivity issues. If AnythingLLM were just more patient, there is a high degree of probability it would all work out.

It would be nice to have this on desktop too.. because why not? It's a simple field.

Seems it would also be good for people running local models that might be a bit on the slow side for performance.

The mobile app times out so fast its dumb as hell.

Originally created by @mgraffam on GitHub (Dec 1, 2025). Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/4700 ### What would you like to see? It's crucial for mobile, less so for desktop: allow configuring the time-out on the model response. When I have bad service on mobile, models time out just because of connectivity issues. If AnythingLLM were just more patient, there is a high degree of probability it would all work out. It would be nice to have this on desktop too.. because why not? It's a simple field. Seems it would also be good for people running local models that might be a bit on the slow side for performance. The mobile app times out so fast its dumb as hell.
yindo added the enhancementfeature requestOS: Mobile labels 2026-02-22 18:32:07 -05:00
yindo commented 2026-02-22 18:32:07 -05:00
Author
Owner
Copy Link
Copy Source

@Sinkingdev commented on GitHub (Dec 27, 2025):

I don't know about the phone instance. But as for desktop, at least the Docker instance.
You can set the timeout limit, although it's placed in a fairly hidden spot.

For workspace settings:

  1. Go to Settings → open the provider dropdown.
  2. Hover over the provider you’re using and click the ⚙️ gear icon that appears on the right.
  3. In that menu you can set timeout-limit (e.g. 120000 ms) before the model request gives up.

For system settings:

  1. Go to Settings → LLM → show advanced control.
  2. Set set timeout-limit to your preferred limit (e.g. 120000 ms).

This is something that I think should be changed; the timeout is both too hidden and too short by default. For me, many models just wouldn’t work at all. @timothycarambat, what do you think? Could we surface this control directly on the main LLM preference and workspace preference pages instead of hiding it behind a dropdown? It’s a clever spot, but I don't think most users interact with the provider dropdown often enough to stumble on it.

I also searched the docs for “timeout” and “provider timeout” without finding any guidance on how to do this. I might do a write-up for the docs, but I also think the UI change would be the better fix. Which solution would you prefer? UI changes are probably beyond my scope, so I’d be happy to document the current workflow if that helps in the meantime.

@Sinkingdev commented on GitHub (Dec 27, 2025): I don't know about the phone instance. But as for desktop, at least the Docker instance. You can set the timeout limit, although it's placed in a fairly hidden spot. For workspace settings: 1. Go to Settings → open the provider dropdown. 3. Hover over the provider you’re using and click the ⚙️ gear icon that appears on the right. 4. In that menu you can set timeout-limit (e.g. 120000 ms) before the model request gives up. For system settings: 1. Go to Settings → LLM → show advanced control. 2. Set set timeout-limit to your preferred limit (e.g. 120000 ms). This is something that I think should be changed; the timeout is both too hidden and too short by default. For me, many models just wouldn’t work at all. @timothycarambat, what do you think? Could we surface this control directly on the main LLM preference and workspace preference pages instead of hiding it behind a dropdown? It’s a clever spot, but I don't think most users interact with the provider dropdown often enough to stumble on it. I also searched the docs for “timeout” and “provider timeout” without finding any guidance on how to do this. I might do a write-up for the docs, but I also think the UI change would be the better fix. Which solution would you prefer? UI changes are probably beyond my scope, so I’d be happy to document the current workflow if that helps in the meantime.
yindo changed title from [FEAT]: Allow configuring time-out to [GH-ISSUE #4700] [FEAT]: Allow configuring time-out 2026-06-05 14:49:36 -04:00
yindo commented 2026-06-05 14:49:37 -04:00
Author
Owner
Copy Link
Copy Source

@MikeDuncan1 commented on GitHub (Mar 25, 2026):

This is a huge issue with a seemingly simple fix, when I get the model pre loaded it works so well but when it unloads it's another trip to the desktop to load it up again. I would keep it loaded in memory on the desktop but other models need to load periodically for tasks and a man can only afford so much vram

<!-- gh-comment-id:4131156126 --> @MikeDuncan1 commented on GitHub (Mar 25, 2026): This is a huge issue with a seemingly simple fix, when I get the model pre loaded it works so well but when it unloads it's another trip to the desktop to load it up again. I would keep it loaded in memory on the desktop but other models need to load periodically for tasks and a man can only afford so much vram
yindo commented 2026-06-05 14:49:37 -04:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Mar 26, 2026):

I would keep it loaded in memory on the desktop but other models need to load periodically for tasks and a man can only afford so much vram

What provider are you using? If the VRAM is competitive and other models are pushing it out we cannot keep it in memory, but we can (and do) have an option for keepAlive - at least for Ollama. Other providers have different ways of handling or specifying this.

The other comments are talking about the overall timeout, like for the HTTP request, not the model unload

<!-- gh-comment-id:4138133714 --> @timothycarambat commented on GitHub (Mar 26, 2026): > I would keep it loaded in memory on the desktop but other models need to load periodically for tasks and a man can only afford so much vram What provider are you using? If the VRAM is competitive and other models are pushing it out we cannot keep it in memory, but we can (and do) have an option for `keepAlive` - at least for Ollama. Other providers have different ways of handling or specifying this. The other comments are talking about the overall timeout, like for the HTTP request, not the model unload
yindo commented 2026-06-05 14:49:37 -04:00
Author
Owner
Copy Link
Copy Source

@elevatingcreativity commented on GitHub (Apr 2, 2026):

I am having the same issue as the OP, and have a proposed fix to submit for @timothycarambat's review.

More details:

Provider: LM Studio

Symptom: Chat returns "Could not respond to message / Socket Timeout" (or similar)
when running large local models (e.g. Qwen2.5-72B+ or similar multi-hundred GB models)
on long prompts. The error occurs before the model finishes generating.

Root cause: The LM Studio provider uses the OpenAI SDK with no custom timeout
configured, so it inherits the SDK's hardcoded default of 10 minutes (timeout:
600000). For large local models, prompt processing alone (before the first token is
produced) can easily exceed this — there is no data flowing on the connection during
this phase, so there's nothing to listen to; the timeout simply fires.

Comparison with other providers: Ollama already has OLLAMA_RESPONSE_TIMEOUT,
OpenRouter has OPENROUTER_TIMEOUT_MS, and Novita has NOVITA_LLM_TIMEOUT_MS. LM Studio
has none.

Proposed fix: Add LMSTUDIO_RESPONSE_TIMEOUT (in ms) env var support, defaulting to 2
hours.

Working branch: fix/lmstudio-response-timeout on https://github.com/elevatingcreativity/anything-llm. Will submit a PR once tested.

<!-- gh-comment-id:4180981309 --> @elevatingcreativity commented on GitHub (Apr 2, 2026): I am having the same issue as the OP, and have a proposed fix to submit for @timothycarambat's review. More details: **Provider**: LM Studio **Symptom**: Chat returns "Could not respond to message / Socket Timeout" (or similar) when running large local models (e.g. Qwen2.5-72B+ or similar multi-hundred GB models) on long prompts. The error occurs before the model finishes generating. **Root cause**: The LM Studio provider uses the OpenAI SDK with no custom timeout configured, so it inherits the SDK's hardcoded default of 10 minutes (timeout: 600000). For large local models, prompt processing alone (before the first token is produced) can easily exceed this — there is no data flowing on the connection during this phase, so there's nothing to listen to; the timeout simply fires. **Comparison with other providers**: Ollama already has OLLAMA_RESPONSE_TIMEOUT, OpenRouter has OPENROUTER_TIMEOUT_MS, and Novita has NOVITA_LLM_TIMEOUT_MS. LM Studio has none. **Proposed fix**: Add LMSTUDIO_RESPONSE_TIMEOUT (in ms) env var support, defaulting to 2 hours. Working branch: fix/lmstudio-response-timeout on https://github.com/elevatingcreativity/anything-llm. Will submit a PR once tested.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Desktop
Docker
Integration Request
Integration Request
OS: Linux
OS: Mobile
OS: Windows
UI/UX
blocked
bug
bug
core-team-only
documentation
duplicate
embed-widget
enhancement
feature request
github_actions
good first issue
investigating
needs info / can't replicate
possible bug
pull-request
Mirrored from GitHub Pull Request
 question
stage: specifications
wontfix
No Label OS: Mobile enhancement feature request
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Mintplex-Labs/anything-llm#2979
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?