Mintplex-Labs/anything-llm
1
0
Fork 0
mirror of https://github.com/Mintplex-Labs/anything-llm.git synced 2026-07-19 14:13:35 -04:00
Code Issues 328 Packages Projects Releases 10 Wiki Activity

[GH-ISSUE #2640] [BUG]: When I leave a chat that's still getting a response, then go back, it has disconnected. #1711

New Issue
Closed
opened 2026-02-22 18:26:11 -05:00 by yindo · 3 comments
yindo commented 2026-02-22 18:26:11 -05:00
Owner
Copy Link
Copy Source

Originally created by @phazei on GitHub (Nov 17, 2024).
Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/2640

How are you running AnythingLLM?

Docker (local)

What happened?

I asked for a response, then went to another chat, then went back.

I'd expect it to keep the socket open unless I hit stop. That way I'm not stuck on the chat screen waiting for a response when I could have other conversations going. It really slows down it's utility.

Are there known steps to reproduce?

Just chat and ask for something that would return a long response. While it's responding, leave the chat, then go back.

Originally created by @phazei on GitHub (Nov 17, 2024). Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/2640 ### How are you running AnythingLLM? Docker (local) ### What happened? I asked for a response, then went to another chat, then went back. I'd expect it to keep the socket open unless I hit stop. That way I'm not stuck on the chat screen waiting for a response when I could have other conversations going. It really slows down it's utility. ### Are there known steps to reproduce? Just chat and ask for something that would return a long response. While it's responding, leave the chat, then go back.
yindo added the possible bug label 2026-02-22 18:26:11 -05:00
yindo closed this issue 2026-02-22 18:26:11 -05:00
yindo commented 2026-02-22 18:26:11 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Nov 18, 2024):

When you leave the chat screen we disconnect the client connection, which in turn stops the streaming on the server side, which should propagate up all the way to the LLM to prevent infinite running LLM responses since a disconnect should propagate to a "STOP INFERENCE" for the LLM.

This in intentional right now

@timothycarambat commented on GitHub (Nov 18, 2024): When you leave the chat screen we disconnect the client connection, which in turn stops the streaming on the server side, which should propagate up all the way to the LLM to prevent infinite running LLM responses since a disconnect should propagate to a "STOP INFERENCE" for the LLM. This in intentional right now
yindo commented 2026-02-22 18:26:11 -05:00
Author
Owner
Copy Link
Copy Source

@phazei commented on GitHub (Nov 18, 2024):

I did a lot of testing with vLLM and llama.cpp. Hitting "stop", or leaving the page, or refreshing, none of those actually stop the inference, the process still runs until completion. I've tried with prompts that provide exceptionally long responses and verified it doesn't stop. It could be an issue with the upstream probably.

But other than clicking 'stop', I think this would be more functional if leaving the chat didn't disconnect the client.

@phazei commented on GitHub (Nov 18, 2024): I did a lot of testing with vLLM and llama.cpp. Hitting "stop", or leaving the page, or refreshing, none of those actually stop the inference, the process still runs until completion. I've tried with prompts that provide exceptionally long responses and verified it doesn't stop. It could be an issue with the upstream probably. But other than clicking 'stop', I think this would be more functional if leaving the chat didn't disconnect the client.
yindo commented 2026-02-22 18:26:11 -05:00
Author
Owner
Copy Link
Copy Source

@tmas commented on GitHub (Jan 17, 2026):

@timothycarambat Sorry to necro an old issue, but I'm experiencing the issue and I can confirm that the behavior @phazei reported (the conversation is lost in AnythingLLM but inference continues and the response from the LLM goes nowhere) is still happening, at least with LM Studio.

This is a bit of a pain when waiting for models to load, as I sometimes click away to read through another chat and then go back to find that my prompt has been lost along with the generated response from the LLM.

Even if the inference result is still discarded (I wouldn't be surprised if some inference providers genuinely support cancelling the request), maybe the prompt could be saved? That could potentially involve saving a "pending" state on the thread, or showing the user message as sent and handling a state where the most recent message is from the user and displaying a "Generate response" button to re-send the inference request.

@tmas commented on GitHub (Jan 17, 2026): @timothycarambat Sorry to necro an old issue, but I'm experiencing the issue and I can confirm that the behavior @phazei reported (the conversation is lost in AnythingLLM but inference continues and the response from the LLM goes nowhere) is still happening, at least with LM Studio. This is a bit of a pain when waiting for models to load, as I sometimes click away to read through another chat and then go back to find that my prompt has been lost along with the generated response from the LLM. Even if the inference result is still discarded (I wouldn't be surprised if some inference providers genuinely support cancelling the request), maybe the prompt could be saved? That could potentially involve saving a "pending" state on the thread, or showing the user message as sent and handling a state where the most recent message is from the user and displaying a "Generate response" button to re-send the inference request.
yindo changed title from [BUG]: When I leave a chat that's still getting a response, then go back, it has disconnected. to [GH-ISSUE #2640] [BUG]: When I leave a chat that's still getting a response, then go back, it has disconnected. 2026-06-05 14:42:16 -04:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Desktop
Docker
Integration Request
Integration Request
OS: Linux
OS: Mobile
OS: Windows
UI/UX
blocked
bug
bug
core-team-only
documentation
duplicate
embed-widget
enhancement
feature request
github_actions
good first issue
investigating
needs info / can't replicate
possible bug
pull-request
Mirrored from GitHub Pull Request
 question
stage: specifications
wontfix
No Label possible bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Mintplex-Labs/anything-llm#1711
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?