Mintplex-Labs/anything-llm
1
0
Fork 0
mirror of https://github.com/Mintplex-Labs/anything-llm.git synced 2026-07-21 01:25:28 -04:00
Code Issues 328 Packages Projects Releases 10 Wiki Activity

[GH-ISSUE #540] Possible to add "STOP" button if user does not like the AI's response? #317

New Issue
Closed
opened 2026-02-22 18:18:55 -05:00 by yindo · 5 comments
yindo commented 2026-02-22 18:18:55 -05:00
Owner
Copy Link
Copy Source

Originally created by @jameschen83 on GitHub (Jan 5, 2024).
Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/540

Originally assigned to: @shatfield4 on GitHub.

In ChatGPT, user can click "stop generation" button to stop LLM streaming. Is it available in AnythingLLM, or Possible to add this feature in AnythingLLM?

If no, I plan to fork the branch to add the feature.

Originally created by @jameschen83 on GitHub (Jan 5, 2024). Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/540 Originally assigned to: @shatfield4 on GitHub. In ChatGPT, user can click "stop generation" button to stop LLM streaming. Is it available in AnythingLLM, or Possible to add this feature in AnythingLLM? If no, I plan to fork the branch to add the feature.
yindo added the enhancementgood first issueUI/UX labels 2026-02-22 18:18:55 -05:00
yindo closed this issue 2026-02-22 18:18:55 -05:00
yindo commented 2026-02-22 18:18:55 -05:00
Author
Owner
Copy Link
Copy Source

@duc-gp commented on GitHub (Feb 16, 2024):

When I would pick this up to implement, how would I start and how is the process regarding UI / UX? Are there any design specs for this or can the developer propose something in their Pull request?

@duc-gp commented on GitHub (Feb 16, 2024): When I would pick this up to implement, how would I start and how is the process regarding UI / UX? Are there any design specs for this or can the developer propose something in their Pull request?
yindo commented 2026-02-22 18:18:55 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Feb 16, 2024):

We do not have a design for this yet. The main limitation here is that all this would do is disconnect the client from the response stream - it would not terminate the request at the LLM side - so an infinite response loop would still continue on the LLM side and it would stay occupied until it finished.

If the client just wants to "stop" seeing the output then we can break the response stream but I think this will lead to more issues with localLLMs that go wild and keep streaming and the user sends a new prompt and now the chat is hanging because the LocalLLM is still stream the old response.

@timothycarambat commented on GitHub (Feb 16, 2024): We do not have a design for this yet. The main limitation here is that all this would do is disconnect the client from the response stream - it would not terminate the request at the LLM side - so an infinite response loop would still continue on the LLM side and it would stay occupied until it finished. If the client just wants to "stop" seeing the output then we can break the response stream but I think this will lead to more issues with localLLMs that go wild and keep streaming and the user sends a new prompt and now the chat is hanging because the LocalLLM is still stream the old response.
yindo commented 2026-02-22 18:18:55 -05:00
Author
Owner
Copy Link
Copy Source

@duc-gp commented on GitHub (Feb 17, 2024):

We do not have a design for this yet. The main limitation here is that all this would do is disconnect the client from the response stream - it would not terminate the request at the LLM side - so an infinite response loop would still continue on the LLM side and it would stay occupied until it finished.

If the client just wants to "stop" seeing the output then we can break the response stream but I think this will lead to more issues with localLLMs that go wild and keep streaming and the user sends a new prompt and now the chat is hanging because the LocalLLM is still stream the old response.

I dig a little bit into the code, so I think a way to disconnect the client would be to declare a abortcontroller instance outside of https://github.com/Mintplex-Labs/anything-llm/blob/f4b09a8c794f4e1ae0a21a2f47790b680579e087/frontend/src/models/workspace.js#L76 and passing it, so it can be called from outside where its used? Would this be the correct way to implement this?

But then I end up having still a blinking cursor at the end, see screenshot:
Screenshot 2024-02-17 at 23 06 09

Then I thought about passing a chatResult with type: "finalizeResponseStream" but for that I would need the chatId ?

Can you provide some guidance on this topic, maybe I'm on a complete wrong way?

Thanks

@duc-gp commented on GitHub (Feb 17, 2024): > We do not have a design for this yet. The main limitation here is that all this would do is disconnect the client from the response stream - it would not terminate the request at the LLM side - so an infinite response loop would still continue on the LLM side and it would stay occupied until it finished. > > If the client just wants to "stop" seeing the output then we can break the response stream but I think this will lead to more issues with localLLMs that go wild and keep streaming and the user sends a new prompt and now the chat is hanging because the LocalLLM is still stream the old response. I dig a little bit into the code, so I think a way to disconnect the client would be to declare a abortcontroller instance outside of https://github.com/Mintplex-Labs/anything-llm/blob/f4b09a8c794f4e1ae0a21a2f47790b680579e087/frontend/src/models/workspace.js#L76 and passing it, so it can be called from outside where its used? Would this be the correct way to implement this? But then I end up having still a blinking cursor at the end, see screenshot: ![Screenshot 2024-02-17 at 23 06 09](https://github.com/Mintplex-Labs/anything-llm/assets/40763918/699805eb-9ff1-4d01-ad79-9a493f97a56d) Then I thought about passing a chatResult with `type: "finalizeResponseStream"` but for that I would need the `chatId` ? Can you provide some guidance on this topic, maybe I'm on a complete wrong way? Thanks
yindo commented 2026-02-22 18:18:56 -05:00
Author
Owner
Copy Link
Copy Source

@sumitsodhi88 commented on GitHub (Feb 18, 2024):

has it been added? i cant see it.

@sumitsodhi88 commented on GitHub (Feb 18, 2024): has it been added? i cant see it.
yindo commented 2026-02-22 18:18:56 -05:00
Author
Owner
Copy Link
Copy Source

@duc-gp commented on GitHub (Feb 18, 2024):

has it been added? i cant see it.

no it hasnt been added yet afaik. I think about implementing it but I'm not sure what the correct approach would be thats why I asked for guidance here https://github.com/Mintplex-Labs/anything-llm/issues/540#issuecomment-1950430063

@duc-gp commented on GitHub (Feb 18, 2024): > has it been added? i cant see it. no it hasnt been added yet afaik. I think about implementing it but I'm not sure what the correct approach would be thats why I asked for guidance here https://github.com/Mintplex-Labs/anything-llm/issues/540#issuecomment-1950430063
yindo changed title from Possible to add "STOP" button if user does not like the AI's response? to [GH-ISSUE #540] Possible to add "STOP" button if user does not like the AI's response? 2026-06-05 14:34:40 -04:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Desktop
Docker
Integration Request
Integration Request
OS: Linux
OS: Mobile
OS: Windows
UI/UX
blocked
bug
bug
core-team-only
documentation
duplicate
embed-widget
enhancement
feature request
github_actions
good first issue
investigating
needs info / can't replicate
possible bug
pull-request
Mirrored from GitHub Pull Request
 question
stage: specifications
wontfix
No Label UI/UX enhancement good first issue
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Mintplex-Labs/anything-llm#317
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?