Mintplex-Labs/anything-llm
1
0
Fork 0
mirror of https://github.com/Mintplex-Labs/anything-llm.git synced 2026-07-19 22:23:50 -04:00
Code Issues 328 Packages Projects Releases 10 Wiki Activity

[GH-ISSUE #1807] [BUG]: Configuration to Enable/Disable LLM Streaming #1177

New Issue
Closed
opened 2026-02-22 18:23:33 -05:00 by yindo · 4 comments
yindo commented 2026-02-22 18:23:33 -05:00
Owner
Copy Link
Copy Source

Originally created by @grundc on GitHub (Jul 3, 2024).
Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/1807

Originally assigned to: @timothycarambat on GitHub.

What would you like to see?

I have an issue using AnythingLLM with our companies AzureOpenAI endpoint. When trying to start a chat the docker container is crashing and shutting down. In the log I can see the following error.

undefined
2024-07-03 16:20:18 /app/server/endpoints/chat.js:117
2024-07-03 16:20:18 error: e.message,
2024-07-03 16:20:18 ^
2024-07-03 16:20:18
2024-07-03 16:20:18 TypeError: Cannot read properties of undefined (reading 'message')
2024-07-03 16:20:18 at /app/server/endpoints/chat.js:117:20
2024-07-03 16:20:18 at process.processTicksAndRejections (node:internal/process/task_queues:95:5)

Not sure what exactly is the reason for it. But I got to notice that my company disabled "streaming" on the endpoint and something is telling me, that it migh be related to that.

I am wondering if there is a possibility to prevent anythingLLM from using the streaming approach.

Originally created by @grundc on GitHub (Jul 3, 2024). Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/1807 Originally assigned to: @timothycarambat on GitHub. ### What would you like to see? I have an issue using AnythingLLM with our companies AzureOpenAI endpoint. When trying to start a chat the docker container is crashing and shutting down. In the log I can see the following error. undefined 2024-07-03 16:20:18 /app/server/endpoints/chat.js:117 2024-07-03 16:20:18 error: e.message, 2024-07-03 16:20:18 ^ 2024-07-03 16:20:18 2024-07-03 16:20:18 TypeError: Cannot read properties of undefined (reading 'message') 2024-07-03 16:20:18 at /app/server/endpoints/chat.js:117:20 2024-07-03 16:20:18 at process.processTicksAndRejections (node:internal/process/task_queues:95:5) Not sure what exactly is the reason for it. But I got to notice that my company **disabled "streaming"** on the endpoint and something is telling me, that it migh be related to that. I am wondering if there is a possibility to prevent anythingLLM from using the streaming approach.
yindo added the bugcore-team-onlyinvestigating labels 2026-02-22 18:23:33 -05:00
yindo closed this issue 2026-02-22 18:23:33 -05:00
yindo commented 2026-02-22 18:23:33 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Jul 3, 2024):

I didnt even know you could disable streaming for Azure endpoints. if that is the case - this is certainly the reason you are getting an error here

@timothycarambat commented on GitHub (Jul 3, 2024): I didnt even know you could disable streaming for Azure endpoints. if that is the case - this is certainly the reason you are getting an error here
yindo commented 2026-02-22 18:23:33 -05:00
Author
Owner
Copy Link
Copy Source

@grundc commented on GitHub (Jul 4, 2024):

I didnt even know you could disable streaming for Azure endpoints. if that is the case - this is certainly the reason you are getting an error here

May be to clarify this. My company decided to put a proxy inbetween. So I am not consuming the Azure-Endpoint directly. So it might be that the endpoint handler has this disabled and not directly the Azure-Endpoint.

@grundc commented on GitHub (Jul 4, 2024): > I didnt even know you could disable streaming for Azure endpoints. if that is the case - this is certainly the reason you are getting an error here May be to clarify this. My company decided to put a proxy inbetween. So I am not consuming the Azure-Endpoint directly. So it might be that the endpoint handler has this disabled and not directly the Azure-Endpoint.
yindo commented 2026-02-22 18:23:33 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Jul 5, 2024):

This is almost certainly the culprit then. As an example, default nginx settings as a proxy will actually block streaming response and they will build in buffer and send as once completion

@timothycarambat commented on GitHub (Jul 5, 2024): This is almost certainly the culprit then. As an example, default nginx settings as a proxy will actually block streaming response and they will build in buffer and send as once completion
yindo commented 2026-02-22 18:23:33 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Dec 11, 2024):

moved to #2804

@timothycarambat commented on GitHub (Dec 11, 2024): moved to #2804
yindo changed title from [BUG]: Configuration to Enable/Disable LLM Streaming to [GH-ISSUE #1807] [BUG]: Configuration to Enable/Disable LLM Streaming 2026-06-05 14:39:23 -04:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Desktop
Docker
Integration Request
Integration Request
OS: Linux
OS: Mobile
OS: Windows
UI/UX
blocked
bug
bug
core-team-only
documentation
duplicate
embed-widget
enhancement
feature request
github_actions
good first issue
investigating
needs info / can't replicate
possible bug
pull-request
Mirrored from GitHub Pull Request
 question
stage: specifications
wontfix
No Label bug core-team-only investigating
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Mintplex-Labs/anything-llm#1177
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?