Mintplex-Labs/anything-llm
1
0
Fork 0
mirror of https://github.com/Mintplex-Labs/anything-llm.git synced 2026-07-19 14:13:35 -04:00
Code Issues 328 Packages Projects Releases 10 Wiki Activity

[GH-ISSUE #670] [BUG]: exception on calling LLM causes whole process to restart and hurts other clients #382

New Issue
Closed
opened 2026-02-22 18:19:14 -05:00 by yindo · 0 comments
yindo commented 2026-02-22 18:19:14 -05:00
Owner
Copy Link
Copy Source

Originally created by @mkhludnev on GitHub (Feb 3, 2024).
Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/670

How are you running AnythingLLM?

Docker (local), as well as linux/macos service/

What happened?

In my setup AnythingLLM calls Ollama, which is lagging for a while. I see timeout in anything llm container log

node:internal/deps/undici/undici:11730
    Error.captureStackTrace(err, this);
...
TypeError: fetch failed\
   at Object.fetch (node:internal/deps/undici/undici:11730:11)
   at async createOllamaStream (/app/server/node_modules/langchain/dist/util/
...
 cause: HeadersTimeoutError: Headers Timeout Error\n"
...
    at Timeout.onParserTimeout [as callback] (node:internal/deps/undici/undici:8228:32)\n"

code: 'UND_ERR_HEADERS_TIMEOUT'\n

it's perfectly clear and makes to questions. But then it's followed by

Node.js v18.19.0
Collector hot directory and tmp storage wiped!
Document processor app listening on port 8888
Environment variables loaded from .env
Prisma schema loaded from prisma/schema.prisma

It seems like restarting the process on exception occurs in one client thread.

Are there known steps to reproduce?

I setup simple health check spin
while true ; do curl -s -v localhost:3001 -o /dev/null >>/tmp/out 2>>/tmp/out; gdate +"%Y-%m-%d %T.%N" >>/tmp/out;done

Then I send request to chat and bounce ollama, got exception and red error message in chat. That's expected

2024-02-03 13:12:20 node:internal/deps/undici/undici:11730
2024-02-03 13:12:20     Error.captureStackTrace(err, this);
2024-02-03 13:12:20           ^
2024-02-03 13:12:20 
2024-02-03 13:12:20 TypeError: fetch failed
2024-02-03 13:12:20     at Object.fetch (node:internal/deps/undici/undici:11730:11)
2024-02-03 13:12:20     at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
2024-02-03 13:12:20     at async createOllamaStream (/app/server/node_modules/langchain/dist/util/ollama.cjs:12:22)
2024-02-03 13:12:20     at async ChatOllama._streamResponseChunks (/app/server/node_modules/langchain/dist/chat_models/ollama.cjs:337:26)
2024-02-03 13:12:20     at async ChatOllama._streamIterator (/app/server/node_modules/@langchain/core/dist/language_models/chat_models.cjs:82:34)
2024-02-03 13:12:20     at async ChatOllama.transform (/app/server/node_modules/@langchain/core/dist/runnables/base.cjs:307:9)
2024-02-03 13:12:20     at async wrapInputForTracing (/app/server/node_modules/@langchain/core/dist/runnables/base.cjs:208:30)
2024-02-03 13:12:20     at async StringOutputParser._transform (/app/server/node_modules/@langchain/core/dist/output_parsers/transform.cjs:13:26)
2024-02-03 13:12:20     at async StringOutputParser._transformStreamWithConfig (/app/server/node_modules/@langchain/core/dist/runnables/base.cjs:235:30)
2024-02-03 13:12:20     at async StringOutputParser.transform (/app/server/node_modules/@langchain/core/dist/output_parsers/transform.cjs:37:9) {
2024-02-03 13:12:20   cause: Error: connect ECONNREFUSED 192.168.65.254:11434
2024-02-03 13:12:20       at TCPConnectWrap.afterConnect [as oncomplete] (node:net:1555:16) {
2024-02-03 13:12:20     errno: -111,
2024-02-03 13:12:20     code: 'ECONNREFUSED',
2024-02-03 13:12:20     syscall: 'connect',
2024-02-03 13:12:20     address: '192.168.65.254',
2024-02-03 13:12:20     port: 11434
2024-02-03 13:12:20   }
2024-02-03 13:12:20 }

in the poller log

2024-02-03 13:12:19.533350000
*   Trying [::1]:3001...
* Connected to localhost (::1) port 3001
> GET / HTTP/1.1
> Host: localhost:3001
> User-Agent: curl/8.4.0
> Accept: */*
> 
< HTTP/1.1 200 OK
< X-Powered-By: Express
< Vary: Origin
< Accept-Ranges: bytes
< Cache-Control: public, max-age=0
< Last-Modified: Fri, 12 Jan 2024 22:24:49 GMT
< ETag: W/"695-18cffc71f68"
< Content-Type: text/html; charset=UTF-8
< Content-Length: 1685
< Date: Sat, 03 Feb 2024 10:12:20 GMT
< Connection: keep-alive
< Keep-Alive: timeout=5
< 
{ [1685 bytes data]
* Connection #0 to host localhost left intact

^ the last successful check. Then it start failing:

2024-02-03 13:12:20.220928000
*   Trying [::1]:3001...
* Connected to localhost (::1) port 3001
> GET / HTTP/1.1
> Host: localhost:3001
> User-Agent: curl/8.4.0
> Accept: */*
> 
* Empty reply from server
* Closing connection
2024-02-03 13:12:20.307283000
*   Trying [::1]:3001...
* Connected to localhost (::1) port 3001
> GET / HTTP/1.1
> Host: localhost:3001
> User-Agent: curl/8.4.0
> Accept: */*
> 
* Recv failure: Connection reset by peer
* Closing connection
2024-02-03 13:12:20.364953000
*   Trying [::1]:3001...
* Connected to localhost (::1) port 3001
> GET / HTTP/1.1
> Host: localhost:3001
> User-Agent: curl/8.4.0
> Accept: */*
> 
* Recv failure: Connection reset by peer
* Closing connection

meanwhile Node is rebooting

2024-02-03 13:12:20 
2024-02-03 13:12:20 Node.js v18.19.0
Originally created by @mkhludnev on GitHub (Feb 3, 2024). Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/670 ### How are you running AnythingLLM? Docker (local), as well as linux/macos service/ ### What happened? In my setup AnythingLLM calls Ollama, which is lagging for a while. I see timeout in anything llm container log ``` node:internal/deps/undici/undici:11730 Error.captureStackTrace(err, this); ... TypeError: fetch failed\ at Object.fetch (node:internal/deps/undici/undici:11730:11) at async createOllamaStream (/app/server/node_modules/langchain/dist/util/ ... cause: HeadersTimeoutError: Headers Timeout Error\n" ... at Timeout.onParserTimeout [as callback] (node:internal/deps/undici/undici:8228:32)\n" code: 'UND_ERR_HEADERS_TIMEOUT'\n ``` it's perfectly clear and makes to questions. But then it's followed by ``` Node.js v18.19.0 Collector hot directory and tmp storage wiped! Document processor app listening on port 8888 Environment variables loaded from .env Prisma schema loaded from prisma/schema.prisma ``` It seems like restarting the process on exception occurs in one client thread. ### Are there known steps to reproduce? I setup simple health check spin `while true ; do curl -s -v localhost:3001 -o /dev/null >>/tmp/out 2>>/tmp/out; gdate +"%Y-%m-%d %T.%N" >>/tmp/out;done` Then I send request to chat and bounce ollama, got exception and red error message in chat. That's expected ``` 2024-02-03 13:12:20 node:internal/deps/undici/undici:11730 2024-02-03 13:12:20 Error.captureStackTrace(err, this); 2024-02-03 13:12:20 ^ 2024-02-03 13:12:20 2024-02-03 13:12:20 TypeError: fetch failed 2024-02-03 13:12:20 at Object.fetch (node:internal/deps/undici/undici:11730:11) 2024-02-03 13:12:20 at process.processTicksAndRejections (node:internal/process/task_queues:95:5) 2024-02-03 13:12:20 at async createOllamaStream (/app/server/node_modules/langchain/dist/util/ollama.cjs:12:22) 2024-02-03 13:12:20 at async ChatOllama._streamResponseChunks (/app/server/node_modules/langchain/dist/chat_models/ollama.cjs:337:26) 2024-02-03 13:12:20 at async ChatOllama._streamIterator (/app/server/node_modules/@langchain/core/dist/language_models/chat_models.cjs:82:34) 2024-02-03 13:12:20 at async ChatOllama.transform (/app/server/node_modules/@langchain/core/dist/runnables/base.cjs:307:9) 2024-02-03 13:12:20 at async wrapInputForTracing (/app/server/node_modules/@langchain/core/dist/runnables/base.cjs:208:30) 2024-02-03 13:12:20 at async StringOutputParser._transform (/app/server/node_modules/@langchain/core/dist/output_parsers/transform.cjs:13:26) 2024-02-03 13:12:20 at async StringOutputParser._transformStreamWithConfig (/app/server/node_modules/@langchain/core/dist/runnables/base.cjs:235:30) 2024-02-03 13:12:20 at async StringOutputParser.transform (/app/server/node_modules/@langchain/core/dist/output_parsers/transform.cjs:37:9) { 2024-02-03 13:12:20 cause: Error: connect ECONNREFUSED 192.168.65.254:11434 2024-02-03 13:12:20 at TCPConnectWrap.afterConnect [as oncomplete] (node:net:1555:16) { 2024-02-03 13:12:20 errno: -111, 2024-02-03 13:12:20 code: 'ECONNREFUSED', 2024-02-03 13:12:20 syscall: 'connect', 2024-02-03 13:12:20 address: '192.168.65.254', 2024-02-03 13:12:20 port: 11434 2024-02-03 13:12:20 } 2024-02-03 13:12:20 } ``` in the poller log ``` 2024-02-03 13:12:19.533350000 * Trying [::1]:3001... * Connected to localhost (::1) port 3001 > GET / HTTP/1.1 > Host: localhost:3001 > User-Agent: curl/8.4.0 > Accept: */* > < HTTP/1.1 200 OK < X-Powered-By: Express < Vary: Origin < Accept-Ranges: bytes < Cache-Control: public, max-age=0 < Last-Modified: Fri, 12 Jan 2024 22:24:49 GMT < ETag: W/"695-18cffc71f68" < Content-Type: text/html; charset=UTF-8 < Content-Length: 1685 < Date: Sat, 03 Feb 2024 10:12:20 GMT < Connection: keep-alive < Keep-Alive: timeout=5 < { [1685 bytes data] * Connection #0 to host localhost left intact ``` ^ the last successful check. Then it start failing: ``` 2024-02-03 13:12:20.220928000 * Trying [::1]:3001... * Connected to localhost (::1) port 3001 > GET / HTTP/1.1 > Host: localhost:3001 > User-Agent: curl/8.4.0 > Accept: */* > * Empty reply from server * Closing connection 2024-02-03 13:12:20.307283000 * Trying [::1]:3001... * Connected to localhost (::1) port 3001 > GET / HTTP/1.1 > Host: localhost:3001 > User-Agent: curl/8.4.0 > Accept: */* > * Recv failure: Connection reset by peer * Closing connection 2024-02-03 13:12:20.364953000 * Trying [::1]:3001... * Connected to localhost (::1) port 3001 > GET / HTTP/1.1 > Host: localhost:3001 > User-Agent: curl/8.4.0 > Accept: */* > * Recv failure: Connection reset by peer * Closing connection ``` meanwhile Node is rebooting ``` 2024-02-03 13:12:20 2024-02-03 13:12:20 Node.js v18.19.0 ```
yindo added the bug label 2026-02-22 18:19:14 -05:00
yindo closed this issue 2026-02-22 18:19:14 -05:00
yindo changed title from [BUG]: exception on calling LLM causes whole process to restart and hurts other clients to [GH-ISSUE #670] [BUG]: exception on calling LLM causes whole process to restart and hurts other clients 2026-06-05 14:35:00 -04:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Desktop
Docker
Integration Request
Integration Request
OS: Linux
OS: Mobile
OS: Windows
UI/UX
blocked
bug
bug
core-team-only
documentation
duplicate
embed-widget
enhancement
feature request
github_actions
good first issue
investigating
needs info / can't replicate
possible bug
pull-request
Mirrored from GitHub Pull Request
 question
stage: specifications
wontfix
No Label bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Mintplex-Labs/anything-llm#382
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?