Mintplex-Labs/anything-llm
1
0
Fork 0
mirror of https://github.com/Mintplex-Labs/anything-llm.git synced 2026-07-19 14:13:35 -04:00
Code Issues 328 Packages Projects Releases 10 Wiki Activity

[GH-ISSUE #2431] [BUG]: An error occurred while streaming response. NetworkError when attempting to fetch resource. #1581

New Issue
Closed
opened 2026-02-22 18:25:30 -05:00 by yindo · 8 comments
yindo commented 2026-02-22 18:25:30 -05:00
Owner
Copy Link
Copy Source

Originally created by @joestump on GitHub (Oct 5, 2024).
Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/2431

How are you running AnythingLLM?

Docker (local)

What happened?

I have Anything LLM running pointed at my local Ollama. My local Ollama is working fine in Open WebUI and when I curl it:

$ curl https://ollama 
Ollama is running%

Also, it would appear Anything LLM is communicating in some form with Ollama as it shows my models:

Screenshot 2024-10-05 at 7 58 55 PM

There are no error in the logs either:

No pending migrations to apply.
[backend] info: [EncryptionManager] Loaded existing key & salt for encrypting arbitrary data.
[backend] info: [TELEMETRY ENABLED] Anonymous Telemetry enabled. Telemetry helps Mintplex Labs Inc improve AnythingLLM.
[backend] info: prisma:info Starting a sqlite pool with 17 connections.
[backend] info: [TELEMETRY SENT] {"event":"server_boot","distinctId":"50247cfa-0b04-4800-89b5-af27967efe6e","properties":{"commit":"--","runtime":"docker"}}
[backend] info: [CommunicationKey] RSA key pair generated for signed payloads within AnythingLLM services.
[backend] info: [EncryptionManager] Loaded existing key & salt for encrypting arbitrary data.
[backend] info: Primary server in HTTP mode listening on port 3001
[backend] info: [BackgroundWorkerService] Feature is not enabled and will not be started.

But when I go to use the frontend I get the following error:

Screenshot 2024-10-05 at 8 01 24 PM

When I look in the console I see the following error:

ns_error_net_interrupt
Screenshot 2024-10-05 at 8 03 24 PM

I'm wondering if this is because I'm serving AnythingLLM behind a Caddy proxy? I looked in the example env file and didn't see anything related to proxy settings.

I can curl Ollama from the AnythingLLM container as well.

Are there known steps to reproduce?

No response

Originally created by @joestump on GitHub (Oct 5, 2024). Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/2431 ### How are you running AnythingLLM? Docker (local) ### What happened? I have Anything LLM running pointed at my local Ollama. My local Ollama is working fine in Open WebUI and when I `curl` it: ``` $ curl https://ollama Ollama is running% ``` Also, it would appear Anything LLM is communicating in some form with Ollama as it shows my models: <img width="800" alt="Screenshot 2024-10-05 at 7 58 55 PM" src="https://github.com/user-attachments/assets/1f0fa1b7-ff03-4b30-9eea-7a389f1566a5"> There are no error in the logs either: ``` No pending migrations to apply. [backend] info: [EncryptionManager] Loaded existing key & salt for encrypting arbitrary data. [backend] info: [TELEMETRY ENABLED] Anonymous Telemetry enabled. Telemetry helps Mintplex Labs Inc improve AnythingLLM. [backend] info: prisma:info Starting a sqlite pool with 17 connections. [backend] info: [TELEMETRY SENT] {"event":"server_boot","distinctId":"50247cfa-0b04-4800-89b5-af27967efe6e","properties":{"commit":"--","runtime":"docker"}} [backend] info: [CommunicationKey] RSA key pair generated for signed payloads within AnythingLLM services. [backend] info: [EncryptionManager] Loaded existing key & salt for encrypting arbitrary data. [backend] info: Primary server in HTTP mode listening on port 3001 [backend] info: [BackgroundWorkerService] Feature is not enabled and will not be started. ``` But when I go to use the frontend I get the following error: <img width="799" alt="Screenshot 2024-10-05 at 8 01 24 PM" src="https://github.com/user-attachments/assets/b243f303-4dcc-419a-83f0-12b9c6053780"> When I look in the console I see the following error: ``` ns_error_net_interrupt ``` <img width="933" alt="Screenshot 2024-10-05 at 8 03 24 PM" src="https://github.com/user-attachments/assets/83f60c2e-b7de-4f61-a90b-421e02217de0"> I'm wondering if this is because I'm serving AnythingLLM behind a Caddy proxy? I looked in the example env file and didn't see anything related to proxy settings. I can `curl` Ollama from the AnythingLLM container as well. ### Are there known steps to reproduce? _No response_
yindo added the needs info / can't replicate label 2026-02-22 18:25:30 -05:00
yindo closed this issue 2026-02-22 18:25:30 -05:00
yindo commented 2026-02-22 18:25:31 -05:00
Author
Owner
Copy Link
Copy Source

@joestump commented on GitHub (Oct 5, 2024):

Pretty sure this has to do with Caddy as the proxy. The desktop app works fine with my Ollama.

@joestump commented on GitHub (Oct 5, 2024): Pretty sure this has to do with Caddy as the proxy. The desktop app works fine with my Ollama.
yindo commented 2026-02-22 18:25:31 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Oct 7, 2024):

I would agree, if you are in docker you must use the host gateway binding:
https://host.docker.internal:PORT on Mac/Windows or http://172.17.0.1:11434

What is http://ollama:11434 mapping to? interesting that the /model seems to work but chats are not?

@timothycarambat commented on GitHub (Oct 7, 2024): I would agree, if you are in docker you _must use_ the host gateway binding: `https://host.docker.internal:PORT` on Mac/Windows or `http://172.17.0.1:11434` What is `http://ollama:11434` mapping to? interesting that the `/model` seems to work but chats are not?
yindo commented 2026-02-22 18:25:31 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Oct 7, 2024):

Also, I realize now that this is about streaming. Does your proxy support web sockets? That is how chat works and 99% of the time the proxy is blocking this and drops connection instantly.

In nginx:

# In server block
.....
# Enable websocket connections for agent protocol.
	location ~* ^/api/agent-invocation/(.*) {
		proxy_pass http://localhost:3001;
		proxy_http_version 1.1;
		proxy_set_header Upgrade $http_upgrade;
		proxy_set_header Connection "Upgrade";
	}

# Regular streaming/HTTP support	
location / {
		proxy_connect_timeout       605;
    proxy_send_timeout          605;
    proxy_read_timeout          605;
    send_timeout                605;
    keepalive_timeout           605;
    proxy_buffering off;
    proxy_cache off;
    proxy_pass         http://ip.address.goes.here:3001$request_uri;
  }
....
@timothycarambat commented on GitHub (Oct 7, 2024): Also, I realize now that this is about streaming. Does your proxy support web sockets? That is how chat works and 99% of the time the proxy is blocking this and drops connection instantly. In nginx: ``` # In server block ..... # Enable websocket connections for agent protocol. location ~* ^/api/agent-invocation/(.*) { proxy_pass http://localhost:3001; proxy_http_version 1.1; proxy_set_header Upgrade $http_upgrade; proxy_set_header Connection "Upgrade"; } # Regular streaming/HTTP support location / { proxy_connect_timeout 605; proxy_send_timeout 605; proxy_read_timeout 605; send_timeout 605; keepalive_timeout 605; proxy_buffering off; proxy_cache off; proxy_pass http://ip.address.goes.here:3001$request_uri; } .... ```
yindo commented 2026-02-22 18:25:31 -05:00
Author
Owner
Copy Link
Copy Source

@joestump commented on GitHub (Oct 8, 2024):

@timothycarambat I'm using Caddy. Specifically, caddy-docker-proxy. I've had no issues using Ollama behind Caddy with other tooling: Open WebUI, Enchanted on iOS, various Paperless-ngx POCs, etc. all work fine. The Anything LLM Mac client works fine with my Ollama behind Caddy as well, so this feels isolated to the Docker version.

http://ollama is just hitting the internal Docker container for Ollama over the Caddy Docker network. I also tried https://ollama.mydomain.com which works with the other Ollama clients I mentioned above, but it also had the same error.

Would there be any issues with AnythingLLM behind Caddy? Wondering if you meant Ollama or AnythingLLM must use the host port bindings.

I should mention my working Open WebUI is also behind Caddy using the same setup.

@joestump commented on GitHub (Oct 8, 2024): @timothycarambat I'm using Caddy. Specifically, [`caddy-docker-proxy`](https://github.com/lucaslorentz/caddy-docker-proxy). I've had no issues using Ollama behind Caddy with other tooling: Open WebUI, Enchanted on iOS, various Paperless-ngx POCs, etc. all work fine. The Anything LLM Mac client works fine with my Ollama behind Caddy as well, so this feels isolated to the Docker version. `http://ollama` is just hitting the internal Docker container for Ollama over the Caddy Docker network. I also tried `https://ollama.mydomain.com` which works with the other Ollama clients I mentioned above, but it also had the same error. Would there be any issues with AnythingLLM behind Caddy? Wondering if you meant Ollama or AnythingLLM must use the host port bindings. I should mention my working Open WebUI is also behind Caddy using the same setup.
yindo commented 2026-02-22 18:25:31 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Oct 9, 2024):

In general if you have a setup of

  • Docker container on host running on port 11434
  • AnythingLLM in another container on host 3001

To connect AnythingLLM to the service on 11434 you would need to use this URL in AnythingLLM
http://host.docker.internal:11434 this would use the network bridge to then go from
AnythingLLM<->host network<->container2

I have never used caddy and do not exactly know or understand what that entails. However, the /models endpoint works and the only distinctive difference between calling /models and /stream-chat is that stream chat uses SSR. Which might explain the NS_ERROR_NET_INTERRUPT.

So clearly the https://ollama endpoint is reachable over traditional REST/HTTP methods, but the client disconnects when the request is SSR
https://github.com/Mintplex-Labs/anything-llm/blob/b658f5012de0658272fb0853a1c00a1fda2ccf1f/server/endpoints/chat.js#L45-L49

Does caddy have issues with text/event-stream header flushing instantly? Might be just the right combo of the docker +SRR+caddy all together
https://github.com/caddyserver/caddy/issues/3765

@timothycarambat commented on GitHub (Oct 9, 2024): In general if you have a setup of - Docker container on host running on port 11434 - AnythingLLM in another container on host 3001 To connect AnythingLLM to the service on 11434 you would need to use this URL in AnythingLLM `http://host.docker.internal:11434` this would use the network bridge to then go from `AnythingLLM<->host network<->container2` I have never used caddy and do not exactly know or understand what that entails. However, the `/models` endpoint works and the only distinctive difference between calling `/models` and `/stream-chat` is that stream chat uses `SSR`. Which might explain the `NS_ERROR_NET_INTERRUPT`. So clearly the `https://ollama` endpoint is reachable over traditional REST/HTTP methods, but the client disconnects when the request is SSR https://github.com/Mintplex-Labs/anything-llm/blob/b658f5012de0658272fb0853a1c00a1fda2ccf1f/server/endpoints/chat.js#L45-L49 Does caddy have issues with `text/event-stream` header flushing instantly? Might be just the right combo of the docker +SRR+caddy all together https://github.com/caddyserver/caddy/issues/3765
yindo commented 2026-02-22 18:25:31 -05:00
Author
Owner
Copy Link
Copy Source

@joestump commented on GitHub (Oct 10, 2024):

@timothycarambat I tried flush_interval -1 without any success. I don't do host ports like you describe for any of my services. Have you gotten this to work behind any reverse proxy? If so, mind sharing the config? Worth noting the Caddy documentation says the flush_interval option is ignored and headers are flushed immediately when Content-Type: text/event-stream.

I have found another lead though: https://github.com/leafac/caddy-express-sse which suggests this as a workaround:

res.type("text/event-stream").write(":\n\n");

See the Caddy bug filed here: https://github.com/leafac/caddy-express-sse?tab=readme-ov-file

There's also this open issue here about compression: https://github.com/caddyserver/caddy/issues/6293

I also found this issue that looks similarly related: https://github.com/Giveth/giveth-dapps-v2/issues/4620

@joestump commented on GitHub (Oct 10, 2024): @timothycarambat I tried `flush_interval -1` without any success. I don't do host ports like you describe for any of my services. Have you gotten this to work behind _any_ reverse proxy? If so, mind sharing the config? Worth noting the [Caddy documentation](https://caddyserver.com/docs/caddyfile/directives/reverse_proxy#flush_interval) says the `flush_interval` option is ignored and headers are flushed immediately when `Content-Type: text/event-stream`. I have found another lead though: https://github.com/leafac/caddy-express-sse which suggests this as a workaround: ```javascript res.type("text/event-stream").write(":\n\n"); ``` See the Caddy bug filed here: https://github.com/leafac/caddy-express-sse?tab=readme-ov-file There's also this open issue here about compression: https://github.com/caddyserver/caddy/issues/6293 I also found this issue that looks similarly related: https://github.com/Giveth/giveth-dapps-v2/issues/4620
yindo commented 2026-02-22 18:25:31 -05:00
Author
Owner
Copy Link
Copy Source

@joestump commented on GitHub (Oct 10, 2024):

Here's where Caddy automatically sets flush_interval to -1 for text/event-stream: https://github.com/cmfcmf/caddy/blob/master/modules/caddyhttp/reverseproxy/streaming.go#L88-L107

@joestump commented on GitHub (Oct 10, 2024): Here's where Caddy automatically sets `flush_interval` to `-1` for `text/event-stream`: https://github.com/cmfcmf/caddy/blob/master/modules/caddyhttp/reverseproxy/streaming.go#L88-L107
yindo commented 2026-02-22 18:25:31 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Oct 11, 2024):

I have gotten AnythingLLM to run fine behind a reverse proxy, but for Nginx

# Default server configuration
# Example config for regular setup + SSL + Websockets.
server {
	listen 80;
	server_name domain-here.com;
	return 301 https://domain-here.com$request_uri;
}

server {
	listen 443 ssl;
	ssl on;
	server_name domain-here.com;

  # Enable websocket connections for agent protocol.
	location ~* ^/api/agent-invocation/(.*) {
		proxy_pass http://localhost:3001;
		proxy_http_version 1.1;
		proxy_set_header Upgrade $http_upgrade;
		proxy_set_header Connection "Upgrade";
	}


	location / {
		proxy_connect_timeout       605;
    proxy_send_timeout          605;
    proxy_read_timeout          605;
    send_timeout                605;
    keepalive_timeout           605;
    proxy_buffering off;
    proxy_cache off;
    proxy_pass         http://public-ip-address-here:3001$request_uri;
  }
}

This has NGINX in front of the service running on 3001 on an Ubuntu machine. Then has http://domain-here.com on 80 proxy to 3001 (can enable SSL if you need to also with the ssl_cert directives

@timothycarambat commented on GitHub (Oct 11, 2024): I have gotten AnythingLLM to run fine behind a reverse proxy, but for Nginx ``` # Default server configuration # Example config for regular setup + SSL + Websockets. server { listen 80; server_name domain-here.com; return 301 https://domain-here.com$request_uri; } server { listen 443 ssl; ssl on; server_name domain-here.com; # Enable websocket connections for agent protocol. location ~* ^/api/agent-invocation/(.*) { proxy_pass http://localhost:3001; proxy_http_version 1.1; proxy_set_header Upgrade $http_upgrade; proxy_set_header Connection "Upgrade"; } location / { proxy_connect_timeout 605; proxy_send_timeout 605; proxy_read_timeout 605; send_timeout 605; keepalive_timeout 605; proxy_buffering off; proxy_cache off; proxy_pass http://public-ip-address-here:3001$request_uri; } } ``` This has NGINX in front of the service running on 3001 on an Ubuntu machine. Then has `http://domain-here.com` on 80 proxy to 3001 (can enable SSL if you need to also with the `ssl_cert` directives
yindo changed title from [BUG]: An error occurred while streaming response. NetworkError when attempting to fetch resource. to [GH-ISSUE #2431] [BUG]: An error occurred while streaming response. NetworkError when attempting to fetch resource. 2026-06-05 14:41:33 -04:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Desktop
Docker
Integration Request
Integration Request
OS: Linux
OS: Mobile
OS: Windows
UI/UX
blocked
bug
bug
core-team-only
documentation
duplicate
embed-widget
enhancement
feature request
github_actions
good first issue
investigating
needs info / can't replicate
possible bug
pull-request
Mirrored from GitHub Pull Request
 question
stage: specifications
wontfix
No Label needs info / can't replicate
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Mintplex-Labs/anything-llm#1581
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?