Mintplex-Labs/anything-llm
1
0
Fork 0
mirror of https://github.com/Mintplex-Labs/anything-llm.git synced 2026-07-19 22:23:50 -04:00
Code Issues 328 Packages Projects Releases 10 Wiki Activity

[GH-ISSUE #3656] Cannot connect to Ollama in Docker Compose setup - UI shows '--loading available models--' #2357

New Issue
Closed
opened 2026-02-22 18:29:19 -05:00 by yindo · 3 comments
yindo commented 2026-02-22 18:29:19 -05:00
Owner
Copy Link
Copy Source

Originally created by @witzker on GitHub (Apr 16, 2025).
Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/3656

Image

Hi AnythingLLM Team and Community,

I'm encountering an issue connecting AnythingLLM to Ollama when running both services via Docker Compose. After successfully resolving a previous LanceDB::Invalid ENV settings error by correcting volume mounts and recreating the AnythingLLM volume, I'm now facing a new problem: AnythingLLM seems unable to fetch the list of available models from Ollama.

Environment:

Host OS: [Your OS, e.g., Ubuntu 22.04]

Docker Setup: Docker Compose

AnythingLLM Image: mintplexlabs/anythingllm:latest

Ollama Image: ollama/ollama:latest

Docker Compose (docker-compose.yml):

version: '3.9' # Specifies the Docker Compose version

services:

--- Ollama Service ---

ollama:
image: ollama/ollama:latest
container_name: ollama
ports:
- "11434:11434" # Exposes Ollama API port
volumes:
- llm-stack_ollama_data:/root/.ollama # Persists Ollama models (Using correct prefixed name)
restart: unless-stopped

--- AnythingLLM Service ---

anythingllm:
image: mintplexlabs/anythingllm:latest
container_name: anythingllm
ports:
- "3001:3001" # Exposes AnythingLLM Web UI port
volumes:
# Mounts the named volume for ALL persistent data (DB, logs, config, uploaded documents)
- llm-stack_anythingllm_storage:/app/server/storage
environment:
# Points AnythingLLM to the Ollama service
- LLM_PROVIDER=ollama
- OLLAMA_BASE_URL=http://ollama:11434
# Add other ENV vars if needed later
# Runs the container process with the host user's permissions
user: "1000:1000" # Matches host user ID
depends_on:
- ollama # Ensures Ollama starts first
restart: unless-stopped

--- Top-Level Volume Definitions ---

volumes:
llm-stack_ollama_data: {} # Using correct prefixed name, {} indicates it's managed here
llm-stack_anythingllm_storage: # Using correct prefixed name
external: false # Explicitly state it's not externally created (Compose manages it)
Use code with caution.
Yaml
Problem Description:

After starting the stack with docker compose up -d, both containers (ollama and anythingllm) are running correctly (confirmed via docker ps).

I performed the initial setup for AnythingLLM (since the volume was recreated).

In Settings -> LLM Preference:

I selected "Ollama" as the LLM Provider.

I entered http://ollama:11434 as the Ollama Base URL (verified no typos, no trailing slash, using http).

I clicked "Save".

The dropdown menu for Ollama Model (both Chat and Embedding) continuously shows "--loading available models--" and never populates with the models available in Ollama. The text "Models will load after entering a valid Ollama URL." is displayed below.

Troubleshooting Steps Taken:

Confirmed both containers are running via docker ps.

Stopped and restarted the containers (docker compose down && docker compose up -d).

Verified the OLLAMA_BASE_URL (http://ollama:11434) is correctly set and saved in the AnythingLLM UI multiple times.

Confirmed the AnythingLLM volume (llm-stack_llm-stack_anythingllm_storage) was successfully removed and recreated (initial setup screen appeared as expected).

Checked Ollama logs (docker logs ollama --tail 50) - no obvious errors related to connection attempts were visible, but I can provide them if needed.

Checked AnythingLLM logs (docker logs anythingllm --tail 50) - no obvious errors indicating a connection failure, but the models just don't load. Can provide logs if needed.

Tested network connectivity from AnythingLLM to Ollama:

/llm-stack$ docker exec -it anythingllm /bin/sh -c "curl http://ollama:11434"
Ollama is running

adminaqu@ki:~/llm-stack$ docker exec ollama ollama list
NAME ID SIZE MODIFIED
no model!

Request:

Despite the containers running and the configuration appearing correct, AnythingLLM fails to communicate with Ollama to retrieve the model list. The result of the curl test seems crucial here.

Could there be another configuration issue, a networking problem within Docker Compose that I'm missing, or a potential bug? Any guidance on what to check next would be greatly appreciated.

Thanks!

Originally created by @witzker on GitHub (Apr 16, 2025). Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/3656 ![Image](https://github.com/user-attachments/assets/9788ce92-697a-4587-862e-e3a63a3a2879) Hi AnythingLLM Team and Community, I'm encountering an issue connecting AnythingLLM to Ollama when running both services via Docker Compose. After successfully resolving a previous LanceDB::Invalid ENV settings error by correcting volume mounts and recreating the AnythingLLM volume, I'm now facing a new problem: AnythingLLM seems unable to fetch the list of available models from Ollama. Environment: Host OS: [Your OS, e.g., Ubuntu 22.04] Docker Setup: Docker Compose AnythingLLM Image: mintplexlabs/anythingllm:latest Ollama Image: ollama/ollama:latest Docker Compose (docker-compose.yml): version: '3.9' # Specifies the Docker Compose version services: # --- Ollama Service --- ollama: image: ollama/ollama:latest container_name: ollama ports: - "11434:11434" # Exposes Ollama API port volumes: - llm-stack_ollama_data:/root/.ollama # Persists Ollama models (Using correct prefixed name) restart: unless-stopped # --- AnythingLLM Service --- anythingllm: image: mintplexlabs/anythingllm:latest container_name: anythingllm ports: - "3001:3001" # Exposes AnythingLLM Web UI port volumes: # Mounts the named volume for ALL persistent data (DB, logs, config, uploaded documents) - llm-stack_anythingllm_storage:/app/server/storage environment: # Points AnythingLLM to the Ollama service - LLM_PROVIDER=ollama - OLLAMA_BASE_URL=http://ollama:11434 # Add other ENV vars if needed later # Runs the container process with the host user's permissions user: "1000:1000" # Matches host user ID depends_on: - ollama # Ensures Ollama starts first restart: unless-stopped # --- Top-Level Volume Definitions --- volumes: llm-stack_ollama_data: {} # Using correct prefixed name, {} indicates it's managed here llm-stack_anythingllm_storage: # Using correct prefixed name external: false # Explicitly state it's not externally created (Compose manages it) Use code with caution. Yaml Problem Description: After starting the stack with docker compose up -d, both containers (ollama and anythingllm) are running correctly (confirmed via docker ps). I performed the initial setup for AnythingLLM (since the volume was recreated). In Settings -> LLM Preference: I selected "Ollama" as the LLM Provider. I entered http://ollama:11434 as the Ollama Base URL (verified no typos, no trailing slash, using http). I clicked "Save". The dropdown menu for Ollama Model (both Chat and Embedding) continuously shows "--loading available models--" and never populates with the models available in Ollama. The text "Models will load after entering a valid Ollama URL." is displayed below. Troubleshooting Steps Taken: Confirmed both containers are running via docker ps. Stopped and restarted the containers (docker compose down && docker compose up -d). Verified the OLLAMA_BASE_URL (http://ollama:11434) is correctly set and saved in the AnythingLLM UI multiple times. Confirmed the AnythingLLM volume (llm-stack_llm-stack_anythingllm_storage) was successfully removed and recreated (initial setup screen appeared as expected). Checked Ollama logs (docker logs ollama --tail 50) - no obvious errors related to connection attempts were visible, but I can provide them if needed. Checked AnythingLLM logs (docker logs anythingllm --tail 50) - no obvious errors indicating a connection failure, but the models just don't load. Can provide logs if needed. Tested network connectivity from AnythingLLM to Ollama: /llm-stack$ docker exec -it anythingllm /bin/sh -c "curl http://ollama:11434" Ollama is running adminaqu@ki:~/llm-stack$ docker exec ollama ollama list NAME ID SIZE MODIFIED no model! Request: Despite the containers running and the configuration appearing correct, AnythingLLM fails to communicate with Ollama to retrieve the model list. The result of the curl test seems crucial here. Could there be another configuration issue, a networking problem within Docker Compose that I'm missing, or a potential bug? Any guidance on what to check next would be greatly appreciated. Thanks!
yindo closed this issue 2026-02-22 18:29:19 -05:00
yindo commented 2026-02-22 18:29:20 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Apr 16, 2025):

https://docs.anythingllm.com/ollama-connection-troubleshooting

TLDR; you are using the wrong URl. your http://ollama hosts config does not work in the AnythingLLM container. That is a host level mapping. You want to use http://172.17.0.1:PORT to reach out to the host and then find a locally running service on that port. In this case, 11434.

This whole document exists explictly for people who run into this: https://docs.anythingllm.com/ollama-connection-troubleshooting#troubleshooting-docker

@timothycarambat commented on GitHub (Apr 16, 2025): https://docs.anythingllm.com/ollama-connection-troubleshooting TLDR; you are using the wrong URl. your `http://ollama` hosts config does not work in the AnythingLLM container. That is a host level mapping. You want to use `http://172.17.0.1:PORT` to reach out to the host and then find a locally running service on that port. In this case, `11434`. This whole document exists explictly for people who run into this: https://docs.anythingllm.com/ollama-connection-troubleshooting#troubleshooting-docker
yindo commented 2026-02-22 18:29:20 -05:00
Author
Owner
Copy Link
Copy Source

@witzker commented on GitHub (Apr 16, 2025):

THX But I Think you got It WRONG!?

explanation from the docs (http://172.17.0.1:11434/):

This IP address (172.17.0.1) is usually the default IP of the Docker host as seen from within a container in the default bridge network.

The recommendation to use this IP mainly applies in scenarios where:

Ollama is running directly on the host operating system (not in a Docker container).

Ollama is running in a different Docker network than AnythingLLM, and one tries to access the host from inside a container to reach Ollama.

Why this (probably) does NOT apply to MY setup and why http://ollama:11434/ is correct:

I’m using docker-compose.yml. By default, Docker Compose creates a dedicated network for all services defined in this file (in my case, probably llm-stack_default).

Within this network, containers can reach each other via their service names. The anythingllm service can reach the ollama service using the hostname ollama. Docker Compose handles the name resolution (DNS) inside this network.

My successful curl http://ollama:11434 test from within the anythingllm container proves that this name resolution and basic network connection between the containers using the service name is working!

Why did http://172.17.0.1:11434/ not work in my case?

Because the Ollama container is not listening on IP 172.17.0.1 (which is the host IP from the container’s perspective), but rather on its own IP within the llm-stack_default network. The service name ollama is correctly resolved by Docker to that internal IP.

My conclusion:

The documentation is describing a different scenario. For my configuration using Docker Compose, http://ollama:11434 is the correct approach.

Since the basic curl test with http://ollama:11434 worked, the issue is not due to connectivity or a wrong URL.

The problem must lie elsewhere:

Either something is blocking the specific request to /api/tags (the model list).

Or there's a bug in the current version of AnythingLLM that prevents the models from loading/displaying even though they might technically be accessible.

Or Ollama is not returning a valid response to /api/tags for some reason.

What do you think?
BUG?

@witzker commented on GitHub (Apr 16, 2025): THX But I Think you got It WRONG!? explanation from the docs (http://172.17.0.1:11434/): This IP address (172.17.0.1) is usually the default IP of the Docker host as seen from within a container in the default bridge network. The recommendation to use this IP mainly applies in scenarios where: Ollama is running directly on the host operating system (not in a Docker container). Ollama is running in a different Docker network than AnythingLLM, and one tries to access the host from inside a container to reach Ollama. Why this (probably) does NOT apply to MY setup and why http://ollama:11434/ is correct: I’m using docker-compose.yml. By default, Docker Compose creates a dedicated network for all services defined in this file (in my case, probably llm-stack_default). Within this network, containers can reach each other via their service names. The anythingllm service can reach the ollama service using the hostname ollama. Docker Compose handles the name resolution (DNS) inside this network. My successful curl http://ollama:11434 test from within the anythingllm container proves that this name resolution and basic network connection between the containers using the service name is working! Why did http://172.17.0.1:11434/ not work in my case? Because the Ollama container is not listening on IP 172.17.0.1 (which is the host IP from the container’s perspective), but rather on its own IP within the llm-stack_default network. The service name ollama is correctly resolved by Docker to that internal IP. My conclusion: The documentation is describing a different scenario. For my configuration using Docker Compose, http://ollama:11434 is the correct approach. Since the basic curl test with http://ollama:11434 worked, the issue is not due to connectivity or a wrong URL. The problem must lie elsewhere: Either something is blocking the specific request to /api/tags (the model list). Or there's a bug in the current version of AnythingLLM that prevents the models from loading/displaying even though they might technically be accessible. Or Ollama is not returning a valid response to /api/tags for some reason. What do you think? BUG?
yindo commented 2026-02-22 18:29:20 -05:00
Author
Owner
Copy Link
Copy Source

@rozek commented on GitHub (May 20, 2025):

did you solve your issue? I have the same problem with AnythinLLM Desktop: curl http://127.0.0.1:11434 says Ollama is alive, but AnythingLLM cannot even load the list of models

@rozek commented on GitHub (May 20, 2025): did you solve your issue? I have the same problem with AnythinLLM Desktop: `curl http://127.0.0.1:11434` says Ollama is alive, but AnythingLLM cannot even load the list of models
yindo changed title from Cannot connect to Ollama in Docker Compose setup - UI shows '--loading available models--' to [GH-ISSUE #3656] Cannot connect to Ollama in Docker Compose setup - UI shows '--loading available models--' 2026-06-05 14:46:06 -04:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Desktop
Docker
Integration Request
Integration Request
OS: Linux
OS: Mobile
OS: Windows
UI/UX
blocked
bug
bug
core-team-only
documentation
duplicate
embed-widget
enhancement
feature request
github_actions
good first issue
investigating
needs info / can't replicate
possible bug
pull-request
Mirrored from GitHub Pull Request
 question
stage: specifications
wontfix
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Mintplex-Labs/anything-llm#2357
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?