Mintplex-Labs/anything-llm
1
0
Fork 0
mirror of https://github.com/Mintplex-Labs/anything-llm.git synced 2026-07-19 14:13:35 -04:00
Code Issues 328 Packages Projects Releases 10 Wiki Activity

[GH-ISSUE #4348] [BUG]: AnythingLLM cannot reach Ollama from Docker despite following instructions #2769

New Issue
Closed
opened 2026-02-22 18:31:10 -05:00 by yindo · 3 comments
yindo commented 2026-02-22 18:31:10 -05:00
Owner
Copy Link
Copy Source

Originally created by @RnoldR on GitHub (Sep 1, 2025).
Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/4348

How are you running AnythingLLM?

Docker (local)

What happened?

I use AnythingLLM on ubuntu 24.04 using docker 28.3.3. When I try to chat I get the following error message:

Your Ollama instance could not be reached or is not responding. Please make sure it is running the API server and your connection information is correct in AnythingLLM.

The documentation warns for some pitfalls and as far as I can see I have avoided these.

Are there known steps to reproduce?

Ollama

  • Install Ollama locally as a service: curl -fsSL https://ollama.com/install.sh | sh
  • start it: ollama pull stablelm2:12b
  • check its status: sudo systemctl status ollama
  • verify you can chat with it (I could)
  • check its URL: http://172.17.0.1:11434, got the message: Ollama is running

AnytingLLM

  • Download and install AnythingLLM on docker. I used the following instructions:

export STORAGE_LOCATION=$HOME/anythingllm && \
mkdir -p $STORAGE_LOCATION && \
touch "$STORAGE_LOCATION/.env" && \
docker run -d -p 3001:3001 \
--name any-llm \
--cap-add SYS_ADMIN \
-v ${STORAGE_LOCATION}:/app/server/storage \
-v ${STORAGE_LOCATION}/.env:/app/server/.env \
-e STORAGE_DIR="/app/server/storage" \
--restart="always" \
mintplexlabs/anythingllm

  • use docker ps -a to check it is running healthy
  • if so connect to it: localhost:3001
  • go thru the intro section
  • create a new workspace, choose LLM provider: Ollama
  • Ollama URL: http://172.17.0.1:11434
  • leave other settings as they are
  • the model stablelm2 is not acessible in the models list
  • Choose workspace in main screen, try to chat. I got the above mentioned message.

I tried to use open-webui and that worked using the same ollama service. I stopped the open-webui container after that because I am not sure what happens when to services putting requests to Ollama.

What am I doing wrong and how can I get it to work?

Originally created by @RnoldR on GitHub (Sep 1, 2025). Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/4348 ### How are you running AnythingLLM? Docker (local) ### What happened? I use AnythingLLM on ubuntu 24.04 using docker 28.3.3. When I try to chat I get the following error message: Your Ollama instance could not be reached or is not responding. Please make sure it is running the API server and your connection information is correct in AnythingLLM. The documentation warns for some pitfalls and as far as I can see I have avoided these. ### Are there known steps to reproduce? Ollama - Install Ollama locally as a service: curl -fsSL https://ollama.com/install.sh | sh - start it: ollama pull stablelm2:12b - check its status: sudo systemctl status ollama - verify you can chat with it (I could) - check its URL: http://172.17.0.1:11434, got the message: Ollama is running AnytingLLM - Download and install AnythingLLM on docker. I used the following instructions: export STORAGE_LOCATION=$HOME/anythingllm && \\ mkdir -p $STORAGE_LOCATION && \\ touch "$STORAGE_LOCATION/.env" && \\ docker run -d -p 3001:3001 \\ --name any-llm \\ --cap-add SYS_ADMIN \\ -v ${STORAGE_LOCATION}:/app/server/storage \\ -v ${STORAGE_LOCATION}/.env:/app/server/.env \\ -e STORAGE_DIR="/app/server/storage" \\ --restart="always" \\ mintplexlabs/anythingllm - use docker ps -a to check it is running healthy - if so connect to it: localhost:3001 - go thru the intro section - create a new workspace, choose LLM provider: Ollama - Ollama URL: http://172.17.0.1:11434 - leave other settings as they are - the model stablelm2 is not acessible in the models list - Choose workspace in main screen, try to chat. I got the above mentioned message. I tried to use open-webui and that worked using the same ollama service. I stopped the open-webui container after that because I am not sure what happens when to services putting requests to Ollama. What am I doing wrong and how can I get it to work?
yindo added the needs info / can't replicate label 2026-02-22 18:31:10 -05:00
yindo closed this issue 2026-02-22 18:31:10 -05:00
yindo commented 2026-02-22 18:31:10 -05:00
Author
Owner
Copy Link
Copy Source

@dreambleumer commented on GitHub (Sep 2, 2025):

I'm also encountering this issue—it shows the prompt: "Ollama service could not be reached. Is Ollama running?" My AnythingLLM version is 1.8.5. However, the 1.8.4 version on my other server doesn’t have this problem.

@dreambleumer commented on GitHub (Sep 2, 2025): I'm also encountering this issue—it shows the prompt: "Ollama service could not be reached. Is Ollama running?" My AnythingLLM version is 1.8.5. However, the 1.8.4 version on my other server doesn’t have this problem.
yindo commented 2026-02-22 18:31:10 -05:00
Author
Owner
Copy Link
Copy Source

@shatfield4 commented on GitHub (Sep 2, 2025):

@RnoldR Are you able to try pulling in version 1.8.4 and confirming that Ollama is working for you in the previous version? You can do so by running this command to pull it in:

docker pull mintplexlabs/anythingllm:1.8.4

Then after it's pulled in run this to start 1.8.4:

export STORAGE_LOCATION=$HOME/anythingllm && \
mkdir -p $STORAGE_LOCATION && \
touch "$STORAGE_LOCATION/.env" && \
docker run -d -p 3001:3001 \
--name any-llm \
--cap-add SYS_ADMIN \
-v ${STORAGE_LOCATION}:/app/server/storage \
-v ${STORAGE_LOCATION}/.env:/app/server/.env \
-e STORAGE_DIR="/app/server/storage" \
--restart="always" \
mintplexlabs/anythingllm:1.8.4
@shatfield4 commented on GitHub (Sep 2, 2025): @RnoldR Are you able to try pulling in version 1.8.4 and confirming that Ollama is working for you in the previous version? You can do so by running this command to pull it in: `docker pull mintplexlabs/anythingllm:1.8.4` Then after it's pulled in run this to start 1.8.4: ``` export STORAGE_LOCATION=$HOME/anythingllm && \ mkdir -p $STORAGE_LOCATION && \ touch "$STORAGE_LOCATION/.env" && \ docker run -d -p 3001:3001 \ --name any-llm \ --cap-add SYS_ADMIN \ -v ${STORAGE_LOCATION}:/app/server/storage \ -v ${STORAGE_LOCATION}/.env:/app/server/.env \ -e STORAGE_DIR="/app/server/storage" \ --restart="always" \ mintplexlabs/anythingllm:1.8.4 ```
yindo commented 2026-02-22 18:31:10 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Sep 3, 2025):

I use AnythingLLM on ubuntu 24.04 using docker 28.3.3.
Ollama URL: http://172.17.0.1:11434 (on host - non containerized)

https://docs.anythingllm.com/ollama-connection-troubleshooting#localhost-and-127001-do-not-work-on-docker

172.17.0.1 is a special IP for Docker on Ubuntu/Linux specifically. Can you start ollama bound to 127.0.0.1 - then using http://172.17.0.1:11434 in the AnythingLLM container should work. This is an IP resolution conflict due to how Docker does networking between container<>host most likely.

@timothycarambat commented on GitHub (Sep 3, 2025): > I use AnythingLLM on ubuntu 24.04 using docker 28.3.3. > Ollama URL: http://172.17.0.1:11434 (on host - non containerized) https://docs.anythingllm.com/ollama-connection-troubleshooting#localhost-and-127001-do-not-work-on-docker 172.17.0.1 is a special IP for Docker on Ubuntu/Linux specifically. Can you start ollama bound to 127.0.0.1 - then using `http://172.17.0.1:11434` in the AnythingLLM container should work. This is an IP resolution conflict due to how Docker does networking between container<>host most likely.
yindo changed title from [BUG]: AnythingLLM cannot reach Ollama from Docker despite following instructions to [GH-ISSUE #4348] [BUG]: AnythingLLM cannot reach Ollama from Docker despite following instructions 2026-06-05 14:48:27 -04:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Desktop
Docker
Integration Request
Integration Request
OS: Linux
OS: Mobile
OS: Windows
UI/UX
blocked
bug
bug
core-team-only
documentation
duplicate
embed-widget
enhancement
feature request
github_actions
good first issue
investigating
needs info / can't replicate
possible bug
pull-request
Mirrored from GitHub Pull Request
 question
stage: specifications
wontfix
No Label needs info / can't replicate
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Mintplex-Labs/anything-llm#2769
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?