Mintplex-Labs/anything-llm
1
0
Fork 0
mirror of https://github.com/Mintplex-Labs/anything-llm.git synced 2026-07-25 04:25:45 -04:00
Code Issues 328 Packages Projects Releases 10 Wiki Activity

[GH-ISSUE #1215] [BUG]: Unable to call model through Ollama #756

New Issue
Closed
opened 2026-02-22 18:21:11 -05:00 by yindo · 10 comments
yindo commented 2026-02-22 18:21:11 -05:00
Owner
Copy Link
Copy Source

Originally created by @hcl27 on GitHub (Apr 28, 2024).
Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/1215

How are you running AnythingLLM?

Docker (local)

What happened?

Local deployment of ollama + anythinglm (Docker)

bug1
I tried the following settings, but it still shows:
127.0.0.1:11434
172.17.0.1:11434
localhost:11434
host.docker.interal:11434

ollama serve ollama normal operation

Who can help me solve this problem? Thank you very much

Are there known steps to reproduce?

No response

Originally created by @hcl27 on GitHub (Apr 28, 2024). Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/1215 ### How are you running AnythingLLM? Docker (local) ### What happened? ### Local deployment of ollama + anythinglm (Docker) ![bug1](https://github.com/Mintplex-Labs/anything-llm/assets/168281399/ee0cc0f8-17c8-406c-974a-0b67f23364d8) I tried the following settings, but it still shows: 127.0.0.1:11434 172.17.0.1:11434 localhost:11434 host.docker.interal:11434 ollama serve ollama normal operation Who can help me solve this problem? Thank you very much ### Are there known steps to reproduce? _No response_
yindo added the needs info / can't replicate label 2026-02-22 18:21:11 -05:00
yindo closed this issue 2026-02-22 18:21:12 -05:00
yindo commented 2026-02-22 18:21:12 -05:00
Author
Owner
Copy Link
Copy Source

@vdamov commented on GitHub (Apr 28, 2024):

Do you have docker instance of Ollama server with the default port running on your machine ?

@vdamov commented on GitHub (Apr 28, 2024): Do you have docker instance of Ollama server with the default port running on your machine ?
yindo commented 2026-02-22 18:21:12 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Apr 28, 2024):

I am assuming you have ollama serve running on the host? Also which operating system as host.docker.internal does not work on Linux Docker, if that is the case. Otherwise, you should be fine.

The only other caveat is Docker on Windows trying to communicate with Ollama running on WSL. WSL does not allow the connection to bridge.

@timothycarambat commented on GitHub (Apr 28, 2024): I am assuming you have `ollama serve` running on the host? Also which operating system as `host.docker.internal` does not work on Linux Docker, if that is the case. Otherwise, you should be fine. The only other caveat is Docker on Windows trying to communicate with Ollama running on WSL. WSL does not allow the connection to bridge.
yindo commented 2026-02-22 18:21:12 -05:00
Author
Owner
Copy Link
Copy Source

@hcl27 commented on GitHub (Apr 28, 2024):

Do you have docker instance of Ollama server with the default port running on your machine ?

Sorry, I may not have expressed it accurately.
Ollama runs locally on the server, not on Docker,
Anything llm runs on Docker
Curl http://127.0.0.1:11434 Display Ollama is running

@hcl27 commented on GitHub (Apr 28, 2024): > Do you have docker instance of Ollama server with the default port running on your machine ? Sorry, I may not have expressed it accurately. Ollama runs locally on the server, not on Docker, Anything llm runs on Docker Curl http://127.0.0.1:11434 Display Ollama is running
yindo commented 2026-02-22 18:21:12 -05:00
Author
Owner
Copy Link
Copy Source

@hcl27 commented on GitHub (Apr 28, 2024):

I am assuming you have ollama serve running on the host? Also which operating system as host.docker.internal does not work on Linux Docker, if that is the case. Otherwise, you should be fine.

The only other caveat is Docker on Windows trying to communicate with Ollama running on WSL. WSL does not allow the connection to bridge.

yes，Ollama runs on host, and the local system is Ubuntu.

@hcl27 commented on GitHub (Apr 28, 2024): > I am assuming you have `ollama serve` running on the host? Also which operating system as `host.docker.internal` does not work on Linux Docker, if that is the case. Otherwise, you should be fine. > > The only other caveat is Docker on Windows trying to communicate with Ollama running on WSL. WSL does not allow the connection to bridge. yes，Ollama runs on host, and the local system is Ubuntu.
yindo commented 2026-02-22 18:21:12 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Apr 29, 2024):

Where are you running Ollama? On the same server as AnythingLLM? If so what operating system

@timothycarambat commented on GitHub (Apr 29, 2024): Where are you running Ollama? On the same server as AnythingLLM? If so what operating system
yindo commented 2026-02-22 18:21:12 -05:00
Author
Owner
Copy Link
Copy Source

@hcl27 commented on GitHub (Apr 29, 2024):

你在哪里运行 Ollama？与 AnythingLLM 在同一台服务器上？如果是这样，什么操作系统

Yes, the same server, ubuntu system.

@hcl27 commented on GitHub (Apr 29, 2024): > 你在哪里运行 Ollama？与 AnythingLLM 在同一台服务器上？如果是这样，什么操作系统 Yes, the same server, ubuntu system.
yindo commented 2026-02-22 18:21:12 -05:00
Author
Owner
Copy Link
Copy Source

@mmsrubar commented on GitHub (Apr 29, 2024):

Hi, I was having the very same issue. The problem for me was that the docker container was trying to connect to docker interface IP address when using http://host.docker.internal:11434 and ollama was listening on 127.0.0.1:11434 (localhost) on the host. So I think the solution is either to configure the ollama to listen also on the docker IP or use for example socat to redirect all packets from 172.17.0.1:11434 to 127.0.0.1:11434 where 172.17.0.1 was IP of my docker0 interface.

socat TCP4-LISTEN:11434,bind=172.17.0.1,fork TCP4:127.0.0.1:11434

@mmsrubar commented on GitHub (Apr 29, 2024): Hi, I was having the very same issue. The problem for me was that the docker container was trying to connect to docker interface IP address when using http://host.docker.internal:11434 and ollama was listening on 127.0.0.1:11434 (localhost) on the host. So I think the solution is either to configure the ollama to listen also on the docker IP or use for example socat to redirect all packets from 172.17.0.1:11434 to 127.0.0.1:11434 where 172.17.0.1 was IP of my docker0 interface. socat TCP4-LISTEN:11434,bind=172.17.0.1,fork TCP4:127.0.0.1:11434
yindo commented 2026-02-22 18:21:12 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Apr 29, 2024):

Additionally, Linux does not support host.docker.internal as I mentioned previously and there is already a doc for this
https://github.com/Mintplex-Labs/anything-llm/blob/master/server/utils/AiProviders/ollama/README.md#common-issues-with-ollama

@timothycarambat commented on GitHub (Apr 29, 2024): Additionally, Linux does not support `host.docker.internal` as I mentioned previously and there is already a doc for this https://github.com/Mintplex-Labs/anything-llm/blob/master/server/utils/AiProviders/ollama/README.md#common-issues-with-ollama
yindo commented 2026-02-22 18:21:12 -05:00
Author
Owner
Copy Link
Copy Source

@hcl27 commented on GitHub (Apr 29, 2024):

Hi, I was having the very same issue. The problem for me was that the docker container was trying to connect to docker interface IP address when using http://host.docker.internal:11434 and ollama was listening on 127.0.0.1:11434 (localhost) on the host. So I think the solution is either to configure the ollama to listen also on the docker IP or use for example socat to redirect all packets from 172.17.0.1:11434 to 127.0.0.1:11434 where 172.17.0.1 was IP of my docker0 interface.

socat TCP4-LISTEN:11434,bind=172.17.0.1,fork TCP4:127.0.0.1:11434

Thank you very much. I'll give it a try

@hcl27 commented on GitHub (Apr 29, 2024): > Hi, I was having the very same issue. The problem for me was that the docker container was trying to connect to docker interface IP address when using http://host.docker.internal:11434 and ollama was listening on 127.0.0.1:11434 (localhost) on the host. So I think the solution is either to configure the ollama to listen also on the docker IP or use for example socat to redirect all packets from 172.17.0.1:11434 to 127.0.0.1:11434 where 172.17.0.1 was IP of my docker0 interface. > > socat TCP4-LISTEN:11434,bind=172.17.0.1,fork TCP4:127.0.0.1:11434 Thank you very much. I'll give it a try
yindo commented 2026-02-22 18:21:13 -05:00
Author
Owner
Copy Link
Copy Source

@hcl27 commented on GitHub (Apr 29, 2024):

Additionally, Linux does not support host.docker.internal as I mentioned previously and there is already a doc for this https://github.com/Mintplex-Labs/anything-llm/blob/master/server/utils/AiProviders/ollama/README.md#common-issues-with-ollama

Thank you for your help. Based on the documentation you provided, I have resolved this issue

@hcl27 commented on GitHub (Apr 29, 2024): > Additionally, Linux does not support `host.docker.internal` as I mentioned previously and there is already a doc for this https://github.com/Mintplex-Labs/anything-llm/blob/master/server/utils/AiProviders/ollama/README.md#common-issues-with-ollama Thank you for your help. Based on the documentation you provided, I have resolved this issue
yindo changed title from [BUG]: Unable to call model through Ollama to [GH-ISSUE #1215] [BUG]: Unable to call model through Ollama 2026-06-05 14:37:01 -04:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Desktop
Docker
Integration Request
Integration Request
OS: Linux
OS: Mobile
OS: Windows
UI/UX
blocked
bug
bug
core-team-only
documentation
duplicate
embed-widget
enhancement
feature request
github_actions
good first issue
investigating
needs info / can't replicate
possible bug
pull-request
Mirrored from GitHub Pull Request
 question
stage: specifications
wontfix
No Label needs info / can't replicate
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Mintplex-Labs/anything-llm#756
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?