Mintplex-Labs/anything-llm
1
0
Fork 0
mirror of https://github.com/Mintplex-Labs/anything-llm.git synced 2026-07-19 22:23:50 -04:00
Code Issues 328 Packages Projects Releases 10 Wiki Activity

[GH-ISSUE #1987] Using hosts file in-container for communication with Ollama #1293

New Issue
Closed
opened 2026-02-22 18:24:08 -05:00 by yindo · 6 comments
yindo commented 2026-02-22 18:24:08 -05:00
Owner
Copy Link
Copy Source

Originally created by @JayCroghan on GitHub (Jul 28, 2024).
Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/1987

How are you running AnythingLLM?

Docker (local)

What happened?

I have Ollama running on a Windows GPU machine which is on the local network, due to circumstances outside of my control this machines IP changes from time to time.

AnythingLLM is running on docker on Ubuntu 22.041 and if I manually change the IP in settings it works fine, it propagates to the projects set to use system default.

Now what I have done successfully with OpenWebUI and am trying with AnthingLLM is to use a local hostsfile using -v for volumes in my docker run line I have -v /home/jay/dynip/hosts:/etc/hosts - when the IP changes I modify the hosts file on the host and it works fine on OpenWebUI and it also works fine in AnythingLLM Admin Settings. The domain I am using is desktop-jay.local and when I use that in the AnythingLLM Admin settings for Ollama it shows me the list of models available and I click save.

But, when I go to ask a question in old or newly created spaces it tells me Ollama service could not be reached. Is Ollama running?

Ollama is running on 0.0.0.0 and desktop-jay.local is its a real address, just inside containers there is no wins so it can't resolve the windows hostname and this is my workaround.

Any ideas?

Are there known steps to reproduce?

No response

Originally created by @JayCroghan on GitHub (Jul 28, 2024). Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/1987 ### How are you running AnythingLLM? Docker (local) ### What happened? I have Ollama running on a Windows GPU machine which is on the local network, due to circumstances outside of my control this machines IP changes from time to time. AnythingLLM is running on docker on Ubuntu 22.041 and if I manually change the IP in settings it works fine, it propagates to the projects set to use system default. Now what I have done successfully with OpenWebUI and am trying with AnthingLLM is to use a local hostsfile using -v for volumes in my docker run line I have `-v /home/jay/dynip/hosts:/etc/hosts` - when the IP changes I modify the hosts file on the host and it works fine on OpenWebUI **and** it also works fine in AnythingLLM Admin Settings. The domain I am using is desktop-jay.local and when I use that in the AnythingLLM Admin settings for Ollama it shows me the list of models available and I click save. But, when I go to ask a question in old or newly created spaces it tells me `Ollama service could not be reached. Is Ollama running?` Ollama is running on 0.0.0.0 and `desktop-jay.local` is its a real address, just inside containers there is no wins so it can't resolve the windows hostname and this is my workaround. Any ideas? ### Are there known steps to reproduce? _No response_
yindo added the question label 2026-02-22 18:24:08 -05:00
yindo closed this issue 2026-02-22 18:24:08 -05:00
yindo commented 2026-02-22 18:24:08 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Jul 29, 2024):

Interesting environment. So to be clear the thing you type into the connection is http://desktop-jay.local and that works, but when you roll the IP for the host entry - the connector no longer resolves the new IP?

Host file changes are applied immediately and we simply use the node process.env strings so the string we will use for connection is always ${process.env.OLLAMA_BASE_PATH}/api/...; which which should concat to http://desktop-jay.local/api/... and then network layer should handle the rest.

@timothycarambat commented on GitHub (Jul 29, 2024): Interesting environment. So to be clear the thing you type into the connection is `http://desktop-jay.local` and that works, but when you roll the IP for the host entry - the connector no longer resolves the new IP? Host file changes are applied immediately and we simply use the node `process.env` strings so the string we will use for connection is always `${process.env.OLLAMA_BASE_PATH}/api/...`; which which should concat to `http://desktop-jay.local/api/...` and then network layer should handle the rest.
yindo commented 2026-02-22 18:24:08 -05:00
Author
Owner
Copy Link
Copy Source

@JayCroghan commented on GitHub (Jul 29, 2024):

Yeah I'm using the same setup for all containers rather than installing
wins into the container. It works for a variety of other tools instantly as
you said, such as OpenWebUI and Flaresolverr but yeah just as you said,
when I type it into the connection entry on the Admin Settings it will load
the list of models i.e. definitely working - but then it fails. I don't
have better logs than just failed, is there a verbose option?

On Tue, 30 Jul 2024 at 02:21, Timothy Carambat @.***>
wrote:

Interesting environment. So to be clear the thing you type into the
connection is http://desktop-jay.local and that works, but when you roll
the IP for the host entry - the connector no longer resolves the new IP?

Host file changes are applied immediately and we simply use the node
process.env strings so the string we will use for connection is always
${process.env.OLLAMA_BASE_PATH}/api/...; which which should contat to
http://desktop-jay.local/api/... and then network layer should handle the
rest.

—
Reply to this email directly, view it on GitHub
https://github.com/Mintplex-Labs/anything-llm/issues/1987#issuecomment-2256615788,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAI55TC2CRHJAHSG37ETL2TZO2B3PAVCNFSM6AAAAABLSYRN26VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDENJWGYYTKNZYHA
.
You are receiving this because you authored the thread.Message ID:
@.***>

@JayCroghan commented on GitHub (Jul 29, 2024): Yeah I'm using the same setup for all containers rather than installing wins into the container. It works for a variety of other tools instantly as you said, such as OpenWebUI and Flaresolverr but yeah just as you said, when I type it into the connection entry on the Admin Settings it will load the list of models i.e. definitely working - but then it fails. I don't have better logs than just failed, is there a verbose option? On Tue, 30 Jul 2024 at 02:21, Timothy Carambat ***@***.***> wrote: > Interesting environment. So to be clear the thing you type into the > connection is http://desktop-jay.local and that works, but when you roll > the IP for the host entry - the connector no longer resolves the new IP? > > Host file changes are applied immediately and we simply use the node > process.env strings so the string we will use for connection is always > ${process.env.OLLAMA_BASE_PATH}/api/...; which which should contat to > http://desktop-jay.local/api/... and then network layer should handle the > rest. > > — > Reply to this email directly, view it on GitHub > <https://github.com/Mintplex-Labs/anything-llm/issues/1987#issuecomment-2256615788>, > or unsubscribe > <https://github.com/notifications/unsubscribe-auth/AAI55TC2CRHJAHSG37ETL2TZO2B3PAVCNFSM6AAAAABLSYRN26VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDENJWGYYTKNZYHA> > . > You are receiving this because you authored the thread.Message ID: > ***@***.***> >
yindo commented 2026-02-22 18:24:08 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Jul 29, 2024):

That particular log is for embedder models - are you getting this issue with embedder values or the Ollama as your LLM? The reason i ask is that they are managed separately in the UI and it was never really clear that this issue is occurring on the LLM selection page or the embedder selection page?

Regardless yes, if the models show then for sure the connection works so it make no sense as to how only the embedder model connection fails since that is the only place that message can come from

@timothycarambat commented on GitHub (Jul 29, 2024): That particular log is for embedder models - are you getting this issue with embedder values or the Ollama as your LLM? The reason i ask is that they are managed separately in the UI and it was never really clear that this issue is occurring on the LLM selection page or the embedder selection page? Regardless yes, if the models show then _for sure_ the connection works so it make no sense as to how only the embedder model connection fails since that is the only place that message can come from
yindo commented 2026-02-22 18:24:08 -05:00
Author
Owner
Copy Link
Copy Source

@JayCroghan commented on GitHub (Jul 29, 2024):

Ah you hit the nail on the head, I had forgotten to change host on the embedder, it all works fine now I changed that too. Brain fart!

@JayCroghan commented on GitHub (Jul 29, 2024): Ah you hit the nail on the head, I had forgotten to change host on the embedder, it all works fine now I changed that too. Brain fart!
yindo commented 2026-02-22 18:24:08 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Jul 29, 2024):

It happens! Glad it was an easy fix :)

@timothycarambat commented on GitHub (Jul 29, 2024): It happens! Glad it was an easy fix :)
yindo commented 2026-02-22 18:24:08 -05:00
Author
Owner
Copy Link
Copy Source

@JayCroghan commented on GitHub (Jul 30, 2024):

Me too! Setting it up for OpenWebUI in the first instance took a lot of messing about, but now I have a quick and dirty NodeJS API that takes the Win GPU machine IP via REST and modifies that hosts file on the AnythingLLM docker host machine with the up-to-date IP and it automatically propagates to all of my docker containers :) Neat. It's a lot easier than editing docker containers to install WINS on each of them! The API was a couple of lines of code, using a modified NPM package named Hostile which I've allowed to take a file param to edit non-standard host file locations and the Windows GPU Server has a C# app I had written anyway that sits in my task bar for various other shortcuts I need during the day with a timer to check and update the IP periodically in the background.

Win-Win!

@JayCroghan commented on GitHub (Jul 30, 2024): Me too! Setting it up for OpenWebUI in the first instance took a lot of messing about, but now I have a quick and dirty NodeJS API that takes the Win GPU machine IP via REST and modifies that hosts file on the AnythingLLM docker host machine with the up-to-date IP and it automatically propagates to all of my docker containers :) Neat. It's a lot easier than editing docker containers to install `WINS` on each of them! The API was a couple of lines of code, using a modified NPM package named `Hostile` which I've allowed to take a file param to edit non-standard host file locations and the Windows GPU Server has a C# app I had written anyway that sits in my task bar for various other shortcuts I need during the day with a timer to check and update the IP periodically in the background. Win-Win!
yindo changed title from Using hosts file in-container for communication with Ollama to [GH-ISSUE #1987] Using hosts file in-container for communication with Ollama 2026-06-05 14:39:58 -04:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Desktop
Docker
Integration Request
Integration Request
OS: Linux
OS: Mobile
OS: Windows
UI/UX
blocked
bug
bug
core-team-only
documentation
duplicate
embed-widget
enhancement
feature request
github_actions
good first issue
investigating
needs info / can't replicate
possible bug
pull-request
Mirrored from GitHub Pull Request
 question
stage: specifications
wontfix
No Label question
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Mintplex-Labs/anything-llm#1293
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?