Mintplex-Labs/anything-llm
1
0
Fork 0
mirror of https://github.com/Mintplex-Labs/anything-llm.git synced 2026-07-19 14:13:35 -04:00
Code Issues 328 Packages Projects Releases 10 Wiki Activity

[GH-ISSUE #2523] [BUG]: cannot load models from ollama #1630

New Issue
Closed
opened 2026-02-22 18:25:46 -05:00 by yindo · 10 comments
yindo commented 2026-02-22 18:25:46 -05:00
Owner
Copy Link
Copy Source

Originally created by @jqxiao12 on GitHub (Oct 22, 2024).
Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/2523

How are you running AnythingLLM?

Docker (local)

What happened?

cannot select Ollama Model.

AnythingLLM is install per https://docs.anythingllm.com/installation-docker/local-docker

AnythingLLM is running in the docker on a Linux machine,
Distributor ID: Ubuntu
Description: Ubuntu 20.04.6 LTS
Release: 20.04

Tried to change the Ollama base url
http://host.docker.internal:11434
http://localhost:11434
http://172.17.0.1:11434

Ollama is running
curl localhost:11434
Ollama is running

image

Are there known steps to reproduce?

No response

Originally created by @jqxiao12 on GitHub (Oct 22, 2024). Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/2523 ### How are you running AnythingLLM? Docker (local) ### What happened? cannot select Ollama Model. AnythingLLM is install per https://docs.anythingllm.com/installation-docker/local-docker AnythingLLM is running in the docker on a Linux machine, Distributor ID: Ubuntu Description: Ubuntu 20.04.6 LTS Release: 20.04 Tried to change the Ollama base url http://host.docker.internal:11434 http://localhost:11434 http://172.17.0.1:11434 Ollama is running curl localhost:11434 Ollama is running <img width="871" alt="image" src="https://github.com/user-attachments/assets/3ae890d7-a42e-4d45-8dad-71661e8f4f09"> ### Are there known steps to reproduce? _No response_
yindo added the possible bug label 2026-02-22 18:25:46 -05:00
yindo closed this issue 2026-02-22 18:25:46 -05:00
yindo commented 2026-02-22 18:25:47 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Oct 22, 2024):

Distributor ID: Ubuntu
Description: Ubuntu 20.04.6 LTS

Screenshot 2024-10-22 at 1 32 56 PM

https://docs.anythingllm.com/ollama-connection-troubleshooting#troubleshooting

Please try http://172.17.0.1:11434

@timothycarambat commented on GitHub (Oct 22, 2024): > Distributor ID: Ubuntu > Description: Ubuntu 20.04.6 LTS <img width="872" alt="Screenshot 2024-10-22 at 1 32 56 PM" src="https://github.com/user-attachments/assets/86cef434-1d5f-4641-b530-001a5c33a309"> https://docs.anythingllm.com/ollama-connection-troubleshooting#troubleshooting Please try `http://172.17.0.1:11434`
yindo commented 2026-02-22 18:25:47 -05:00
Author
Owner
Copy Link
Copy Source

@szur1 commented on GitHub (Oct 22, 2024):

This is still happening for me - even using http://host.docker.internal:11434/

Windows 11
AnythingLLM (Latest) is running in Docker - Verified http://localhost:3001/
Ollama (Latest) is running in windows - Verified http://localhost:11434
Ollama has Llama3.1 model installed - verified with the command C:\Windows\System32>ollama list
Ollama return chat using C:\Windows\System32>ollama run llama3.1

Have tried all these - No models able to be selected - pulldown is disabled

http://host.docker.internal:11434/
http://localhost:11434/
http://172.17.0.1:11434/

@szur1 commented on GitHub (Oct 22, 2024): This is still happening for me - even using http://host.docker.internal:11434/ Windows 11 AnythingLLM (Latest) is running in Docker - Verified http://localhost:3001/ Ollama (Latest) is running in windows - Verified http://localhost:11434 Ollama has Llama3.1 model installed - verified with the command C:\Windows\System32>ollama list Ollama return chat using C:\Windows\System32>ollama run llama3.1 Have tried all these - No models able to be selected - pulldown is disabled http://host.docker.internal:11434/ http://localhost:11434/ http://172.17.0.1:11434/
yindo commented 2026-02-22 18:25:47 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Oct 22, 2024):

@szur1 you need to be running ollama serve - only this command starts the server in the backend. However, I presume that is working since the localhost:11434 is up.

How are you running the docker container (command) since this is a network layer thing with the docker container being able to reach the host network.

Also just for sanity sake, does removing the trailing / matter here?

@timothycarambat commented on GitHub (Oct 22, 2024): @szur1 you need to be running `ollama serve` - only this command starts the server in the backend. However, I presume that is working since the localhost:11434 is up. How are you running the docker container (command) since this is a network layer thing with the docker container being able to reach the host network. Also just for sanity sake, does removing the trailing `/` matter here?
yindo commented 2026-02-22 18:25:47 -05:00
Author
Owner
Copy Link
Copy Source

@szur1 commented on GitHub (Oct 22, 2024):

I am running C:\Windows\System32>ollama serve

C:\Windows\System32>ollama serve
2024/10/22 13:38:09 routes.go:1158: INFO server config env="map[CUDA_VISIBLE_DEVICES: GPU_DEVICE_ORDINAL: HIP_VISIBLE_DEVICES: HSA_OVERRIDE_GFX_VERSION: HTTPS_PROXY: HTTP_PROXY: NO_PROXY: OLLAMA_DEBUG:false OLLAMA_FLASH_ATTENTION:false OLLAMA_GPU_OVERHEAD:0 OLLAMA_HOST:http://127.0.0.1:11434 OLLAMA_INTEL_GPU:false OLLAMA_KEEP_ALIVE:5m0s OLLAMA_LLM_LIBRARY: OLLAMA_LOAD_TIMEOUT:5m0s OLLAMA_MAX_LOADED_MODELS:0 OLLAMA_MAX_QUEUE:512 OLLAMA_MODELS:C:\Users\szur\.ollama\models OLLAMA_MULTIUSER_CACHE:false OLLAMA_NOHISTORY:false OLLAMA_NOPRUNE:false OLLAMA_NUM_PARALLEL:0 OLLAMA_ORIGINS:[http://localhost https://localhost http://localhost:* https://localhost:* http://127.0.0.1 https://127.0.0.1 http://127.0.0.1:* https://127.0.0.1:* http://0.0.0.0 https://0.0.0.0 http://0.0.0.0:* https://0.0.0.0:* app://* file://* tauri://*] OLLAMA_SCHED_SPREAD:false OLLAMA_TMPDIR: ROCR_VISIBLE_DEVICES:]"
time=2024-10-22T13:38:09.577-07:00 level=INFO source=images.go:754 msg="total blobs: 16"
time=2024-10-22T13:38:09.596-07:00 level=INFO source=images.go:761 msg="total unused blobs removed: 0"
time=2024-10-22T13:38:09.604-07:00 level=INFO source=routes.go:1205 msg="Listening on 127.0.0.1:11434 (version 0.3.13)"
time=2024-10-22T13:38:09.606-07:00 level=INFO source=common.go:49 msg="Dynamic LLM libraries" runners="[rocm_v6.1 cpu cpu_avx cpu_avx2 cuda_v11 cuda_v12]"
time=2024-10-22T13:38:09.607-07:00 level=INFO source=gpu.go:199 msg="looking for compatible GPUs"
time=2024-10-22T13:38:09.721-07:00 level=INFO source=gpu.go:347 msg="no compatible GPUs were discovered"
time=2024-10-22T13:38:09.722-07:00 level=INFO source=types.go:107 msg="inference compute" id=0 library=cpu variant=avx2 compute="" driver=0.0 name="" total="39.8 GiB" available="16.3 GiB"
[GIN] 2024/10/22 - 13:38:25 | 200 | 0s | 127.0.0.1 | GET "/"
[GIN] 2024/10/22 - 13:47:56 | 200 | 0s | 127.0.0.1 | GET "/"

@szur1 commented on GitHub (Oct 22, 2024): I am running C:\Windows\System32>ollama serve C:\Windows\System32>ollama serve 2024/10/22 13:38:09 routes.go:1158: INFO server config env="map[CUDA_VISIBLE_DEVICES: GPU_DEVICE_ORDINAL: HIP_VISIBLE_DEVICES: HSA_OVERRIDE_GFX_VERSION: HTTPS_PROXY: HTTP_PROXY: NO_PROXY: OLLAMA_DEBUG:false OLLAMA_FLASH_ATTENTION:false OLLAMA_GPU_OVERHEAD:0 OLLAMA_HOST:http://127.0.0.1:11434 OLLAMA_INTEL_GPU:false OLLAMA_KEEP_ALIVE:5m0s OLLAMA_LLM_LIBRARY: OLLAMA_LOAD_TIMEOUT:5m0s OLLAMA_MAX_LOADED_MODELS:0 OLLAMA_MAX_QUEUE:512 OLLAMA_MODELS:C:\\Users\\szur\\.ollama\\models OLLAMA_MULTIUSER_CACHE:false OLLAMA_NOHISTORY:false OLLAMA_NOPRUNE:false OLLAMA_NUM_PARALLEL:0 OLLAMA_ORIGINS:[http://localhost https://localhost http://localhost:* https://localhost:* http://127.0.0.1 https://127.0.0.1 http://127.0.0.1:* https://127.0.0.1:* http://0.0.0.0 https://0.0.0.0 http://0.0.0.0:* https://0.0.0.0:* app://* file://* tauri://*] OLLAMA_SCHED_SPREAD:false OLLAMA_TMPDIR: ROCR_VISIBLE_DEVICES:]" time=2024-10-22T13:38:09.577-07:00 level=INFO source=images.go:754 msg="total blobs: 16" time=2024-10-22T13:38:09.596-07:00 level=INFO source=images.go:761 msg="total unused blobs removed: 0" time=2024-10-22T13:38:09.604-07:00 level=INFO source=routes.go:1205 msg="Listening on 127.0.0.1:11434 (version 0.3.13)" time=2024-10-22T13:38:09.606-07:00 level=INFO source=common.go:49 msg="Dynamic LLM libraries" runners="[rocm_v6.1 cpu cpu_avx cpu_avx2 cuda_v11 cuda_v12]" time=2024-10-22T13:38:09.607-07:00 level=INFO source=gpu.go:199 msg="looking for compatible GPUs" time=2024-10-22T13:38:09.721-07:00 level=INFO source=gpu.go:347 msg="no compatible GPUs were discovered" time=2024-10-22T13:38:09.722-07:00 level=INFO source=types.go:107 msg="inference compute" id=0 library=cpu variant=avx2 compute="" driver=0.0 name="" total="39.8 GiB" available="16.3 GiB" [GIN] 2024/10/22 - 13:38:25 | 200 | 0s | 127.0.0.1 | GET "/" [GIN] 2024/10/22 - 13:47:56 | 200 | 0s | 127.0.0.1 | GET "/"
yindo commented 2026-02-22 18:25:47 -05:00
Author
Owner
Copy Link
Copy Source

@jqxiao12 commented on GitHub (Oct 22, 2024):

still not working with http://172.17.0.1:11434

anythingllm docker log
[backend] error: TypeError: fetch failed
at node:internal/deps/undici/undici:12618:11
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async ollamaAIModels (/app/server/utils/helpers/customModels.js:283:18)
at async getCustomModels (/app/server/utils/helpers/customModels.js:36:14)
at async /app/server/endpoints/system.js:910:35

image
@jqxiao12 commented on GitHub (Oct 22, 2024): still not working with http://172.17.0.1:11434 anythingllm docker log [backend] error: TypeError: fetch failed at node:internal/deps/undici/undici:12618:11 at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async ollamaAIModels (/app/server/utils/helpers/customModels.js:283:18) at async getCustomModels (/app/server/utils/helpers/customModels.js:36:14) at async /app/server/endpoints/system.js:910:35 <img width="886" alt="image" src="https://github.com/user-attachments/assets/c8f00426-3017-4706-8a4a-8df2b4caf48f">
yindo commented 2026-02-22 18:25:47 -05:00
Author
Owner
Copy Link
Copy Source

@jqxiao12 commented on GitHub (Oct 22, 2024):

if I do
curl 172.17.0.1:11434
curl: (7) Failed to connect to 172.17.0.1 port 11434: Connection refused

but
curl localhost:11434
Ollama is running

@jqxiao12 commented on GitHub (Oct 22, 2024): if I do curl 172.17.0.1:11434 curl: (7) Failed to connect to 172.17.0.1 port 11434: Connection refused but curl localhost:11434 Ollama is running
yindo commented 2026-02-22 18:25:47 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Oct 22, 2024):

if I do
curl 172.17.0.1:11434
curl: (7) Failed to connect to 172.17.0.1 port 11434: Connection refused

Are you doing this inside of the anythingllm docker container or on your host? That IP only resolves inside of the container. Also how are you running the docker container?

@timothycarambat commented on GitHub (Oct 22, 2024): > if I do curl 172.17.0.1:11434 curl: (7) Failed to connect to 172.17.0.1 port 11434: Connection refused Are you doing this inside of the anythingllm docker container or on your host? That IP only resolves inside of the container. Also how are you running the docker container?
yindo commented 2026-02-22 18:25:47 -05:00
Author
Owner
Copy Link
Copy Source

@szur1 commented on GitHub (Oct 22, 2024):

curl 172.17.0.1:11434
curl: (7) Failed to connect to 172.17.0.1 port 11434 after 0 ms: Connection refused

From within the docker image. Running using Docker Desktop

@szur1 commented on GitHub (Oct 22, 2024): curl 172.17.0.1:11434 curl: (7) Failed to connect to 172.17.0.1 port 11434 after 0 ms: Connection refused From within the docker image. Running using Docker Desktop
yindo commented 2026-02-22 18:25:47 -05:00
Author
Owner
Copy Link
Copy Source

@jqxiao12 commented on GitHub (Oct 22, 2024):

follow this fixed the issue for me - https://github.com/Mintplex-Labs/anything-llm/blob/master/server/utils/AiProviders/ollama/README.md

now it is returning running
curl 172.17.0.1:11434
Ollama is running

@jqxiao12 commented on GitHub (Oct 22, 2024): follow this fixed the issue for me - https://github.com/Mintplex-Labs/anything-llm/blob/master/server/utils/AiProviders/ollama/README.md now it is returning running curl 172.17.0.1:11434 Ollama is running
yindo commented 2026-02-22 18:25:48 -05:00
Author
Owner
Copy Link
Copy Source

@yaoxi-std commented on GitHub (Feb 18, 2025):

For me this error occurs because there's no model downloaded in Ollama, and I finally solved this after pulling one. I think AnythingLLM didn't handle the empty model list correctly.

@yaoxi-std commented on GitHub (Feb 18, 2025): For me this error occurs because there's no model downloaded in Ollama, and I finally solved this after pulling one. I think AnythingLLM didn't handle the empty model list correctly.
yindo changed title from [BUG]: cannot load models from ollama to [GH-ISSUE #2523] [BUG]: cannot load models from ollama 2026-06-05 14:41:50 -04:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Desktop
Docker
Integration Request
Integration Request
OS: Linux
OS: Mobile
OS: Windows
UI/UX
blocked
bug
bug
core-team-only
documentation
duplicate
embed-widget
enhancement
feature request
github_actions
good first issue
investigating
needs info / can't replicate
possible bug
pull-request
Mirrored from GitHub Pull Request
 question
stage: specifications
wontfix
No Label possible bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Mintplex-Labs/anything-llm#1630
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?