Mintplex-Labs/anything-llm
1
0
Fork 0
mirror of https://github.com/Mintplex-Labs/anything-llm.git synced 2026-07-21 01:25:28 -04:00
Code Issues 328 Packages Projects Releases 10 Wiki Activity

[GH-ISSUE #495] Ollma stuck on -- loading available models -- #288

New Issue
Closed
opened 2026-02-22 18:18:46 -05:00 by yindo · 23 comments
yindo commented 2026-02-22 18:18:46 -05:00
Owner
Copy Link
Copy Source

Originally created by @ThatOneCalculator on GitHub (Dec 27, 2023).
Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/495

When setting up with Ollama, it seems to not be able to find any of the models that I have installed.

image

Originally created by @ThatOneCalculator on GitHub (Dec 27, 2023). Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/495 When setting up with Ollama, it seems to not be able to find any of the models that I have installed. ![image](https://cdn.discordapp.com/attachments/1114740395159597068/1189770183171125269/image.png)
yindo closed this issue 2026-02-22 18:18:46 -05:00
yindo commented 2026-02-22 18:18:46 -05:00
Author
Owner
Copy Link
Copy Source

@Abdoulaye-Sayouti commented on GitHub (Dec 28, 2023):

If you are running it using Docker, you have to access first the docker image shell with:
docker exec -u 0 -it docker_image_id /bin/bash

Then you need to install ollama :
curl https://ollama.ai/install.sh | sh

Then pull the model you want. From what I understood, anythingllm docker image can't have access to the ollama server installed on the host machine

@Abdoulaye-Sayouti commented on GitHub (Dec 28, 2023): If you are running it using Docker, you have to access first the docker image shell with: docker exec -u 0 -it docker_image_id /bin/bash Then you need to install ollama : curl https://ollama.ai/install.sh | sh Then pull the model you want. From what I understood, anythingllm docker image can't have access to the ollama server installed on the host machine
yindo commented 2026-02-22 18:18:46 -05:00
Author
Owner
Copy Link
Copy Source

@ThatOneCalculator commented on GitHub (Dec 28, 2023):

Oh, well then that kind of makes anything-llm a bit useless for ollama users. Why would I want to reinstall ollama and have a duplicate of all my models? Other docker based frontends can access ollama from the host just fine.

@ThatOneCalculator commented on GitHub (Dec 28, 2023): Oh, well then that kind of makes anything-llm a bit useless for ollama users. Why would I want to reinstall ollama and have a duplicate of all my models? Other docker based frontends can access ollama from the host just fine.
yindo commented 2026-02-22 18:18:46 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Dec 28, 2023):

You need to use http://host.docker.internal:11434 to be able to connect to localhost from outside the AnythingLLM docker container. Using 127.0.0.x or localhost will only resolve inside of the docker container running AnythingLLM while your Ollama instance is running on your host machine.

@timothycarambat commented on GitHub (Dec 28, 2023): You need to use `http://host.docker.internal:11434` to be able to connect to localhost from outside the AnythingLLM docker container. Using 127.0.0.x or localhost will only resolve inside of the docker container running AnythingLLM while your Ollama instance is running on your host machine.
yindo commented 2026-02-22 18:18:46 -05:00
Author
Owner
Copy Link
Copy Source

@drmetro09 commented on GitHub (Dec 28, 2023):

I have an issue where my ollama models are loading, but when i type anything in chat box of anything LLM the container of anything LLM crashes automatically any fix for this?

@drmetro09 commented on GitHub (Dec 28, 2023): I have an issue where my ollama models are loading, but when i type anything in chat box of anything LLM the container of anything LLM crashes automatically any fix for this?
yindo commented 2026-02-22 18:18:46 -05:00
Author
Owner
Copy Link
Copy Source

@ThatOneCalculator commented on GitHub (Dec 28, 2023):

@timothycarambat why is this the case? Other Docker-based Ollama UIs can connect to 0.0.0.0:11434 normally

@ThatOneCalculator commented on GitHub (Dec 28, 2023): @timothycarambat why is this the case? Other Docker-based Ollama UIs can connect to 0.0.0.0:11434 normally
yindo commented 2026-02-22 18:18:46 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Dec 28, 2023):

@ThatOneCalculator AnythingLLM uses a network bridge and not the host network itself because we run other services within the container that resolve via localhost/loopback. In order to prevent port conflicts we only bridge to the host.

@timothycarambat commented on GitHub (Dec 28, 2023): @ThatOneCalculator AnythingLLM uses a network bridge and not the host network itself because we run other services within the container that resolve via localhost/loopback. In order to prevent port conflicts we only bridge to the host.
yindo commented 2026-02-22 18:18:46 -05:00
Author
Owner
Copy Link
Copy Source

@ThatOneCalculator commented on GitHub (Dec 28, 2023):

Tested and works. Please consider adding a warning like this when selecting Ollama:

image

@ThatOneCalculator commented on GitHub (Dec 28, 2023): Tested and ~~works~~. Please consider adding a warning like this when selecting Ollama: ![image](https://github.com/Mintplex-Labs/anything-llm/assets/44733677/7dbaef99-d576-410c-8796-89b0ce4402fd)
yindo commented 2026-02-22 18:18:46 -05:00
Author
Owner
Copy Link
Copy Source

@ThatOneCalculator commented on GitHub (Dec 28, 2023):

Welp, doesn't work. Tried a single message and it crashed.

image

From the podman container:

[TELEMETRY SENT] {
  event: 'server_boot',
  distinctId: 'ad98c37a-8a86-4625-bc10-a76a89ba3024',
  properties: { commit: '--' }
}
Primary server listening on port 3001
/app/server/utils/chats/stream.js:240
        fullText += chunk.message.content;
                                  ^

TypeError: Cannot read properties of undefined (reading 'content')
    at /app/server/utils/chats/stream.js:240:35
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
@ThatOneCalculator commented on GitHub (Dec 28, 2023): Welp, doesn't work. Tried a single message and it crashed. ![image](https://github.com/Mintplex-Labs/anything-llm/assets/44733677/eccce662-2f69-4e35-9505-eb06a7bf60e5) From the podman container: ``` [TELEMETRY SENT] { event: 'server_boot', distinctId: 'ad98c37a-8a86-4625-bc10-a76a89ba3024', properties: { commit: '--' } } Primary server listening on port 3001 /app/server/utils/chats/stream.js:240 fullText += chunk.message.content; ^ TypeError: Cannot read properties of undefined (reading 'content') at /app/server/utils/chats/stream.js:240:35 at process.processTicksAndRejections (node:internal/process/task_queues:95:5) ```
yindo commented 2026-02-22 18:18:46 -05:00
Author
Owner
Copy Link
Copy Source

@moritztim commented on GitHub (Jan 8, 2024):

I have the same issue with LocalAI
image
The URL isn't fully in view, this is what I put in: http://host.docker.internal:8080/v1
LocalAI is running:

 ┌───────────────────────────────────────────────────┐ 
 │                   Fiber v2.50.0                   │ 
 │               http://127.0.0.1:8080               │ 
 │       (bound on host 0.0.0.0 and port 8080)       │ 
 │                                                   │ 
 │ Handlers ............ 73  Processes ........... 1 │ 
 │ Prefork ....... Disabled  PID ................ 13 │ 
 └───────────────────────────────────────────────────┘
@moritztim commented on GitHub (Jan 8, 2024): I have the same issue with LocalAI ![image](https://github.com/Mintplex-Labs/anything-llm/assets/90388353/ded20bbf-4727-44b0-a104-49dee45c94aa) The URL isn't fully in view, this is what I put in: `http://host.docker.internal:8080/v1` LocalAI is running: ``` ┌───────────────────────────────────────────────────┐ │ Fiber v2.50.0 │ │ http://127.0.0.1:8080 │ │ (bound on host 0.0.0.0 and port 8080) │ │ │ │ Handlers ............ 73 Processes ........... 1 │ │ Prefork ....... Disabled PID ................ 13 │ └───────────────────────────────────────────────────┘ ```
yindo commented 2026-02-22 18:18:46 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Jan 8, 2024):

@moritz-t-w
Do you see any relevant logs in the AnythingLLM docker container or the LocalAI container? If the request is going through and failing you would see logs on each container. Something from AnythingLLM and LocalAI reporting the request was denied and its status code.

@timothycarambat commented on GitHub (Jan 8, 2024): @moritz-t-w Do you see any relevant logs in the AnythingLLM docker container or the LocalAI container? If the request is going through and failing you would see logs on each container. Something from AnythingLLM and LocalAI reporting the request was denied and its status code.
yindo commented 2026-02-22 18:18:46 -05:00
Author
Owner
Copy Link
Copy Source

@moritztim commented on GitHub (Jan 10, 2024):

@timothycarambat
The LocalAI container did not recieve any requests, that's all I know. I don't know how to see the logs from the AnythingLLM container.

@moritztim commented on GitHub (Jan 10, 2024): @timothycarambat The LocalAI container did not recieve any requests, that's all I know. I don't know how to see the logs from the AnythingLLM container.
yindo commented 2026-02-22 18:18:46 -05:00
Author
Owner
Copy Link
Copy Source

@bokey007 commented on GitHub (Jan 12, 2024):

its the same issue with me

Primary server in HTTP mode listening on port 3001
TypeError: fetch failed
at Object.fetch (node:internal/deps/undici/undici:11730:11)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async ollamaAIModels (/app/server/utils/helpers/customModels.js:84:18)
at async getCustomModels (/app/server/utils/helpers/customModels.js:20:14)
at async /app/server/endpoints/system.js:769:35 {
cause: Error: getaddrinfo ENOTFOUND host.docker.internal
at GetAddrInfoReqWrap.onlookup [as oncomplete] (node:dns:107:26) {
errno: -3008,
code: 'ENOTFOUND',
syscall: 'getaddrinfo',
hostname: 'host.docker.internal'
}
}

@timothycarambat I am facing this issue with both ollama and localAi

I think, this problem is specifically for linux machines.

@bokey007 commented on GitHub (Jan 12, 2024): its the same issue with me Primary server in HTTP mode listening on port 3001 TypeError: fetch failed at Object.fetch (node:internal/deps/undici/undici:11730:11) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async ollamaAIModels (/app/server/utils/helpers/customModels.js:84:18) at async getCustomModels (/app/server/utils/helpers/customModels.js:20:14) at async /app/server/endpoints/system.js:769:35 { cause: Error: getaddrinfo ENOTFOUND host.docker.internal at GetAddrInfoReqWrap.onlookup [as oncomplete] (node:dns:107:26) { errno: -3008, code: 'ENOTFOUND', syscall: 'getaddrinfo', hostname: 'host.docker.internal' } } @timothycarambat I am facing this issue with both ollama and localAi I think, this problem is specifically for linux machines.
yindo commented 2026-02-22 18:18:47 -05:00
Author
Owner
Copy Link
Copy Source

@mdkberry commented on GitHub (Apr 7, 2024):

this is closed, but if this info helps anyone else it had me confused for a bit. In the browser you put http://localhost:11434/ and it shows Ollama running but AnythingLLM doesnt find it like that and needs http://127.0.0.1:11434

I ran into this issue on Windows 10 with the download install of Ollama and AnythingLLM (not the docker version).

Instructions elsewhere had said to run powershell and type "ollama serve" it gives 127.0.0.1:11434 but without the http bit, so when you try to add that in the field it refuses it as it wants a URL (hence why I assumed localhost would solve it but it doesnt). I had to manually change it to http://127.0.0.1:11434 and then it found the models.

A bit confusing that it wont work with "localhost" since that is what works in a browser, but this fixed the issue if anyone else hits it.

@mdkberry commented on GitHub (Apr 7, 2024): this is closed, but if this info helps anyone else it had me confused for a bit. In the browser you put http://localhost:11434/ and it shows Ollama running but AnythingLLM doesnt find it like that and needs http://127.0.0.1:11434 I ran into this issue on Windows 10 with the download install of Ollama and AnythingLLM (not the docker version). Instructions elsewhere had said to run powershell and type "ollama serve" it gives 127.0.0.1:11434 but without the http bit, so when you try to add that in the field it refuses it as it wants a URL (hence why I assumed localhost would solve it but it doesnt). I had to manually change it to http://127.0.0.1:11434 and then it found the models. A bit confusing that it wont work with "localhost" since that is what works in a browser, but this fixed the issue if anyone else hits it.
yindo commented 2026-02-22 18:18:47 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Apr 7, 2024):

Yeah, you have to specify OLLAMA_HOST as an env var to bind it to localhost, otherwise it defaults to 127.0.0.1

@timothycarambat commented on GitHub (Apr 7, 2024): Yeah, you have to specify `OLLAMA_HOST` as an env var to bind it to localhost, otherwise it defaults to 127.0.0.1
yindo commented 2026-02-22 18:18:47 -05:00
Author
Owner
Copy Link
Copy Source

@kalilfagundes commented on GitHub (May 14, 2024):

Try 0.0.0.0 instead of localhost or 127.0.0.1. Worked here.

@kalilfagundes commented on GitHub (May 14, 2024): Try 0.0.0.0 instead of localhost or 127.0.0.1. Worked here.
yindo commented 2026-02-22 18:18:47 -05:00
Author
Owner
Copy Link
Copy Source

@MilitantHitchhiker commented on GitHub (May 20, 2024):

None of these solutions worked for me. My LLM models in Ollama are local .gguf models that I've imported.

@MilitantHitchhiker commented on GitHub (May 20, 2024): None of these solutions worked for me. My LLM models in Ollama are local .gguf models that I've imported.
yindo commented 2026-02-22 18:18:47 -05:00
Author
Owner
Copy Link
Copy Source

@ParshvLimbad commented on GitHub (Jun 17, 2024):

image
what is the issue here please help

@ParshvLimbad commented on GitHub (Jun 17, 2024): ![image](https://github.com/Mintplex-Labs/anything-llm/assets/103508696/d83a28bb-841a-437c-a54e-ce71dd0694ae) what is the issue here please help
yindo commented 2026-02-22 18:18:47 -05:00
Author
Owner
Copy Link
Copy Source

@MilitantHitchhiker commented on GitHub (Jun 19, 2024):

I have fixed the issue.

On Mon., 17 Jun. 2024, 11:37 pm Parshv Limbad, @.***>
wrote:

image.png (view on web)
https://github.com/Mintplex-Labs/anything-llm/assets/103508696/d83a28bb-841a-437c-a54e-ce71dd0694ae
what is the issue here please help

—
Reply to this email directly, view it on GitHub
https://github.com/Mintplex-Labs/anything-llm/issues/495#issuecomment-2173419928,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/BICISXY2IGVKES6TBAMAAO3ZH3RBPAVCNFSM6AAAAABBFAPHMGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCNZTGQYTSOJSHA
.
You are receiving this because you commented.Message ID:
@.***>

@MilitantHitchhiker commented on GitHub (Jun 19, 2024): I have fixed the issue. On Mon., 17 Jun. 2024, 11:37 pm Parshv Limbad, ***@***.***> wrote: > image.png (view on web) > <https://github.com/Mintplex-Labs/anything-llm/assets/103508696/d83a28bb-841a-437c-a54e-ce71dd0694ae> > what is the issue here please help > > — > Reply to this email directly, view it on GitHub > <https://github.com/Mintplex-Labs/anything-llm/issues/495#issuecomment-2173419928>, > or unsubscribe > <https://github.com/notifications/unsubscribe-auth/BICISXY2IGVKES6TBAMAAO3ZH3RBPAVCNFSM6AAAAABBFAPHMGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCNZTGQYTSOJSHA> > . > You are receiving this because you commented.Message ID: > ***@***.***> >
yindo commented 2026-02-22 18:18:47 -05:00
Author
Owner
Copy Link
Copy Source

@AIlvxiaobu commented on GitHub (Sep 25, 2024):

use --network=host

export STORAGE_LOCATION=/disk/anythingllm && \
rm -rf /disk/anythingllm && \
mkdir -p $STORAGE_LOCATION && \
chmod 777 $STORAGE_LOCATION && \
touch "$STORAGE_LOCATION/.env" && \
sudo docker run -d \
 --name anythingllm \
 --network=host \
 --env STORAGE_DIR=/app/server/storage \
 --health-cmd "/bin/bash /usr/local/bin/docker-healthcheck.sh || exit 1" \
 --health-interval 60s \
 --health-start-period 60s \
 --health-timeout 10s \
 --cap-add SYS_ADMIN \
 -p 3001:3001/tcp \
 --restart=always \
 --user anythingllm \
 -v ${STORAGE_LOCATION}:/app/server/storage \
 -v ${STORAGE_LOCATION}/.env:/app/server/storage/.env \
 -w /app \
 mintplexlabs/anythingllm
@AIlvxiaobu commented on GitHub (Sep 25, 2024): use --network=host ```shell export STORAGE_LOCATION=/disk/anythingllm && \ rm -rf /disk/anythingllm && \ mkdir -p $STORAGE_LOCATION && \ chmod 777 $STORAGE_LOCATION && \ touch "$STORAGE_LOCATION/.env" && \ sudo docker run -d \ --name anythingllm \ --network=host \ --env STORAGE_DIR=/app/server/storage \ --health-cmd "/bin/bash /usr/local/bin/docker-healthcheck.sh || exit 1" \ --health-interval 60s \ --health-start-period 60s \ --health-timeout 10s \ --cap-add SYS_ADMIN \ -p 3001:3001/tcp \ --restart=always \ --user anythingllm \ -v ${STORAGE_LOCATION}:/app/server/storage \ -v ${STORAGE_LOCATION}/.env:/app/server/storage/.env \ -w /app \ mintplexlabs/anythingllm ```
yindo commented 2026-02-22 18:18:47 -05:00
Author
Owner
Copy Link
Copy Source

@blackpyramid88 commented on GitHub (Mar 18, 2025):

Image

I can't get the models to load either. I had it working fine for weeks, I upgraded Ollama to a newer version and it wont load the models. So i uninstalled Anything LLM and reinstalled hoping it would reconnect and this didn't work. Also it wont allow me to change the port. Ollama shows it's serving in my browser like normal

@blackpyramid88 commented on GitHub (Mar 18, 2025): <img width="525" alt="Image" src="https://github.com/user-attachments/assets/3dcf1faf-3c2a-4e64-bbbe-616b91002763" /> I can't get the models to load either. I had it working fine for weeks, I upgraded Ollama to a newer version and it wont load the models. So i uninstalled Anything LLM and reinstalled hoping it would reconnect and this didn't work. Also it wont allow me to change the port. Ollama shows it's serving in my browser like normal
yindo commented 2026-02-22 18:18:47 -05:00
Author
Owner
Copy Link
Copy Source

@blackpyramid88 commented on GitHub (Mar 18, 2025):

Image This is on a WIndows 10 system. Not sure why it quit working
@blackpyramid88 commented on GitHub (Mar 18, 2025): <img width="466" alt="Image" src="https://github.com/user-attachments/assets/1cd5fff4-7a9e-4fa9-bbae-97cb5e98f5e8" /> This is on a WIndows 10 system. Not sure why it quit working
yindo changed title from Ollma stuck on `-- loading available models --` to [GH-ISSUE #495] Ollma stuck on `-- loading available models --` 2026-06-05 14:34:29 -04:00
yindo commented 2026-06-05 14:34:30 -04:00
Author
Owner
Copy Link
Copy Source

@jacfalcon0 commented on GitHub (Apr 7, 2026):

I'm also having the problem, nothing above solves it.

<!-- gh-comment-id:4202725044 --> @jacfalcon0 commented on GitHub (Apr 7, 2026): I'm also having the problem, nothing above solves it.
yindo commented 2026-06-05 14:34:30 -04:00
Author
Owner
Copy Link
Copy Source

@RoLex commented on GitHub (Jun 1, 2026):

Same. Has this been fixed yet?

<!-- gh-comment-id:4595542650 --> @RoLex commented on GitHub (Jun 1, 2026): Same. Has this been fixed yet?
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Desktop
Docker
Integration Request
Integration Request
OS: Linux
OS: Mobile
OS: Windows
UI/UX
blocked
bug
bug
core-team-only
documentation
duplicate
embed-widget
enhancement
feature request
github_actions
good first issue
investigating
needs info / can't replicate
possible bug
pull-request
Mirrored from GitHub Pull Request
 question
stage: specifications
wontfix
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Mintplex-Labs/anything-llm#288
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?