Mintplex-Labs/anything-llm
1
0
Fork 0
mirror of https://github.com/Mintplex-Labs/anything-llm.git synced 2026-07-19 14:13:35 -04:00
Code Issues 328 Packages Projects Releases 10 Wiki Activity

[GH-ISSUE #4190] [BUG]: Can't select models in external Ollama server #2666

New Issue
Closed
opened 2026-02-22 18:30:41 -05:00 by yindo · 9 comments
yindo commented 2026-02-22 18:30:41 -05:00
Owner
Copy Link
Copy Source

Originally created by @jhsmith409 on GitHub (Jul 23, 2025).
Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/4190

Originally assigned to: @timothycarambat on GitHub.

How are you running AnythingLLM?

Docker (local)

What happened?

After pulling 1.8.4, I am unable to select models from an external Ollama server. Localhost ollama servers still work. Reverting to 1.8.3 resolved the issue.

Are there known steps to reproduce?

No response

Originally created by @jhsmith409 on GitHub (Jul 23, 2025). Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/4190 Originally assigned to: @timothycarambat on GitHub. ### How are you running AnythingLLM? Docker (local) ### What happened? After pulling 1.8.4, I am unable to select models from an external Ollama server. Localhost ollama servers still work. Reverting to 1.8.3 resolved the issue. ### Are there known steps to reproduce? _No response_
yindo added the possible bugneeds info / can't replicateinvestigating labels 2026-02-22 18:30:41 -05:00
yindo closed this issue 2026-02-22 18:30:41 -05:00
yindo commented 2026-02-22 18:30:41 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Jul 23, 2025):

Unable to replicate. Hosting ollama on another machine and using NGROK to replicate remote destination. Using the BaseURL (no path) I can see all models listed on the remote machine as well as send chats and agent calls.

@timothycarambat commented on GitHub (Jul 23, 2025): Unable to replicate. Hosting ollama on another machine and using NGROK to replicate remote destination. Using the BaseURL (no path) I can see all models listed on the remote machine as well as send chats and agent calls.
yindo commented 2026-02-22 18:30:42 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Jul 23, 2025):

Given you are on Docker this could be networking related. I would assume the remote URL is HTTP and the self-hosted docker is also HTTP. Even then, we request models via the backend so CORS should not be an issue.

@timothycarambat commented on GitHub (Jul 23, 2025): Given you are on Docker this could be networking related. I would assume the remote URL is HTTP and the self-hosted docker is also HTTP. Even then, we request models via the backend so CORS should not be an issue.
yindo commented 2026-02-22 18:30:42 -05:00
Author
Owner
Copy Link
Copy Source

@jhsmith409 commented on GitHub (Jul 23, 2025):

Yes, hosted on docker using docker compose. Same docker compose file used with both 1.8.3 and 1.8.4. Anything llm hosted on 192.168.30.209. Ollama hosted on 192.168.30.211. Ollama version 0.9.6. Only change was moving from 1.8.3 to 1.8.4 with a docker compose down, docker compose pull, docker compose up -d. Changed back by editing docker-compose.yml to specify 1.8.3 instead of latest. The problem survived a reboot before I switched back to 1.8.3.

docker-compose.yml posted below:

services:
anythingllm:
image: mintplexlabs/anythingllm:1.8.3
container_name: anythingllm
restart: unless-stopped
cap_add:
- SYS_ADMIN
ports:
- "3001:3001"
environment:
- NODE_ENV=production
- STORAGE_DIR=/app/server/storage
- COMMUNITY_HUB_BUNDLE_DOWNLOADS_ENABLED=1
- UID=1000
- GID=1000
- LLM_PROVIDER=ollama
- OLLAMA_BASE_PATH=http://host.docker.internal:11434
- EMBEDDING_ENGINE=ollama
- EMBEDDING_BASE_PATH=http://host.docker.internal:11434
- VECTOR_DB=lancedb
extra_hosts:
- "host.docker.internal:host-gateway"
volumes:
- ./storage:/app/server/storage
- ./.env:/app/server/.env

Image. This pulldown box says something like it can't find any models under 1.8.4 but works under 1.8.3. I will try to reproduce and post another message.

@jhsmith409 commented on GitHub (Jul 23, 2025): Yes, hosted on docker using docker compose. Same docker compose file used with both 1.8.3 and 1.8.4. Anything llm hosted on 192.168.30.209. Ollama hosted on 192.168.30.211. Ollama version 0.9.6. Only change was moving from 1.8.3 to 1.8.4 with a docker compose down, docker compose pull, docker compose up -d. Changed back by editing docker-compose.yml to specify 1.8.3 instead of latest. The problem survived a reboot before I switched back to 1.8.3. docker-compose.yml posted below: services: anythingllm: image: mintplexlabs/anythingllm:1.8.3 container_name: anythingllm restart: unless-stopped cap_add: - SYS_ADMIN ports: - "3001:3001" environment: - NODE_ENV=production - STORAGE_DIR=/app/server/storage - COMMUNITY_HUB_BUNDLE_DOWNLOADS_ENABLED=1 - UID=1000 - GID=1000 - LLM_PROVIDER=ollama - OLLAMA_BASE_PATH=http://host.docker.internal:11434 - EMBEDDING_ENGINE=ollama - EMBEDDING_BASE_PATH=http://host.docker.internal:11434 - VECTOR_DB=lancedb extra_hosts: - "host.docker.internal:host-gateway" volumes: - ./storage:/app/server/storage - ./.env:/app/server/.env <img width="279" height="142" alt="Image" src="https://github.com/user-attachments/assets/bad7c0eb-7685-4b52-9958-b8ac5e4b6e08" />. This pulldown box says something like it can't find any models under 1.8.4 but works under 1.8.3. I will try to reproduce and post another message.
yindo commented 2026-02-22 18:30:42 -05:00
Author
Owner
Copy Link
Copy Source

@jhsmith409 commented on GitHub (Jul 23, 2025):

Changed the version to 1.8.4, docker compose down, docker compose pull, docker compose up and it can't read the files from 192.168.30.211

Image

No changes to the .211 machine, no reboot, just a nano and those 3 docker commands. Some error below cite a custom model. I do have a local version of one medgemma model. Perhaps related to that?

nano docker-compose.yml
jhsmith@xeon:/anythingllm$ docker compose pull
[+] Pulling 14/14
✔ anythingllm Pulled 61.2s
✔ 3713021b0277 Already exists 0.0s
✔ 4f4fb700ef54 Pull complete 0.4s
✔ 7da6e67eb7e3 Already exists 0.0s
✔ 3cdb165adebd Already exists 0.0s
✔ 648ddedec995 Already exists 0.0s
✔ 9ae0933ba01c Already exists 0.0s
✔ 2e3891f345d8 Already exists 0.0s
✔ 505ffbc2f160 Already exists 0.0s
✔ da15cc1f8504 Pull complete 0.4s
✔ b30793a7029b Pull complete 36.1s
✔ 570e03ab36a0 Pull complete 36.2s
✔ 07e32c9c4ce0 Pull complete 60.3s
✔ 060dcd79d942 Pull complete 60.6s
jhsmith@xeon:/anythingllm$ docker compose up
[+] Running 2/2
✔ Network anythingllm_default Created 0.1s
✔ Container anythingllm Created 0.7s
Attaching to anythingllm
anythingllm | [collector] info: [TikTokenTokenizer] Initialized new TikTokenTokenizer instance.
anythingllm | [collector] info: Collector hot directory and tmp storage wiped!
anythingllm | [collector] info: Document processor app listening on port 8888
anythingllm | Environment variables loaded from .env
anythingllm | Prisma schema loaded from prisma/schema.prisma
anythingllm |
anythingllm | ✔ Generated Prisma Client (v5.3.1) to ./node_modules/@prisma/client in 406ms
anythingllm |
anythingllm | Start using Prisma Client in Node.js (See: https://pris.ly/d/client)
anythingllm | anythingllm | import { PrismaClient } from '@prisma/client' anythingllm | const prisma = new PrismaClient() anythingllm |
anythingllm | or start using Prisma Client at the edge (See: https://pris.ly/d/accelerate)
anythingllm | anythingllm | import { PrismaClient } from '@prisma/client/edge' anythingllm | const prisma = new PrismaClient() anythingllm |
anythingllm |
anythingllm | See other ways of importing Prisma Client: http://pris.ly/d/importing-client
anythingllm |
anythingllm | npm notice
anythingllm | npm notice New major version of npm available! 10.8.2 -> 11.4.2
anythingllm | npm notice Changelog: https://github.com/npm/cli/releases/tag/v11.4.2
anythingllm | npm notice To update run: npm install -g npm@11.4.2
anythingllm | npm notice
anythingllm | Environment variables loaded from .env
anythingllm | Prisma schema loaded from prisma/schema.prisma
anythingllm | Datasource "db": SQLite database "anythingllm.db" at "file:../storage/anythingllm.db"
anythingllm |
anythingllm | 31 migrations found in prisma/migrations
anythingllm |
anythingllm |
anythingllm | No pending migrations to apply.
anythingllm | ┌─────────────────────────────────────────────────────────┐
anythingllm | │ Update available 5.3.1 -> 6.12.0 │
anythingllm | │ │
anythingllm | │ This is a major update - please follow the guide at │
anythingllm | │ https://pris.ly/d/major-version-upgrade │
anythingllm | │ │
anythingllm | │ Run the following to update │
anythingllm | │ npm i --save-dev prisma@latest │
anythingllm | │ npm i @prisma/client@latest │
anythingllm | └─────────────────────────────────────────────────────────┘
anythingllm | [backend] info: [EncryptionManager] Loaded existing key & salt for encrypting arbitrary data.
anythingllm | [backend] info: [TokenManager] Initialized new TokenManager instance for model: gpt-3.5-turbo
anythingllm | [backend] info: [TokenManager] Returning existing instance for model: gpt-3.5-turbo
anythingllm | [backend] info: [TELEMETRY ENABLED] Anonymous Telemetry enabled. Telemetry helps Mintplex Labs Inc improve AnythingLLM.
anythingllm | [backend] info: prisma:info Starting a sqlite pool with 41 connections.
anythingllm | [backend] info: [TELEMETRY SENT] {"event":"server_boot","distinctId":"44f9983a-04f2-4ee1-89a8-b0b6b472a475","properties":{"commit":"--","runtime":"docker"}}
anythingllm | [backend] info: [CommunicationKey] RSA key pair generated for signed payloads within AnythingLLM services.
anythingllm | [backend] info: [EncryptionManager] Loaded existing key & salt for encrypting arbitrary data.
anythingllm | [backend] info: Primary server in HTTP mode listening on port 3001
anythingllm | [backend] info: [BackgroundWorkerService] Feature is not enabled and will not be started.
anythingllm | [backend] info: [MetaGenerator] fetching custom meta tag settings...
anythingllm | [backend] info: [Event Logged] - update_llm_provider
anythingllm | [backend] error: TypeError: fetch failed
anythingllm | at node:internal/deps/undici/undici:12637:11
anythingllm | at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
anythingllm | at async ollamaAIModels (/app/server/utils/helpers/customModels.js:352:18)
anythingllm | at async getCustomModels (/app/server/utils/helpers/customModels.js:50:14)
anythingllm | at async /app/server/endpoints/system.js:976:35

@jhsmith409 commented on GitHub (Jul 23, 2025): Changed the version to 1.8.4, docker compose down, docker compose pull, docker compose up and it can't read the files from 192.168.30.211 <img width="854" height="612" alt="Image" src="https://github.com/user-attachments/assets/3653bf1f-b9ed-4d90-ade9-2c9e7e07bb2d" /> No changes to the .211 machine, no reboot, just a nano and those 3 docker commands. Some error below cite a custom model. I do have a local version of one medgemma model. Perhaps related to that? nano docker-compose.yml jhsmith@xeon:~/anythingllm$ docker compose pull [+] Pulling 14/14 ✔ anythingllm Pulled 61.2s ✔ 3713021b0277 Already exists 0.0s ✔ 4f4fb700ef54 Pull complete 0.4s ✔ 7da6e67eb7e3 Already exists 0.0s ✔ 3cdb165adebd Already exists 0.0s ✔ 648ddedec995 Already exists 0.0s ✔ 9ae0933ba01c Already exists 0.0s ✔ 2e3891f345d8 Already exists 0.0s ✔ 505ffbc2f160 Already exists 0.0s ✔ da15cc1f8504 Pull complete 0.4s ✔ b30793a7029b Pull complete 36.1s ✔ 570e03ab36a0 Pull complete 36.2s ✔ 07e32c9c4ce0 Pull complete 60.3s ✔ 060dcd79d942 Pull complete 60.6s jhsmith@xeon:~/anythingllm$ docker compose up [+] Running 2/2 ✔ Network anythingllm_default Created 0.1s ✔ Container anythingllm Created 0.7s Attaching to anythingllm anythingllm | [collector] info: [TikTokenTokenizer] Initialized new TikTokenTokenizer instance. anythingllm | [collector] info: Collector hot directory and tmp storage wiped! anythingllm | [collector] info: Document processor app listening on port 8888 anythingllm | Environment variables loaded from .env anythingllm | Prisma schema loaded from prisma/schema.prisma anythingllm | anythingllm | ✔ Generated Prisma Client (v5.3.1) to ./node_modules/@prisma/client in 406ms anythingllm | anythingllm | Start using Prisma Client in Node.js (See: https://pris.ly/d/client) anythingllm | ``` anythingllm | import { PrismaClient } from '@prisma/client' anythingllm | const prisma = new PrismaClient() anythingllm | ``` anythingllm | or start using Prisma Client at the edge (See: https://pris.ly/d/accelerate) anythingllm | ``` anythingllm | import { PrismaClient } from '@prisma/client/edge' anythingllm | const prisma = new PrismaClient() anythingllm | ``` anythingllm | anythingllm | See other ways of importing Prisma Client: http://pris.ly/d/importing-client anythingllm | anythingllm | npm notice anythingllm | npm notice New major version of npm available! 10.8.2 -> 11.4.2 anythingllm | npm notice Changelog: https://github.com/npm/cli/releases/tag/v11.4.2 anythingllm | npm notice To update run: npm install -g npm@11.4.2 anythingllm | npm notice anythingllm | Environment variables loaded from .env anythingllm | Prisma schema loaded from prisma/schema.prisma anythingllm | Datasource "db": SQLite database "anythingllm.db" at "file:../storage/anythingllm.db" anythingllm | anythingllm | 31 migrations found in prisma/migrations anythingllm | anythingllm | anythingllm | No pending migrations to apply. anythingllm | ┌─────────────────────────────────────────────────────────┐ anythingllm | │ Update available 5.3.1 -> 6.12.0 │ anythingllm | │ │ anythingllm | │ This is a major update - please follow the guide at │ anythingllm | │ https://pris.ly/d/major-version-upgrade │ anythingllm | │ │ anythingllm | │ Run the following to update │ anythingllm | │ npm i --save-dev prisma@latest │ anythingllm | │ npm i @prisma/client@latest │ anythingllm | └─────────────────────────────────────────────────────────┘ anythingllm | [backend] info: [EncryptionManager] Loaded existing key & salt for encrypting arbitrary data. anythingllm | [backend] info: [TokenManager] Initialized new TokenManager instance for model: gpt-3.5-turbo anythingllm | [backend] info: [TokenManager] Returning existing instance for model: gpt-3.5-turbo anythingllm | [backend] info: [TELEMETRY ENABLED] Anonymous Telemetry enabled. Telemetry helps Mintplex Labs Inc improve AnythingLLM. anythingllm | [backend] info: prisma:info Starting a sqlite pool with 41 connections. anythingllm | [backend] info: [TELEMETRY SENT] {"event":"server_boot","distinctId":"44f9983a-04f2-4ee1-89a8-b0b6b472a475","properties":{"commit":"--","runtime":"docker"}} anythingllm | [backend] info: [CommunicationKey] RSA key pair generated for signed payloads within AnythingLLM services. anythingllm | [backend] info: [EncryptionManager] Loaded existing key & salt for encrypting arbitrary data. anythingllm | [backend] info: Primary server in HTTP mode listening on port 3001 anythingllm | [backend] info: [BackgroundWorkerService] Feature is not enabled and will not be started. anythingllm | [backend] info: [MetaGenerator] fetching custom meta tag settings... anythingllm | [backend] info: [Event Logged] - update_llm_provider anythingllm | [backend] error: TypeError: fetch failed anythingllm | at node:internal/deps/undici/undici:12637:11 anythingllm | at process.processTicksAndRejections (node:internal/process/task_queues:95:5) anythingllm | at async ollamaAIModels (/app/server/utils/helpers/customModels.js:352:18) anythingllm | at async getCustomModels (/app/server/utils/helpers/customModels.js:50:14) anythingllm | at async /app/server/endpoints/system.js:976:35
yindo commented 2026-02-22 18:30:42 -05:00
Author
Owner
Copy Link
Copy Source

@jhsmith409 commented on GitHub (Jul 23, 2025):

I asked Claude Code to look into it and got this answer:

Based on my analysis, the error TypeError: fetch failed in version 1.8.4 when connecting to your
Ollama server at 192.168.30.211 appears to be a network connectivity issue. Here's what I
found:

Root Cause Analysis

The error occurs in customModels.js:352 in the ollamaAIModels function when it tries to fetch
from ${url}/api/tags. The function constructs the URL from either:

  • The basePath parameter, or
  • The OLLAMA_BASE_PATH environment variable

Potential Issues in 1.8.4

  1. URL Construction Changes: Recent commits show improvements to Ollama auth token handling and
    header passing
  2. Network Request Changes: The fetch request now includes authentication headers that might be
    causing connection issues
  3. Docker Networking: The error might be related to how the Docker container resolves the IP
    address 192.168.30.211
@jhsmith409 commented on GitHub (Jul 23, 2025): I asked Claude Code to look into it and got this answer: Based on my analysis, the error TypeError: fetch failed in version 1.8.4 when connecting to your Ollama server at 192.168.30.211 appears to be a network connectivity issue. Here's what I found: Root Cause Analysis The error occurs in customModels.js:352 in the ollamaAIModels function when it tries to fetch from ${url}/api/tags. The function constructs the URL from either: - The basePath parameter, or - The OLLAMA_BASE_PATH environment variable Potential Issues in 1.8.4 1. URL Construction Changes: Recent commits show improvements to Ollama auth token handling and header passing 2. Network Request Changes: The fetch request now includes authentication headers that might be causing connection issues 3. Docker Networking: The error might be related to how the Docker container resolves the IP address 192.168.30.211
yindo commented 2026-02-22 18:30:42 -05:00
Author
Owner
Copy Link
Copy Source

@jhsmith409 commented on GitHub (Jul 23, 2025):

Claude suggested this test:

docker exec -it anythingllm curl http://192.168.30.211:11434/api/tags

{"models":[{"name":"alibayram/medgemma:4b","model":"alibayram/medgemma:4b","modified_at":"2025-07-22T01:42:01.12387678Z","size":2489895540,"digest":"970df2db7db7aa5ee240462d488302ea718ce3c3af53e3f5b453a350efc661d2","details":{"parent_model":"","format":"gguf","family":"gemma3","families":["gemma3"],"parameter_size":"3.9B","quantization_level":"Q4_K_M"}},{"name":"medgemma:27b","model":"medgemma:27b","modified_at":"2025-07-22T01:30:28.312153846Z","size":16546405712,"digest":"cbf55bbb8357ef1021c310452827ebbf3a377b3e0fe331a8ebf7e57993a2fc33","details":{"parent_model":"","format":"gguf","family":"gemma3","families":["gemma3"],"parameter_size":"27.0B","quantization_level":"Q4_K_M"}},{"name":"qwen3:30b-a3b","model":"qwen3:30b-a3b","modified_at":"2025-07-21T20:26:15.0275801Z","size":18622563179,"digest":"0b28110b7a3364296d5cef609095537189fbb81547761de69fd22dd68d2166c2","details":{"parent_model":"","format":"gguf","family":"qwen3moe","families":["qwen3moe"],"parameter_size":"30.5B","quantization_level":"Q4_K_M"}},{"name":"mistral:7b-instruct","model":"mistral:7b-instruct","modified_at":"2025-07-21T20:26:14.030577929Z","size":4372824384,"digest":"6577803aa9a036369e481d648a2baebb381ebc6e897f2bb9a766a2aa7bfbc1cf","details":{"parent_model":"","format":"gguf","family":"llama","families":["llama"],"parameter_size":"7.2B","quantization_level":"Q4_K_M"}},{"name":"nomic-embed-text:v1.5","model":"nomic-embed-text:v1.5","modified_at":"2025-07-21T20:26:14.779579558Z","size":274302450,"digest":"0a109f422b47e3a30ba2b10eca18548e944e8a23073ee3f3e947efcf3c45e59f","details":{"parent_model":"","format":"gguf","family":"nomic-bert","families":["nomic-bert"],"parameter_size":"137M","quantization_level":"F16"}},{"name":"devstral:latest","model":"devstral:latest","modified_at":"2025-07-21T20:26:14.25957843Z","size":14333928046,"digest":"9bd74193e93935e9d8564d88607b220a9d341c4a36b748cffcbd9ad4f47a9ca9","details":{"parent_model":"","format":"gguf","family":"llama","families":["llama"],"parameter_size":"23.6B","quantization_level":"Q4_K_M"}},{"name":"gemma3:27b-it-qat","model":"gemma3:27b-it-qat","modified_at":"2025-07-21T20:26:14.526579013Z","size":18087370898,"digest":"29eb0b9aeda35295ed728124d341b27e0c6771ea5c586fcabfb157884224fa93","details":{"parent_model":"","format":"gguf","family":"gemma3","families":["gemma3"],"parameter_size":"27.4B","quantization_level":"Q4_0"}}]}

and concludes

Perfect! The curl command works, showing that network connectivity from the container to your
Ollama server is fine. This confirms the issue is specifically in the AnythingLLM code changes
between versions 1.8.3 and 1.8.4.

So, I changed the environment variable in docker-compose.yml from OLLAMA_BASE_PATH=http://host.docker.internal:11434
to
OLLAMA_BASE_PATH=http://192.168.30.211:11434
and 1.8.4 now works.
So something changed in 1.8.3 to 1.8.4 on how it deals with the base path vs. me updating the path from the gui.
So that makes the GUI option to change it sort of useless but I know how to make it work now. So, I would now say the bug is that the GUI option to change the source for Ollama doesn't work like it used to.

@jhsmith409 commented on GitHub (Jul 23, 2025): Claude suggested this test: docker exec -it anythingllm curl http://192.168.30.211:11434/api/tags {"models":[{"name":"alibayram/medgemma:4b","model":"alibayram/medgemma:4b","modified_at":"2025-07-22T01:42:01.12387678Z","size":2489895540,"digest":"970df2db7db7aa5ee240462d488302ea718ce3c3af53e3f5b453a350efc661d2","details":{"parent_model":"","format":"gguf","family":"gemma3","families":["gemma3"],"parameter_size":"3.9B","quantization_level":"Q4_K_M"}},{"name":"medgemma:27b","model":"medgemma:27b","modified_at":"2025-07-22T01:30:28.312153846Z","size":16546405712,"digest":"cbf55bbb8357ef1021c310452827ebbf3a377b3e0fe331a8ebf7e57993a2fc33","details":{"parent_model":"","format":"gguf","family":"gemma3","families":["gemma3"],"parameter_size":"27.0B","quantization_level":"Q4_K_M"}},{"name":"qwen3:30b-a3b","model":"qwen3:30b-a3b","modified_at":"2025-07-21T20:26:15.0275801Z","size":18622563179,"digest":"0b28110b7a3364296d5cef609095537189fbb81547761de69fd22dd68d2166c2","details":{"parent_model":"","format":"gguf","family":"qwen3moe","families":["qwen3moe"],"parameter_size":"30.5B","quantization_level":"Q4_K_M"}},{"name":"mistral:7b-instruct","model":"mistral:7b-instruct","modified_at":"2025-07-21T20:26:14.030577929Z","size":4372824384,"digest":"6577803aa9a036369e481d648a2baebb381ebc6e897f2bb9a766a2aa7bfbc1cf","details":{"parent_model":"","format":"gguf","family":"llama","families":["llama"],"parameter_size":"7.2B","quantization_level":"Q4_K_M"}},{"name":"nomic-embed-text:v1.5","model":"nomic-embed-text:v1.5","modified_at":"2025-07-21T20:26:14.779579558Z","size":274302450,"digest":"0a109f422b47e3a30ba2b10eca18548e944e8a23073ee3f3e947efcf3c45e59f","details":{"parent_model":"","format":"gguf","family":"nomic-bert","families":["nomic-bert"],"parameter_size":"137M","quantization_level":"F16"}},{"name":"devstral:latest","model":"devstral:latest","modified_at":"2025-07-21T20:26:14.25957843Z","size":14333928046,"digest":"9bd74193e93935e9d8564d88607b220a9d341c4a36b748cffcbd9ad4f47a9ca9","details":{"parent_model":"","format":"gguf","family":"llama","families":["llama"],"parameter_size":"23.6B","quantization_level":"Q4_K_M"}},{"name":"gemma3:27b-it-qat","model":"gemma3:27b-it-qat","modified_at":"2025-07-21T20:26:14.526579013Z","size":18087370898,"digest":"29eb0b9aeda35295ed728124d341b27e0c6771ea5c586fcabfb157884224fa93","details":{"parent_model":"","format":"gguf","family":"gemma3","families":["gemma3"],"parameter_size":"27.4B","quantization_level":"Q4_0"}}]} and concludes Perfect! The curl command works, showing that network connectivity from the container to your Ollama server is fine. This confirms the issue is specifically in the AnythingLLM code changes between versions 1.8.3 and 1.8.4. So, I changed the environment variable in docker-compose.yml from OLLAMA_BASE_PATH=http://host.docker.internal:11434 to OLLAMA_BASE_PATH=http://192.168.30.211:11434 and 1.8.4 now works. So something changed in 1.8.3 to 1.8.4 on how it deals with the base path vs. me updating the path from the gui. So that makes the GUI option to change it sort of useless but I know how to make it work now. So, I would now say the bug is that the GUI option to change the source for Ollama doesn't work like it used to.
yindo commented 2026-02-22 18:30:42 -05:00
Author
Owner
Copy Link
Copy Source

@jhsmith409 commented on GitHub (Jul 23, 2025):

Thank you for looking at this Tim.

@jhsmith409 commented on GitHub (Jul 23, 2025): Thank you for looking at this Tim.
yindo commented 2026-02-22 18:30:42 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Jul 28, 2025):

It seems like these kind of Docker Networking things never rest 😢 - we have this doc:
https://docs.anythingllm.com/ollama-connection-troubleshooting

Which is tangentially related since it has to do with resolution. I really dont think we changed anything about the connector itself at all between 1.8.3 -> 1.8.4, but without a deep review it would be hard to know for certain. My hunch comes from the

extra_hosts:
- "host.docker.internal:host-gateway"

In combination with:
- OLLAMA_BASE_PATH=http://host.docker.internal:11434/ in the ENV doing some kind of non-transparent URL resolution. Thank you for taking the initivative to solve the issue - it helps a ton at least so people can find issues that might mimic their exact setups - since what you are doing is by no means an edge case.

@timothycarambat commented on GitHub (Jul 28, 2025): It seems like these kind of Docker Networking things never rest 😢 - we have this doc: https://docs.anythingllm.com/ollama-connection-troubleshooting Which is _tangentially_ related since it has to do with resolution. I really dont think we changed anything about the connector itself at all between 1.8.3 -> 1.8.4, but without a deep review it would be hard to know for certain. My hunch comes from the ``` extra_hosts: - "host.docker.internal:host-gateway" ``` In combination with: `- OLLAMA_BASE_PATH=http://host.docker.internal:11434/` in the ENV doing some kind of non-transparent URL resolution. Thank you for taking the initivative to solve the issue - it helps a ton at least so people can find issues that might mimic their exact setups - since what you are doing is by no means an edge case.
yindo commented 2026-02-22 18:30:42 -05:00
Author
Owner
Copy Link
Copy Source

@skrantz7104 commented on GitHub (Oct 7, 2025):

Had the same problem not able to use dropdown and choose....
I changed to http://127.0.0.1:11434 and it worked.

@skrantz7104 commented on GitHub (Oct 7, 2025): Had the same problem not able to use dropdown and choose.... I changed to http://127.0.0.1:11434 and it worked.
yindo changed title from [BUG]: Can't select models in external Ollama server to [GH-ISSUE #4190] [BUG]: Can't select models in external Ollama server 2026-06-05 14:47:51 -04:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Desktop
Docker
Integration Request
Integration Request
OS: Linux
OS: Mobile
OS: Windows
UI/UX
blocked
bug
bug
core-team-only
documentation
duplicate
embed-widget
enhancement
feature request
github_actions
good first issue
investigating
needs info / can't replicate
possible bug
pull-request
Mirrored from GitHub Pull Request
 question
stage: specifications
wontfix
No Label investigating needs info / can't replicate possible bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Mintplex-Labs/anything-llm#2666
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?