Mintplex-Labs/anything-llm
1
0
Fork 0
mirror of https://github.com/Mintplex-Labs/anything-llm.git synced 2026-07-19 14:13:35 -04:00
Code Issues 328 Packages Projects Releases 10 Wiki Activity

[GH-ISSUE #4810] [BUG]: Ollama Embedder falls back to Native Embedder despite correct configuration #3030

New Issue
Closed
opened 2026-02-22 18:32:19 -05:00 by yindo · 2 comments
yindo commented 2026-02-22 18:32:19 -05:00
Owner
Copy Link
Copy Source

Originally created by @torstenzenk on GitHub (Dec 25, 2025).
Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/4810

How are you running AnythingLLM?

Docker (local)

What happened?

Problem Description

AnythingLLM claims to initialize OllamaEmbedder but actually uses the Native Embedder for embedding operations. This happens despite correct database configuration and environment variables.

Expected Behavior

When EmbeddingEngine is set to ollama, AnythingLLM should:

  1. Use Ollama API at http://localhost:11434/api/embed
  2. Use model nomic-embed-text:latest
  3. NOT fall back to Native Embedder

Actual Behavior

Logs show initialization with Ollama but execution with Native Embedder:

[backend] info: [OllamaEmbedder] initialized with model nomic-embed-text:latest at http://localhost:11434. Batch size: 1, num_ctx: 8192
[backend] info: [OllamaEmbedder] Embedding 3 chunks of text with nomic-embed-text:latest in batches of 1.
[backend] info: [OllamaEmbedder] do embedding request: Post "http://127.0.0.1:35179/embedding": EOF
[backend] error: addDocumentToNamespace Ollama Failed to embed: do embedding request: Post "http://127.0.0.1:35179/embedding": EOF

Note the discrepancy:

  • Initializes: http://localhost:11434 ✅ (Ollama)
  • Executes: http://127.0.0.1:35179/embedding ❌ (Native Embedder)

Port 35179 is ephemeral Native Embedder port, endpoint /embedding (not Ollama's /api/embed)

Configuration

Database state (verified):

sqlite> SELECT label, value FROM system_settings WHERE label LIKE 'Embedding%';
EmbeddingBasePath|http://localhost:11434
EmbeddingEngine|ollama
EmbeddingModelMaxChunkLength|8192
EmbeddingModelPref|nomic-embed-text:latest

docker-compose.yml:

version: "3.8"
services:
  anythingllm:
    image: mintplexlabs/anythingllm:latest
    network_mode: host
    environment:
      - EMBEDDING_ENGINE=ollama
      - EMBEDDING_BASE_PATH=http://localhost:11434
      - EMBEDDING_MODEL_PREF=nomic-embed-text:latest
      - EMBEDDING_MODEL_MAX_CHUNK_LENGTH=8192
      - VECTOR_DB=qdrant
      - QDRANT_ENDPOINT=http://localhost:6333
    volumes:
      - /var/lib/anythingllm/storage:/app/server/storage

Ollama verification (works!):

docker exec anythingllm curl http://localhost:11434/api/embed \
  -d '{"model":"nomic-embed-text:latest","input":"test"}'
# Returns: {"model":"nomic-embed-text:latest","embeddings":[[0.123,...]]}

Reproduction Steps

  1. Set up Ollama on host (systemd service on port 11434)
  2. Pull model: ollama pull nomic-embed-text:latest
  3. Configure AnythingLLM with EMBEDDING_ENGINE=ollama via environment AND database
  4. Verify database has correct settings
  5. Restart container: docker compose restart
  6. Try to embed any document
  7. Check logs - shows initialization with Ollama but execution with Native Embedder

Additional Context

Additional Context

Configuration attempts made (ALL ignored):

  1. Environment Variables in docker-compose.yml:

    • Set EMBEDDING_ENGINE=ollama, EMBEDDING_BASE_PATH, etc.
    • Container restarted → Still uses Native Embedder

  2. UI Settings:

    • Configured via Settings → Embedding → Provider: Ollama
    • Clicked "Save Changes"
    • Settings NOT persisted to database! (separate bug?)
    • Container restarted → Still uses Native Embedder

  3. Manual Database INSERT:

    • Directly wrote to SQLite database:
   INSERT INTO system_settings (label, value, ...) 
   VALUES ('EmbeddingEngine', 'ollama', ...);
   -- Same for EmbeddingBasePath, EmbeddingModelPref, etc.
  • Verified with SELECT - values correctly in database
  • Container FULLY restarted with docker compose down && docker compose up -d
  • Still uses Native Embedder!

This means:

  • Environment variables are ignored ❌
  • UI settings don't persist ❌
  • Manual database entries are ignored ❌
  • Even after complete container rebuild, wrong embedder is used ❌

Success case

Only ONE document (Erinnerungen_Chatgpt.txt) embedded successfully once with Ollama. All subsequent documents fail and use Native Embedder. Cannot reproduce the success.

Network verification

  • Ollama is reachable from container: curl http://localhost:11434/api/tags works
  • Qdrant is reachable: curl http://localhost:6333/collections works
  • Container uses network_mode: host - no network isolation

Conclusion

This is NOT a configuration or caching issue. The code path for embedding execution is fundamentally broken and ignores all configuration sources (env, database, UI).

Possible Root Cause

The initialization code reads from database/environment correctly and logs "OllamaEmbedder initialized", but the actual embedding execution uses a different code path that falls back to Native Embedder (possibly cached instance or hardcoded fallback).

System Info

  • OS: Arch Linux (CachyOS)
  • Ollama: systemd service on host:11434
  • Vector DB: Qdrant (Docker)
  • Docker Network: host mode

Are there known steps to reproduce?

No response

Originally created by @torstenzenk on GitHub (Dec 25, 2025). Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/4810 ### How are you running AnythingLLM? Docker (local) ### What happened? ## Problem Description AnythingLLM claims to initialize OllamaEmbedder but actually uses the Native Embedder for embedding operations. This happens despite correct database configuration and environment variables. ## Expected Behavior When `EmbeddingEngine` is set to `ollama`, AnythingLLM should: 1. Use Ollama API at `http://localhost:11434/api/embed` 2. Use model `nomic-embed-text:latest` 3. NOT fall back to Native Embedder ## Actual Behavior Logs show initialization with Ollama but execution with Native Embedder: ```log [backend] info: [OllamaEmbedder] initialized with model nomic-embed-text:latest at http://localhost:11434. Batch size: 1, num_ctx: 8192 [backend] info: [OllamaEmbedder] Embedding 3 chunks of text with nomic-embed-text:latest in batches of 1. [backend] info: [OllamaEmbedder] do embedding request: Post "http://127.0.0.1:35179/embedding": EOF [backend] error: addDocumentToNamespace Ollama Failed to embed: do embedding request: Post "http://127.0.0.1:35179/embedding": EOF ``` **Note the discrepancy:** - Initializes: `http://localhost:11434` ✅ (Ollama) - Executes: `http://127.0.0.1:35179/embedding` ❌ (Native Embedder) Port 35179 is ephemeral Native Embedder port, endpoint `/embedding` (not Ollama's `/api/embed`) ## Configuration **Database state (verified):** ```sql sqlite> SELECT label, value FROM system_settings WHERE label LIKE 'Embedding%'; EmbeddingBasePath|http://localhost:11434 EmbeddingEngine|ollama EmbeddingModelMaxChunkLength|8192 EmbeddingModelPref|nomic-embed-text:latest ``` **docker-compose.yml:** ```yaml version: "3.8" services: anythingllm: image: mintplexlabs/anythingllm:latest network_mode: host environment: - EMBEDDING_ENGINE=ollama - EMBEDDING_BASE_PATH=http://localhost:11434 - EMBEDDING_MODEL_PREF=nomic-embed-text:latest - EMBEDDING_MODEL_MAX_CHUNK_LENGTH=8192 - VECTOR_DB=qdrant - QDRANT_ENDPOINT=http://localhost:6333 volumes: - /var/lib/anythingllm/storage:/app/server/storage ``` **Ollama verification (works!):** ```bash docker exec anythingllm curl http://localhost:11434/api/embed \ -d '{"model":"nomic-embed-text:latest","input":"test"}' # Returns: {"model":"nomic-embed-text:latest","embeddings":[[0.123,...]]} ``` ## Reproduction Steps 1. Set up Ollama on host (systemd service on port 11434) 2. Pull model: `ollama pull nomic-embed-text:latest` 3. Configure AnythingLLM with `EMBEDDING_ENGINE=ollama` via environment AND database 4. Verify database has correct settings 5. Restart container: `docker compose restart` 6. Try to embed any document 7. Check logs - shows initialization with Ollama but execution with Native Embedder ## Additional Context ## Additional Context ### Configuration attempts made (ALL ignored): 1. **Environment Variables in docker-compose.yml:** - Set `EMBEDDING_ENGINE=ollama`, `EMBEDDING_BASE_PATH`, etc. - Container restarted → Still uses Native Embedder 2. **UI Settings:** - Configured via Settings → Embedding → Provider: Ollama - Clicked "Save Changes" - **Settings NOT persisted to database!** (separate bug?) - Container restarted → Still uses Native Embedder 3. **Manual Database INSERT:** - Directly wrote to SQLite database: ```sql INSERT INTO system_settings (label, value, ...) VALUES ('EmbeddingEngine', 'ollama', ...); -- Same for EmbeddingBasePath, EmbeddingModelPref, etc. ``` - Verified with SELECT - values correctly in database - Container FULLY restarted with `docker compose down && docker compose up -d` - **Still uses Native Embedder!** **This means:** - Environment variables are ignored ❌ - UI settings don't persist ❌ - Manual database entries are ignored ❌ - Even after complete container rebuild, wrong embedder is used ❌ ### Success case Only ONE document (`Erinnerungen_Chatgpt.txt`) embedded successfully once with Ollama. All subsequent documents fail and use Native Embedder. Cannot reproduce the success. ### Network verification - Ollama is reachable from container: `curl http://localhost:11434/api/tags` works - Qdrant is reachable: `curl http://localhost:6333/collections` works - Container uses `network_mode: host` - no network isolation ## Conclusion This is NOT a configuration or caching issue. The code path for embedding execution is fundamentally broken and ignores all configuration sources (env, database, UI). ## Possible Root Cause The initialization code reads from database/environment correctly and logs "OllamaEmbedder initialized", but the actual embedding execution uses a different code path that falls back to Native Embedder (possibly cached instance or hardcoded fallback). ## System Info - OS: Arch Linux (CachyOS) - Ollama: systemd service on host:11434 - Vector DB: Qdrant (Docker) - Docker Network: host mode ### Are there known steps to reproduce? _No response_
yindo added the possible bugneeds info / can't replicate labels 2026-02-22 18:32:19 -05:00
yindo closed this issue 2026-02-22 18:32:19 -05:00
yindo commented 2026-02-22 18:32:20 -05:00
Author
Owner
Copy Link
Copy Source

@rogue07 commented on GitHub (Dec 27, 2025):

I get the same error. Once landing on the /settings/branding page the error generates.

[ANY] [backend] error: Error: No embedding base path was set.

Here is my db settings:

================================================================
AnythingLLM Database Settings Viewer

Database location: /home/lucifer/anythingllm/storage/anythingllm.db

================================================================

  1. DATABASE SCHEMA
    ================================================================

System Settings Table Structure:
CREATE TABLE IF NOT EXISTS "system_settings" (
"id" INTEGER NOT NULL PRIMARY KEY AUTOINCREMENT,
"label" TEXT NOT NULL,
"value" TEXT,
"createdAt" DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
"lastUpdatedAt" DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP
);
CREATE UNIQUE INDEX "system_settings_label_key" ON "system_settings"("label");

================================================================
2. ALL SYSTEM SETTINGS (Formatted)

Setting Name Value Last Updated

EmbedderPreference {"provider":"ollama","model":"nomic-embed-text:... 2025-12-27 16:22:31
EmbeddingBasePath http://host.docker.internal:11434 2025-12-27 16:36:29
OllamaBasePath http://host.docker.internal:11434 2025-12-27 16:36:29
agent_skill_complete true 2025-12-27 16:35:45
default_agent_skills ["save-file-to-browser"]
disabled_agent_skills []
embedding_provider ollama 2025-12-27 16:35:45
llm_provider ollama 2025-12-27 16:35:45
meta_page_favicon
meta_page_title Mouseion
multi_user_mode false 2025-12-27 16:35:45
onboarding_complete true 2025-12-27 16:35:45
telemetry_id disabled 2025-12-27 16:35:45
vector_db lancedb 2025-12-27 16:35:45

================================================================
3. SETTINGS BY CATEGORY

--- BRANDING SETTINGS ---

--- OLLAMA SETTINGS ---
OllamaBasePath|http://host.docker.internal:11434

--- EMBEDDING SETTINGS ---
EmbedderPreference|{"provider":"ollama","model":"nomic-embed-text:latest","basePath":"http://host.docker.internal:11434"}
embedding_provider|ollama
EmbeddingBasePath|http://host.docker.internal:11434

--- LLM SETTINGS ---
llm_provider|ollama

--- VECTOR DATABASE SETTINGS ---
vector_db|lancedb

--- USER/AUTH SETTINGS ---
multi_user_mode|false

--- SETUP/ONBOARDING SETTINGS ---
onboarding_complete|true

@rogue07 commented on GitHub (Dec 27, 2025): I get the same error. Once landing on the /settings/branding page the error generates. [ANY] [backend] error: Error: No embedding base path was set. Here is my db settings: ================================================================ AnythingLLM Database Settings Viewer ================================================================ Database location: /home/lucifer/anythingllm/storage/anythingllm.db ================================================================ 1. DATABASE SCHEMA ================================================================ System Settings Table Structure: CREATE TABLE IF NOT EXISTS "system_settings" ( "id" INTEGER NOT NULL PRIMARY KEY AUTOINCREMENT, "label" TEXT NOT NULL, "value" TEXT, "createdAt" DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP, "lastUpdatedAt" DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP ); CREATE UNIQUE INDEX "system_settings_label_key" ON "system_settings"("label"); ================================================================ 2. ALL SYSTEM SETTINGS (Formatted) ================================================================ Setting Name Value Last Updated ------------------------------ -------------------------------------------------- -------------------- EmbedderPreference {"provider":"ollama","model":"nomic-embed-text:... 2025-12-27 16:22:31 EmbeddingBasePath http://host.docker.internal:11434 2025-12-27 16:36:29 OllamaBasePath http://host.docker.internal:11434 2025-12-27 16:36:29 agent_skill_complete true 2025-12-27 16:35:45 default_agent_skills ["save-file-to-browser"] disabled_agent_skills [] embedding_provider ollama 2025-12-27 16:35:45 llm_provider ollama 2025-12-27 16:35:45 meta_page_favicon meta_page_title Mouseion multi_user_mode false 2025-12-27 16:35:45 onboarding_complete true 2025-12-27 16:35:45 telemetry_id disabled 2025-12-27 16:35:45 vector_db lancedb 2025-12-27 16:35:45 ================================================================ 3. SETTINGS BY CATEGORY ================================================================ --- BRANDING SETTINGS --- --- OLLAMA SETTINGS --- OllamaBasePath|http://host.docker.internal:11434 --- EMBEDDING SETTINGS --- EmbedderPreference|{"provider":"ollama","model":"nomic-embed-text:latest","basePath":"http://host.docker.internal:11434"} embedding_provider|ollama EmbeddingBasePath|http://host.docker.internal:11434 --- LLM SETTINGS --- llm_provider|ollama --- VECTOR DATABASE SETTINGS --- vector_db|lancedb --- USER/AUTH SETTINGS --- multi_user_mode|false --- SETUP/ONBOARDING SETTINGS --- onboarding_complete|true
yindo commented 2026-02-22 18:32:20 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Jan 2, 2026):

You issue is the setup is doing this in the compose

 environment:
      - EMBEDDING_ENGINE=ollama
      - EMBEDDING_BASE_PATH=http://localhost:11434
      - EMBEDDING_MODEL_PREF=nomic-embed-text:latest
      - EMBEDDING_MODEL_MAX_CHUNK_LENGTH=8192
      - VECTOR_DB=qdrant
      - QDRANT_ENDPOINT=http://localhost:6333

A valid compose looks like

services:
  anythingllm:
    # Use 'latest' for latest features or specific version tag (see GH releases)
    image: mintplexlabs/anythingllm:1.9.1
    container_name: anythingllm
    restart: always
    
    # Enable SYS_ADMIN capability
    cap_add:
      - SYS_ADMIN

    ports:
      - "3001:3001"

    # Standard storage mapping for persistence
    volumes:
      - /home/username/anythingllm:/app/server/storage
      - /home/username/anythingllm/.env:/app/server/.env
    environment:
      - STORAGE_DIR=/app/server/storage
    extra_hosts:
      - "host.docker.internal:host-gateway"

This binds the local file env to the one we actually use. The only real ENV needed is the STORAGE_DIR, since everything about the system should be in there.

This part:

Note the discrepancy:

Initializes: http://localhost:11434 ✅ (Ollama)
Executes: http://127.0.0.1:35179/embedding ❌ (Native Embedder)

Doesnt make sense, since the local native embedder is a not a ip:port process. It runs in the main server thread as a sub-process so it doesnt have an endpoint or a thread. I dont see where you see

[backend] error: addDocumentToNamespace Ollama Failed to embed: do embedding request: Post "http://127.0.0.1:35179/embedding": EOF

Since that log doesnt make sense as nowhere do we show an error log of do embedding request: Post "http://127.0.0.1:35179/embedding": EOF like that isnt in the codebase. Are you on a fork?

There is also an issue with how you are accessing ollama. Ollama is outside of the container, so localhost cannot work which is why http://127.0.0.1:35179/embedding does not resolve. We have a whole docs page about this https://docs.anythingllm.com/ollama-connection-troubleshooting#troubleshooting-docker

So your Ollama URL, by setting it in the UI, not the compose ENV should be http://host.docker.internal:11434. We use the offical Ollama SDK for inference and embedding and dont print the endpoint to the logs so I am not sure how you know this information.

I just pulled nomic text embed on the latest ollama, set the URL correctly in the UI and embedded a document without issue. What docker tag are you on or are you on a custom fork?

@timothycarambat commented on GitHub (Jan 2, 2026): You issue is the setup is doing this in the compose ``` environment: - EMBEDDING_ENGINE=ollama - EMBEDDING_BASE_PATH=http://localhost:11434 - EMBEDDING_MODEL_PREF=nomic-embed-text:latest - EMBEDDING_MODEL_MAX_CHUNK_LENGTH=8192 - VECTOR_DB=qdrant - QDRANT_ENDPOINT=http://localhost:6333 ``` A valid compose looks like ``` services: anythingllm: # Use 'latest' for latest features or specific version tag (see GH releases) image: mintplexlabs/anythingllm:1.9.1 container_name: anythingllm restart: always # Enable SYS_ADMIN capability cap_add: - SYS_ADMIN ports: - "3001:3001" # Standard storage mapping for persistence volumes: - /home/username/anythingllm:/app/server/storage - /home/username/anythingllm/.env:/app/server/.env environment: - STORAGE_DIR=/app/server/storage extra_hosts: - "host.docker.internal:host-gateway" ``` This binds the local file env to the one we actually use. The only real ENV needed is the STORAGE_DIR, since everything about the system should be in there. This part: > Note the discrepancy: > > Initializes: http://localhost:11434 ✅ (Ollama) > Executes: http://127.0.0.1:35179/embedding ❌ (Native Embedder) Doesnt make sense, since the local native embedder is a not a ip:port process. It runs in the main server thread as a sub-process so it doesnt have an endpoint or a thread. I dont see where you see ``` [backend] error: addDocumentToNamespace Ollama Failed to embed: do embedding request: Post "http://127.0.0.1:35179/embedding": EOF ``` Since that log doesnt make sense as nowhere do we show an error log of `do embedding request: Post "http://127.0.0.1:35179/embedding": EOF` like that isnt in the codebase. Are you on a fork? There is also an issue with how you are accessing ollama. Ollama is outside of the container, so localhost cannot work which is why `http://127.0.0.1:35179/embedding` does not resolve. We have a whole docs page about this https://docs.anythingllm.com/ollama-connection-troubleshooting#troubleshooting-docker So your Ollama URL, by setting it in the UI, not the compose ENV should be `http://host.docker.internal:11434`. We use the offical Ollama SDK for[ inference and embedding](https://github.com/Mintplex-Labs/anything-llm/blob/e287fab56089cf8fcea9ba579a3ecdeca0daa313/server/utils/EmbeddingEngines/ollama/index.js#L22) and dont print the endpoint to the logs so I am not sure how you know this information. I just pulled nomic text embed on the latest ollama, set the URL correctly in the UI and embedded a document without issue. What docker tag are you on or are you on a custom fork?
yindo changed title from [BUG]: Ollama Embedder falls back to Native Embedder despite correct configuration to [GH-ISSUE #4810] [BUG]: Ollama Embedder falls back to Native Embedder despite correct configuration 2026-06-05 14:49:54 -04:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Desktop
Docker
Integration Request
Integration Request
OS: Linux
OS: Mobile
OS: Windows
UI/UX
blocked
bug
bug
core-team-only
documentation
duplicate
embed-widget
enhancement
feature request
github_actions
good first issue
investigating
needs info / can't replicate
possible bug
pull-request
Mirrored from GitHub Pull Request
 question
stage: specifications
wontfix
No Label needs info / can't replicate possible bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Mintplex-Labs/anything-llm#3030
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?