Mintplex-Labs/anything-llm
1
0
Fork 0
mirror of https://github.com/Mintplex-Labs/anything-llm.git synced 2026-07-19 14:13:35 -04:00
Code Issues 328 Packages Projects Releases 10 Wiki Activity

[GH-ISSUE #5510] [BUG]: On initial start, UI config of EMBEDDING_ENGINE is not working properly with ollama #5137

New Issue
Closed
opened 2026-06-05 14:52:13 -04:00 by yindo · 5 comments
yindo commented 2026-06-05 14:52:13 -04:00
Owner
Copy Link
Copy Source

Originally created by @vdmitriyev on GitHub (Apr 24, 2026).
Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/5510

How are you running AnythingLLM?

Docker (local)

What happened?

After initial deployment of the anything-llm with Docker and Docker Compose, it is not possible to change embedding engine using web UI. After a workaround has been applied (see below), the embedding with ollama start to work again.

Are there known steps to reproduce?

Create compose.yaml with ollama and anythingllm

Build

docker compose build --no-cache

Run

docker compose up -d

Check logs for VectorDB::LanceDb:

docker compose logs -f

LLM Provider & Model (if applicable)

  - OLLAMA_BASE_PATH=http://ollama:11434
  - OLLAMA_MODEL_PREF=llama3.3:latest
  - EMBEDDING_ENGINE=ollama

Embedder Provider & Model (if applicable)

Ollama with all-minilm:l6-v2

Workaround to overcome the bug

Use the following configuration, to set the correct ollama (in this case ollama runs in the same compose namespace as anythingllm)

service:

  ollama:
    image: ollama/ollama:latest
    container_name: ollama
    volumes:
      -./ollama_data:/root/.ollama
    ports:
      - "127.0.0.1:11434:11434"
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: all
              capabilities: [gpu]
  
  rag-app:
    image: mintplexlabs/anythingllm:latest
    container_name: anythingllm
    environment:
      - STORAGE_DIR=/app/server/storage
      - LLM_PROVIDER=ollama
      - OLLAMA_BASE_PATH=http://ollama:11434
      - OLLAMA_MODEL_PREF=llama3.3:latest
      - EMBEDDING_ENGINE=ollama
      - EMBEDDING_BASE_PATH=http://ollama:11434
      - EMBEDDING_MODEL_PREF=all-minilm:l6-v2
      - EMBEDDING_MODEL_MAX_CHUNK_LENGTH=8192
    volumes:
      - ./anythingllm_storage:/app/server/storage
    ports:
      - "127.0.0.1:3001:3001"
    depends_on:
      - ollama
Originally created by @vdmitriyev on GitHub (Apr 24, 2026). Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/5510 ### How are you running AnythingLLM? Docker (local) ### What happened? After initial deployment of the `anything-llm` with Docker and Docker Compose, it is not possible to change embedding engine using web UI. After a workaround has been applied (see below), the embedding with ollama start to work again. ### Are there known steps to reproduce? Create `compose.yaml` with ollama and `anythingllm` Build ``` docker compose build --no-cache ``` Run ``` docker compose up -d ``` Check logs for `VectorDB::LanceDb`: ``` docker compose logs -f ``` ### LLM Provider & Model (if applicable) ```env - OLLAMA_BASE_PATH=http://ollama:11434 - OLLAMA_MODEL_PREF=llama3.3:latest - EMBEDDING_ENGINE=ollama ``` ### Embedder Provider & Model (if applicable) Ollama with `all-minilm:l6-v2` ### Workaround to overcome the bug Use the following configuration, to set the correct ollama (in this case ollama runs in the same compose namespace as anythingllm) ```yaml service: ollama: image: ollama/ollama:latest container_name: ollama volumes: -./ollama_data:/root/.ollama ports: - "127.0.0.1:11434:11434" deploy: resources: reservations: devices: - driver: nvidia count: all capabilities: [gpu] rag-app: image: mintplexlabs/anythingllm:latest container_name: anythingllm environment: - STORAGE_DIR=/app/server/storage - LLM_PROVIDER=ollama - OLLAMA_BASE_PATH=http://ollama:11434 - OLLAMA_MODEL_PREF=llama3.3:latest - EMBEDDING_ENGINE=ollama - EMBEDDING_BASE_PATH=http://ollama:11434 - EMBEDDING_MODEL_PREF=all-minilm:l6-v2 - EMBEDDING_MODEL_MAX_CHUNK_LENGTH=8192 volumes: - ./anythingllm_storage:/app/server/storage ports: - "127.0.0.1:3001:3001" depends_on: - ollama ```
yindo added the possible bug label 2026-06-05 14:52:13 -04:00
yindo closed this issue 2026-06-05 14:52:13 -04:00
yindo commented 2026-06-05 14:52:14 -04:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Apr 24, 2026):

After initial deployment of the anything-llm with Docker and Docker Compose, it is not possible to change embedding engine using web UI. After a workaround has been applied (see below), the embedding with ollama start to work again.

How is that possible? You should be able to just go to the settings/embedding-preference path. If the model selection dropdown is not showing models it is because http://ollama:11434 does not work in the anythingllm container becuase it would need that network layer.

To use ollama container<>container https://docs.anythingllm.com/ollama-connection-troubleshooting#localhost-and-127001-do-not-work-on-docker

<!-- gh-comment-id:4315931755 --> @timothycarambat commented on GitHub (Apr 24, 2026): > After initial deployment of the anything-llm with Docker and Docker Compose, it is not possible to change embedding engine using web UI. After a workaround has been applied (see below), the embedding with ollama start to work again. How is that possible? You should be able to just go to the `settings/embedding-preference` path. If the model selection dropdown is not showing models it is because `http://ollama:11434` does not work in the anythingllm container becuase it would need that network layer. To use ollama container<>container https://docs.anythingllm.com/ollama-connection-troubleshooting#localhost-and-127001-do-not-work-on-docker
yindo commented 2026-06-05 14:52:14 -04:00
Author
Owner
Copy Link
Copy Source

@vdmitriyev commented on GitHub (Apr 26, 2026):

After initial deployment of the anything-llm with Docker and Docker Compose, it is not possible to change embedding engine using web UI. After a workaround has been applied (see below), the embedding with ollama start to work again.

How is that possible? You should be able to just go to the settings/embedding-preference path. If the model selection dropdown is not showing models it is because http://ollama:11434 does not work in the anythingllm container becuase it would need that network layer.

To use ollama container<>container https://docs.anythingllm.com/ollama-connection-troubleshooting#localhost-and-127001-do-not-work-on-docker

Thanks for your prompt reply and direct links to the documentation. The problem is not UI on its own, UI works fine and lets you select and configure ollama as backend and pick the right model (dropdown is populated after the request has been send to a newly configurated ollama). Thus, all these changes are seen in UI and stored somewhere. However, these configuration and param changes do not effect calls of the embedding engine. So, when the embedding engine starts to perform calls to the configured via UI embedding engine backend, it fails to find the newly configured one. The bug is hard to spot directly, because the default model for the embedding engine would be pulled and used in case of misconfiguration.

A workaround, which sets these parameters directly on the start via environmental variables works fine.

An assumption would be that there is a issues with not overtaking/overriding the parameters coming from the web UI, while calls are made to the embedding engine.

UPDATE:

In this source file, which is most probably related to the issue, the values are taken directly from the environment variables. Therefore, if the environment variables are not overridden at the start of the process, then ignoring the configuration from the web UI is highly likely.
server/utils/EmbeddingEngines/ollama/index.js

Important note: I have limited knowledge of the source code, so I'm not sure how re-reading the environmental variables in this project are happening and how these updates are passed around.

@timothycarambat just pinging you here, so could see assumption about the bug above

<!-- gh-comment-id:4322531011 --> @vdmitriyev commented on GitHub (Apr 26, 2026): > > After initial deployment of the anything-llm with Docker and Docker Compose, it is not possible to change embedding engine using web UI. After a workaround has been applied (see below), the embedding with ollama start to work again. > > How is that possible? You should be able to just go to the `settings/embedding-preference` path. If the model selection dropdown is not showing models it is because `http://ollama:11434` does not work in the anythingllm container becuase it would need that network layer. > > To use ollama container<>container https://docs.anythingllm.com/ollama-connection-troubleshooting#localhost-and-127001-do-not-work-on-docker Thanks for your prompt reply and direct links to the documentation. The problem is not UI on its own, UI works fine and lets you select and configure ollama as backend and pick the right model (dropdown is populated after the request has been send to a newly configurated ollama). Thus, all these changes are seen in UI and stored somewhere. However, these configuration and param changes do not effect calls of the `embedding engine`. So, when the `embedding engine` starts to perform calls to the configured via UI embedding engine backend, it fails to find the newly configured one. The bug is hard to spot directly, because the default model for the embedding engine would be pulled and used in case of misconfiguration. A workaround, which sets these parameters directly on the start via environmental variables works fine. An assumption would be that there is a issues with not overtaking/overriding the parameters coming from the web UI, while calls are made to the `embedding engine`. UPDATE: > In this source file, which is most probably related to the issue, the values are taken directly from the environment variables. Therefore, if the environment variables are not overridden at the start of the process, then ignoring the configuration from the web UI is highly likely. > `server/utils/EmbeddingEngines/ollama/index.js` > **Important note**: I have limited knowledge of the source code, so I'm not sure how re-reading the environmental variables in this project are happening and how these updates are passed around. @timothycarambat just pinging you here, so could see assumption about the bug above
yindo commented 2026-06-05 14:52:14 -04:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Apr 28, 2026):

@vdmitriyev So to be absolutely clear on the scope of the bug the way to reproduce this is:

  • Clean install
  • Configure Ollama as LLM on onboarding
  • After onboarding, go to Embedder, select Ollama, using same connection details as LLM now pick an embedding model
  • Upload a document for RAG
  • Note that the embedding engine does not use my ollama config and instead uses another.
  • Therefore, my only solution is to set this via ENV directly in config.yaml

Am I understanding this bug correctly? I think the issue is much more simple to solve!

rag-app:
    image: mintplexlabs/anythingllm:latest
    container_name: anythingllm
    environment:
      - STORAGE_DIR=/app/server/storage
      # You dont specify these, they are handled via the /app/server/storage/.env file!
      # - LLM_PROVIDER=ollama
      # - OLLAMA_BASE_PATH=http://ollama:11434
      # - OLLAMA_MODEL_PREF=llama3.3:latest
      # - EMBEDDING_ENGINE=ollama
      # - EMBEDDING_BASE_PATH=http://ollama:11434
      # - EMBEDDING_MODEL_PREF=all-minilm:l6-v2
      # - EMBEDDING_MODEL_MAX_CHUNK_LENGTH=8192
    volumes:
      - ./anythingllm_storage:/app/server/storage
      - ./anythingllm_storage:/app/server/storage/.env:/app/server/.env  # <-- Missing (should pre-exist on start)
    ports:
      - "127.0.0.1:3001:3001"
    depends_on:
      - ollama

Without the above YAML changes, your docker ENV is colliding with the attempted to be loaded file-based ENV and its leading to this weird conflict. Likely the docker ENV override the file and take precedence.

<!-- gh-comment-id:4337615723 --> @timothycarambat commented on GitHub (Apr 28, 2026): @vdmitriyev So to be absolutely clear on the scope of the bug the way to reproduce this is: - Clean install - Configure Ollama as LLM on onboarding - After onboarding, go to Embedder, select Ollama, using same connection details as LLM now pick an embedding model - Upload a document for RAG - Note that the embedding engine **does not** use my ollama config and instead uses another. - Therefore, my only solution is to set this via ENV directly in config.yaml Am I understanding this bug correctly? I think the issue is much more simple to solve! ```yaml rag-app: image: mintplexlabs/anythingllm:latest container_name: anythingllm environment: - STORAGE_DIR=/app/server/storage # You dont specify these, they are handled via the /app/server/storage/.env file! # - LLM_PROVIDER=ollama # - OLLAMA_BASE_PATH=http://ollama:11434 # - OLLAMA_MODEL_PREF=llama3.3:latest # - EMBEDDING_ENGINE=ollama # - EMBEDDING_BASE_PATH=http://ollama:11434 # - EMBEDDING_MODEL_PREF=all-minilm:l6-v2 # - EMBEDDING_MODEL_MAX_CHUNK_LENGTH=8192 volumes: - ./anythingllm_storage:/app/server/storage - ./anythingllm_storage:/app/server/storage/.env:/app/server/.env # <-- Missing (should pre-exist on start) ports: - "127.0.0.1:3001:3001" depends_on: - ollama ``` Without the above YAML changes, your docker ENV is colliding with the attempted to be loaded file-based ENV and its leading to this weird conflict. Likely the docker ENV override the file and take precedence.
yindo commented 2026-06-05 14:52:14 -04:00
Author
Owner
Copy Link
Copy Source

@vdmitriyev commented on GitHub (Apr 28, 2026):

Thank you for your response and for providing me with the modified compose.yaml file.
The bug summary is correct - this is exactly what is happening..

I have tried to run your version of compose.yaml, but it looks line there is an issue with the line:

 - ./anythingllm_storage:/app/server/storage/.env:/app/server/.env  # <-- Missing (should pre-exist on start)

There is an issue with linking proper .env file, thus following error happens:

error during container init: error mounting
...
...merged/app/server/storage/.env": not a directory: unknown: 
Are you trying to mount a directory onto a file (or vice-versa)? 
Check if the specified host path exists and is the expected type

To fix the problem with the compose.yaml, the line must be adjusted:

rag-app:
    image: mintplexlabs/anythingllm:latest
    container_name: anythingllm
    environment:
      - STORAGE_DIR=/app/server/storage
      # Dont specify any envs here, they are handled via the /app/server/storage/.env file!
    volumes:
      - ./anythingllm_storage:/app/server/storage
      - ./anythingllm_storage/.env:/app/server/.env # fixed here
    ports:
      - "127.0.0.1:3001:3001"
    depends_on:
      - ollama

Basically, one MUST use an .env file from the very beginning, so the applied will be consistent.

NOTE: The original .env file will be modified by anythingllm on the first start (e.g., SIG_KEY and SIG_SALT will be added to it.

@timothycarambat thanks again for the help, I really appreciate it 🚀 👍

<!-- gh-comment-id:4338103591 --> @vdmitriyev commented on GitHub (Apr 28, 2026): Thank you for your response and for providing me with the modified `compose.yaml` file. The bug summary is correct - this is exactly what is happening.. I have tried to run your version of `compose.yaml`, but it looks line there is an issue with the line: ```yaml - ./anythingllm_storage:/app/server/storage/.env:/app/server/.env # <-- Missing (should pre-exist on start) ``` There is an issue with linking proper `.env` file, thus following error happens: ```txt error during container init: error mounting ... ...merged/app/server/storage/.env": not a directory: unknown: Are you trying to mount a directory onto a file (or vice-versa)? Check if the specified host path exists and is the expected type ``` To fix the problem with the `compose.yaml`, the line must be adjusted: ```yaml rag-app: image: mintplexlabs/anythingllm:latest container_name: anythingllm environment: - STORAGE_DIR=/app/server/storage # Dont specify any envs here, they are handled via the /app/server/storage/.env file! volumes: - ./anythingllm_storage:/app/server/storage - ./anythingllm_storage/.env:/app/server/.env # fixed here ports: - "127.0.0.1:3001:3001" depends_on: - ollama ``` Basically, one MUST use an `.env` file from the very beginning, so the applied will be consistent. NOTE: The original `.env` file will be modified by `anythingllm` on the first start (e.g., `SIG_KEY` and `SIG_SALT` will be added to it. @timothycarambat thanks again for the help, I really appreciate it 🚀 👍
yindo commented 2026-06-05 14:52:14 -04:00
Author
Owner
Copy Link
Copy Source

@vdmitriyev commented on GitHub (Apr 28, 2026):

Related to the issue documentation can be found here - https://docs.anythingllm.com/installation-docker/quickstart#start-anythingllm-via-docker

But make sure, you will create and modify .env file BEFORE the start

<!-- gh-comment-id:4338117205 --> @vdmitriyev commented on GitHub (Apr 28, 2026): Related to the issue documentation can be found here - https://docs.anythingllm.com/installation-docker/quickstart#start-anythingllm-via-docker But make sure, you will create and modify `.env` file BEFORE the start
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Desktop
Docker
Integration Request
Integration Request
OS: Linux
OS: Mobile
OS: Windows
UI/UX
blocked
bug
bug
core-team-only
documentation
duplicate
embed-widget
enhancement
feature request
github_actions
good first issue
investigating
needs info / can't replicate
possible bug
pull-request
Mirrored from GitHub Pull Request
 question
stage: specifications
wontfix
No Label possible bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Mintplex-Labs/anything-llm#5137
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?