Mintplex-Labs/anything-llm
1
0
Fork 0
mirror of https://github.com/Mintplex-Labs/anything-llm.git synced 2026-07-19 14:13:35 -04:00
Code Issues 328 Packages Projects Releases 10 Wiki Activity

[GH-ISSUE #3122] [FEAT]: num_ctx for Ollama embedder #2003

New Issue
Closed
opened 2026-02-22 18:27:42 -05:00 by yindo · 1 comment
yindo commented 2026-02-22 18:27:42 -05:00
Owner
Copy Link
Copy Source

Originally created by @alpilotx on GitHub (Feb 5, 2025).
Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/3122

Originally assigned to: @timothycarambat on GitHub.

How are you running AnythingLLM?

Docker (remote machine)

What happened?

I have started a new document embedding run and wanted to have a look on the amchine running ollama, how it uses cores and also check which parameters have been used to start ollama process.

Then I saw this:
Image
/usr/lib/ollama/runners/cpu_avx2/ollama_llama_server runner --model /root/.ollama/models/blobs/sha256-daec91ffb5dd0c27411bd71f29932917c49cf529a641d0168496c3a501e3062c --ctx-size 2048 --batch-size 512 --threads 64 --no-mmap --parallel 1

And I was surprised to see --ctx-size 2048 !

Because the embedding model I use is bge-m3 and for one it supports larger context windows AND I also explicitly told in the configuration to have "Max Embedding Chunk Length=4096"
Image

And also I let my documents be splitted to have up to 4096 tokens per chunk.
But by having Ollama run with --ctx-size 2048 , it will - to my understanding - not "see" anything from my chunks going beyond 2048 tokens.

QUESTION: So, is this a bug, that AnythingLLM does not run the embedding model with the correct configured size set in "Max Embedding Chunk Length=xxx" (in my case the 4096) ???

Are there known steps to reproduce?

No response

Originally created by @alpilotx on GitHub (Feb 5, 2025). Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/3122 Originally assigned to: @timothycarambat on GitHub. ### How are you running AnythingLLM? Docker (remote machine) ### What happened? I have started a new document embedding run and wanted to have a look on the amchine running ollama, how it uses cores and also check which parameters have been used to start ollama process. Then I saw this: ![Image](https://github.com/user-attachments/assets/e3a98389-9c16-4570-b91b-d7d3c5042754) `/usr/lib/ollama/runners/cpu_avx2/ollama_llama_server runner --model /root/.ollama/models/blobs/sha256-daec91ffb5dd0c27411bd71f29932917c49cf529a641d0168496c3a501e3062c --ctx-size 2048 --batch-size 512 --threads 64 --no-mmap --parallel 1 ` And I was surprised to see **--ctx-size 2048** ! Because the embedding model I use is bge-m3 and for one it supports larger context windows AND I also explicitly told in the configuration to have **"Max Embedding Chunk Length=4096"** ![Image](https://github.com/user-attachments/assets/b470fb92-19dc-4552-9823-49cdfd90f4c7) And also I let my documents be splitted to have up to 4096 tokens per chunk. But by having Ollama run with **--ctx-size 2048** , it will - to my understanding - not "see" anything from my chunks going beyond 2048 tokens. QUESTION: So, is this a bug, that AnythingLLM does not run the embedding model with the correct configured size set in **"Max Embedding Chunk Length=xxx"** (in my case the 4096) ??? ### Are there known steps to reproduce? _No response_
yindo added the enhancement label 2026-02-22 18:27:42 -05:00
yindo closed this issue 2026-02-22 18:27:42 -05:00
yindo commented 2026-02-22 18:27:42 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Feb 5, 2025):

The default context size for any ollama model is 2048, we need to explicitly override it to take larger values. That setting is for text splitting, not the embedder itself. So not a bug, but an enhancement to be made for sure.

@timothycarambat commented on GitHub (Feb 5, 2025): The default context size for [any ollama model is 2048](https://github.com/ollama/ollama/issues/2714), we need to explicitly override it to take larger values. That setting is for text splitting, not the embedder itself. So not a bug, but an enhancement to be made for sure.
yindo changed title from [FEAT]: `num_ctx` for Ollama embedder to [GH-ISSUE #3122] [FEAT]: `num_ctx` for Ollama embedder 2026-06-05 14:43:58 -04:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Desktop
Docker
Integration Request
Integration Request
OS: Linux
OS: Mobile
OS: Windows
UI/UX
blocked
bug
bug
core-team-only
documentation
duplicate
embed-widget
enhancement
feature request
github_actions
good first issue
investigating
needs info / can't replicate
possible bug
pull-request
Mirrored from GitHub Pull Request
 question
stage: specifications
wontfix
No Label enhancement
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Mintplex-Labs/anything-llm#2003
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?