Mintplex-Labs/anything-llm
1
0
Fork 0
mirror of https://github.com/Mintplex-Labs/anything-llm.git synced 2026-07-21 01:25:28 -04:00
Code Issues 328 Packages Projects Releases 10 Wiki Activity

[GH-ISSUE #2633] [FEAT]: Max. chunk size should be overridable #1705

New Issue
Open
opened 2026-02-22 18:26:09 -05:00 by yindo · 3 comments
yindo commented 2026-02-22 18:26:09 -05:00
Owner
Copy Link
Copy Source

Originally created by @TeaAlc on GitHub (Nov 14, 2024).
Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/2633

Originally assigned to: @timothycarambat on GitHub.

What would you like to see?

Feature: To not only support one model that was somehow selected at development time and not have to maintain lists for every embedding model it would be great to have a option to overrule the hardcoded embeddingMaxChunkLength.

Explaination:
In Text splitting & Chunking Preferences the max chunk size seems to be embedding provider dependend rather than embedding model dependent, this leads through a max length of characters that does not fit to every model.

e.G.
image
is set for the Azure OpenAI Embedding Provider.
The chunk size seems to be from
https://github.com/Mintplex-Labs/anything-llm/blob/da3d0283ffee9c592e5b81d2be6a848722df298f/server/utils/EmbeddingEngines/azureOpenAi/index.js#L22C10-L22C34
The model that was used as base seems to be text-embedding-ada-002, but there are already newer models like text-embedding-3-large.

Also it seems that the AnythingLLM embedder counts characters rather than tokens, reducing the amount of data in a vector even further.

Originally created by @TeaAlc on GitHub (Nov 14, 2024). Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/2633 Originally assigned to: @timothycarambat on GitHub. ### What would you like to see? **Feature:** To not only support one model that was somehow selected at development time and not have to maintain lists for every embedding model it would be great to have a option to overrule the hardcoded embeddingMaxChunkLength. **Explaination:** In _Text splitting & Chunking Preferences_ the max chunk size seems to be embedding provider dependend rather than embedding model dependent, this leads through a max length of characters that does not fit to every model. e.G. ![image](https://github.com/user-attachments/assets/5fd09cc5-c172-43c9-9e90-beb2ce23dac1) is set for the Azure OpenAI Embedding Provider. The chunk size seems to be from https://github.com/Mintplex-Labs/anything-llm/blob/da3d0283ffee9c592e5b81d2be6a848722df298f/server/utils/EmbeddingEngines/azureOpenAi/index.js#L22C10-L22C34 The model that was used as base seems to be _text-embedding-ada-002_, but there are already newer models like _text-embedding-3-large_. Also it seems that the AnythingLLM embedder counts characters rather than tokens, reducing the amount of data in a vector even further.
yindo added the enhancementfeature request labels 2026-02-22 18:26:09 -05:00
yindo commented 2026-02-22 18:26:09 -05:00
Author
Owner
Copy Link
Copy Source

@kalle07 commented on GitHub (Jan 21, 2025):

can i change also the number of chunks

lets say

8 numbers
1024 size
32 overlap

Image

or is always used the larges that the model can handle?

@kalle07 commented on GitHub (Jan 21, 2025): can i change also the number of chunks lets say 8 numbers 1024 size 32 overlap ![Image](https://github.com/user-attachments/assets/6dafeb4a-aa5d-4833-b485-b3538016f3c7) or is always used the larges that the model can handle?
yindo commented 2026-02-22 18:26:10 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Jan 21, 2025):

If you dont set it or swap the embedder model, we use the max by default. If you set it - we will not change it.

@timothycarambat commented on GitHub (Jan 21, 2025): If you dont set it or swap the embedder model, we use the max by default. If you set it - we will not change it.
yindo commented 2026-02-22 18:26:10 -05:00
Author
Owner
Copy Link
Copy Source

@kalle07 commented on GitHub (Jan 21, 2025):

lets say the model can 16000

i set
1024 chunk size
and
overlap 32

so and now?
you multiply the number of chunks to get 16000 ?

@kalle07 commented on GitHub (Jan 21, 2025): lets say the model can 16000 i set 1024 chunk size and overlap 32 so and now? you multiply the number of chunks to get 16000 ?
yindo changed title from [FEAT]: Max. chunk size should be overridable to [GH-ISSUE #2633] [FEAT]: Max. chunk size should be overridable 2026-06-05 14:42:15 -04:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Desktop
Docker
Integration Request
Integration Request
OS: Linux
OS: Mobile
OS: Windows
UI/UX
blocked
bug
bug
core-team-only
documentation
duplicate
embed-widget
enhancement
feature request
github_actions
good first issue
investigating
needs info / can't replicate
possible bug
pull-request
Mirrored from GitHub Pull Request
 question
stage: specifications
wontfix
No Label enhancement feature request
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Mintplex-Labs/anything-llm#1705
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?