Mintplex-Labs/anything-llm
1
0
Fork 0
mirror of https://github.com/Mintplex-Labs/anything-llm.git synced 2026-07-19 14:13:35 -04:00
Code Issues 328 Packages Projects Releases 10 Wiki Activity

[GH-ISSUE #658] [FEAT]: Multilingual Native Embedder #373

New Issue
Closed
opened 2026-02-22 18:19:11 -05:00 by yindo · 10 comments
yindo commented 2026-02-22 18:19:11 -05:00
Owner
Copy Link
Copy Source

Originally created by @timothycarambat on GitHub (Jan 30, 2024).
Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/658

What would you like to see?

Currently, the built-in embedder uses the ONNX all-MiniLM-L6-v2 embedder, which does okay for most use cases and is much smaller to download.

There should be support for the larger multilingual-e5-large model (ONNX HERE) for multi-lingual support.

This should not be the default, but it should be something the user can opt to select. They may have to wait for the download to completely download for the embedder change to be saved as we cannot afford the latency to download the model at runtime.

Originally created by @timothycarambat on GitHub (Jan 30, 2024). Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/658 ### What would you like to see? Currently, the built-in embedder uses the ONNX [all-MiniLM-L6-v2](https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2) embedder, which does okay for most use cases and is much smaller to download. There should be support for the larger [multilingual-e5-large](https://huggingface.co/intfloat/multilingual-e5-large) model ([ONNX HERE](https://huggingface.co/Xenova/multilingual-e5-large)) for multi-lingual support. This should not be the default, but it should be something the user can opt to select. They may have to wait for the download to completely download for the embedder change to be saved as we cannot afford the latency to download the model at runtime.
yindo added the enhancementfeature request labels 2026-02-22 18:19:11 -05:00
yindo closed this issue 2026-02-22 18:19:11 -05:00
yindo commented 2026-02-22 18:19:11 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Jan 30, 2024):

Also, we don't want to pre-pack the docker image with models people may not use, so we will not be doing that in the future to keep the docker image portable enough for a reasonable size.

@timothycarambat commented on GitHub (Jan 30, 2024): Also, we don't want to pre-pack the docker image with models people may not use, so we will not be doing that in the future to keep the docker image portable enough for a reasonable size.
yindo commented 2026-02-22 18:19:12 -05:00
Author
Owner
Copy Link
Copy Source

@vlbosch commented on GitHub (Mar 5, 2024):

I would also like the option to add another local embeddings model, like for example BGE-M3. I tried adding it in the models-folder myself, but couldn't get it to work yet, unfortunately. Hopefully this feature can be added on the short term, so that we don't need to really on OpenAI's models for multilingual documents. Thanks in advance! :-)

@vlbosch commented on GitHub (Mar 5, 2024): I would also like the option to add another local embeddings model, like for example BGE-M3. I tried adding it in the models-folder myself, but couldn't get it to work yet, unfortunately. Hopefully this feature can be added on the short term, so that we don't need to really on OpenAI's models for multilingual documents. Thanks in advance! :-)
yindo commented 2026-02-22 18:19:12 -05:00
Author
Owner
Copy Link
Copy Source

@oscar-7000 commented on GitHub (Mar 23, 2024):

bge-m3 would be nice

@oscar-7000 commented on GitHub (Mar 23, 2024): bge-m3 would be nice
yindo commented 2026-02-22 18:19:12 -05:00
Author
Owner
Copy Link
Copy Source

@sweco-nlmdek commented on GitHub (Apr 29, 2024):

This would be a very welcome feature. i see in this thread : https://github.com/Mintplex-Labs/anything-llm/issues/645 someone tried multilingual-e5-large and it seems to help allot.

@sweco-nlmdek commented on GitHub (Apr 29, 2024): This would be a very welcome feature. i see in this thread : https://github.com/Mintplex-Labs/anything-llm/issues/645 someone tried [multilingual-e5-large](https://huggingface.co/Xenova/multilingual-e5-large) and it seems to help allot.
yindo commented 2026-02-22 18:19:12 -05:00
Author
Owner
Copy Link
Copy Source

@Herz3h commented on GitHub (Jan 29, 2025):

Is there a manual way to use a multilingual embedding model in the mean time ? or one from sentence-transformers?

@Herz3h commented on GitHub (Jan 29, 2025): Is there a manual way to use a multilingual embedding model in the mean time ? or one from sentence-transformers?
yindo commented 2026-02-22 18:19:12 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Jan 29, 2025):

@Herz3h, yes, we support Ollama or LMStudio for embedder endpoints, where you can use any embedder you like to fit your use case.

@timothycarambat commented on GitHub (Jan 29, 2025): @Herz3h, yes, we support Ollama or LMStudio for embedder endpoints, where you can use any embedder you like to fit your use case.
yindo commented 2026-02-22 18:19:12 -05:00
Author
Owner
Copy Link
Copy Source

@Herz3h commented on GitHub (Jan 30, 2025):

Thanks, however neither of ollama/lmstudio seem to support sentence-transformers, is there a way to still use them? since some of them rank very high in https://huggingface.co/spaces/mteb/leaderboard

@Herz3h commented on GitHub (Jan 30, 2025): Thanks, however neither of ollama/lmstudio seem to support sentence-transformers, is there a way to still use them? since some of them rank very high in https://huggingface.co/spaces/mteb/leaderboard
yindo commented 2026-02-22 18:19:12 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Jan 30, 2025):

Ollama and LMstudio both support embedders - you need to use the GGUF version since they cannot run tensorflow or pytorch models.

Ollamas select is certainly more limited since it relies on their registry, but you can import HF models directly into Ollama.
Example: https://huggingface.co/models?library=gguf&sort=downloads&search=embedding
Click on any of those and you will see the ability to pull it in Ollama or LMStudio.

Both of which support /embedding - so it works.

Image
@timothycarambat commented on GitHub (Jan 30, 2025): Ollama and LMstudio both support embedders - you need to use the GGUF version since they cannot run tensorflow or pytorch models. Ollamas select is certainly more limited since it relies on their registry, but you can import HF models directly into Ollama. Example: https://huggingface.co/models?library=gguf&sort=downloads&search=embedding Click on any of those and you will see the ability to pull it in Ollama _or_ LMStudio. Both of which support /embedding - so it works. <img width="363" alt="Image" src="https://github.com/user-attachments/assets/4ed1f91c-15b6-4ba4-94b9-17269b0b6edb" />
yindo commented 2026-02-22 18:19:12 -05:00
Author
Owner
Copy Link
Copy Source

@Herz3h commented on GitHub (Jan 31, 2025):

Oh didn't know there was an option from huggingface to pull anymodel in ollama. Thank you very much !

@Herz3h commented on GitHub (Jan 31, 2025): Oh didn't know there was an option from huggingface to pull anymodel in ollama. Thank you very much !
yindo commented 2026-02-22 18:19:13 -05:00
Author
Owner
Copy Link
Copy Source

@JosefAschauer commented on GitHub (Apr 17, 2025):

I opened another request for this which was closed since this one exists. Yes, a multilingual embedder would be fantastic, I love arctic snowflake l v2 which ranks fantastic but this also requires sentence transformers to unfold it's full potential. So I created my own dockerized sentence transformers embedder but cannot use it with anythingllm (which i really love!) since it doesn't seem to support this as an external embedder. Maybe someone could point me to how I can integrate this? Localai or ollama are not an option

@JosefAschauer commented on GitHub (Apr 17, 2025): I opened another request for this which was closed since this one exists. Yes, a multilingual embedder would be fantastic, I love arctic snowflake l v2 which ranks fantastic but this also requires sentence transformers to unfold it's full potential. So I created my own dockerized sentence transformers embedder but cannot use it with anythingllm (which i really love!) since it doesn't seem to support this as an external embedder. Maybe someone could point me to how I can integrate this? Localai or ollama are not an option
yindo changed title from [FEAT]: Multilingual Native Embedder to [GH-ISSUE #658] [FEAT]: Multilingual Native Embedder 2026-06-05 14:34:58 -04:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Desktop
Docker
Integration Request
Integration Request
OS: Linux
OS: Mobile
OS: Windows
UI/UX
blocked
bug
bug
core-team-only
documentation
duplicate
embed-widget
enhancement
feature request
github_actions
good first issue
investigating
needs info / can't replicate
possible bug
pull-request
Mirrored from GitHub Pull Request
 question
stage: specifications
wontfix
No Label enhancement feature request
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Mintplex-Labs/anything-llm#373
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?