Mintplex-Labs/anything-llm
1
0
Fork 0
mirror of https://github.com/Mintplex-Labs/anything-llm.git synced 2026-07-19 14:13:35 -04:00
Code Issues 328 Packages Projects Releases 10 Wiki Activity

[PR #3835] [MERGED] Native Embedder model selection (incl: Multilingual support) #4427

New Issue
Closed
opened 2026-02-22 18:35:48 -05:00 by yindo · 0 comments
yindo commented 2026-02-22 18:35:48 -05:00
Owner
Copy Link
Copy Source

📋 Pull Request Information

Original PR: https://github.com/Mintplex-Labs/anything-llm/pull/3835
Author: @timothycarambat
Created: 5/15/2025
Status: ✅ Merged
Merged: 7/22/2025
Merged by: @timothycarambat

Base: master ← Head: multilingual-native-embedder-selection

📝 Commits (10+)

  • f2701bc WIP on embedder selection
  • 4fe3bbe wip on upsert
  • 537bb6b Merge branch 'master' of github.com:Mintplex-Labs/anything-llm into multilingual-native-embedder-selection
  • 59eb31a Support base model
  • 71201fe norm translations
  • a12684b move supported models to constants
  • 3bdeb1a dev
  • 9b8c9a8 Merge branch 'master' into multilingual-native-embedder-selection
  • d4bf946 patch text splitter maximums for now
  • 9f6f609 Merge branch 'master' into multilingual-native-embedder-selection

📊 Changes

44 files changed (+463 additions, -80 deletions)

View changed files

📝 .github/workflows/dev-build.yaml (+1 -1)
📝 docker/.env.example (+4 -0)
📝 frontend/src/components/EmbeddingSelection/NativeEmbeddingOptions/index.jsx (+95 -7)
📝 frontend/src/components/WorkspaceChat/ChatContainer/ChatHistory/Citation/index.jsx (+1 -1)
📝 frontend/src/locales/ar/common.js (+0 -2)
📝 frontend/src/locales/da/common.js (+0 -2)
📝 frontend/src/locales/de/common.js (+0 -2)
📝 frontend/src/locales/en/common.js (+0 -2)
📝 frontend/src/locales/es/common.js (+0 -2)
📝 frontend/src/locales/et/common.js (+0 -2)
📝 frontend/src/locales/fa/common.js (+0 -2)
📝 frontend/src/locales/fr/common.js (+0 -2)
📝 frontend/src/locales/he/common.js (+0 -2)
📝 frontend/src/locales/it/common.js (+0 -2)
📝 frontend/src/locales/ja/common.js (+0 -2)
📝 frontend/src/locales/ko/common.js (+0 -2)
📝 frontend/src/locales/lv/common.js (+0 -2)
📝 frontend/src/locales/nl/common.js (+0 -2)
📝 frontend/src/locales/normalizeEn.mjs (+5 -0)
📝 frontend/src/locales/pl/common.js (+0 -2)

...and 24 more files

📄 Description

Pull Request Type

  • ✨ feat
  • 🐛 fix
  • ♻️ refactor
  • 💄 style
  • 🔨 chore
  • 📝 docs

Relevant Issues

resolves #658

What is in this change?

Introduces 2 additional embedding models selected and downloadable for the default embedder engine:

Additional Information

Models are also mirrored on the AnythingLLM CDN for VPN and geo-locked users
Prefixing for both embedding and query vectorizations are applied depending on the model requirement.
Model will download in the background to local storage if requested and not already present in user storage.

Developer Validations

  • I ran yarn lint from the root of the repo & committed changes
  • Relevant documentation has been updated
  • I have tested my code functionality
  • Docker build succeeds locally

🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.

## 📋 Pull Request Information **Original PR:** https://github.com/Mintplex-Labs/anything-llm/pull/3835 **Author:** [@timothycarambat](https://github.com/timothycarambat) **Created:** 5/15/2025 **Status:** ✅ Merged **Merged:** 7/22/2025 **Merged by:** [@timothycarambat](https://github.com/timothycarambat) **Base:** `master` ← **Head:** `multilingual-native-embedder-selection` --- ### 📝 Commits (10+) - [`f2701bc`](https://github.com/Mintplex-Labs/anything-llm/commit/f2701bc939a5490794006d7de3e9bacfaad856b5) WIP on embedder selection - [`4fe3bbe`](https://github.com/Mintplex-Labs/anything-llm/commit/4fe3bbe75f00e8d55fb7b6ffc4d4a839ac33ac36) wip on upsert - [`537bb6b`](https://github.com/Mintplex-Labs/anything-llm/commit/537bb6bb49fc6021b1727503f8f5c72bab1e38be) Merge branch 'master' of github.com:Mintplex-Labs/anything-llm into multilingual-native-embedder-selection - [`59eb31a`](https://github.com/Mintplex-Labs/anything-llm/commit/59eb31a577bf953d499dd253664a242af2dcff2c) Support base model - [`71201fe`](https://github.com/Mintplex-Labs/anything-llm/commit/71201fed7de5ab675b0210ea57b1e8ff51886241) norm translations - [`a12684b`](https://github.com/Mintplex-Labs/anything-llm/commit/a12684b8c6bfbd1ad32b367b6944982ad340aa58) move supported models to constants - [`3bdeb1a`](https://github.com/Mintplex-Labs/anything-llm/commit/3bdeb1a1c1c972bfcdcd9fee8364058d9d68944a) dev - [`9b8c9a8`](https://github.com/Mintplex-Labs/anything-llm/commit/9b8c9a82ad9c5016634e3d77535d24e3dae819ac) Merge branch 'master' into multilingual-native-embedder-selection - [`d4bf946`](https://github.com/Mintplex-Labs/anything-llm/commit/d4bf946cda14cf248fbe549e14e332561d42d131) patch text splitter maximums for now - [`9f6f609`](https://github.com/Mintplex-Labs/anything-llm/commit/9f6f60952911f8729c857237ac9105344c499cb4) Merge branch 'master' into multilingual-native-embedder-selection ### 📊 Changes **44 files changed** (+463 additions, -80 deletions) <details> <summary>View changed files</summary> 📝 `.github/workflows/dev-build.yaml` (+1 -1) 📝 `docker/.env.example` (+4 -0) 📝 `frontend/src/components/EmbeddingSelection/NativeEmbeddingOptions/index.jsx` (+95 -7) 📝 `frontend/src/components/WorkspaceChat/ChatContainer/ChatHistory/Citation/index.jsx` (+1 -1) 📝 `frontend/src/locales/ar/common.js` (+0 -2) 📝 `frontend/src/locales/da/common.js` (+0 -2) 📝 `frontend/src/locales/de/common.js` (+0 -2) 📝 `frontend/src/locales/en/common.js` (+0 -2) 📝 `frontend/src/locales/es/common.js` (+0 -2) 📝 `frontend/src/locales/et/common.js` (+0 -2) 📝 `frontend/src/locales/fa/common.js` (+0 -2) 📝 `frontend/src/locales/fr/common.js` (+0 -2) 📝 `frontend/src/locales/he/common.js` (+0 -2) 📝 `frontend/src/locales/it/common.js` (+0 -2) 📝 `frontend/src/locales/ja/common.js` (+0 -2) 📝 `frontend/src/locales/ko/common.js` (+0 -2) 📝 `frontend/src/locales/lv/common.js` (+0 -2) 📝 `frontend/src/locales/nl/common.js` (+0 -2) 📝 `frontend/src/locales/normalizeEn.mjs` (+5 -0) 📝 `frontend/src/locales/pl/common.js` (+0 -2) _...and 24 more files_ </details> ### 📄 Description ### Pull Request Type <!-- For change type, change [ ] to [x]. --> - [x] ✨ feat - [ ] 🐛 fix - [ ] ♻️ refactor - [ ] 💄 style - [ ] 🔨 chore - [ ] 📝 docs ### Relevant Issues <!-- Use "resolves #xxx" to auto resolve on merge. Otherwise, please use "connect #xxx" --> resolves #658 ### What is in this change? Introduces 2 additional embedding models selected and downloadable for the default embedder engine: - [Xenova/all-MiniLM-L6-v2](https://huggingface.co/Xenova/all-MiniLM-L6-v2) - default; English - [Xenova/nomic-embed-text-v1](https://huggingface.co/Xenova/nomic-embed-text-v1) - English - [intfloat/multilingual-e5-small](https://huggingface.co/intfloat/multilingual-e5-small) - 100+ languages ### Additional Information Models are also mirrored on the AnythingLLM CDN for VPN and geo-locked users Prefixing for both embedding and query vectorizations are applied depending on the model requirement. Model will download in the background to local storage if requested and not already present in user storage. <!-- Add any other context about the Pull Request here that was not captured above. --> ### Developer Validations <!-- All of the applicable items should be checked. --> - [x] I ran `yarn lint` from the root of the repo & committed changes - [ ] Relevant documentation has been updated - [x] I have tested my code functionality - [x] Docker build succeeds locally --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>
yindo added the pull-request label 2026-02-22 18:35:48 -05:00
yindo closed this issue 2026-02-22 18:35:48 -05:00
yindo changed title from [PR #3835] Native Embedder model selection (incl: Multilingual support) to [PR #3835] [MERGED] Native Embedder model selection (incl: Multilingual support) 2026-06-05 15:18:31 -04:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Desktop
Docker
Integration Request
Integration Request
OS: Linux
OS: Mobile
OS: Windows
UI/UX
blocked
bug
bug
core-team-only
documentation
duplicate
embed-widget
enhancement
feature request
github_actions
good first issue
investigating
needs info / can't replicate
possible bug
pull-request
Mirrored from GitHub Pull Request
 question
stage: specifications
wontfix
No Label pull-request
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Mintplex-Labs/anything-llm#4427
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?