Mintplex-Labs/anything-llm
1
0
Fork 0
mirror of https://github.com/Mintplex-Labs/anything-llm.git synced 2026-07-19 14:13:35 -04:00
Code Issues 328 Packages Projects Releases 10 Wiki Activity

[PR #413] [MERGED] [Feature] AnythingLLM use locally hosted Llama.cpp and GGUF files for inferencing #3345

New Issue
Closed
opened 2026-02-22 18:33:35 -05:00 by yindo · 0 comments
yindo commented 2026-02-22 18:33:35 -05:00
Owner
Copy Link
Copy Source

📋 Pull Request Information

Original PR: https://github.com/Mintplex-Labs/anything-llm/pull/413
Author: @timothycarambat
Created: 12/6/2023
Status: ✅ Merged
Merged: 12/7/2023
Merged by: @timothycarambat

Base: master ← Head: built-in-llm

📝 Commits (10+)

  • 6c63918 Implement use of native embedder (all-Mini-L6-v2)
  • f038a65 Add native embedder as an available embedder selection
  • ab451f8 wrap model loader in try/catch
  • 48680a6 Merge branch 'master' of github.com:Mintplex-Labs/anything-llm into built-in-embedder
  • 0618196 print progress on download
  • 73795df add built-in LLM support (expiermental)
  • f7bc1fc Update to progress output for embedder
  • e3bfaf9 move embedder selection options to component
  • c1baa1f Merge branch 'built-in-embedder' of github.com:Mintplex-Labs/anything-llm into built-in-llm
  • 6afc5d8 merge conf

📊 Changes

22 files changed (+1304 additions, -99 deletions)

View changed files

📝 README.md (+16 -12)
📝 docker/Dockerfile (+6 -1)
➕ frontend/src/components/LLMSelection/NativeLLMOptions/index.jsx (+84 -0)
📝 frontend/src/components/PrivateRoute/index.jsx (+4 -6)
📝 frontend/src/pages/GeneralSettings/LLMPreference/index.jsx (+15 -0)
📝 frontend/src/pages/OnboardingFlow/OnboardingModal/Steps/DataHandling/index.jsx (+7 -0)
📝 frontend/src/pages/OnboardingFlow/OnboardingModal/Steps/LLMSelection/index.jsx (+11 -0)
➖ images/screenshots/SCREENSHOTS.md (+0 -18)
➖ images/screenshots/document.png (+0 -0)
➖ images/screenshots/home.png (+0 -0)
➖ images/screenshots/llm_selection.png (+0 -0)
➖ images/screenshots/uploading_doc.gif (+0 -0)
➖ images/screenshots/vector_databases.png (+0 -0)
📝 server/models/systemSettings.js (+7 -2)
📝 server/package.json (+3 -2)
📝 server/storage/models/README.md (+24 -4)
➕ server/utils/AiProviders/native/index.js (+196 -0)
📝 server/utils/chats/stream.js (+30 -0)
📝 server/utils/helpers/customModels.js (+23 -1)
📝 server/utils/helpers/index.js (+3 -0)

...and 2 more files

📄 Description

  • Add support for the 2,000+ GGUF models that can be used as the LLM of choice. Making AnythingLLM now 100% privately running.
  • Support sync chat and streaming from models.
  • Add documentation to support this feature
  • Hide selection when on hosted domain

🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.

## 📋 Pull Request Information **Original PR:** https://github.com/Mintplex-Labs/anything-llm/pull/413 **Author:** [@timothycarambat](https://github.com/timothycarambat) **Created:** 12/6/2023 **Status:** ✅ Merged **Merged:** 12/7/2023 **Merged by:** [@timothycarambat](https://github.com/timothycarambat) **Base:** `master` ← **Head:** `built-in-llm` --- ### 📝 Commits (10+) - [`6c63918`](https://github.com/Mintplex-Labs/anything-llm/commit/6c63918b87ee587ec1c74d62c8b17c9e12507311) Implement use of native embedder (all-Mini-L6-v2) - [`f038a65`](https://github.com/Mintplex-Labs/anything-llm/commit/f038a65ff5b50a8137b27a8fa756a1ff75017c42) Add native embedder as an available embedder selection - [`ab451f8`](https://github.com/Mintplex-Labs/anything-llm/commit/ab451f893beb4552c89ccc7aa21daee8fb80c41f) wrap model loader in try/catch - [`48680a6`](https://github.com/Mintplex-Labs/anything-llm/commit/48680a6d9c71f61b2a021d7319e16c785dedc7ac) Merge branch 'master' of github.com:Mintplex-Labs/anything-llm into built-in-embedder - [`0618196`](https://github.com/Mintplex-Labs/anything-llm/commit/061819691f80f05c9b38e498a091747b13105283) print progress on download - [`73795df`](https://github.com/Mintplex-Labs/anything-llm/commit/73795df434cb75385e717c0787f6ab586ec7ab26) add built-in LLM support (expiermental) - [`f7bc1fc`](https://github.com/Mintplex-Labs/anything-llm/commit/f7bc1fc35c545452ad06c98a0cc2f2cb92adfe92) Update to progress output for embedder - [`e3bfaf9`](https://github.com/Mintplex-Labs/anything-llm/commit/e3bfaf9e58329b1fc05f78408d035e02665dea38) move embedder selection options to component - [`c1baa1f`](https://github.com/Mintplex-Labs/anything-llm/commit/c1baa1f4996d4a2fe67a4059132ad1c0cc43fc04) Merge branch 'built-in-embedder' of github.com:Mintplex-Labs/anything-llm into built-in-llm - [`6afc5d8`](https://github.com/Mintplex-Labs/anything-llm/commit/6afc5d8cff211dbeca870ab83d6142ff1b6168ce) merge conf ### 📊 Changes **22 files changed** (+1304 additions, -99 deletions) <details> <summary>View changed files</summary> 📝 `README.md` (+16 -12) 📝 `docker/Dockerfile` (+6 -1) ➕ `frontend/src/components/LLMSelection/NativeLLMOptions/index.jsx` (+84 -0) 📝 `frontend/src/components/PrivateRoute/index.jsx` (+4 -6) 📝 `frontend/src/pages/GeneralSettings/LLMPreference/index.jsx` (+15 -0) 📝 `frontend/src/pages/OnboardingFlow/OnboardingModal/Steps/DataHandling/index.jsx` (+7 -0) 📝 `frontend/src/pages/OnboardingFlow/OnboardingModal/Steps/LLMSelection/index.jsx` (+11 -0) ➖ `images/screenshots/SCREENSHOTS.md` (+0 -18) ➖ `images/screenshots/document.png` (+0 -0) ➖ `images/screenshots/home.png` (+0 -0) ➖ `images/screenshots/llm_selection.png` (+0 -0) ➖ `images/screenshots/uploading_doc.gif` (+0 -0) ➖ `images/screenshots/vector_databases.png` (+0 -0) 📝 `server/models/systemSettings.js` (+7 -2) 📝 `server/package.json` (+3 -2) 📝 `server/storage/models/README.md` (+24 -4) ➕ `server/utils/AiProviders/native/index.js` (+196 -0) 📝 `server/utils/chats/stream.js` (+30 -0) 📝 `server/utils/helpers/customModels.js` (+23 -1) 📝 `server/utils/helpers/index.js` (+3 -0) _...and 2 more files_ </details> ### 📄 Description - Add support for the 2,000+ [GGUF models](https://huggingface.co/models?library=gguf&sort=trending) that can be used as the LLM of choice. Making AnythingLLM now 100% privately running. - Support sync chat and streaming from models. - Add documentation to support this feature - Hide selection when on hosted domain --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>
yindo added the pull-request label 2026-02-22 18:33:35 -05:00
yindo closed this issue 2026-02-22 18:33:35 -05:00
yindo changed title from [PR #413] [Feature] AnythingLLM use locally hosted Llama.cpp and GGUF files for inferencing to [PR #413] [MERGED] [Feature] AnythingLLM use locally hosted Llama.cpp and GGUF files for inferencing 2026-06-05 15:12:55 -04:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Desktop
Docker
Integration Request
Integration Request
OS: Linux
OS: Mobile
OS: Windows
UI/UX
blocked
bug
bug
core-team-only
documentation
duplicate
embed-widget
enhancement
feature request
github_actions
good first issue
investigating
needs info / can't replicate
possible bug
pull-request
Mirrored from GitHub Pull Request
 question
stage: specifications
wontfix
No Label pull-request
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Mintplex-Labs/anything-llm#3345
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?