Mintplex-Labs/anything-llm
1
0
Fork 0
mirror of https://github.com/Mintplex-Labs/anything-llm.git synced 2026-07-19 14:13:35 -04:00
Code Issues 328 Packages Projects Releases 10 Wiki Activity

[GH-ISSUE #4859] [FEAT]: More LLM Models With NPU Support #3063

New Issue
Closed
opened 2026-02-22 18:32:28 -05:00 by yindo · 1 comment
yindo commented 2026-02-22 18:32:28 -05:00
Owner
Copy Link
Copy Source

Originally created by @GaneshHeralgi on GitHub (Jan 13, 2026).
Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/4859

What would you like to see?

I am hoping to see multimodal models also compatible with NPUs. Right now, the only multimodal I am running is Gemma 3, but that's a problem because it runs only on CPU and integrated GPU, which takes up a lot of power and resources, but NPUs don't have that problem.
I am hoping for additional models to be compatible with NPUs like Phi-4 or Gemma 3.
I am using AnythingLLM on a laptop-tablet, and I prioritise portability. Additionally, I don't have a dedicated GPU on my device.

Image
Originally created by @GaneshHeralgi on GitHub (Jan 13, 2026). Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/4859 ### What would you like to see? I am hoping to see multimodal models also compatible with NPUs. Right now, the only multimodal I am running is Gemma 3, but that's a problem because it runs only on CPU and integrated GPU, which takes up a lot of power and resources, but NPUs don't have that problem. I am hoping for additional models to be compatible with NPUs like Phi-4 or Gemma 3. I am using AnythingLLM on a laptop-tablet, and I prioritise portability. Additionally, I don't have a dedicated GPU on my device. <img width="1466" height="1328" alt="Image" src="https://github.com/user-attachments/assets/aaef08ef-faee-41a1-8644-fbd005ed0825" />
yindo added the enhancementfeature request labels 2026-02-22 18:32:28 -05:00
yindo closed this issue 2026-02-22 18:32:28 -05:00
yindo commented 2026-02-22 18:32:28 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Jan 13, 2026):

I am using AnythingLLM on a laptop-tablet, and I prioritise portability. Additionally, I don't have a dedicated GPU on my device.

The one we ship with is specific to Qualcomm models, unfortunatly the process to get more NPU models is bottlenecked by an entire company of lawyers blocking any new models from being published on their hub.

Alternative (that is faster too) is to use Foundry Local, which is another thing to download, but it runs TONS of models on NPU/CPU optimized for your hardware specific for windows.
https://github.com/microsoft/Foundry-Local/releases/tag/v0.8.117

We work with FL and while in beta, you have many, many more models you can use. Next release will have a much cleaner UI for this as well so you dont have to pull models first to see them. Since FL can be high friction to set up it is not the default, but it honestly is much better.

@timothycarambat commented on GitHub (Jan 13, 2026): > I am using AnythingLLM on a laptop-tablet, and I prioritise portability. Additionally, I don't have a dedicated GPU on my device. The one we ship with is specific to Qualcomm models, unfortunatly the process to get more NPU models is bottlenecked by an entire company of lawyers blocking any new models from being published on their hub. Alternative (that is faster too) is to use [Foundry Local](https://github.com/microsoft/Foundry-Local), which is another thing to download, but it runs TONS of models on NPU/CPU optimized for your hardware specific for windows. https://github.com/microsoft/Foundry-Local/releases/tag/v0.8.117 We work with FL and while in beta, you have many, many more models you can use. Next release will have a much cleaner UI for this as well so you dont have to pull models first to see them. Since FL can be high friction to set up it is not the default, but it honestly is much better.
yindo changed title from [FEAT]: More LLM Models With NPU Support to [GH-ISSUE #4859] [FEAT]: More LLM Models With NPU Support 2026-06-05 14:50:06 -04:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Desktop
Docker
Integration Request
Integration Request
OS: Linux
OS: Mobile
OS: Windows
UI/UX
blocked
bug
bug
core-team-only
documentation
duplicate
embed-widget
enhancement
feature request
github_actions
good first issue
investigating
needs info / can't replicate
possible bug
pull-request
Mirrored from GitHub Pull Request
 question
stage: specifications
wontfix
No Label enhancement feature request
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Mintplex-Labs/anything-llm#3063
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?