Mintplex-Labs/anything-llm
1
0
Fork 0
mirror of https://github.com/Mintplex-Labs/anything-llm.git synced 2026-07-19 14:13:35 -04:00
Code Issues 328 Packages Projects Releases 10 Wiki Activity

[GH-ISSUE #5486] [FEAT]: MLX Engine support #5114

New Issue
Closed
opened 2026-06-05 14:52:05 -04:00 by yindo · 3 comments
yindo commented 2026-06-05 14:52:05 -04:00
Owner
Copy Link
Copy Source

Originally created by @computerworxdev on GitHub (Apr 22, 2026).
Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/5486

What would you like to see?

Hello
will MLX native to Apple computers Framework be ever adopted ?

This native most memory efficient not to mentioned Speed Optimized in some cases x2 framework for local models.

It hurts to see Mac Version that is not using the best AI Framework there is.

Please consider to include that in the future release. We need optimization and speed for local models.

Thanks

Originally created by @computerworxdev on GitHub (Apr 22, 2026). Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/5486 ### What would you like to see? Hello will MLX native to Apple computers Framework be ever adopted ? This native most memory efficient not to mentioned Speed Optimized in some cases x2 framework for local models. It hurts to see Mac Version that is not using the best AI Framework there is. Please consider to include that in the future release. We need optimization and speed for local models. Thanks
yindo added the enhancementfeature request labels 2026-06-05 14:52:05 -04:00
yindo closed this issue 2026-06-05 14:52:05 -04:00
yindo commented 2026-06-05 14:52:06 -04:00
Author
Owner
Copy Link
Copy Source

@computerworxdev commented on GitHub (May 19, 2026):

Thanks for this , i will look into its, i just dont like complex setups. Could you share a complete command to run "mlx-community/Qwen3.6-35B-A3B-6bit" mlx model with 64k context window size , i would like to try it but i dont know how to manage my custom HF model location and how to use default mlx model location in cache.

TIM any feedback on MLX built in support?

<!-- gh-comment-id:4488904261 --> @computerworxdev commented on GitHub (May 19, 2026): Thanks for this , i will look into its, i just dont like complex setups. Could you share a complete command to run "mlx-community/Qwen3.6-35B-A3B-6bit" mlx model with 64k context window size , i would like to try it but i dont know how to manage my custom HF model location and how to use default mlx model location in cache. TIM any feedback on MLX built in support?
yindo commented 2026-06-05 14:52:06 -04:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (May 19, 2026):

We do not build or maintain a custom LLM runtime. That is a whole job in and of itself - we focus on the application layer. That being said, you can use mlx-server and then use that via the Generic OAI connector we have and it will run on MLX.

<!-- gh-comment-id:4490743280 --> @timothycarambat commented on GitHub (May 19, 2026): We do not build or maintain a custom LLM runtime. That is a whole job in and of itself - we focus on the application layer. That being said, you can use `mlx-server` and then use that via the Generic OAI connector we have and it will run on MLX.
yindo commented 2026-06-05 14:52:06 -04:00
Author
Owner
Copy Link
Copy Source

@computerworxdev commented on GitHub (May 20, 2026):

Perfect, thats what i was looking for, i did not expect nor required custom LLM runtime when one is available from from Apple, i just wanted to make sure you have in Provider config option to select one. ( it would be nice to have it in the provider list and be able to detect if server mlx is running and allow to choose existing models ) thats it.

<!-- gh-comment-id:4499349156 --> @computerworxdev commented on GitHub (May 20, 2026): Perfect, thats what i was looking for, i did not expect nor required custom LLM runtime when one is available from from Apple, i just wanted to make sure you have in Provider config option to select one. ( it would be nice to have it in the provider list and be able to detect if server mlx is running and allow to choose existing models ) thats it.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Desktop
Docker
Integration Request
Integration Request
OS: Linux
OS: Mobile
OS: Windows
UI/UX
blocked
bug
bug
core-team-only
documentation
duplicate
embed-widget
enhancement
feature request
github_actions
good first issue
investigating
needs info / can't replicate
possible bug
pull-request
Mirrored from GitHub Pull Request
 question
stage: specifications
wontfix
No Label enhancement feature request
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Mintplex-Labs/anything-llm#5114
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?