Mintplex-Labs/anything-llm
1
0
Fork 0
mirror of https://github.com/Mintplex-Labs/anything-llm.git synced 2026-07-19 14:13:35 -04:00
Code Issues 328 Packages Projects Releases 10 Wiki Activity

[GH-ISSUE #4136] [FEAT]: Jan as a backend server option #2631

New Issue
Closed
opened 2026-02-22 18:30:31 -05:00 by yindo · 8 comments
yindo commented 2026-02-22 18:30:31 -05:00
Owner
Copy Link
Copy Source

Originally created by @CrazyAce25 on GitHub (Jul 11, 2025).
Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/4136

Originally assigned to: @shatfield4 on GitHub.

What would you like to see?

First off, thank you all for all of the work on AnyhingLLM, fantastic software! I would like to see Jan https://github.com/menloresearch/jan added to the plug and play list of back end providers for LLM inference within AnythingLLM. Its inference speed is far better than what I get out of LM Studio, so I would really like to make use of it with AnythingLLM.

Originally created by @CrazyAce25 on GitHub (Jul 11, 2025). Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/4136 Originally assigned to: @shatfield4 on GitHub. ### What would you like to see? First off, thank you all for all of the work on AnyhingLLM, fantastic software! I would like to see Jan https://github.com/menloresearch/jan added to the plug and play list of back end providers for LLM inference within AnythingLLM. Its inference speed is far better than what I get out of LM Studio, so I would really like to make use of it with AnythingLLM.
yindo added the wontfix label 2026-02-22 18:30:31 -05:00
yindo closed this issue 2026-02-22 18:30:31 -05:00
yindo commented 2026-02-22 18:30:32 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Jul 14, 2025):

That makes sense - in the interim, does the generic OpenAI connector not function here? Their backend should be fully compatible.

@timothycarambat commented on GitHub (Jul 14, 2025): That makes sense - in the interim, does the generic OpenAI connector not function here? Their backend should be fully compatible.
yindo commented 2026-02-22 18:30:32 -05:00
Author
Owner
Copy Link
Copy Source

@CrazyAce25 commented on GitHub (Jul 14, 2025):

That makes sense - in the interim, does the generic OpenAI connector not function here? Their backend should be fully compatible.

I have not been able to get Jan and AnythingLLM to connect yet. Ive tried using Generic OpenAI under Instance Settings LLM, but when I go to a workspace to try to select that Generic OpenAI instance I configured, it doesnt show up as an option.

@CrazyAce25 commented on GitHub (Jul 14, 2025): > That makes sense - in the interim, does the generic OpenAI connector not function here? Their backend should be fully compatible. I have not been able to get Jan and AnythingLLM to connect yet. Ive tried using Generic OpenAI under Instance Settings LLM, but when I go to a workspace to try to select that Generic OpenAI instance I configured, it doesnt show up as an option.
yindo commented 2026-02-22 18:30:32 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Jul 14, 2025):

Yes, because Generic OpenAI has a single common connection - not many independent ones. So you need to click on the wrench on the sidebar > LLM Preferences, and set it there.

@timothycarambat commented on GitHub (Jul 14, 2025): Yes, because Generic OpenAI has a single common connection - not many independent ones. So you need to click on the wrench on the sidebar > LLM Preferences, and set it there.
yindo commented 2026-02-22 18:30:32 -05:00
Author
Owner
Copy Link
Copy Source

@CrazyAce25 commented on GitHub (Jul 14, 2025):

I believe that is precisely what I did, but then when needing to select that model to use in a workspace, its not there. Its quite possible Im just overlooking something, but still havent had any luck.

Image
@CrazyAce25 commented on GitHub (Jul 14, 2025): I believe that is precisely what I did, but then when needing to select that model to use in a workspace, its not there. Its quite possible Im just overlooking something, but still havent had any luck. <img width="1164" height="467" alt="Image" src="https://github.com/user-attachments/assets/28ade0e7-4117-4d8c-bb2d-76b0c234a4cd" />
yindo commented 2026-02-22 18:30:32 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Jul 14, 2025):

If i understand correctly, since you have that set at the LLM Preference this is now your "System default" so in the workspace, you dont need to do anything else. The workspace provider should just be "System default" and it will use the credentials in that above screenshot and you should be good

@timothycarambat commented on GitHub (Jul 14, 2025): If i understand correctly, since you have that set at the LLM Preference this is now your "System default" so in the workspace, you dont need to do anything else. The workspace provider should just be "System default" and it will use the credentials in that above screenshot and you should be good
yindo commented 2026-02-22 18:30:32 -05:00
Author
Owner
Copy Link
Copy Source

@CrazyAce25 commented on GitHub (Jul 14, 2025):

That is correct as far as I know. I always configure my models via that route under the LLM Preferences, then proceed to the workspace to get things done. When I use LMStudio or KoboldCPP, while in the workspace I can hover over the

Image icon and it will always show the model Im currently using right there. I figured that I would also see the one I added to use via Jan, but thats not the case. This makes sense as Jan is not fully integrated as the other 2 options, but I thought it may show something such as "Generic OpenAI" and display the model in use. In any case, once the model is configured as shown above and then I attempt to type anything in the workspace, I get no response.

@CrazyAce25 commented on GitHub (Jul 14, 2025): That is correct as far as I know. I always configure my models via that route under the LLM Preferences, then proceed to the workspace to get things done. When I use LMStudio or KoboldCPP, while in the workspace I can hover over the <img width="34" height="37" alt="Image" src="https://github.com/user-attachments/assets/900a22e4-d177-4de0-83f5-ff887ef1f21e" /> icon and it will always show the model Im currently using right there. I figured that I would also see the one I added to use via Jan, but thats not the case. This makes sense as Jan is not fully integrated as the other 2 options, but I thought it may show something such as "Generic OpenAI" and display the model in use. In any case, once the model is configured as shown above and then I attempt to type anything in the workspace, I get no response.
yindo commented 2026-02-22 18:30:32 -05:00
Author
Owner
Copy Link
Copy Source

@Admin-coreDevX commented on GitHub (Jul 18, 2025):

Hello, first of all, congratulations on AnythingLLM, an incredible product. Like @CrazyAce25 , I have the same need. However, it works for me with both the OpenIA Generic class interface and even the LiteLLM interface. However, the response message is truncated at the end and then throws a connection error. I have the impression there's a problem with the end of the message that Jan.AI sends at the end of the response.

I also think that, if possible, it would be ideal if Jan.AI were directly added as an access option for Remote and Local LLMs.

@Admin-coreDevX commented on GitHub (Jul 18, 2025): Hello, first of all, congratulations on AnythingLLM, an incredible product. Like @CrazyAce25 , I have the same need. However, it works for me with both the OpenIA Generic class interface and even the LiteLLM interface. However, the response message is truncated at the end and then throws a connection error. I have the impression there's a problem with the end of the message that Jan.AI sends at the end of the response. I also think that, if possible, it would be ideal if Jan.AI were directly added as an access option for Remote and Local LLMs.
yindo commented 2026-02-22 18:30:32 -05:00
Author
Owner
Copy Link
Copy Source

@shatfield4 commented on GitHub (Oct 1, 2025):

Closing as not planned/wontfix due to how the Jan AI backend works. We are unable to load models in and out since the API exposed by Jan will only show the model that is currently loaded in the UI.

@shatfield4 commented on GitHub (Oct 1, 2025): Closing as not planned/wontfix due to how the Jan AI backend works. We are unable to load models in and out since the API exposed by Jan will only show the model that is currently loaded in the UI.
yindo changed title from [FEAT]: Jan as a backend server option to [GH-ISSUE #4136] [FEAT]: Jan as a backend server option 2026-06-05 14:47:38 -04:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Desktop
Docker
Integration Request
Integration Request
OS: Linux
OS: Mobile
OS: Windows
UI/UX
blocked
bug
bug
core-team-only
documentation
duplicate
embed-widget
enhancement
feature request
github_actions
good first issue
investigating
needs info / can't replicate
possible bug
pull-request
Mirrored from GitHub Pull Request
 question
stage: specifications
wontfix
No Label wontfix
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Mintplex-Labs/anything-llm#2631
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?