Mintplex-Labs/anything-llm
1
0
Fork 0
mirror of https://github.com/Mintplex-Labs/anything-llm.git synced 2026-07-19 14:13:35 -04:00
Code Issues 328 Packages Projects Releases 10 Wiki Activity

[GH-ISSUE #4976] [FEAT]: Allow use of custom modelfile system prompt and temperature #3123

New Issue
Closed
opened 2026-02-22 18:32:43 -05:00 by yindo · 3 comments
yindo commented 2026-02-22 18:32:43 -05:00
Owner
Copy Link
Copy Source

Originally created by @RecklessCharger on GitHub (Feb 9, 2026).
Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/4976

What would you like to see?

AnythingLLM seems to override the system prompt contained within a custom Ollama modelfile. I am assuming temperature is also overridden. Is there any way to work around this short of copy-pasting the system prompt and temperature from the modelfile? If not can we please have a checkbox above the system prompt and temperature settings in Anything LLM to allow direct use of modelfile system prompt and temperature (which would then grey out these parameters in the chat settings dialogue)?

Originally created by @RecklessCharger on GitHub (Feb 9, 2026). Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/4976 ### What would you like to see? AnythingLLM seems to override the system prompt contained within a custom Ollama modelfile. I am assuming temperature is also overridden. Is there any way to work around this short of copy-pasting the system prompt and temperature from the modelfile? If not can we please have a checkbox above the system prompt and temperature settings in Anything LLM to allow direct use of modelfile system prompt and temperature (which would then grey out these parameters in the chat settings dialogue)?
yindo added the enhancementfeature request labels 2026-02-22 18:32:43 -05:00
yindo closed this issue 2026-02-22 18:32:43 -05:00
yindo commented 2026-02-22 18:32:43 -05:00
Author
Owner
Copy Link
Copy Source

@bytebycris commented on GitHub (Feb 9, 2026):

To solve this issue, it seems you need to preserve the settings (system prompt and temperature) from the modelfile, as AnythingLLM seems to override them by default. Here's a possible approach:

  • Direct Use of Modelfile Settings: You can check if AnythingLLM offers a feature or flag to directly use the settings from the modelfile (e.g., system prompt and temperature). If it doesn't, you can submit a feature request, as you've mentioned, asking for a checkbox that ensures the settings from the modelfile are used directly, which would disable manual overrides in the chat settings dialogue.

  • Manual Workaround: Until such a feature is implemented, the immediate workaround is to copy-paste the system prompt and temperature values from the modelfile into the chat settings every time you use the model. This may be cumbersome, but it's the most direct approach.

  • Automate via API: If AnythingLLM allows for API interactions, consider writing a small script or wrapper around it to fetch the system prompt and temperature from the modelfile and pass them programmatically when starting a chat session. This way, you can automate the process of copying over these values without manual input.

  • Custom Integration: Depending on the flexibility of AnythingLLM, it may be possible to modify or extend its settings to read directly from the modelfile each time it is used. You would need to explore its configuration files or documentation for more insights on this option.

If AnythingLLM doesn't provide the exact option you're requesting yet, submitting a feature request for the checkbox or another solution would likely be the best approach for long-term ease of use.

@bytebycris commented on GitHub (Feb 9, 2026): To solve this issue, it seems you need to preserve the settings (system prompt and temperature) from the modelfile, as AnythingLLM seems to override them by default. Here's a possible approach: - Direct Use of Modelfile Settings: You can check if AnythingLLM offers a feature or flag to directly use the settings from the modelfile (e.g., system prompt and temperature). If it doesn't, you can submit a feature request, as you've mentioned, asking for a checkbox that ensures the settings from the modelfile are used directly, which would disable manual overrides in the chat settings dialogue. - Manual Workaround: Until such a feature is implemented, the immediate workaround is to copy-paste the system prompt and temperature values from the modelfile into the chat settings every time you use the model. This may be cumbersome, but it's the most direct approach. - Automate via API: If AnythingLLM allows for API interactions, consider writing a small script or wrapper around it to fetch the system prompt and temperature from the modelfile and pass them programmatically when starting a chat session. This way, you can automate the process of copying over these values without manual input. - Custom Integration: Depending on the flexibility of AnythingLLM, it may be possible to modify or extend its settings to read directly from the modelfile each time it is used. You would need to explore its configuration files or documentation for more insights on this option. If AnythingLLM doesn't provide the exact option you're requesting yet, submitting a feature request for the checkbox or another solution would likely be the best approach for long-term ease of use.
yindo commented 2026-02-22 18:32:43 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Feb 9, 2026):

If you have a modelfile that has PARAMETERS like temperature we pass body arguments to the API during inference. The only ones we do pass are
use_mlock
temperature = default 0.7
n_ctx

Anything else will be inherited (or should) from the modelfile definition during inference

@timothycarambat commented on GitHub (Feb 9, 2026): If you have a modelfile that has `PARAMETERS` like temperature we pass body arguments to the API during inference. The only ones we do pass are `use_mlock` `temperature` = default 0.7 `n_ctx` Anything else will be inherited (or should) from the modelfile definition during inference
yindo commented 2026-02-22 18:32:43 -05:00
Author
Owner
Copy Link
Copy Source

@RecklessCharger commented on GitHub (Feb 9, 2026):

Or presumably the system prompt?

@RecklessCharger commented on GitHub (Feb 9, 2026): Or presumably the system prompt?
yindo changed title from [FEAT]: Allow use of custom modelfile system prompt and temperature to [GH-ISSUE #4976] [FEAT]: Allow use of custom modelfile system prompt and temperature 2026-06-05 14:50:26 -04:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Desktop
Docker
Integration Request
Integration Request
OS: Linux
OS: Mobile
OS: Windows
UI/UX
blocked
bug
bug
core-team-only
documentation
duplicate
embed-widget
enhancement
feature request
github_actions
good first issue
investigating
needs info / can't replicate
possible bug
pull-request
Mirrored from GitHub Pull Request
 question
stage: specifications
wontfix
No Label enhancement feature request
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Mintplex-Labs/anything-llm#3123
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?