Mintplex-Labs/anything-llm
1
0
Fork 0
mirror of https://github.com/Mintplex-Labs/anything-llm.git synced 2026-07-21 01:25:28 -04:00
Code Issues 328 Packages Projects Releases 10 Wiki Activity

[GH-ISSUE #1572] [FEAT]: Add ability to rewrite parameters in the json request. #1018

New Issue
Open
opened 2026-02-22 18:22:42 -05:00 by yindo · 1 comment
yindo commented 2026-02-22 18:22:42 -05:00
Owner
Copy Link
Copy Source

Originally created by @goodglitch on GitHub (May 30, 2024).
Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/1572

What would you like to see?

In case when "max_tokens" field is none my LLM server updates the field with too small tokens number. More precisely it is around 500 and most my coding requests got truncated. Please, add a button in the Options/LLM preferences that for each LLM can add item to an update dict with parameter and its value, like "max_tokens: 2048 or "top_k: 3". Then update json request with this dict. On the top there may be button to reset (clear update dict).

This feature will allow to fix many personal user problems by themselves and I think will be very welcomed by the community. Meanwhile, I have to edit LLM server code each time I update it, since all the changes got erased.

Originally created by @goodglitch on GitHub (May 30, 2024). Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/1572 ### What would you like to see? In case when "max_tokens" field is none my LLM server updates the field with too small tokens number. More precisely it is around 500 and most my coding requests got truncated. Please, add a button in the Options/LLM preferences that for each LLM can add item to an update dict with parameter and its value, like "max_tokens: 2048 or "top_k: 3". Then update json request with this dict. On the top there may be button to reset (clear update dict). This feature will allow to fix many personal user problems by themselves and I think will be very welcomed by the community. Meanwhile, I have to edit LLM server code each time I update it, since all the changes got erased.
yindo added the enhancementfeature request labels 2026-02-22 18:22:42 -05:00
yindo changed title from [FEAT]: Add ability to rewrite parameters in the json request. to [GH-ISSUE #1572] [FEAT]: Add ability to rewrite parameters in the json request. 2026-06-05 14:38:30 -04:00
yindo commented 2026-06-05 14:38:31 -04:00
Author
Owner
Copy Link
Copy Source

@ppil commented on GitHub (Mar 9, 2026):

Testing qwen3.5, this feature is more of a requirement to get that model functioning in an instruct mode. Even with reasoning-budget 0 and thinking disabled via template kwargs, the model disobeys and will still generate thought patterns unless the presence_penalty is set. I can set this via cli args on llama-server, but the API request that gets processed from anythingllm has presence_penalty set to 0, which takes precedence. With models becoming more dependent on these parameters, I think this should be revisited.

<!-- gh-comment-id:4028484153 --> @ppil commented on GitHub (Mar 9, 2026): Testing qwen3.5, this feature is more of a requirement to get that model functioning in an instruct mode. Even with reasoning-budget 0 and thinking disabled via template kwargs, the model disobeys and will still generate thought patterns unless the presence_penalty is set. I can set this via cli args on llama-server, but the API request that gets processed from anythingllm has presence_penalty set to 0, which takes precedence. With models becoming more dependent on these parameters, I think this should be revisited.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Desktop
Docker
Integration Request
Integration Request
OS: Linux
OS: Mobile
OS: Windows
UI/UX
blocked
bug
bug
core-team-only
documentation
duplicate
embed-widget
enhancement
feature request
github_actions
good first issue
investigating
needs info / can't replicate
possible bug
pull-request
Mirrored from GitHub Pull Request
 question
stage: specifications
wontfix
No Label enhancement feature request
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Mintplex-Labs/anything-llm#1018
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?