Mintplex-Labs/anything-llm
1
0
Fork 0
mirror of https://github.com/Mintplex-Labs/anything-llm.git synced 2026-07-19 14:13:35 -04:00
Code Issues 328 Packages Projects Releases 10 Wiki Activity

[GH-ISSUE #4045] [BUG]: LiteLLM Drop Params for o-* models #2572

New Issue
Closed
opened 2026-02-22 18:30:19 -05:00 by yindo · 6 comments
yindo commented 2026-02-22 18:30:19 -05:00
Owner
Copy Link
Copy Source

Originally created by @buildgreatthings on GitHub (Jun 24, 2025).
Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/4045

Originally assigned to: @shatfield4 on GitHub.

How are you running AnythingLLM?

Docker (local)

What happened?

The temperature isn't allowed in certain OpenAI models, but still gets sent from AnythingLLM when pointing to a LiteLLm service.

Image

Are there known steps to reproduce?

No response

Originally created by @buildgreatthings on GitHub (Jun 24, 2025). Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/4045 Originally assigned to: @shatfield4 on GitHub. ### How are you running AnythingLLM? Docker (local) ### What happened? The temperature isn't allowed in certain OpenAI models, but still gets sent from AnythingLLM when pointing to a LiteLLm service. ![Image](https://github.com/user-attachments/assets/17d7627b-10a8-4ec3-a265-83d5a6518873) ### Are there known steps to reproduce? _No response_
yindo added the bugIntegration Request labels 2026-02-22 18:30:19 -05:00
yindo closed this issue 2026-02-22 18:30:19 -05:00
yindo commented 2026-02-22 18:30:19 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Jun 24, 2025):

Adding dropParams will likelt cause issues with non-OpenAI providers since if we add more params that do work with some provider but not with other they will be dropped.

That being said, setting temp to 1 should be permitted and you can set that in the workspace settings for now.

note
By default, LiteLLM raises an exception if the openai param being passed in isn't supported.

To drop the param instead, set litellm.drop_params = True or completion(..drop_params=True).

This ONLY DROPS UNSUPPORTED OPENAI PARAMS.

LiteLLM assumes any non-openai param is provider specific and passes it in as a kwarg in the request body

@timothycarambat commented on GitHub (Jun 24, 2025): Adding dropParams will likelt cause issues with non-OpenAI providers since if we add more params that do work with some provider but not with other they will be dropped. That being said, setting temp to `1` should be permitted and you can set that in the workspace settings for now. > note > By default, LiteLLM raises an exception if the openai param being passed in isn't supported. > > To drop the param instead, set litellm.drop_params = True or completion(..drop_params=True). > > This ONLY DROPS UNSUPPORTED OPENAI PARAMS. > > LiteLLM assumes any non-openai param is provider specific and passes it in as a kwarg in the request body
yindo commented 2026-02-22 18:30:19 -05:00
Author
Owner
Copy Link
Copy Source

@buildgreatthings commented on GitHub (Jun 24, 2025):

In our experience, dropParams is added just for LiteLLM calls. LiteLLM has a catalog of known accepted params by provider based on what they've put in, fireworks, together, etc. We can also override that on LiteLLm server if the params are out of date.

The dropParams should not be added for calls directly to OpenAI or someone else from AnythingLLM.

@buildgreatthings commented on GitHub (Jun 24, 2025): In our experience, dropParams is **added just for** LiteLLM calls. LiteLLM has a catalog of known accepted params by provider based on what they've put in, fireworks, together, etc. We can also override that on LiteLLm server if the params are out of date. The dropParams should not be added for calls directly to OpenAI or someone else from AnythingLLM.
yindo commented 2026-02-22 18:30:19 -05:00
Author
Owner
Copy Link
Copy Source

@timothycarambat commented on GitHub (Jun 24, 2025):

Right, so it would exist in something like:
https://github.com/Mintplex-Labs/anything-llm/blob/91e498229cc6893744ae718df3479cd742c0e2e3/server/utils/AiProviders/liteLLM/index.js#L130-L134

My question was if adding that param but then having the liteLLM model be something like openrouter/llama-xyz would cause other provider-specific params to be dropped that arent in the OpenAI spec.

If LiteLLM maintains its own internal list of available params per provider, then drop_params should work as we are discussing, yeah?

@timothycarambat commented on GitHub (Jun 24, 2025): Right, so it would exist in something like: https://github.com/Mintplex-Labs/anything-llm/blob/91e498229cc6893744ae718df3479cd742c0e2e3/server/utils/AiProviders/liteLLM/index.js#L130-L134 My question was if adding that param but then having the liteLLM model be something like openrouter/llama-xyz would cause other provider-specific params to be dropped that arent in the OpenAI spec. If LiteLLM maintains its own internal list of available params per provider, then drop_params should work as we are discussing, yeah?
yindo commented 2026-02-22 18:30:19 -05:00
Author
Owner
Copy Link
Copy Source

@buildgreatthings commented on GitHub (Jun 25, 2025):

yup, should be good. See this for reference.

https://docs.litellm.ai/docs/proxy/configs#set-custom-prompt-templates

@buildgreatthings commented on GitHub (Jun 25, 2025): yup, should be good. See this for reference. https://docs.litellm.ai/docs/proxy/configs#set-custom-prompt-templates
yindo commented 2026-02-22 18:30:19 -05:00
Author
Owner
Copy Link
Copy Source

@buildgreatthings commented on GitHub (Jun 25, 2025):

Your code location also looks right as well.

@buildgreatthings commented on GitHub (Jun 25, 2025): Your code location also looks right as well.
yindo commented 2026-02-22 18:30:20 -05:00
Author
Owner
Copy Link
Copy Source

@shatfield4 commented on GitHub (Jun 26, 2025):

@buildgreatthings After some testing and reading though LiteLLM's docs, I found that you can just add this to the bottom of your config.yaml

litellm_settings:
drop_params: True

This has the same functionality as adding the drop_params: true to the LiteLLM provider class inside of AnythingLLM. Please use that config in your LiteLLM config file. We are hesitant to update the LiteLLM provider code directly inside AnythingLLM because this may cause issues for existing users.

@shatfield4 commented on GitHub (Jun 26, 2025): @buildgreatthings After some testing and reading though LiteLLM's docs, I found that you can just add this to the bottom of your `config.yaml` litellm_settings: drop_params: True This has the same functionality as adding the `drop_params: true` to the LiteLLM provider class inside of AnythingLLM. Please use that config in your LiteLLM config file. We are hesitant to update the LiteLLM provider code directly inside AnythingLLM because this may cause issues for existing users.
yindo changed title from [BUG]: LiteLLM Drop Params for `o-*` models to [GH-ISSUE #4045] [BUG]: LiteLLM Drop Params for `o-*` models 2026-06-05 14:47:20 -04:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Desktop
Docker
Integration Request
Integration Request
OS: Linux
OS: Mobile
OS: Windows
UI/UX
blocked
bug
bug
core-team-only
documentation
duplicate
embed-widget
enhancement
feature request
github_actions
good first issue
investigating
needs info / can't replicate
possible bug
pull-request
Mirrored from GitHub Pull Request
 question
stage: specifications
wontfix
No Label Integration Request bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Mintplex-Labs/anything-llm#2572
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?