langgenius/dify-official-plugins
1
0
Fork 0
mirror of https://github.com/langgenius/dify-official-plugins.git synced 2026-07-22 10:05:23 -04:00
Code Issues 54 Packages Projects Releases Wiki Activity

[openai_api_compatible] Top-level "thinking" breaks compatibility with providers/models that don't support it #918

New Issue
Closed
opened 2026-02-16 10:21:00 -05:00 by yindo · 1 comment
yindo commented 2026-02-16 10:21:00 -05:00
Owner
Copy Link
Copy Source

Originally created by @utsumi-fj on GitHub (Jan 6, 2026).

Self Checks

  • This is only for bug report, if you would like to ask a question, please head to Discussions.
  • I have searched for existing issues Dify issues & Dify Official Plugins, including closed ones.
  • I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
  • [FOR CHINESE USERS] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:）
  • Please do not modify this template :) and fill in all the required fields.

Dify version

1.11.1

Plugin version

0.0.29

Cloud or Self Hosted

Self Hosted (Docker)

Steps to reproduce

When using OpenAI-API-compatible plugin, requests fail on providers/models that do not support the top-level thinking parameter.

Error (screenshot):
Image

This regression was introduced by commit 04dcfdc4d2 (#2363), which adds the top-level thinking parameter to requests. Reverting that commit makes the issue disappear.

In my case, this happens with the model hosted on vLLM accessed via LiteLLM using the hosted_vllm/ prefix.

LiteLLM's drop_params can work around this in my setup, but other providers/models may not have an equivalent option, so this breaks compatibility more broadly.

✔️ Error log

Run failed: req_id: 9c1d871dcb PluginInvokeError: {"args":{"description":"[models] Error: API request failed with status code 400: {\"error\":{\"message\":\"litellm.UnsupportedParamsError: hosted_vllm does not support parameters: ['thinking'], for model=ISTA-DASLab/gemma-3-27b-it-GPTQ-4b-128g. To drop these, set `litellm.drop_params=True` or for proxy:\\n\\n`litellm_settings:\\n drop_params: true`\\n. \\n If you want to use these params dynamically send allowed_openai_params=['thinking'] in your request.. Received Model Group=ISTA-DASLab/gemma-3-27b-it-GPTQ-4b-128g\\nAvailable Model Group Fallbacks=None\",\"type\":\"None\",\"param\":null,\"code\":\"400\"}}"},"error_type":"InvokeError","message":"[models] Error: API request failed with status code 400: {\"error\":{\"message\":\"litellm.UnsupportedParamsError: hosted_vllm does not support parameters: ['thinking'], for model=ISTA-DASLab/gemma-3-27b-it-GPTQ-4b-128g. To drop these, set `litellm.drop_params=True` or for proxy:\\n\\n`litellm_settings:\\n drop_params: true`\\n. \\n If you want to use these params dynamically send allowed_openai_params=['thinking'] in your request.. Received Model Group=ISTA-DASLab/gemma-3-27b-it-GPTQ-4b-128g\\nAvailable Model Group Fallbacks=None\",\"type\":\"None\",\"param\":null,\"code\":\"400\"}}"}
Originally created by @utsumi-fj on GitHub (Jan 6, 2026). ### Self Checks - [x] This is only for bug report, if you would like to ask a question, please head to [Discussions](https://github.com/langgenius/dify/discussions/categories/general). - [x] I have searched for existing issues [Dify issues](https://github.com/langgenius/dify/issues) & [Dify Official Plugins](https://github.com/langgenius/dify-official-plugins/issues), including closed ones. - [x] I confirm that I am using English to submit this report (我已阅读并同意 [Language Policy](https://github.com/langgenius/dify/issues/1542)). - [x] [FOR CHINESE USERS] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:） - [x] Please do not modify this template :) and fill in all the required fields. ### Dify version 1.11.1 ### Plugin version 0.0.29 ### Cloud or Self Hosted Self Hosted (Docker) ### Steps to reproduce When using OpenAI-API-compatible plugin, requests fail on providers/models that do not support the top-level `thinking` parameter. Error (screenshot): <img width="1893" height="944" alt="Image" src="https://github.com/user-attachments/assets/e643252f-6b68-4763-9c1a-e39f54b95a27" /> This regression was introduced by commit 04dcfdc4d2f2e2bba3829eaecb6089d587557267 (#2363), which adds the top-level `thinking` parameter to requests. Reverting that commit makes the issue disappear. In my case, this happens with the model hosted on [vLLM](https://github.com/vllm-project/vllm) accessed via [LiteLLM](https://github.com/BerriAI/litellm) using the [`hosted_vllm/` prefix](https://docs.litellm.ai/docs/providers/vllm). [LiteLLM's drop_params](https://docs.litellm.ai/docs/completion/drop_params) can work around this in my setup, but other providers/models may not have an equivalent option, so this breaks compatibility more broadly. ### ✔️ Error log ``` Run failed: req_id: 9c1d871dcb PluginInvokeError: {"args":{"description":"[models] Error: API request failed with status code 400: {\"error\":{\"message\":\"litellm.UnsupportedParamsError: hosted_vllm does not support parameters: ['thinking'], for model=ISTA-DASLab/gemma-3-27b-it-GPTQ-4b-128g. To drop these, set `litellm.drop_params=True` or for proxy:\\n\\n`litellm_settings:\\n drop_params: true`\\n. \\n If you want to use these params dynamically send allowed_openai_params=['thinking'] in your request.. Received Model Group=ISTA-DASLab/gemma-3-27b-it-GPTQ-4b-128g\\nAvailable Model Group Fallbacks=None\",\"type\":\"None\",\"param\":null,\"code\":\"400\"}}"},"error_type":"InvokeError","message":"[models] Error: API request failed with status code 400: {\"error\":{\"message\":\"litellm.UnsupportedParamsError: hosted_vllm does not support parameters: ['thinking'], for model=ISTA-DASLab/gemma-3-27b-it-GPTQ-4b-128g. To drop these, set `litellm.drop_params=True` or for proxy:\\n\\n`litellm_settings:\\n drop_params: true`\\n. \\n If you want to use these params dynamically send allowed_openai_params=['thinking'] in your request.. Received Model Group=ISTA-DASLab/gemma-3-27b-it-GPTQ-4b-128g\\nAvailable Model Group Fallbacks=None\",\"type\":\"None\",\"param\":null,\"code\":\"400\"}}"} ```
yindo added the bug label 2026-02-16 10:21:00 -05:00
yindo closed this issue 2026-02-16 10:21:00 -05:00
yindo commented 2026-02-16 10:21:00 -05:00
Author
Owner
Copy Link
Copy Source

@dosubot[bot] commented on GitHub (Jan 6, 2026):

The unconditional addition of the top-level thinking parameter in OpenAI-compatible plugin requests breaks compatibility with providers/models that don't support it, like vLLM via LiteLLM, resulting in 400 errors. This regression was introduced by a plugin SDK upgrade and related commits that added support for "thinking mode" in some models, but the parameter is sent to all providers regardless of compatibility (PR #191, PR #1851, PR #2129).

LiteLLM's drop_params=True can filter out unsupported parameters, but not all providers offer this, so the issue affects more than just LiteLLM setups. There is no conditional logic in the Dify core plugin client to add or remove the thinking parameter based on provider compatibility—it is emitted unconditionally if present in the plugin or model config. No official fix or patch is available yet; the only workarounds are to revert the change or filter the parameter at the proxy/provider level if possible.

This is a known pattern with OpenAI-compatible plugins: sending unsupported parameters (like response_format.json_schema, json_schema, or thinking) to providers that don't recognize them causes failures (example issue). The best long-term solution would be for maintainers to add conditional logic so thinking is only sent to providers/models that support it, or to provide a configuration option to disable it. For now, using a proxy that can drop unsupported parameters is the most reliable workaround if reverting is not possible.

To reply, just mention @dosu.

How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other

Open in Cursor Join Discord Share on X

@dosubot[bot] commented on GitHub (Jan 6, 2026): <!-- Answer --> The unconditional addition of the top-level `thinking` parameter in OpenAI-compatible plugin requests breaks compatibility with providers/models that don't support it, like vLLM via LiteLLM, resulting in 400 errors. This regression was introduced by a plugin SDK upgrade and related commits that added support for "thinking mode" in some models, but the parameter is sent to all providers regardless of compatibility ([PR #191](https://github.com/langgenius/dify-official-plugins/pull/191), [PR #1851](https://github.com/langgenius/dify-official-plugins/pull/1851), [PR #2129](https://github.com/langgenius/dify-official-plugins/pull/2129)). LiteLLM's `drop_params=True` can filter out unsupported parameters, but not all providers offer this, so the issue affects more than just LiteLLM setups. There is no conditional logic in the Dify core plugin client to add or remove the `thinking` parameter based on provider compatibility—it is emitted unconditionally if present in the plugin or model config. No official fix or patch is available yet; the only workarounds are to revert the change or filter the parameter at the proxy/provider level if possible. This is a known pattern with OpenAI-compatible plugins: sending unsupported parameters (like `response_format.json_schema`, `json_schema`, or `thinking`) to providers that don't recognize them causes failures ([example issue](https://github.com/langgenius/dify-official-plugins/issues/1568)). The best long-term solution would be for maintainers to add conditional logic so `thinking` is only sent to providers/models that support it, or to provide a configuration option to disable it. For now, using a proxy that can drop unsupported parameters is the most reliable workaround if reverting is not possible. <!-- Dosu Comment Footer --> *To reply, just mention [@dosu](https://go.dosu.dev/dosubot).* --- <sup>How did I do? [Good](https://app.dosu.dev/response-feedback/8ecd3cd5-30a1-4070-8bb6-cd344508c225?feedback_type=great_response) | [Irrelevant](https://app.dosu.dev/response-feedback/8ecd3cd5-30a1-4070-8bb6-cd344508c225?feedback_type=irrelevant_answer) | [Incorrect](https://app.dosu.dev/response-feedback/8ecd3cd5-30a1-4070-8bb6-cd344508c225?feedback_type=incorrect_sources) | [Verbose](https://app.dosu.dev/response-feedback/8ecd3cd5-30a1-4070-8bb6-cd344508c225?feedback_type=too_verbose) | [Hallucination](https://app.dosu.dev/response-feedback/8ecd3cd5-30a1-4070-8bb6-cd344508c225?feedback_type=hallucination) | [Report 🐛](https://app.dosu.dev/response-feedback/8ecd3cd5-30a1-4070-8bb6-cd344508c225?feedback_type=bug_report) | [Other](https://app.dosu.dev/response-feedback/8ecd3cd5-30a1-4070-8bb6-cd344508c225?feedback_type=other)</sup> [![Open in Cursor](https://dosu.dev/dosu-in-cursor.svg)](https://cursor.com/link/prompt?text=The%20unconditional%20addition%20of%20the%20top-level%20%60thinking%60%20parameter%20in%20OpenAI-compatible%20plugin%20requests%20breaks%20compatibility%20with%20providers/models%20that%20don%27t%20support%20it%2C%20like%20vLLM%20via%20LiteLLM%2C%20resulting%20in%20400%20errors.%20This%20regression%20was%20introduced%20by%20a%20plugin%20SDK%20upgrade%20and%20related%20commits%20that%20added%20support%20for%20%22thinking%20mode%22%20in%20some%20models%2C%20but%20the%20parameter%20is%20sent%20to%20all%20providers%20regardless%20of%20compatibility%C2%A0%28%5BPR%20%23191%5D%28https%3A//github.com/langgenius/dify-official-plugins/pull/191%29%2C%20%5BPR%20%231851%5D%28https%3A//github.com/langgenius/dify-official-plugins/pull/1851%29%2C%20%5BPR%20%232129%5D%28https%3A//github.com/langgenius/dify-official-plugins/pull/2129%29%29.%0A%0ALiteLLM%27s%20%60drop_params%3DTrue%60%20can%20filter%20out%20unsupported%20parameters%2C%20but%20not%20all%20providers%20offer%20this%2C%20so%20the%20issue%20affects%20more%20than%20just%20LiteLLM%20setups.%20There%20is%20no%20conditional%20logic%20in%20the%20Dify%20core%20plugin%20client%20to%20add%20or%20remove%20the%20%60thinking%60%20parameter%20based%20on%20provider%20compatibility%E2%80%94it%20is%20emitted%20unconditionally%20if%20present%20in%20the%20plugin%20or%20model%20config.%20No%20official%20fix%20or%20patch%20is%20available%20yet%3B%20the%20only%20workarounds%20are%20to%20revert%20the%20change%20or%20filter%20the%20parameter%20at%20the%20proxy/provider%20level%20if%20possible.%0A%0AThis%20is%20a%20known%20pattern%20with%20OpenAI-compatible%20plugins%3A%20sending%20unsupported%20parameters%20%28like%20%60response_format.json_schema%60%2C%20%60json_schema%60%2C%20or%20%60thinking%60%29%20to%20providers%20that%20don%27t%20recognize%20them%20causes%20failures%C2%A0%28%5Bexample%20issue%5D%28https%3A//github.com/langgenius/dify-official-plugins/issues/1568%29%29.%20The%20best%20long-term%20solution%20would%20be%20for%20maintainers%20to%20add%20conditional%20logic%20so%20%60thinking%60%20is%20only%20sent%20to%20providers/models%20that%20support%20it%2C%20or%20to%20provide%20a%20configuration%20option%20to%20disable%20it.%20For%20now%2C%20using%20a%20proxy%20that%20can%20drop%20unsupported%20parameters%20is%20the%20most%20reliable%20workaround%20if%20reverting%20is%20not%20possible.)&nbsp;[![Join Discord](https://img.shields.io/badge/join-5865F2?logo=discord&logoColor=white&label=)](https://go.dosu.dev/discord-bot)&nbsp;[![Share on X](https://img.shields.io/badge/X-share-black)](https://twitter.com/intent/tweet?text=%40dosu_ai%20helped%20me%20solve%20this%20issue!&url=https%3A//github.com/langgenius/dify-official-plugins/issues/2366)
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
bug
bug
bug
dependencies
documentation
enhancement
good first issue
invalid
pull-request
Mirrored from GitHub Pull Request
 question
stale
No Label bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: langgenius/dify-official-plugins#918
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?