[Enhancement] Align functionality with vertex ai and gemini model #582

New Issue

Closed

opened 2026-02-16 10:19:48 -05:00 by yindo · 7 comments

yindo commented 2026-02-16 10:19:48 -05:00

Owner

Copy Link

Copy Source

Originally created by @AlexChim1231 on GitHub (Aug 22, 2025).

Self Checks

• I have read the Contributing Guide and Language Policy.
• I have searched for existing issues search for existing issues, including closed ones.
• I confirm that I am using English to submit this report, otherwise it will be closed.
• Please do not modify this template :) and fill in all the required fields.

1. Is this request related to a challenge you're experiencing? Tell me about your story.

Currently gemini 2.5 series model have params include_thoughts and thinking_budget but vertex ai doesn't
After checking the code, their implementation is different.

2. Additional context or comments

In Gemini plugin llm.py, it is using GenerateContentConfig to setup the config of the generation, it is ok

in Vertex Ai plugin llm.py, there are two different approach to request gemini, if the grounding set, it is using genai.Client same as Gemini plugin, it accept GenerateContentConfig. But if the grounding is not set, it is using glm.GenerativeModel and did not accept the thinking related params.

3. Can you help us with this feature?

• I am interested in contributing to this feature.

Originally created by @AlexChim1231 on GitHub (Aug 22, 2025). ### Self Checks - [x] I have read the [Contributing Guide](https://github.com/langgenius/dify/blob/main/CONTRIBUTING.md) and [Language Policy](https://github.com/langgenius/dify/issues/1542). - [x] I have searched for existing issues [search for existing issues](https://github.com/langgenius/dify-official-plugins/issues), including closed ones. - [x] I confirm that I am using English to submit this report, otherwise it will be closed. - [x] Please do not modify this template :) and fill in all the required fields. ### 1. Is this request related to a challenge you're experiencing? Tell me about your story. Currently gemini 2.5 series model have params include_thoughts and thinking_budget but vertex ai doesn't After checking the code, their implementation is different. ### 2. Additional context or comments In Gemini plugin llm.py, it is using GenerateContentConfig to setup the config of the generation, it is ok in Vertex Ai plugin llm.py, there are two different approach to request gemini, if the `grounding` set, it is using genai.Client same as Gemini plugin, it accept GenerateContentConfig. But if the `grounding` is not set, it is using glm.GenerativeModel and did not accept the thinking related params. ### 3. Can you help us with this feature? - [x] I am interested in contributing to this feature.

yindo added the enhancement label 2026-02-16 10:19:48 -05:00

yindo closed this issue 2026-02-16 10:19:49 -05:00

yindo commented 2026-02-16 10:19:49 -05:00

Author

Owner

Copy Link

Copy Source

@dosubot[bot] commented on GitHub (Aug 22, 2025):

You're right about the difference: the Gemini plugin uses genai.Client and GenerateContentConfig to support include_thoughts and thinking_budget for Gemini 2.5 models, while the Vertex AI plugin only supports these parameters when grounding is set and falls back to glm.GenerativeModel (which does not support thinking-related parameters) otherwise. This is due to an API limitation—glm.GenerativeModel and its GenerationConfig do not expose or accept these advanced reasoning options, while GenerateContentConfig does source.

To align the functionality, you could refactor the Vertex AI plugin so that all Gemini 2.x model requests use genai.Client and GenerateContentConfig, regardless of the grounding parameter. This would allow you to support include_thoughts and thinking_budget for all relevant models. The main code section to update is the branching logic in models/vertex_ai/models/llm/llm.py where it chooses between genai.Client and glm.GenerativeModel source.

For implementation, look at how the Gemini plugin's _set_thinking_config method sets these parameters on the config object, and apply a similar approach in the Vertex AI plugin. Make sure to handle model-specific compatibility (some models may not support thinking mode) as the Gemini plugin does with its blacklist logic source.

If you want to contribute, focus on refactoring the request logic in Vertex AI's llm.py so that Gemini 2.x models consistently use genai.Client and GenerateContentConfig, and add support for the reasoning parameters. Test with both streaming and non-streaming responses to ensure full compatibility.

Let me know if you want pointers to specific code lines or have questions about edge cases!

To reply, just mention @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}   

@dosubot[bot] commented on GitHub (Aug 22, 2025): <!-- Answer --> You're right about the difference: the Gemini plugin uses `genai.Client` and `GenerateContentConfig` to support `include_thoughts` and `thinking_budget` for Gemini 2.5 models, while the Vertex AI plugin only supports these parameters when `grounding` is set and falls back to `glm.GenerativeModel` (which does not support thinking-related parameters) otherwise. This is due to an API limitation—`glm.GenerativeModel` and its `GenerationConfig` do not expose or accept these advanced reasoning options, while `GenerateContentConfig` does [source](https://github.com/langgenius/dify-official-plugins/blob/b120688c39bfddf8f68274c4698b7c0d52d9cf3e/models/vertex_ai/models/llm/llm.py). To align the functionality, you could refactor the Vertex AI plugin so that all Gemini 2.x model requests use `genai.Client` and `GenerateContentConfig`, regardless of the `grounding` parameter. This would allow you to support `include_thoughts` and `thinking_budget` for all relevant models. The main code section to update is the branching logic in `models/vertex_ai/models/llm/llm.py` where it chooses between `genai.Client` and `glm.GenerativeModel` [source](https://github.com/langgenius/dify-official-plugins/blob/b120688c39bfddf8f68274c4698b7c0d52d9cf3e/models/vertex_ai/models/llm/llm.py). For implementation, look at how the Gemini plugin's `_set_thinking_config` method sets these parameters on the config object, and apply a similar approach in the Vertex AI plugin. Make sure to handle model-specific compatibility (some models may not support thinking mode) as the Gemini plugin does with its blacklist logic [source](https://github.com/langgenius/dify-official-plugins/blob/b120688c39bfddf8f68274c4698b7c0d52d9cf3e/models/gemini/models/llm/llm.py). If you want to contribute, focus on refactoring the request logic in Vertex AI's `llm.py` so that Gemini 2.x models consistently use `genai.Client` and `GenerateContentConfig`, and add support for the reasoning parameters. Test with both streaming and non-streaming responses to ensure full compatibility. Let me know if you want pointers to specific code lines or have questions about edge cases! <!-- Dosu Comment Footer --> *To reply, just mention [@dosu](https://go.dosu.dev/dosubot).* --- ^{How did I do? [Good](https://app.dosu.dev/response-feedback/2b05b41d-c5f4-4b4a-a63f-7c4cd262c736?feedback_type=great_response) | [Irrelevant](https://app.dosu.dev/response-feedback/2b05b41d-c5f4-4b4a-a63f-7c4cd262c736?feedback_type=irrelevant_answer) | [Incorrect](https://app.dosu.dev/response-feedback/2b05b41d-c5f4-4b4a-a63f-7c4cd262c736?feedback_type=incorrect_sources) | [Verbose](https://app.dosu.dev/response-feedback/2b05b41d-c5f4-4b4a-a63f-7c4cd262c736?feedback_type=too_verbose) | [Hallucination](https://app.dosu.dev/response-feedback/2b05b41d-c5f4-4b4a-a63f-7c4cd262c736?feedback_type=hallucination) | [Report 🐛](https://app.dosu.dev/response-feedback/2b05b41d-c5f4-4b4a-a63f-7c4cd262c736?feedback_type=bug_report) | [Other](https://app.dosu.dev/response-feedback/2b05b41d-c5f4-4b4a-a63f-7c4cd262c736?feedback_type=other)}&nbsp;&nbsp;[](https://go.dosu.dev/discord-bot)&nbsp;[](https://twitter.com/intent/tweet?text=%40dosu_ai%20helped%20me%20solve%20this%20issue!&url=https%3A//github.com/langgenius/dify-official-plugins/issues/1555)

yindo commented 2026-02-16 10:19:49 -05:00

Author

Owner

Copy Link

Copy Source

@AlexChim1231 commented on GitHub (Aug 22, 2025):

@dosubot
thank you

anyone can assist for this? i try to modify the vertex ai llm.py but not work, here is my code

elif model.startswith("gemini-2."):
        SCOPES = [
            "https://www.googleapis.com/auth/cloud-platform",
            "https://www.googleapis.com/auth/generative-language"
        ]
        credential = service_account.Credentials.from_service_account_info(
            service_account_info,
            scopes=SCOPES
        )
        client = genai.Client(credentials=credential, project=project_id, location=location, vertexai=True)
        
        config = types.GenerateContentConfig()
        self._set_thinking_config(
            config=config, model_parameters=model_parameters, model_name=model
        )
        
        print(prompt_messages)
        
        response = client.models.generate_content(
            model=model,
            contents=history,
            config=config
        )

@AlexChim1231 commented on GitHub (Aug 22, 2025): @dosubot thank you anyone can assist for this? i try to modify the vertex ai llm.py but not work, here is my code ```py elif model.startswith("gemini-2."): SCOPES = [ "https://www.googleapis.com/auth/cloud-platform", "https://www.googleapis.com/auth/generative-language" ] credential = service_account.Credentials.from_service_account_info( service_account_info, scopes=SCOPES ) client = genai.Client(credentials=credential, project=project_id, location=location, vertexai=True) config = types.GenerateContentConfig() self._set_thinking_config( config=config, model_parameters=model_parameters, model_name=model ) print(prompt_messages) response = client.models.generate_content( model=model, contents=history, config=config ) ```

yindo commented 2026-02-16 10:19:49 -05:00

Author

Owner

Copy Link

Copy Source

@dosubot[bot] commented on GitHub (Aug 22, 2025):

Your code is close, but there are a few key details to check for Gemini 2.x models in Vertex AI:

1. Contents formatting: The contents parameter for client.models.generate_content should be a list of dicts, not just the raw history or prompt_messages. You need to convert each item in history to a dict, like [item.to_dict() for item in history] see example.

2. Thinking config: Your use of self._set_thinking_config is correct if it matches the Gemini plugin's logic, including blacklist handling for unsupported models and proper mapping of include_thoughts and thinking_budget reference.

3. System instruction: If you want to set a system prompt, make sure to extract it from prompt_messages and pass it as system_instruction in GenerateContentConfig.

Here's a pattern you can follow:

elif model.startswith("gemini-2."):
    # ... credential setup ...
    client = genai.Client(credentials=credential, project=project_id, location=location, vertexai=True)
    config = types.GenerateContentConfig()
    self._set_thinking_config(config=config, model_parameters=model_parameters, model_name=model)
    # Build history as a list of glm.Content objects
    # history = ... (convert prompt_messages to glm.Content objects, merging roles as needed)
    response = client.models.generate_content(
        model=model,
        contents=[item.to_dict() for item in history],
        config=config
    )

If your code "did not work," please share the exact error message or behavior you saw—this will help pinpoint the issue. Common problems include passing the wrong format for contents or using unsupported parameters for certain models see related error.

Let me know if you need help with the history construction or want a more complete example!

To reply, just mention @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}   

@dosubot[bot] commented on GitHub (Aug 22, 2025): <!-- Answer --> Your code is close, but there are a few key details to check for Gemini 2.x models in Vertex AI: 1. **Contents formatting**: The `contents` parameter for `client.models.generate_content` should be a list of dicts, not just the raw `history` or `prompt_messages`. You need to convert each item in `history` to a dict, like `[item.to_dict() for item in history]` [see example](https://github.com/langgenius/dify-official-plugins/pull/1039). 2. **Thinking config**: Your use of `self._set_thinking_config` is correct if it matches the Gemini plugin's logic, including blacklist handling for unsupported models and proper mapping of `include_thoughts` and `thinking_budget` [reference](https://github.com/langgenius/dify-official-plugins/blob/b120688c39bfddf8f68274c4698b7c0d52d9cf3e/models/gemini/models/llm/llm.py). 3. **System instruction**: If you want to set a system prompt, make sure to extract it from `prompt_messages` and pass it as `system_instruction` in `GenerateContentConfig`. Here's a pattern you can follow: ```python elif model.startswith("gemini-2."): # ... credential setup ... client = genai.Client(credentials=credential, project=project_id, location=location, vertexai=True) config = types.GenerateContentConfig() self._set_thinking_config(config=config, model_parameters=model_parameters, model_name=model) # Build history as a list of glm.Content objects # history = ... (convert prompt_messages to glm.Content objects, merging roles as needed) response = client.models.generate_content( model=model, contents=[item.to_dict() for item in history], config=config ) ``` If your code "did not work," please share the exact error message or behavior you saw—this will help pinpoint the issue. Common problems include passing the wrong format for `contents` or using unsupported parameters for certain models [see related error](https://github.com/langgenius/dify-official-plugins/issues/1212). Let me know if you need help with the `history` construction or want a more complete example! <!-- Dosu Comment Footer --> *To reply, just mention [@dosu](https://go.dosu.dev/dosubot).* --- ^{How did I do? [Good](https://app.dosu.dev/response-feedback/ca9374d2-c51d-4f0e-852a-780c7e296d84?feedback_type=great_response) | [Irrelevant](https://app.dosu.dev/response-feedback/ca9374d2-c51d-4f0e-852a-780c7e296d84?feedback_type=irrelevant_answer) | [Incorrect](https://app.dosu.dev/response-feedback/ca9374d2-c51d-4f0e-852a-780c7e296d84?feedback_type=incorrect_sources) | [Verbose](https://app.dosu.dev/response-feedback/ca9374d2-c51d-4f0e-852a-780c7e296d84?feedback_type=too_verbose) | [Hallucination](https://app.dosu.dev/response-feedback/ca9374d2-c51d-4f0e-852a-780c7e296d84?feedback_type=hallucination) | [Report 🐛](https://app.dosu.dev/response-feedback/ca9374d2-c51d-4f0e-852a-780c7e296d84?feedback_type=bug_report) | [Other](https://app.dosu.dev/response-feedback/ca9374d2-c51d-4f0e-852a-780c7e296d84?feedback_type=other)}&nbsp;&nbsp;[](https://go.dosu.dev/discord-bot)&nbsp;[](https://twitter.com/intent/tweet?text=%40dosu_ai%20helped%20me%20solve%20this%20issue!&url=https%3A//github.com/langgenius/dify-official-plugins/issues/1555)

yindo commented 2026-02-16 10:19:49 -05:00

Author

Owner

Copy Link

Copy Source

@AlexChim1231 commented on GitHub (Aug 22, 2025):

I guess there has another issue for processing the message.

        elif model.startswith("gemini-2."):
            SCOPES = [
                "https://www.googleapis.com/auth/cloud-platform",
                "https://www.googleapis.com/auth/generative-language"
            ]
            credential = service_account.Credentials.from_service_account_info(
                service_account_info,
                scopes=SCOPES
            )
            client = genai.Client(credentials=credential, project=project_id, location=location, vertexai=True)
            
            if dynamic_threshold is not None:
                tools = self._convert_grounding_to_glm_tool(dynamic_threshold=dynamic_threshold)
            else:
                tools = self._convert_tools_to_glm_tool(tools) if tools else None
            
            config = types.GenerateContentConfig(
                response_modalities=["TEXT"],
                system_instruction=system_instruction,
                tools=tools
            )
            self._set_thinking_config(
                config=config, model_parameters=model_parameters, model_name=model
            )
            
            print("prompt_messages", prompt_messages)
            print("History: ", history)
            print("History after process: ", [item.to_dict() for item in history])
            
            response = client.models.generate_content(
                model=model,
                contents=[item.to_dict() for item in history],
                config=config,
            )

prompt_messages [SystemPromptMessage(role=<PromptMessageRole.SYSTEM: 'system'>, content='test', name='')]
History:  []
History after process:  []
...
ValueError: contents are required.

@AlexChim1231 commented on GitHub (Aug 22, 2025): I guess there has another issue for processing the message. ```py elif model.startswith("gemini-2."): SCOPES = [ "https://www.googleapis.com/auth/cloud-platform", "https://www.googleapis.com/auth/generative-language" ] credential = service_account.Credentials.from_service_account_info( service_account_info, scopes=SCOPES ) client = genai.Client(credentials=credential, project=project_id, location=location, vertexai=True) if dynamic_threshold is not None: tools = self._convert_grounding_to_glm_tool(dynamic_threshold=dynamic_threshold) else: tools = self._convert_tools_to_glm_tool(tools) if tools else None config = types.GenerateContentConfig( response_modalities=["TEXT"], system_instruction=system_instruction, tools=tools ) self._set_thinking_config( config=config, model_parameters=model_parameters, model_name=model ) print("prompt_messages", prompt_messages) print("History: ", history) print("History after process: ", [item.to_dict() for item in history]) response = client.models.generate_content( model=model, contents=[item.to_dict() for item in history], config=config, ) ``` ```bash prompt_messages [SystemPromptMessage(role=<PromptMessageRole.SYSTEM: 'system'>, content='test', name='')] History: [] History after process: [] ... ValueError: contents are required. ``` <img width="1405" height="870" alt="Image" src="https://github.com/user-attachments/assets/5161f985-ac8a-42a9-ba7f-b9a402c942b5" />

yindo commented 2026-02-16 10:19:49 -05:00

Author

Owner

Copy Link

Copy Source

@dosubot[bot] commented on GitHub (Sep 7, 2025):

Hi, @AlexChim1231. I'm Dosu, and I'm helping the dify-official-plugins team manage their backlog and am marking this issue as stale.

Issue Summary:

• You reported a discrepancy between Gemini 2.5 and Vertex AI plugins regarding support for parameters like include_thoughts and thinking_budget.
• The difference arises because Vertex AI uses two approaches, one of which lacks thinking-related support due to API limitations.
• It was suggested to refactor Vertex AI's llm.py to consistently use genai.Client and GenerateContentConfig for Gemini 2.x models to enable full parameter support.
• You tried modifying the code but encountered a "ValueError: contents are required" caused by empty or improperly formatted message history.
• I provided guidance to ensure the contents parameter is a list of dicts representing messages, with offers for further help on message formatting and integration.

Next Steps:

• Please let me know if this issue is still relevant with the latest version of the dify-official-plugins repository by commenting here to keep the discussion open.
• If I do not hear back within 5 days, this issue will be automatically closed to keep the backlog manageable.

Thank you for your understanding and contribution!

@dosubot[bot] commented on GitHub (Sep 7, 2025): Hi, @AlexChim1231. I'm [Dosu](https://dosu.dev), and I'm helping the dify-official-plugins team manage their backlog and am marking this issue as stale. **Issue Summary:** - You reported a discrepancy between Gemini 2.5 and Vertex AI plugins regarding support for parameters like include_thoughts and thinking_budget. - The difference arises because Vertex AI uses two approaches, one of which lacks thinking-related support due to API limitations. - It was suggested to refactor Vertex AI's llm.py to consistently use genai.Client and GenerateContentConfig for Gemini 2.x models to enable full parameter support. - You tried modifying the code but encountered a "ValueError: contents are required" caused by empty or improperly formatted message history. - I provided guidance to ensure the contents parameter is a list of dicts representing messages, with offers for further help on message formatting and integration. **Next Steps:** - Please let me know if this issue is still relevant with the latest version of the dify-official-plugins repository by commenting here to keep the discussion open. - If I do not hear back within 5 days, this issue will be automatically closed to keep the backlog manageable. Thank you for your understanding and contribution!

yindo commented 2026-02-16 10:19:49 -05:00

Author

Owner

Copy Link

Copy Source

@jortegac commented on GitHub (Sep 29, 2025):

Can this be reviewed again? Aligning this would also unlock the ability to use the image generation models in the vertex plugin for dify

@jortegac commented on GitHub (Sep 29, 2025): Can this be reviewed again? Aligning this would also unlock the ability to use the image generation models in the vertex plugin for dify

yindo commented 2026-02-16 10:19:49 -05:00

Author

Owner

Copy Link

Copy Source

@AlexChim1231 commented on GitHub (Sep 29, 2025):

@jortegac maybe you can find @crazywoola for this

@AlexChim1231 commented on GitHub (Sep 29, 2025): @jortegac maybe you can find @crazywoola for this

Sign in to join this conversation.

No Branch/Tag Specified

Branches Tags

main

feat/3479-gemini-flash-models

dependabot/uv/tools/chart/uv-cdf8d38825

feat/moonshot-kimi-k3-k27-code

feat/gemini-system-instruction-semantics

fix/gemini-text-system-parts

dosu/doc-update-3436

feat/openai-gpt-5-6-support

fix/outlook-trigger-readme-permissions

fix/microsoft-oauth-datasource-tenant

fix/byteplus-api-base-url-label

feat/ark-video-generation-custom-models

fix/openai-stream-reasoning-thinking

fix/skip-live-tests-missing-keys

codex/enhance-discord-integrations

codex/harden-trigger-oauth-permissions

update-lemonade-default-port

codex/fix-pr-review-ci

feat/ark-multimodal-polling

copilot/update-mineru-3x-plugin-file-types

feat/action-security-fix

feat/claude-fable5

codex/fix-ollama-model-category

codex/upgrade-ollama-latest

feat/minimax-m3

codex/refine-plugin-readmes

fix/fireworks-model-label-zh-hans

chore/version-bump-plugins-batch-06

chore/version-bump-plugins-batch-05

chore/version-bump-plugins-batch-04

chore/version-bump-plugins-batch-03

chore/version-bump-plugins-batch-02

fix/plugin-sdk-i18n-fields

chore/version-bump-plugins-batch-01

chore/dependencies-high-priority

chore/dependencies-medium-priority

chore/dependencies-low-priority-clean

chore/dependencies-low-priority

fix/tongyi-align-deepseek-model-metadata

codex/update-openrouter-models

codex/azure-ai-studio-cjk-version-bump

feat/ernie-image-tool

claude/cranky-thompson-b6a281

fix/remove-unavailable-volcengine-models

fix/legacy-plugin-daemon-compatibility

copilot/add-deepseek-v4-to-openrouter

codex/merge-tongyi-tts-prs

codex/depot-runners

chore/migrate-selected-plugins-to-uv-lock

fix/repair-failed-plugin-deploys

ci/fix-plugin-change-detection

chore/migrate-aws-s3-storage-to-uv-lock

chore/migrate-somark-dependencies-to-uv-lock

ci/uv-lock-dependency-validation

fix/2911-openai-api-compatible-tool-call-content

chore/remove-deprecated-plugins

copilot/add-more-models-openrouter-provider

copilot/fix-error-connecting-fireworks-ai

fix/openai-compatible-max-completion-tokens-validation

fix/gemini-lockfile

chore/add-React-fallback-desc

feat/add-gemini-embedding-2-preview

copilot/add-claude-sonnet-opus-4-6

codex/update-xai-models-2721

fix/remove-sentence-transformer-for-hubspot

feat/add-gpt-5.4

copilot/fix-ci-errors-and-bump-version

copilot/fix-security-errors-combined

copilot/update-gemini-model-version

fix/anthropic-parallel-tool-call

codex/fix-azure-ai-studio-azure-core-1-38

codex/fix-azure-blob-azure-core-1-38

codex/fix-azure-openai-azure-core-1-38

gitlab-credentials-c

fix/openrouter-validation-model

fix/gemini-null-checks

fix/deepseek-tool-message-ignore

fix/gemini-dependencies

No results found.

Labels

Clear labels

bug

bug

bug

dependencies

documentation

enhancement

good first issue

invalid

pull-request

Mirrored from GitHub Pull Request

question

stale

No Label enhancement

Milestone

No items

No Milestone

Projects

Clear projects

No project

Assignees

Clear assignees

yindo

No Assignees

1 Participants

Notifications

Subscribe

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: langgenius/dify-official-plugins#582