langgenius/dify-official-plugins
1
0
Fork 0
mirror of https://github.com/langgenius/dify-official-plugins.git synced 2026-07-21 17:45:23 -04:00
Code Issues 54 Packages Projects Releases Wiki Activity

[PR #1488] [MERGED] fix(gemini): dealing with a series of nasty bugs that cause the model to perform poorly #1920

New Issue
Closed
opened 2026-02-16 11:15:14 -05:00 by yindo · 0 comments
yindo commented 2026-02-16 11:15:14 -05:00
Owner
Copy Link
Copy Source

📋 Pull Request Information

Original PR: https://github.com/langgenius/dify-official-plugins/pull/1488
Author: @QIN2DIM
Created: 8/11/2025
Status: ✅ Merged
Merged: 8/11/2025
Merged by: @crazywoola

Base: main ← Head: gemini-think-parts

📝 Commits (8)

  • 752cf3a fix(llm): improve Gemini LLM integration methods
  • 98cee91 Merge branch 'main' into gemini-think-parts
  • eecd981 fix(llm): restructure gemini content building and message handling
  • edeaba5 Merge branch 'main' into gemini-think-parts
  • 394fab1 build(manifest): update version to 0.4.0
  • 8700045 feat(gemini-llm): add support for multimodal content handling and grounding sources
  • 2b7deec refactor(llm): remove debug prints from GoogleLargeLanguageModel
  • 9c51b72 test(llm): add comprehensive tests for multimodal content conversion

📊 Changes

4 files changed (+889 additions, -337 deletions)

View changed files

📝 models/gemini/manifest.yaml (+1 -1)
📝 models/gemini/models/llm/llm.py (+218 -271)
📝 models/gemini/models/llm/test_llm.py (+669 -65)
📝 models/gemini/pyproject.toml (+1 -0)

📄 Description

Related Issues or Context

This PR addresses several core issues in the integration between Dify Chatflow and the Gemini GenAI SDK.

1. system_instructions Replay Issue

2. thinking Content Leakage

  • Problem:
    The original flow included reasoning content (e.g., <think>xxx</think>) in the conversation contents.

    • This could introduce unsafe content.
    • It could also cause severe performance degradation in multi-turn or long-context conversations.
    • Notably, in a pure Gemini-only system, storing Gemini’s own reasoning chain in memory does not significantly impact performance. However, in Dify Chatflow, multi-turn interactions may involve different reasoning models. Reasoning traces from non-Gemini models can “pollute” subsequent Gemini outputs.
    • The core cause is that Gemini’s reasoning chain is essentially a concise, length-controlled “thinking summary,” while other models may generate extremely verbose “self-correction” logs.
    • Structural Limitation:
      • Case 1 – Sequential Multi-Model Pipeline:
        As shown in the first example flow, when Gemini 2.5 Flash is placed after another LLM in the same branch, its content cannot be reliably removed. Although regex could strip matching pairs from raw text, this is risky—without a channel-level structure identifier, the block is indistinguishable from legitimate parts of the final answer. image
      • Case 2 – Separate Conditional Branches:
        As shown in the second example flow, when multiple reasoning models operate in different conditional branches (i.e., not sequentially in the same branch), the current method works safely, since reasoning outputs from one model do not enter the conversation state of another. image image

  • Fix:
    When constructing contents, attempt to remove reasoning-channel data from memory.

3. function calling Role Mislabeling

4. Token Counter Miscalculation

  • Problem:
    The Gemini GenAI SDK directly returns the total token usage (including reasoning, responses, and tool calls). The correct behavior is to use this total value directly instead of summing it with other counts.
  • Fix:
    TODO — Some token counting logic is implemented in the lower-level Dify SDK, so this PR does not yet modify that part of the code.

5. Type Hints and SDK Compatibility

  • Problem:
    The source code contained multiple incorrect Gemini type usages (partly due to SDK version changes, where some types are now deprecated).
  • Fix:
    Updated all Gemini type references to their correct values to improve IDE autocomplete and AI-assisted coding accuracy.

6. Comprehensive Tests for contents History State

  • Problem:
    The existing test coverage for conversation contents history state was insufficient to ensure correctness across different scenarios.
  • Fix:
    Added a full set of test cases covering various conversation states, message types, and role assignments to ensure contents is correctly managed across the conversation lifecycle.

This PR contains Changes to Non-Plugin

  • Documentation
  • Other

This PR contains Changes to Non-LLM Models Plugin

  • I have Run Comprehensive Tests Relevant to My Changes

This PR contains Changes to LLM Models Plugin

  • My Changes Affect Message Flow Handling (System Messages and User→Assistant Turn-Taking)
  • My Changes Affect Tool Interaction Flow (Multi-Round Usage and Output Handling, for both Agent App and Agent Node)
  • My Changes Affect Multimodal Input Handling (Images, PDFs, Audio, Video, etc.)
  • My Changes Affect Multimodal Output Generation (Images, Audio, Video, etc.)
  • My Changes Affect Structured Output Format (JSON, XML, etc.)
  • My Changes Affect Token Consumption Metrics
  • My Changes Affect Other LLM Functionalities (Reasoning Process, Grounding, Prompt Caching, etc.)
  • Other Changes (Add New Models, Fix Model Parameters etc.)

Version Control (Any Changes to the Plugin Will Require Bumping the Version)

  • I have Bumped Up the Version in Manifest.yaml (Top-Level Version Field, Not in Meta Section)

Dify Plugin SDK Version

  • I have Ensured dify_plugin>=0.3.0,<0.5.0 is in requirements.txt (SDK docs)

Environment Verification (If Any Code Changes)

Local Deployment Environment

  • Dify Version is: , I have Tested My Changes on Local Deployment Dify with a Clean Environment That Matches the Production Configuration.

SaaS Environment

  • I have Tested My Changes on cloud.dify.ai with a Clean Environment That Matches the Production Configuration

🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.

## 📋 Pull Request Information **Original PR:** https://github.com/langgenius/dify-official-plugins/pull/1488 **Author:** [@QIN2DIM](https://github.com/QIN2DIM) **Created:** 8/11/2025 **Status:** ✅ Merged **Merged:** 8/11/2025 **Merged by:** [@crazywoola](https://github.com/crazywoola) **Base:** `main` ← **Head:** `gemini-think-parts` --- ### 📝 Commits (8) - [`752cf3a`](https://github.com/langgenius/dify-official-plugins/commit/752cf3ac543a108a4a1331aa2ea728cd09ed158d) fix(llm): improve Gemini LLM integration methods - [`98cee91`](https://github.com/langgenius/dify-official-plugins/commit/98cee9173179b0aa53ede0dbc4ffc24a254cd1d7) Merge branch 'main' into gemini-think-parts - [`eecd981`](https://github.com/langgenius/dify-official-plugins/commit/eecd9811459dca7602dfd3f22d03291886a820db) fix(llm): restructure gemini content building and message handling - [`edeaba5`](https://github.com/langgenius/dify-official-plugins/commit/edeaba5f18244082b615f3decf02bd0968fbf4bd) Merge branch 'main' into gemini-think-parts - [`394fab1`](https://github.com/langgenius/dify-official-plugins/commit/394fab1b5ea8fed2043600a2c288bc187d826daf) build(manifest): update version to 0.4.0 - [`8700045`](https://github.com/langgenius/dify-official-plugins/commit/8700045d1362ada402c299e53e15d6385d93ecfb) feat(gemini-llm): add support for multimodal content handling and grounding sources - [`2b7deec`](https://github.com/langgenius/dify-official-plugins/commit/2b7deec0de62fe39ba43177d8405944ab7d9760b) refactor(llm): remove debug prints from GoogleLargeLanguageModel - [`9c51b72`](https://github.com/langgenius/dify-official-plugins/commit/9c51b7202c17e9ce75f342eca1746ce05de14b00) test(llm): add comprehensive tests for multimodal content conversion ### 📊 Changes **4 files changed** (+889 additions, -337 deletions) <details> <summary>View changed files</summary> 📝 `models/gemini/manifest.yaml` (+1 -1) 📝 `models/gemini/models/llm/llm.py` (+218 -271) 📝 `models/gemini/models/llm/test_llm.py` (+669 -65) 📝 `models/gemini/pyproject.toml` (+1 -0) </details> ### 📄 Description ## Related Issues or Context <!-- ⚠️ NOTE: This repository is for Dify Official Plugins only. For community contributions, please submit to https://github.com/langgenius/dify-plugins instead. - Link Related Issues if Applicable: #issue_number - Or Provide Context about Why this Change is Needed --> This PR addresses several core issues in the integration between Dify Chatflow and the Gemini GenAI SDK. --- ### 1. **`system_instructions` Replay Issue** * **Problem**: In Dify, the `system_prompt` was incorrectly parsed as `user_text` and inserted into the conversation `history` contents. This caused the same system prompt to be duplicated in the session state. https://github.com/langgenius/dify-official-plugins/blob/39b6920bf928e44650a75676b0aebaf239fb55e8/models/gemini/models/llm/llm.py#L379-L380 https://github.com/langgenius/dify-official-plugins/blob/39b6920bf928e44650a75676b0aebaf239fb55e8/models/gemini/models/llm/llm.py#L192-L193 * **Fix**: Transmit the system prompt via `config.system_instruction` in the Gemini configuration instead of injecting it into the `history`. --- ### 2. **`thinking` Content Leakage** * **Problem**: The original flow included reasoning content (e.g., `<think>xxx</think>`) in the conversation `contents`. * This could introduce unsafe content. * It could also cause severe performance degradation in multi-turn or long-context conversations. * Notably, in a pure Gemini-only system, storing Gemini’s own reasoning chain in memory does not significantly impact performance. However, in Dify Chatflow, multi-turn interactions may involve different reasoning models. Reasoning traces from non-Gemini models can “pollute” subsequent Gemini outputs. * The core cause is that Gemini’s reasoning chain is essentially a concise, length-controlled “thinking summary,” while other models may generate extremely verbose “self-correction” logs. * **Structural Limitation**: * **Case 1 – Sequential Multi-Model Pipeline**: As shown in the first example flow, when Gemini 2.5 Flash is placed after another LLM in the same branch, its <think> content cannot be reliably removed. Although regex could strip matching pairs from raw text, this is risky—without a channel-level structure identifier, the <think> block is indistinguishable from legitimate parts of the final answer. <img width="958" height="150" alt="image" src="https://github.com/user-attachments/assets/dbe0ba56-0c16-455f-b486-2fece7dcd629" /> * **Case 2 – Separate Conditional Branches**: As shown in the second example flow, when multiple reasoning models operate in different conditional branches (i.e., not sequentially in the same branch), the current method works safely, since reasoning outputs from one model do not enter the conversation state of another. <img width="1524" height="292" alt="image" src="https://github.com/user-attachments/assets/30c523bd-dd31-436e-823a-9a50ebdfeb3c" /> <img width="1835" height="373" alt="image" src="https://github.com/user-attachments/assets/a9c003d5-3178-4958-adca-41e7c51c0026" /> * **Fix**: When constructing `contents`, attempt to remove reasoning-channel data from memory. --- ### 3. **`function calling` Role Mislabeling** * **Problem**: The Gemini GenAI SDK only supports `user` and `model` roles—there is no `function` role. * ToolCall parameters are generated by the `model`. * ToolCall execution results are returned by the `user`. https://googleapis.github.io/python-genai/genai.html#genai.types.Content.role https://github.com/langgenius/dify-official-plugins/blob/39b6920bf928e44650a75676b0aebaf239fb55e8/models/gemini/models/llm/llm.py#L381-L389 * **Fix**: Correct role assignments for ToolCall-related messages. --- ### 4. **Token Counter Miscalculation** * **Problem**: The Gemini GenAI SDK directly returns the total token usage (including reasoning, responses, and tool calls). The correct behavior is to use this total value directly instead of summing it with other counts. * **Fix**: **TODO** — Some token counting logic is implemented in the lower-level Dify SDK, so this PR does not yet modify that part of the code. --- ### 5. **Type Hints and SDK Compatibility** * **Problem**: The source code contained multiple incorrect Gemini type usages (partly due to SDK version changes, where some types are now deprecated). * **Fix**: Updated all Gemini type references to their correct values to improve IDE autocomplete and AI-assisted coding accuracy. ### 6. **Comprehensive Tests for `contents` History State** - **Problem**: The existing test coverage for conversation `contents` history state was insufficient to ensure correctness across different scenarios. - **Fix**: Added a full set of test cases covering various conversation states, message types, and role assignments to ensure `contents` is correctly managed across the conversation lifecycle. ## This PR contains Changes to *Non-Plugin* <!-- Put an `x` in all the boxes that apply by replacing [ ] with [x] For example: - [x] Documentation --> - [ ] Documentation - [ ] Other ## This PR contains Changes to *Non-LLM Models Plugin* - [x] I have Run Comprehensive Tests Relevant to My Changes <!-- 📷 Include Screenshots/Videos Demonstrating the Fix, New Feature, or the Behavior Before/After Breaking Changes. --> ## This PR contains Changes to *LLM Models Plugin* <!-- LLM Models Test Example: --> <!-- https://github.com/langgenius/dify-official-plugins/blob/main/.assets/test-examples/llm-plugin-tests/llm_test_example.md --> - [x] My Changes Affect Message Flow Handling (System Messages and User→Assistant Turn-Taking) <!-- 📷 Include Screenshots/Videos Demonstrating the Fix, New Feature, or the Behavior Before/After Breaking Changes. --> - [ ] My Changes Affect Tool Interaction Flow (Multi-Round Usage and Output Handling, for both Agent App and Agent Node) <!-- 📷 Include Screenshots/Videos Demonstrating the Fix, New Feature, or the Behavior Before/After Breaking Changes. --> - [ ] My Changes Affect Multimodal Input Handling (Images, PDFs, Audio, Video, etc.) <!-- 📷 Include Screenshots/Videos Demonstrating the Fix, New Feature, or the Behavior Before/After Breaking Changes. --> - [ ] My Changes Affect Multimodal Output Generation (Images, Audio, Video, etc.) <!-- 📷 Include Screenshots/Videos Demonstrating the Fix, New Feature, or the Behavior Before/After Breaking Changes. --> - [ ] My Changes Affect Structured Output Format (JSON, XML, etc.) <!-- 📷 Include Screenshots/Videos Demonstrating the Fix, New Feature, or the Behavior Before/After Breaking Changes. --> - [ ] My Changes Affect Token Consumption Metrics <!-- 📷 Include Screenshots/Videos Demonstrating the Fix, New Feature, or the Behavior Before/After Breaking Changes. --> - [x] My Changes Affect Other LLM Functionalities (Reasoning Process, Grounding, Prompt Caching, etc.) <!-- 📷 Include Screenshots/Videos Demonstrating the Fix, New Feature, or the Behavior Before/After Breaking Changes. --> - [ ] Other Changes (Add New Models, Fix Model Parameters etc.) <!-- 📷 Include Screenshots/Videos Demonstrating the Fix, New Feature, or the Behavior Before/After Breaking Changes. --> ## Version Control (Any Changes to the Plugin Will Require Bumping the Version) - [x] I have Bumped Up the Version in Manifest.yaml (Top-Level `Version` Field, Not in Meta Section) <!-- ⚠️ NOTE: Version Format: MAJOR.MINOR.PATCH - MAJOR (0.x.x): Reserved for Significant architectural changes or incompatible API modifications - MINOR (x.0.x): For New feature additions while maintaining backward compatibility - PATCH (x.x.0): For Backward-compatible bug fixes and minor improvements - Note: Each Version Component (MAJOR, MINOR, PATCH) Can Be 2 Digits, e.g., 10.11.22 --> ## Dify Plugin SDK Version - [x] I have Ensured `dify_plugin>=0.3.0,<0.5.0` is in requirements.txt ([SDK docs](https://github.com/langgenius/dify-plugin-sdks/blob/main/python/README.md)) ## Environment Verification (If Any Code Changes) <!-- ⚠️ NOTE: At Least One Environment Must Be Tested. --> ### Local Deployment Environment - [x] Dify Version is: <!-- Specify Your Version (e.g., 1.2.0) -->, I have Tested My Changes on Local Deployment Dify with a Clean Environment That Matches the Production Configuration. <!-- - Python Virtual Env Matching Manifest.yaml & requirements.txt - No Breaking Changes in Dify That May Affect the Testing Result --> ### SaaS Environment - [x] I have Tested My Changes on cloud.dify.ai with a Clean Environment That Matches the Production Configuration <!-- - Python Virtual Env Matching Manifest.yaml & requirements.txt --> --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>
yindo added the pull-request label 2026-02-16 11:15:14 -05:00
yindo closed this issue 2026-02-16 11:15:14 -05:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
bug
bug
bug
dependencies
documentation
enhancement
good first issue
invalid
pull-request
Mirrored from GitHub Pull Request
 question
stale
No Label pull-request
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: langgenius/dify-official-plugins#1920
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?