langgenius/dify-plugin-sdks
1
0
Fork 0
mirror of https://github.com/langgenius/dify-plugin-sdks.git synced 2026-07-22 02:15:22 -04:00
Code Issues 58 Packages Projects Releases 10 Wiki Activity

feat: add optional reasoning_content to LLMResultChunkDelta #63

New Issue
Open
opened 2026-02-15 21:15:31 -05:00 by yindo · 0 comments
yindo commented 2026-02-15 21:15:31 -05:00
Owner
Copy Link
Copy Source

Originally created by @ultramancode on GitHub (Oct 27, 2025).

Summary

Add optional reasoning_content field to LLMResultChunkDelta to support structured streaming of LLM reasoning/thinking processes, separate from the final response text.

This change is part of a three-repository update (sdk, plugin, and main),
with a safe, phased rollout plan detailed below to ensure backward compatibility.

Follow-up to #23313

Motivation

Current inefficient flow (error-prone):

  1. Some vendor APIs return reasoning already separated (e.g., delta.thinking)
  2. Some plugins artificially merge it into message.content using <think> tags
  3. The Dify Main backend re-parses those <think> tags via _split_reasoning() to separate them again
  4. Risk of parsing errors, encoding issues, nested tags

Problems:

  • Unnecessary serialization/deserialization overhead
  • Parsing bugs (malformed tags, edge cases)
  • The Dify Main backend already supports the reasoning_content structure (#23313),
    but currently relies on fragile parsing through _split_reasoning() because the SDK lacks this field and plugin support.

Proposed flow:

  1. Vendor API returns separated data → Plugin preserves separation
  2. Plugin sends via reasoning_content field (no merging)
  3. Main reads directly (no parsing)
  4. Efficient, type-safe, maintainable, less error-prone separation

This aligns the SDK with how vendor APIs actually work.

Proposed Changes

Phase 1: SDK (this issue)

class LLMResultChunkDelta(BaseModel):
    index: int
    message: AssistantPromptMessage
    reasoning_content: str | None = None  # ← NEW
    usage: LLMUsage | None = None
    finish_reason: str | None = None

Compatibility: Backward-compatible (optional field, default None)

Phase 2: Plugins (defensive)

New plugin versions will enforce SDK versions that include this PR or later (via requirements.txt)
Add defensive pop() before Dify Main Backend sends the flag (Phase 3).
This prevents vendor API errors if an older plugin receives updated parameters from the Main backend.

def _chat_generate(self, model_parameters, ...):
    model_parameters = dict(model_parameters)
    model_parameters.pop("_dify_supports_reasoning_content", False)  # defensive
    # ... call vendor API

Affected plugins: All LLM plugins

Phase 3: Dify Main Backend (read & capability signaling)

Main reads reasoning_content from plugin responses:

  • Use delta.reasoning_content directly if present

Main sends capability flag via model_parameters:

  • Plugins detect this and optimize (single-channel vs dual-channel streaming)

Phase 4: Plugins (opt-in implementation)

Provider-by-provider implementation for reasoning-capable models:

My Plan

  1. SDK PR
  2. Plugin PR (Phase 2 defensive)
  3. Main PR (Phase 3)
  4. Plugin PRs (Phase 4 opt-in) → Gradual rollout per provider

Backward Compatibility

No breaking changes at any phase

  • Old plugins work with new SDK (field ignored)
  • New plugins work with old Main (dual-channel fallback)
  • Old Main works with new plugins (defensive pop)

Migration Safety

This phased approach ensures zero-risk migration despite independent release cycles:
no coordination required between repositories.

Why this is safe:

  • No LLM node disruption: All phases use optional fields/parameters
  • SDK-independent: Plugins can upgrade SDK without waiting for Main
  • Main-independent: SDK can release without breaking existing deployments
  • Plugin-independent: Each provider can opt in at its own pace

Version compatibility matrix:

Scenario SDK Main Plugin Result
Old stack Old Old Old Works (current flow)
SDK updated New Old Old Works (field ignored)
SDK + Plugin New Old New Works (defensive pop)
SDK + Main New New Old Works (Main reads field if present)
All updated New New New Optimal (direct reasoning flow)
Originally created by @ultramancode on GitHub (Oct 27, 2025). ## Summary Add optional `reasoning_content` field to `LLMResultChunkDelta` to support structured streaming of LLM reasoning/thinking processes, separate from the final response text. This change is part of a three-repository update (`sdk`, `plugin`, and `main`), with a safe, phased rollout plan detailed below to ensure backward compatibility. Follow-up to [#23313](https://github.com/langgenius/dify/pull/23313) ## Motivation **Current inefficient flow (error-prone):** 1. Some vendor APIs return reasoning already separated (e.g., `delta.thinking`) 2. Some plugins artificially merge it into `message.content` using `<think>` tags 3. The Dify Main backend re-parses those `<think>` tags via `_split_reasoning()` to separate them again 4. Risk of parsing errors, encoding issues, nested tags **Problems:** - Unnecessary serialization/deserialization overhead - Parsing bugs (malformed tags, edge cases) - The Dify Main backend already supports the reasoning_content structure ([#23313](https://github.com/langgenius/dify/pull/23313)), but currently relies on fragile parsing through _split_reasoning() because the SDK lacks this field and plugin support. **Proposed flow:** 1. Vendor API returns separated data → Plugin preserves separation 2. Plugin sends via `reasoning_content` field (no merging) 3. Main reads directly (no parsing) 4. Efficient, type-safe, maintainable, less error-prone separation This aligns the SDK with how vendor APIs actually work. ## Proposed Changes ### Phase 1: SDK (this issue) ```python class LLMResultChunkDelta(BaseModel): index: int message: AssistantPromptMessage reasoning_content: str | None = None # ← NEW usage: LLMUsage | None = None finish_reason: str | None = None ``` **Compatibility:** Backward-compatible (optional field, default `None`) ### Phase 2: Plugins (defensive) New plugin versions will enforce SDK versions that include this PR or later (via requirements.txt) Add defensive pop() before Dify Main Backend sends the flag (Phase 3). This prevents vendor API errors if an older plugin receives updated parameters from the Main backend. ```python def _chat_generate(self, model_parameters, ...): model_parameters = dict(model_parameters) model_parameters.pop("_dify_supports_reasoning_content", False) # defensive # ... call vendor API ``` **Affected plugins:** All LLM plugins ### Phase 3: Dify Main Backend (read & capability signaling) **Main reads `reasoning_content` from plugin responses:** - Use `delta.reasoning_content` directly if present **Main sends capability flag via `model_parameters`:** - Plugins detect this and optimize (single-channel vs dual-channel streaming) ### Phase 4: Plugins (opt-in implementation) Provider-by-provider implementation for reasoning-capable models: ## My Plan 1. **SDK PR** 2. **Plugin PR (Phase 2 defensive)** 3. **Main PR (Phase 3)** 4. **Plugin PRs (Phase 4 opt-in)** → Gradual rollout per provider ## Backward Compatibility No breaking changes at any phase - Old plugins work with new SDK (field ignored) - New plugins work with old Main (dual-channel fallback) - Old Main works with new plugins (defensive pop) ## Migration Safety This phased approach ensures **zero-risk migration** despite independent release cycles: no coordination required between repositories. **Why this is safe:** - **No LLM node disruption**: All phases use optional fields/parameters - **SDK-independent**: Plugins can upgrade SDK without waiting for Main - **Main-independent**: SDK can release without breaking existing deployments - **Plugin-independent**: Each provider can opt in at its own pace **Version compatibility matrix:** | Scenario | SDK | Main | Plugin | Result | |----------|-----|------|--------|--------| | Old stack | Old | Old | Old | Works (current flow) | | SDK updated | New | Old | Old | Works (field ignored) | | SDK + Plugin | New | Old | New | Works (defensive pop) | | SDK + Main | New | New | Old | Works (Main reads field if present) | | All updated | New | New | New | Optimal (direct reasoning flow) |
yindo added the enhancement label 2026-02-15 21:15:31 -05:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
bug
documentation
enhancement
pull-request
Mirrored from GitHub Pull Request
 question
No Label enhancement
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: langgenius/dify-plugin-sdks#63
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?