langgenius/dify
1
0
Fork 0
mirror of https://github.com/langgenius/dify.git synced 2026-07-21 12:05:30 -04:00
Code Issues 775 Packages Projects Releases 10 Wiki Activity

[PR #21897] feat: enhance LLMNode with selective <think> tag and reasoning block removal #29786

New Issue
Closed
opened 2026-02-21 20:46:14 -05:00 by yindo · 0 comments
yindo commented 2026-02-21 20:46:14 -05:00
Owner
Copy Link
Copy Source

Original Pull Request: https://github.com/langgenius/dify/pull/21897

State: closed
Merged: No

Important

  1. Make sure you have read our contribution guidelines
  2. Ensure there is an associated issue and you have been assigned to it
  3. Use the correct syntax to link this PR: Fixes #<issue number>.

Summary

Fixes #21896
Related: #13107, #13419, #18556

reasoning-enabled models (e.g. Qwen-2.5) emit <think> … </think> blocks,
while others (e.g. Gemini-1.5) do not.
When these tags reach downstream nodes (code-executor, JSON parser, etc.) they break
parsing and produce runtime errors.

This PR introduces an environment flag:

Variable Default Behavior
LLM_NODE_THINKING_TAGS_ENABLED true true → keep tags and reasoning blocks(current behavior)
false → strip tags reasoning blocks inside LLMNode

Deployment notes

  • .env.sample now includes LLM_NODE_THINKING_TAGS_ENABLED=true
  • docker-compose.yaml exposes the variable: 
    LLM_NODE_THINKING_TAGS_ENABLED: ${LLM_NODE_THINKING_TAGS_ENABLED:-true}

This PR adds a safe fallback mechanism for mixed-model flows.

While it's occasionally possible to manually suppress output via prompt tricks in ASSISTANT mode, such behavior is not reliable across all models or input patterns.

That causes issues when strict JSON or code output parsing is involved.
The new env‑flag ensures the <think> block is stripped entirely at the LLMNode level.

For example, with ASSISTANT Mode, the output can sometimes include reasoning like:
"Okay, the user is asking, ... The capital of South Korea is Seoul."

In contrast, with this PR’s behavior, the output becomes:
"The capital of South Korea is Seoul."

Note: While Qwen3 supports prompt-based toggles like /no_think, they sometimes leave behind empty <think></think> tags or retain full reasoning text, and their behavior is not always reliable. This means they don’t guarantee clean output.

스크린샷 2025-07-11 오전 1 11 21

In contrast, setting LLM_NODE_THINKING_TAGS_ENABLED=false unconditionally removes the entire <think>…</think> block—including reasoning content—at the code level, ensuring that downstream components always receive a clean, parser-safe final answer.

Only the text field returned by LLMNode changes when the flag is false;
the collapsible “thinking” panel itself remains exactly as before.

Screenshots

Before (true, think tag visible) After (false, think tag removed)
before after

Checklist

  • This change requires a documentation update, included: Dify Document
  • I understand that this PR may be closed in case there was no previous discussion or issues. (This doesn't apply to typos!)
  • I've added a test for each change that was introduced, and I tried as much as possible to make a single atomic change.
  • I've updated the documentation accordingly.
  • I ran dev/reformat(backend) and cd web && npx lint-staged(frontend) to appease the lint gods
**Original Pull Request:** https://github.com/langgenius/dify/pull/21897 **State:** closed **Merged:** No --- > [!IMPORTANT] > > 1. Make sure you have read our [contribution guidelines](https://github.com/langgenius/dify/blob/main/CONTRIBUTING.md) > 2. Ensure there is an associated issue and you have been assigned to it > 3. Use the correct syntax to link this PR: `Fixes #<issue number>`. ## Summary Fixes #21896 Related: #13107, #13419, #18556 reasoning-enabled models (e.g. **Qwen-2.5**) emit `<think> … </think>` blocks, while others (e.g. **Gemini-1.5**) do not. When these tags reach downstream nodes (code-executor, JSON parser, etc.) they break parsing and produce runtime errors. This PR introduces an environment flag: | Variable | Default | Behavior | |----------|---------|-----------| | `LLM_NODE_THINKING_TAGS_ENABLED` | `true` | `true` → keep tags and reasoning blocks(current behavior) <br>`false` → strip tags reasoning blocks inside `LLMNode` | ### Deployment notes * `.env.sample` now includes **LLM_NODE_THINKING_TAGS_ENABLED=true** * `docker-compose.yaml` exposes the variable: ```yaml LLM_NODE_THINKING_TAGS_ENABLED: ${LLM_NODE_THINKING_TAGS_ENABLED:-true} ``` This PR adds a safe fallback mechanism for mixed-model flows. While it's occasionally possible to manually suppress <think> output via prompt tricks in ASSISTANT mode, such behavior is not reliable across all models or input patterns. That causes issues when strict JSON or code output parsing is involved. The new env‑flag ensures the `<think>` block is stripped entirely at the LLMNode level. > For example, with ASSISTANT Mode, the output can sometimes include reasoning like: > "Okay, the user is asking, ... The capital of South Korea is **Seoul**." > > In contrast, with this PR’s behavior, the output becomes: > "The capital of South Korea is **Seoul**." **Note**: While Qwen3 supports prompt-based toggles like `/no_think`, they **sometimes leave behind empty `<think></think>` tags or retain full reasoning text**, and their behavior is not always reliable. This means they don’t guarantee clean output. <img width="788" height="554" alt="스크린샷 2025-07-11 오전 1 11 21" src="https://github.com/user-attachments/assets/6627230b-ce54-4263-aa2f-3775a7ca56e8" /> In contrast, setting `LLM_NODE_THINKING_TAGS_ENABLED=false` **unconditionally removes** the entire `<think>…</think>` block—including reasoning content—**at the code level**, ensuring that downstream components always receive a **clean, parser-safe final answer**. *Only the `text` field returned by **LLMNode** changes when the flag is `false`; the collapsible “thinking” panel itself remains exactly as before.* ## Screenshots | Before (`true`, think tag visible) | After (`false`, think tag removed) | |--------|-------| | <img width="817" alt="before" src="https://github.com/user-attachments/assets/ba71d2c6-e7a4-4812-895e-f5900c937194" /> | <img width="818" alt="after" src="https://github.com/user-attachments/assets/2221880c-3b3b-4c01-bd2a-67a9864c9597" />| ## Checklist - [ ] This change requires a documentation update, included: [Dify Document](https://github.com/langgenius/dify-docs) - [x] I understand that this PR may be closed in case there was no previous discussion or issues. (This doesn't apply to typos!) - [x] I've added a test for each change that was introduced, and I tried as much as possible to make a single atomic change. - [x] I've updated the documentation accordingly. - [x] I ran `dev/reformat`(backend) and `cd web && npx lint-staged`(frontend) to appease the lint gods
yindo added the pull-request label 2026-02-21 20:46:14 -05:00
yindo closed this issue 2026-02-21 20:46:14 -05:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Knowledge Pipeline
Proposal
WIP
cloud
dependencies
feat:webapp
good first issue
outdated
performance
pull-request
Mirrored from GitHub Pull Request
 python:uv
refactor
size:L
stale
status: accepting prs
status: blocked
status: in discussion
stt & tts
vulnerablity
web
wont-fix
🌊 feat:workflow
🌚 invalid
🐞 bug
👻 feat:rag
💪 enhancement
📚 documentation
🔨 feat:tools
🙋‍♂️ question
🙏 help wanted
🤔 cant-reproduce
🤖 feat:agent
No Label pull-request
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: langgenius/dify#29786
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?