langgenius/dify
1
0
Fork 0
mirror of https://github.com/langgenius/dify.git synced 2026-07-21 20:15:31 -04:00
Code Issues 775 Packages Projects Releases 10 Wiki Activity

Questions about ReAct mode Agent #1133

New Issue
Closed
opened 2026-02-21 17:30:09 -05:00 by yindo · 7 comments
yindo commented 2026-02-21 17:30:09 -05:00
Owner
Copy Link
Copy Source

Originally created by @sunxichen on GitHub (Feb 29, 2024).

Originally assigned to: @takatost, @Yeuoly on GitHub.

Self Checks

  • I have searched for existing issues search for existing issues, including closed ones.
  • I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
  • Pleas do not modify this template :) and fill in all the required fields.

Provide a description of the help you need

  1. In ReAct mode, the Action prefix, Observation, Thought are also streamed. Is it by designed or a bug related to agent output parsing? Because user may only concern about final output, so I'm expecting only Final Answer should be streamed.
    截屏2024-02-29 15 34 59

  2. OpenAI/Azure OpenAI only support Function Calling mode, some other issues like #2517 also discussed this. I understand that function_calling may perform better than ReAct, maybe it would be better if it could be made changeable.

Originally created by @sunxichen on GitHub (Feb 29, 2024). Originally assigned to: @takatost, @Yeuoly on GitHub. ### Self Checks - [X] I have searched for existing issues [search for existing issues](https://github.com/langgenius/dify/issues), including closed ones. - [X] I confirm that I am using English to submit this report (我已阅读并同意 [Language Policy](https://github.com/langgenius/dify/issues/1542)). - [X] Pleas do not modify this template :) and fill in all the required fields. ### Provide a description of the help you need 1. In ReAct mode, the Action prefix, Observation, Thought are also streamed. Is it by designed or a bug related to agent output parsing? Because user may only concern about final output, so I'm expecting only Final Answer should be streamed. ![截屏2024-02-29 15 34 59](https://github.com/langgenius/dify/assets/18310222/ba54cfd5-d47f-44a8-bda1-8e38fc24334c) 2. OpenAI/Azure OpenAI only support Function Calling mode, some other issues like #2517 also discussed this. I understand that function_calling may perform better than ReAct, maybe it would be better if it could be made changeable.
yindo closed this issue 2026-02-21 17:30:09 -05:00
yindo commented 2026-02-21 17:30:09 -05:00
Author
Owner
Copy Link
Copy Source

@dosubot[bot] commented on GitHub (Feb 29, 2024):

🤖

Hey there @sunxichen! Great to see you back with another insightful question. Hope you're doing well!

Based on the information provided in the context, the streaming of Action prefix, Observation, and Thought in ReAct mode is a designed feature, not a bug. This is evident from the structured way in which the AssistantCotApplicationRunner class handles the generation of agent scratchpad units, which include fields for action_str, observation, and thought. The class method _handle_stream_react specifically processes chunks of responses from the language model, distinguishing between different types of content (e.g., actions, observations, thoughts) and handling them accordingly. This indicates that the handling of these elements is an intentional part of the system's design to parse and utilize the output from the language model in a structured manner. You can refer to the source code here for more details.

As for your second question, the provided code does not explicitly include a mechanism for dynamically switching between different operational modes based on performance metrics or other criteria. The mode is determined by the configuration (app_orchestration_config.model_config.mode) and is used to conditionally modify behavior (e.g., appending 'Observation' to the stop list if not already present when in "completion" mode). However, the code structure and design patterns suggest that extending the system to support dynamic mode switching based on performance or other criteria could be feasible with additional development. This would likely involve implementing logic to evaluate the performance or suitability of different modes based on specific criteria and dynamically adjusting the mode accordingly.

Regarding the possibility of choosing whether to stream the Action prefix, Observation, and Thought in ReAct mode, I wasn't able to find a specific configuration setting or method in the codebase that allows for this. It's possible that this feature might be available elsewhere or I might have missed it. If you could provide more information or context, I might be able to give a more accurate answer.

Sources

About Dosu

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

@dosubot[bot] commented on GitHub (Feb 29, 2024): _🤖_ <!-- Greeting --> Hey there @sunxichen! Great to see you back with another insightful question. Hope you're doing well! <!-- Answer --> Based on the information provided in the context, the streaming of Action prefix, Observation, and Thought in ReAct mode is a designed feature, not a bug. This is evident from the structured way in which the `AssistantCotApplicationRunner` class handles the generation of agent scratchpad units, which include fields for `action_str`, `observation`, and `thought`. The class method `_handle_stream_react` specifically processes chunks of responses from the language model, distinguishing between different types of content (e.g., actions, observations, thoughts) and handling them accordingly. This indicates that the handling of these elements is an intentional part of the system's design to parse and utilize the output from the language model in a structured manner. You can refer to the source code [here](https://github.com/langgenius/dify/blob/main/api/core/features/assistant_cot_runner.py) for more details. As for your second question, the provided code does not explicitly include a mechanism for dynamically switching between different operational modes based on performance metrics or other criteria. The mode is determined by the configuration (`app_orchestration_config.model_config.mode`) and is used to conditionally modify behavior (e.g., appending 'Observation' to the stop list if not already present when in "completion" mode). However, the code structure and design patterns suggest that extending the system to support dynamic mode switching based on performance or other criteria could be feasible with additional development. This would likely involve implementing logic to evaluate the performance or suitability of different modes based on specific criteria and dynamically adjusting the mode accordingly. Regarding the possibility of choosing whether to stream the Action prefix, Observation, and Thought in ReAct mode, I wasn't able to find a specific configuration setting or method in the codebase that allows for this. It's possible that this feature might be available elsewhere or I might have missed it. If you could provide more information or context, I might be able to give a more accurate answer. ## Sources - [api/core/features/assistant_cot_runner.py](https://github.com/langgenius/dify/blob/main/api/core/features/assistant_cot_runner.py) <!-- Dosu Comment Footer --> <details open> <summary> #### About Dosu </summary> This response is meant to be useful and save you time. It is **not meant to be a precise solution**, but rather a starting point for your own research. Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant. If you want to continue the conversation, start your reply with @dosu-bot. </details>
yindo commented 2026-02-21 17:30:09 -05:00
Author
Owner
Copy Link
Copy Source

@takatost commented on GitHub (Mar 1, 2024):

  1. This should be related to the bug in ReACT and the output parse. The lack of precision in ReACT's output has caused many parsing issues.

    @Yeuoly, can you take a look at this issue? 🤔

  2. As mentioned in 1, ReACT is not stable. Could you please explain why you prefer using ReACT instead of function calling?

@takatost commented on GitHub (Mar 1, 2024): 1. This should be related to the bug in ReACT and the output parse. The lack of precision in ReACT's output has caused many parsing issues. @Yeuoly, can you take a look at this issue? 🤔 3. As mentioned in 1, ReACT is not stable. Could you please explain why you prefer using ReACT instead of function calling?
yindo commented 2026-02-21 17:30:09 -05:00
Author
Owner
Copy Link
Copy Source

@sunxichen commented on GitHub (Mar 3, 2024):

2. As mentioned in 1, ReACT is not stable. Could you please explain why you prefer using ReACT instead of function calling?

In my project, I can only use open-source LLMs to do reasoning using the ReAct mode. Sometimes, I need to use OpenAI models like gpt-3.5/gpt-4 to test the impact of different ReAct prompts, function/input parameter descriptions on agent performance first. But using the OpenAI model can only limit my use of function calling mode. Although I can temporarily bypass this limitation by using the openai_api_compatible model in Dify which actually connected to gpt-3.5/gpt-4, it still feels a bit inconvenient, so I think it might be better to switch between the two.

@sunxichen commented on GitHub (Mar 3, 2024): > 2\. As mentioned in 1, ReACT is not stable. Could you please explain why you prefer using ReACT instead of function calling? In my project, I can only use open-source LLMs to do reasoning using the ReAct mode. Sometimes, I need to use OpenAI models like gpt-3.5/gpt-4 to test the impact of different ReAct prompts, function/input parameter descriptions on agent performance first. But using the OpenAI model can only limit my use of function calling mode. Although I can temporarily bypass this limitation by using the openai_api_compatible model in Dify which actually connected to gpt-3.5/gpt-4, it still feels a bit inconvenient, so I think it might be better to switch between the two.
yindo commented 2026-02-21 17:30:09 -05:00
Author
Owner
Copy Link
Copy Source

@Yeuoly commented on GitHub (Mar 3, 2024):

  1. This should be related to the bug in ReACT and the output parse. The lack of precision in ReACT's output has caused many parsing issues.
    @Yeuoly, can you take a look at this issue? 🤔
  2. As mentioned in 1, ReACT is not stable. Could you please explain why you prefer using ReACT instead of function calling?
  1. It's actually a bug related to ReACT output parser. And as we known, different prompts and different LLMs could lead different output structures, such as
action: current_time
action_input:

{"action":"current_time", "action_input":{}}

A simple output parser hardcoded could not handle a wide variety of prompts, the final result is what you saw, have you got any good idea of doing that?

@Yeuoly commented on GitHub (Mar 3, 2024): > 1. This should be related to the bug in ReACT and the output parse. The lack of precision in ReACT's output has caused many parsing issues. > @Yeuoly, can you take a look at this issue? 🤔 > 2. As mentioned in 1, ReACT is not stable. Could you please explain why you prefer using ReACT instead of function calling? 1. It's actually a bug related to ReACT output parser. And as we known, different prompts and different LLMs could lead different output structures, such as ```yaml action: current_time action_input: ``` ```json {"action":"current_time", "action_input":{}} ``` A simple output parser hardcoded could not handle a wide variety of prompts, the final result is what you saw, have you got any good idea of doing that?
yindo commented 2026-02-21 17:30:09 -05:00
Author
Owner
Copy Link
Copy Source

@Yeuoly commented on GitHub (Mar 3, 2024):

@vikeychen help take a look at this requirement?

@Yeuoly commented on GitHub (Mar 3, 2024): @vikeychen help take a look at this requirement?
yindo commented 2026-02-21 17:30:09 -05:00
Author
Owner
Copy Link
Copy Source

@sunxichen commented on GitHub (Mar 4, 2024):

A simple output parser hardcoded could not handle a wide variety of prompts

I agree with you, It is hard to implement a general output parser. Now that I know this is not by designed, my doubts have been resolved.

have you got any good idea of doing that?

You can consider using a guided generation library (like Outlines) to implement ReAct, so that every step during the ReAct process is controllable. You can limit the output of LLM to be exactly the same as the format you want by using regex (and also limit the output of LLM to conform to the defined JSON format, which is important for action input), so you don't have to spend a lot of efforts on output parsers. Furthermore, you can refer to this implementation example.

@sunxichen commented on GitHub (Mar 4, 2024): > A simple output parser hardcoded could not handle a wide variety of prompts I agree with you, It is hard to implement a general output parser. Now that I know this is not by designed, my doubts have been resolved. > have you got any good idea of doing that? You can consider using a guided generation library (like [Outlines](https://github.com/outlines-dev/outlines)) to implement ReAct, so that every step during the ReAct process is controllable. You can limit the output of LLM to be exactly the same as the format you want by using regex (and also limit the output of LLM to conform to the defined JSON format, which is important for action input), so you don't have to spend a lot of efforts on output parsers. Furthermore, you can refer to this implementation [example](https://github.com/outlines-dev/outlines/blob/main/examples/react.py).
yindo commented 2026-02-21 17:30:09 -05:00
Author
Owner
Copy Link
Copy Source

@vikeychen commented on GitHub (Mar 4, 2024):

@vikeychen help take a look at this requirement?

yes ReAct is not stable on almost all models. In code ReAct prompt can easily produce Thought/Action/Observation strings although ask for single action only, the prompt also says follow Question/Thought/Action/Observation.

@vikeychen commented on GitHub (Mar 4, 2024): > @vikeychen help take a look at this requirement? yes ReAct is not stable on almost all models. In code ReAct prompt can easily produce Thought/Action/Observation strings although ask for single action only, the prompt also says follow Question/Thought/Action/Observation.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Knowledge Pipeline
Proposal
WIP
cloud
dependencies
feat:webapp
good first issue
outdated
performance
pull-request
Mirrored from GitHub Pull Request
 python:uv
refactor
size:L
stale
status: accepting prs
status: blocked
status: in discussion
stt & tts
vulnerablity
web
wont-fix
🌊 feat:workflow
🌚 invalid
🐞 bug
👻 feat:rag
💪 enhancement
📚 documentation
🔨 feat:tools
🙋‍♂️ question
🙏 help wanted
🤔 cant-reproduce
🤖 feat:agent
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: langgenius/dify#1133
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?