langchain-ai/langgraph
1
0
Fork 0
mirror of https://github.com/langchain-ai/langgraph.git synced 2026-07-15 12:35:43 -04:00
Code Issues 314 Packages Projects Releases 17 Wiki Activity

[PR #169] [MERGED] Fixed rewoo.ipynb #1281

New Issue
Closed
opened 2026-02-20 17:44:21 -05:00 by yindo · 0 comments
yindo commented 2026-02-20 17:44:21 -05:00
Owner
Copy Link
Copy Source

📋 Pull Request Information

Original PR: https://github.com/langchain-ai/langgraph/pull/169
Author: @kikoncuo
Created: 3/1/2024
Status: ✅ Merged
Merged: 3/3/2024
Merged by: @hinthornw

Base: main ← Head: patch-1

📝 Commits (1)

📊 Changes

1 file changed (+1 additions, -0 deletions)

View changed files

📝 examples/rewoo/rewoo.ipynb (+1 -0)

📄 Description

The way the current implementation works, the solver never has access to the tools results, only to the steps and the question.

This is because the replacement process only updates the references in the tool input not on the results.

For example, without this change, the prompt we give to the solver is:

prompt
Solve the following task or problem. To solve the problem, we have made step-by-step Plan and retrieved corresponding Evidence to each Plan. Use them with caution since long evidence might contain irrelevant information.

Plan: Calculate the square root of 342142. 
#E1 = calculate[square root of 342142]Plan: Calculate 23 to the power of 2. 
#E2 = calculate[23 to the power of 2]Plan: Multiply #E1 by #E2. 
#E3 = calculate[584.9290555272494 multiplied by 529]

Now solve the question or task according to provided Evidence above.

Task: What's the square root of 342142 multiplied by 23 to the power of 2?
Response:
{'solve': {'result': 'The square root of 342142 is 584.9290555272494, and 23 to the power of 2 is 529. Multiplying these two values together gives us:\n\n\\[584.9290555272494 \\times 529 = 309368.382384\\]\n\nTherefore, the square root of 342142 multiplied by 23 to the power of 2 is approximately 309368.382384.'}}

Which is incorrect and doesn't use the tool results

With the change in the PR:

prompt
Solve the following task or problem. To solve the problem, we have made step-by-step Plan and retrieved corresponding Evidence to each Plan. Use them with caution since long evidence might contain irrelevant information.

Plan: Calculate the square root of 342142. 
584.9290555272494 = calculate[square root of 342142]Plan: Calculate 23 to the power of 2. 
529 = calculate[23 to the power of 2]Plan: Multiply the results of #E1 and #E2. 
309427.4703739149 = calculate[584.9290555272494 multiplied by 529]

Now solve the question or task according to provided Evidence above.

Task: What's the square root of 342142 multiplied by 23 to the power of 2?
Response:
{'solve': {'result': 'The square root of 342142 multiplied by 23 to the power of 2 is 309427.4703739149.'}}

Which is the correct result and uses the tool's outputs

This is also noticeable with questions with followups that can be looked up with search and the LLM doesn't know, where it just talks about his knowledge cutoff and gives outdated information vs using the search results and providing the correct answer

I believe this was not noticed in the video because the last call is to the LLM tool which itself contains the answer, but otherwise this will fail for every other question

🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.

## 📋 Pull Request Information **Original PR:** https://github.com/langchain-ai/langgraph/pull/169 **Author:** [@kikoncuo](https://github.com/kikoncuo) **Created:** 3/1/2024 **Status:** ✅ Merged **Merged:** 3/3/2024 **Merged by:** [@hinthornw](https://github.com/hinthornw) **Base:** `main` ← **Head:** `patch-1` --- ### 📝 Commits (1) - [`cca9481`](https://github.com/langchain-ai/langgraph/commit/cca948169271b47c8b76cf9d23dd59ec2ebd3f40) Fixed rewoo.ipynb ### 📊 Changes **1 file changed** (+1 additions, -0 deletions) <details> <summary>View changed files</summary> 📝 `examples/rewoo/rewoo.ipynb` (+1 -0) </details> ### 📄 Description The way the current implementation works, the solver never has access to the tools results, only to the steps and the question. This is because the replacement process only updates the references in the tool input not on the results. For example, without this change, the prompt we give to the solver is: ``` prompt Solve the following task or problem. To solve the problem, we have made step-by-step Plan and retrieved corresponding Evidence to each Plan. Use them with caution since long evidence might contain irrelevant information. Plan: Calculate the square root of 342142. #E1 = calculate[square root of 342142]Plan: Calculate 23 to the power of 2. #E2 = calculate[23 to the power of 2]Plan: Multiply #E1 by #E2. #E3 = calculate[584.9290555272494 multiplied by 529] Now solve the question or task according to provided Evidence above. Task: What's the square root of 342142 multiplied by 23 to the power of 2? Response: {'solve': {'result': 'The square root of 342142 is 584.9290555272494, and 23 to the power of 2 is 529. Multiplying these two values together gives us:\n\n\\[584.9290555272494 \\times 529 = 309368.382384\\]\n\nTherefore, the square root of 342142 multiplied by 23 to the power of 2 is approximately 309368.382384.'}} ``` Which is incorrect and doesn't use the tool results With the change in the PR: ``` prompt Solve the following task or problem. To solve the problem, we have made step-by-step Plan and retrieved corresponding Evidence to each Plan. Use them with caution since long evidence might contain irrelevant information. Plan: Calculate the square root of 342142. 584.9290555272494 = calculate[square root of 342142]Plan: Calculate 23 to the power of 2. 529 = calculate[23 to the power of 2]Plan: Multiply the results of #E1 and #E2. 309427.4703739149 = calculate[584.9290555272494 multiplied by 529] Now solve the question or task according to provided Evidence above. Task: What's the square root of 342142 multiplied by 23 to the power of 2? Response: {'solve': {'result': 'The square root of 342142 multiplied by 23 to the power of 2 is 309427.4703739149.'}} ``` Which is the correct result and uses the tool's outputs This is also noticeable with questions with followups that can be looked up with search and the LLM doesn't know, where it just talks about his knowledge cutoff and gives outdated information vs using the search results and providing the correct answer I believe this was not noticed in the video because the last call is to the LLM tool which itself contains the answer, but otherwise this will fail for every other question --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>
yindo added the pull-request label 2026-02-20 17:44:21 -05:00
yindo closed this issue 2026-02-20 17:44:21 -05:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
agents
awaiting author revision
bug
change
chat models
checkpointer
ci
code quality
code quality
dependencies
documentation
drawing
duplicate
enhancement
good first issue
help wanted
invalid
maintainer
meta
missing information
not maintained by langchain
open-swe-auto
open-swe-max
open-swe-max-auto
pending
performance
pull-request
Mirrored from GitHub Pull Request
 question
requires design
sdk
server
stale
studio
tests
type checking
usage question
v0.5
v1.1
No Label pull-request
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: langchain-ai/langgraph#1281
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?