langchain-ai/langgraph
1
0
Fork 0
mirror of https://github.com/langchain-ai/langgraph.git synced 2026-07-15 12:35:43 -04:00
Code Issues 314 Packages Projects Releases 17 Wiki Activity

[PR #6620] feat(prebuilt): Add True ReAct Primitives for Explicit Reasoning Traces #5172

New Issue
Closed
opened 2026-02-20 17:51:20 -05:00 by yindo · 0 comments
yindo commented 2026-02-20 17:51:20 -05:00
Owner
Copy Link
Copy Source

Original Pull Request: https://github.com/langchain-ai/langgraph/pull/6620

State: closed
Merged: No

Add True ReAct Primitives with reasoning=True Integration

TL;DR: Add reasoning=True parameter to create_react_agent for automatic capture of Thought → Action → Observation cycles. Makes agent decision-making visible, debuggable, and measurable with one parameter.

📄 RFC: #6619

The Problem: Agents Are Black Boxes

Today, when you run an agent, you only see messages:

result = agent.invoke({"messages": [HumanMessage("What's the weather in Paris?")]})

print(result["messages"])
# [HumanMessage("What's the weather in Paris?"),
#  AIMessage(tool_calls=[{"name": "get_weather", ...}]),
#  ToolMessage("22°C, sunny"),
#  AIMessage("The weather in Paris is 22°C and sunny!")]

What You CAN'T See:

  • ❌ WHY did the agent decide to call get_weather? (reasoning hidden in LLM)
  • ❌ HOW LONG did the tool take? (no timing captured)
  • ❌ DID IT FAIL before succeeding? (no error history)
  • ❌ HOW TO DEBUG when something breaks? (manual log analysis)

The Solution: One Parameter Changes Everything

from langgraph.prebuilt import create_react_agent

# Just add reasoning=True
agent = create_react_agent(
    model=model,
    tools=[get_weather, search],
    reasoning=True  # ← That's it!
)

result = agent.invoke({"messages": [HumanMessage("What's the weather in Paris?")]})

# NOW YOU SEE EVERYTHING
trace = result["reasoning_trace"]
for step in trace:
    print(f"🧠 Thought: {step.thought.content if step.thought else 'N/A'}")
    print(f"🔧 Action: {step.action['name'] if step.action else 'Final answer'}")
    print(f"📊 Result: {step.observation.tool_output if step.observation else 'N/A'}")
    print(f"⏱️  Duration: {step.observation.duration_ms if step.observation else 'N/A'}ms")
    print(f"✅ Success: {step.observation.success if step.observation else 'N/A'}")
    print()

What You Get

1. Instant Debugging

from langgraph.prebuilt import find_failed_steps

# Find exactly what went wrong
failed = find_failed_steps(trace)
for step in failed:
    print(f"❌ Tool '{step.observation.tool_name}' failed:")
    print(f"   Error: {step.observation.error}")
    print(f"   Duration: {step.observation.duration_ms}ms")
    print(f"   Input: {step.action['args']}")

2. Production Metrics

from langgraph.prebuilt import calculate_trace_metrics

metrics = calculate_trace_metrics(trace)
print(f"Total steps: {metrics.total_steps}")
print(f"Success rate: {metrics.successful_tool_calls / metrics.tool_calls:.1%}")
print(f"Avg latency: {metrics.average_duration_ms:.1f}ms")
print(f"Tools used: {metrics.tool_names_used}")

3. User Transparency

from langgraph.prebuilt import render_reasoning_trace

# Show users what the agent is doing
print(render_reasoning_trace(trace))
# Step 0:
#   💭 Thought: User wants weather info. I'll use the get_weather tool.
#   🔧 Action: get_weather(city="Paris")
#   📊 Observation: Tool 'get_weather' succeeded in 145.3ms
#        Output: 22°C, sunny

4. Quality Analysis

from langgraph.prebuilt import get_tool_usage_summary

summary = get_tool_usage_summary(trace)
for tool_name, stats in summary.items():
    print(f"{tool_name}:")
    print(f"  Calls: {stats['calls']}")
    print(f"  Failures: {stats['failures']}")
    print(f"  Avg duration: {stats['avg_duration_ms']:.1f}ms")

Before/After Comparison

Capability BEFORE (Today) AFTER (With reasoning=True)
See agent reasoning ❌ Hidden in LLM ✅ step.thought.content
Debug failures ❌ Read chat logs ✅ find_failed_steps(trace)
Tool timing ❌ Not captured ✅ observation.duration_ms
Success/failure tracking ❌ Not captured ✅ observation.success
Confidence scores ❌ Not captured ✅ thought.confidence
Aggregate metrics ❌ Manual counting ✅ calculate_trace_metrics()
Production monitoring ❌ Custom implementation ✅ Built-in utilities

API Design

Simple Integration

def create_react_agent(
    model: LanguageModelLike,
    tools: Sequence[BaseTool] | ToolNode,
    *,
    # ... existing parameters ...
    reasoning: bool = False,  # ← NEW: Enable reasoning capture
) -> CompiledStateGraph:

Automatic State Schema

When reasoning=True, state automatically includes reasoning_trace:

# Without reasoning (default)
result = agent.invoke({...})
# result = {"messages": [...]}

# With reasoning=True
result = agent.invoke({...})
# result = {"messages": [...], "reasoning_trace": [...]}

Works with response_format

from pydantic import BaseModel

class WeatherResponse(BaseModel):
    city: str
    temperature: str

agent = create_react_agent(
    model=model,
    tools=[get_weather],
    reasoning=True,
    response_format=WeatherResponse  # Works together!
)

result = agent.invoke({...})
# result = {
#   "messages": [...],
#   "structured_response": WeatherResponse(...),
#   "reasoning_trace": [...]
# }

Files Changed

New Files (2)

  1. langgraph/prebuilt/reasoning.py

    • Core primitives: Thought, Observation, ReActStep
    • State reducer: add_reasoning_steps

  2. langgraph/prebuilt/reasoning_helpers.py

    • ReasoningCapture helper class
    • Utility functions: metrics, rendering, debugging

Modified Files (2)

  1. langgraph/prebuilt/chat_agent_executor.py

    • Add reasoning: bool = False parameter
    • Add new state schemas with reasoning_trace
    • Integrate ReasoningCapture when reasoning=True

  2. langgraph/prebuilt/__init__.py

    • Export new primitives and utilities

Testing

Test Suite: 59 tests, 100% passing

Execution time: ~0.11 seconds ⚡

Test breakdown:

  • Core primitives: 15 tests
  • State management: 5 tests
  • Helper utilities: 13 tests
  • LangGraph integration: 3 tests
  • Serialization: 3 tests
  • Edge cases: 7 tests
  • ReasoningCapture: 8 tests
  • create_react_agent integration: 5 tests ⭐

Testing Strategy

Following repository conventions:

  • ✅ All tests use FakeToolCallingModel
  • ✅ Fast, deterministic, no API calls
  • ✅ No integration tests with real APIs

New Test Class: TestCreateReactAgentWithReasoning

1. test_reasoning_true_has_correct_state_schema
   → Verifies reasoning_trace field exists in result

2. test_reasoning_false_no_trace_in_state
   → Ensures backwards compatibility

3. test_reasoning_with_response_format_has_both_fields
   → Tests reasoning + structured output together

4. test_reasoning_true_rejects_prebuilt_tool_node
   → Validates error handling

5. test_backward_compatibility_reasoning_false
   → Proves existing code works unchanged

Backwards Compatibility

100% backwards compatible:

Aspect Impact
Existing code ✅ Works unchanged - reasoning=False by default
State schema ✅ Only extended when reasoning=True
Performance ✅ Zero overhead when disabled
API surface ✅ No breaking changes
Return type ✅ Same format, optional new field 
# Existing code - works exactly as before
agent = create_react_agent(model, tools)
result = agent.invoke({"messages": [...]})
# ✅ result["messages"] exists
# ✅ result["reasoning_trace"] does NOT exist (no surprise fields)

# Opt-in - get new functionality
agent = create_react_agent(model, tools, reasoning=True)
result = agent.invoke({"messages": [...]})
# ✅ result["messages"] exists (same as before)
# ✅ result["reasoning_trace"] exists (new!)

Performance

Zero overhead when disabled:

  • reasoning=False (default): No capture logic executed
  • reasoning=True: Microsecond-level overhead per step

Benchmark (reasoning=True overhead):

  • Thought creation: ~0.25μs
  • ReActStep creation: ~0.74μs
  • Reducer per step: ~0.71μs

vs. LLM latency: 100-5000ms → overhead is negligible

Commits

  1. ✅ feat(prebuilt): add True ReAct primitives for explicit reasoning traces

    • Core primitives implementation

  2. ✅ test(prebuilt): add comprehensive tests for ReAct primitives

    • 54 tests for primitives and helpers

  3. ✅ feat(prebuilt): integrate reasoning=True into create_react_agent

    • One-parameter integration

  4. ✅ test(prebuilt): add tests for reasoning=True parameter

    • 5 integration tests using FakeToolCallingModel

Summary

This PR makes agent decision-making visible, debuggable, and measurable with a single reasoning=True parameter. It provides:

  • 🔍 Visibility: See WHY agents make decisions
  • 🐛 Debuggability: Instant failure diagnosis
  • 📊 Observability: Production metrics out-of-the-box
  • 👥 Transparency: Show users what's happening
  • ⚡ Zero overhead: When disabled (default)
  • ✅ 100% backwards compatible: No breaking changes

Ready for review!

**Original Pull Request:** https://github.com/langchain-ai/langgraph/pull/6620 **State:** closed **Merged:** No --- # Add True ReAct Primitives with `reasoning=True` Integration > **TL;DR**: Add `reasoning=True` parameter to `create_react_agent` for automatic capture of Thought → Action → Observation cycles. Makes agent decision-making **visible, debuggable, and measurable** with one parameter. 📄 **RFC**: #6619 --- ## The Problem: Agents Are Black Boxes Today, when you run an agent, you only see messages: ```python result = agent.invoke({"messages": [HumanMessage("What's the weather in Paris?")]}) print(result["messages"]) # [HumanMessage("What's the weather in Paris?"), # AIMessage(tool_calls=[{"name": "get_weather", ...}]), # ToolMessage("22°C, sunny"), # AIMessage("The weather in Paris is 22°C and sunny!")] ``` ### What You CAN'T See: - ❌ **WHY** did the agent decide to call `get_weather`? (reasoning hidden in LLM) - ❌ **HOW LONG** did the tool take? (no timing captured) - ❌ **DID IT FAIL** before succeeding? (no error history) - ❌ **HOW TO DEBUG** when something breaks? (manual log analysis) --- ## The Solution: One Parameter Changes Everything ```python from langgraph.prebuilt import create_react_agent # Just add reasoning=True agent = create_react_agent( model=model, tools=[get_weather, search], reasoning=True # ← That's it! ) result = agent.invoke({"messages": [HumanMessage("What's the weather in Paris?")]}) # NOW YOU SEE EVERYTHING trace = result["reasoning_trace"] for step in trace: print(f"🧠 Thought: {step.thought.content if step.thought else 'N/A'}") print(f"🔧 Action: {step.action['name'] if step.action else 'Final answer'}") print(f"📊 Result: {step.observation.tool_output if step.observation else 'N/A'}") print(f"⏱️ Duration: {step.observation.duration_ms if step.observation else 'N/A'}ms") print(f"✅ Success: {step.observation.success if step.observation else 'N/A'}") print() ``` --- ## What You Get ### 1. Instant Debugging ```python from langgraph.prebuilt import find_failed_steps # Find exactly what went wrong failed = find_failed_steps(trace) for step in failed: print(f"❌ Tool '{step.observation.tool_name}' failed:") print(f" Error: {step.observation.error}") print(f" Duration: {step.observation.duration_ms}ms") print(f" Input: {step.action['args']}") ``` ### 2. Production Metrics ```python from langgraph.prebuilt import calculate_trace_metrics metrics = calculate_trace_metrics(trace) print(f"Total steps: {metrics.total_steps}") print(f"Success rate: {metrics.successful_tool_calls / metrics.tool_calls:.1%}") print(f"Avg latency: {metrics.average_duration_ms:.1f}ms") print(f"Tools used: {metrics.tool_names_used}") ``` ### 3. User Transparency ```python from langgraph.prebuilt import render_reasoning_trace # Show users what the agent is doing print(render_reasoning_trace(trace)) # Step 0: # 💭 Thought: User wants weather info. I'll use the get_weather tool. # 🔧 Action: get_weather(city="Paris") # 📊 Observation: Tool 'get_weather' succeeded in 145.3ms # Output: 22°C, sunny ``` ### 4. Quality Analysis ```python from langgraph.prebuilt import get_tool_usage_summary summary = get_tool_usage_summary(trace) for tool_name, stats in summary.items(): print(f"{tool_name}:") print(f" Calls: {stats['calls']}") print(f" Failures: {stats['failures']}") print(f" Avg duration: {stats['avg_duration_ms']:.1f}ms") ``` --- ## Before/After Comparison | Capability | BEFORE (Today) | AFTER (With `reasoning=True`) | |------------|----------------|-------------------------------| | See agent reasoning | ❌ Hidden in LLM | ✅ `step.thought.content` | | Debug failures | ❌ Read chat logs | ✅ `find_failed_steps(trace)` | | Tool timing | ❌ Not captured | ✅ `observation.duration_ms` | | Success/failure tracking | ❌ Not captured | ✅ `observation.success` | | Confidence scores | ❌ Not captured | ✅ `thought.confidence` | | Aggregate metrics | ❌ Manual counting | ✅ `calculate_trace_metrics()` | | Production monitoring | ❌ Custom implementation | ✅ Built-in utilities | --- ## API Design ### Simple Integration ```python def create_react_agent( model: LanguageModelLike, tools: Sequence[BaseTool] | ToolNode, *, # ... existing parameters ... reasoning: bool = False, # ← NEW: Enable reasoning capture ) -> CompiledStateGraph: ``` ### Automatic State Schema When `reasoning=True`, state automatically includes `reasoning_trace`: ```python # Without reasoning (default) result = agent.invoke({...}) # result = {"messages": [...]} # With reasoning=True result = agent.invoke({...}) # result = {"messages": [...], "reasoning_trace": [...]} ``` ### Works with `response_format` ```python from pydantic import BaseModel class WeatherResponse(BaseModel): city: str temperature: str agent = create_react_agent( model=model, tools=[get_weather], reasoning=True, response_format=WeatherResponse # Works together! ) result = agent.invoke({...}) # result = { # "messages": [...], # "structured_response": WeatherResponse(...), # "reasoning_trace": [...] # } ``` --- ## Files Changed ### New Files (2) 1. **`langgraph/prebuilt/reasoning.py`** - Core primitives: `Thought`, `Observation`, `ReActStep` - State reducer: `add_reasoning_steps` 2. **`langgraph/prebuilt/reasoning_helpers.py`** - `ReasoningCapture` helper class - Utility functions: metrics, rendering, debugging ### Modified Files (2) 1. **`langgraph/prebuilt/chat_agent_executor.py`** - Add `reasoning: bool = False` parameter - Add new state schemas with `reasoning_trace` - Integrate `ReasoningCapture` when `reasoning=True` 2. **`langgraph/prebuilt/__init__.py`** - Export new primitives and utilities --- ## Testing ### Test Suite: **59 tests, 100% passing** **Execution time**: ~0.11 seconds ⚡ **Test breakdown:** - Core primitives: 15 tests - State management: 5 tests - Helper utilities: 13 tests - LangGraph integration: 3 tests - Serialization: 3 tests - Edge cases: 7 tests - `ReasoningCapture`: 8 tests - **`create_react_agent` integration**: 5 tests ⭐ ### Testing Strategy Following repository conventions: - ✅ All tests use `FakeToolCallingModel` - ✅ Fast, deterministic, no API calls - ✅ No integration tests with real APIs **New Test Class**: `TestCreateReactAgentWithReasoning` ```python 1. test_reasoning_true_has_correct_state_schema → Verifies reasoning_trace field exists in result 2. test_reasoning_false_no_trace_in_state → Ensures backwards compatibility 3. test_reasoning_with_response_format_has_both_fields → Tests reasoning + structured output together 4. test_reasoning_true_rejects_prebuilt_tool_node → Validates error handling 5. test_backward_compatibility_reasoning_false → Proves existing code works unchanged ``` --- ## Backwards Compatibility **100% backwards compatible:** | Aspect | Impact | |--------|--------| | **Existing code** | ✅ Works unchanged - `reasoning=False` by default | | **State schema** | ✅ Only extended when `reasoning=True` | | **Performance** | ✅ Zero overhead when disabled | | **API surface** | ✅ No breaking changes | | **Return type** | ✅ Same format, optional new field | ```python # Existing code - works exactly as before agent = create_react_agent(model, tools) result = agent.invoke({"messages": [...]}) # ✅ result["messages"] exists # ✅ result["reasoning_trace"] does NOT exist (no surprise fields) # Opt-in - get new functionality agent = create_react_agent(model, tools, reasoning=True) result = agent.invoke({"messages": [...]}) # ✅ result["messages"] exists (same as before) # ✅ result["reasoning_trace"] exists (new!) ``` --- ## Performance **Zero overhead when disabled:** - `reasoning=False` (default): No capture logic executed - `reasoning=True`: Microsecond-level overhead per step **Benchmark** (reasoning=True overhead): - Thought creation: **~0.25μs** - ReActStep creation: **~0.74μs** - Reducer per step: **~0.71μs** vs. LLM latency: **100-5000ms** → overhead is **negligible** --- ## Commits 1. ✅ `feat(prebuilt): add True ReAct primitives for explicit reasoning traces` - Core primitives implementation 2. ✅ `test(prebuilt): add comprehensive tests for ReAct primitives` - 54 tests for primitives and helpers 3. ✅ `feat(prebuilt): integrate reasoning=True into create_react_agent` - One-parameter integration 4. ✅ `test(prebuilt): add tests for reasoning=True parameter` - 5 integration tests using FakeToolCallingModel --- ## Summary This PR makes agent decision-making **visible, debuggable, and measurable** with a single `reasoning=True` parameter. It provides: - 🔍 **Visibility**: See WHY agents make decisions - 🐛 **Debuggability**: Instant failure diagnosis - 📊 **Observability**: Production metrics out-of-the-box - 👥 **Transparency**: Show users what's happening - ⚡ **Zero overhead**: When disabled (default) - ✅ **100% backwards compatible**: No breaking changes **Ready for review!**
yindo added the pull-request label 2026-02-20 17:51:20 -05:00
yindo closed this issue 2026-02-20 17:51:20 -05:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
agents
awaiting author revision
bug
change
chat models
checkpointer
ci
code quality
code quality
dependencies
documentation
drawing
duplicate
enhancement
good first issue
help wanted
invalid
maintainer
meta
missing information
not maintained by langchain
open-swe-auto
open-swe-max
open-swe-max-auto
pending
performance
pull-request
Mirrored from GitHub Pull Request
 question
requires design
sdk
server
stale
studio
tests
type checking
usage question
v0.5
v1.1
No Label pull-request
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: langchain-ai/langgraph#5172
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?