langchain-ai/langgraph
1
0
Fork 0
mirror of https://github.com/langchain-ai/langgraph.git synced 2026-07-15 12:35:43 -04:00
Code Issues 314 Packages Projects Releases 17 Wiki Activity

RFC: True ReAct Primitives for Explicit Reasoning Traces #1103

New Issue
Closed
opened 2026-02-20 17:43:05 -05:00 by yindo · 3 comments
yindo commented 2026-02-20 17:43:05 -05:00
Owner
Copy Link
Copy Source

Originally created by @fede-kamel on GitHub (Dec 22, 2025).

RFC: True ReAct Primitives for LangGraph

Summary

Add explicit reasoning primitives (Thought, Observation, ReActStep) to LangGraph and integrate them into create_react_agent via a simple reasoning=True parameter. This makes agent decision-making visible, debuggable, and measurable without changing any existing APIs.

The Problem

Agents Are Black Boxes

When you run a LangGraph agent today, you only see the message history:

result = agent.invoke({"messages": [HumanMessage("What's the weather in Paris?")]})

print(result["messages"])
# [HumanMessage("What's the weather in Paris?"),
#  AIMessage(tool_calls=[{"name": "get_weather", "args": {"city": "Paris"}}]),
#  ToolMessage("22°C, sunny"),
#  AIMessage("The weather in Paris is 22°C and sunny!")]

What you CAN'T see:

  • ❌ WHY did the agent decide to call get_weather? (reasoning is hidden in the LLM)
  • ❌ How long did the tool take to execute? (no timing captured)
  • ❌ Did any tools fail before succeeding? (no error history)
  • ❌ What was the agent's confidence in its decision? (not captured)
  • ❌ How can I debug when something goes wrong? (read through message logs manually)

Real-World Pain Points

1. Production Debugging is Painful

# Agent failed - what happened?
# - Read through 50+ messages manually
# - Guess which tool call failed
# - No structured error information
# - Can't measure failure rates

2. No Observability

# Questions you can't answer today:
# - Which tools are called most often?
# - What's the average tool execution time?
# - What's the tool success rate?
# - Where do agents get stuck?

3. Quality Evaluation is Manual

# To evaluate agent quality, you must:
# - Manually review chat logs
# - Count tool calls by hand
# - Estimate reasoning quality subjectively
# - No structured data for analysis

4. Users Can't See Agent Thinking

# Users see magic:
# "Let me check that for you..." → [10 seconds] → "Here's the answer"
#
# They don't know:
# - What the agent is doing
# - Why it's taking time
# - What steps it's following

The Solution

Explicit ReAct Primitives

Make the Thought → Action → Observation cycle first-class and observable:

from langgraph.prebuilt import create_react_agent

# ONE PARAMETER: reasoning=True
agent = create_react_agent(
    model=model,
    tools=[get_weather, search],
    reasoning=True  # That's it!
)

result = agent.invoke({"messages": [HumanMessage("What's the weather in Paris?")]})

# NOW YOU CAN SEE EVERYTHING
trace = result["reasoning_trace"]
print(trace)
# [ReActStep(
#     thought=Thought(
#         content="User wants weather info. I'll use the get_weather tool.",
#         confidence=0.95
#     ),
#     action={"name": "get_weather", "args": {"city": "Paris"}},
#     observation=Observation(
#         tool_name="get_weather",
#         tool_output="22°C, sunny",
#         success=True,
#         duration_ms=145.3
#     ),
#     step_number=0,
#     timestamp=datetime(2025, 1, 15, 10, 30, 0)
# )]

What You Gain

1. Instant Debugging

from langgraph.prebuilt import find_failed_steps

# Find exactly what went wrong
failed = find_failed_steps(trace)
for step in failed:
    print(f"Tool {step.observation.tool_name} failed:")
    print(f"  Error: {step.observation.error}")
    print(f"  After {step.observation.duration_ms}ms")

2. Production Metrics

from langgraph.prebuilt import calculate_trace_metrics

metrics = calculate_trace_metrics(trace)
print(f"Total steps: {metrics.total_steps}")
print(f"Success rate: {metrics.successful_tool_calls / metrics.tool_calls:.1%}")
print(f"Avg duration: {metrics.average_duration_ms:.1f}ms")
print(f"Tools used: {metrics.tool_names_used}")

3. User Transparency

from langgraph.prebuilt import render_reasoning_trace

# Show users what the agent is doing
for step in trace:
    if step.thought:
        print(f"💭 Thinking: {step.thought.content}")
    if step.action:
        print(f"🔧 Using tool: {step.action['name']}")
    if step.observation:
        print(f"📊 Result: {step.observation.tool_output[:50]}...")

4. Quality Analysis

from langgraph.prebuilt import get_tool_usage_summary

# Understand agent behavior patterns
summary = get_tool_usage_summary(trace)
for tool_name, stats in summary.items():
    print(f"{tool_name}:")
    print(f"  Calls: {stats['count']}")
    print(f"  Failures: {stats['failures']}")
    print(f"  Avg time: {stats['avg_duration_ms']:.1f}ms")

Core Primitives

1. Thought - Agent Reasoning

@dataclass
class Thought:
    """Captures agent's reasoning before taking action."""
    content: str                      # The reasoning text
    confidence: float | None = None   # 0.0-1.0 if model provides it
    metadata: dict[str, Any] = field(default_factory=dict)

Why it matters:

  • See WHY the agent made decisions
  • Detect reasoning patterns
  • Identify when agents are uncertain (low confidence)

2. Observation - Tool Execution Results

@dataclass
class Observation:
    """Captures what happened when a tool was executed."""
    tool_name: str
    tool_input: dict[str, Any]
    tool_output: Any
    success: bool
    error: str | None = None
    duration_ms: float | None = None

Why it matters:

  • Track tool performance (latency, failure rates)
  • Debug tool errors with full context
  • Measure production SLAs

3. ReActStep - Complete Reasoning Cycle

@dataclass
class ReActStep:
    """One complete Thought → Action → Observation cycle."""
    thought: Thought | None
    action: dict[str, Any] | None      # Tool call details
    observation: Observation | None
    step_number: int
    timestamp: datetime

    def is_tool_call(self) -> bool:
        """True if this step called a tool."""
        return self.action is not None

    def is_final_response(self) -> bool:
        """True if this is the agent's final answer."""
        return self.action is None and self.thought is not None

Why it matters:

  • Complete audit trail of agent behavior
  • Reconstruct exactly what happened
  • Identify bottlenecks and failure points

4. add_reasoning_steps - State Accumulation

def add_reasoning_steps(
    existing: list[ReActStep] | None,
    new: list[ReActStep] | None,
) -> list[ReActStep]:
    """LangGraph reducer for accumulating reasoning trace."""
    return (existing or []) + (new or [])

Why it matters:

  • Automatically accumulates trace across graph iterations
  • Works seamlessly with LangGraph's state management
  • No manual tracking required

Integration with create_react_agent

Simple API: One Parameter

def create_react_agent(
    model: LanguageModelLike,
    tools: Sequence[BaseTool] | ToolNode,
    *,
    # ... existing parameters ...
    reasoning: bool = False,  # NEW: Enable reasoning capture
) -> CompiledStateGraph:
    """Create a ReAct agent with optional reasoning trace capture."""

Automatic State Schema Selection

When reasoning=True, the state schema automatically includes reasoning_trace:

# Without reasoning (default)
class AgentState(TypedDict):
    messages: Annotated[list, add_messages]

# With reasoning=True
class AgentStateWithReasoning(TypedDict):
    messages: Annotated[list, add_messages]
    reasoning_trace: Annotated[list[ReActStep], add_reasoning_steps]

# With reasoning=True AND response_format
class AgentStateWithStructuredResponseAndReasoning(TypedDict):
    messages: Annotated[list, add_messages]
    structured_response: dict[str, Any]
    reasoning_trace: Annotated[list[ReActStep], add_reasoning_steps]

How It Works

When reasoning=True:

  1. Tool Node Wrapping: Tools are automatically wrapped to capture execution

    capture = ReasoningCapture()
tool_node = ToolNode(tools=tools, wrap_tool_call=capture.wrap_tool_call)

  2. Automatic Capture: Each tool call creates an Observation

    observation = Observation(
    tool_name=tool.name,
    tool_input=tool_call.args,
    tool_output=result,
    success=not isinstance(result, Exception),
    duration_ms=elapsed_ms
)

  3. State Updates: Steps are automatically added to reasoning_trace

    return {
    "messages": [result_message],
    "reasoning_trace": [step]  # Accumulated by add_reasoning_steps
}

Helper Utilities

Debugging

def find_failed_steps(trace: list[ReActStep]) -> list[ReActStep]:
    """Find all steps where tools failed."""
    return [s for s in trace if s.observation and not s.observation.success]

def render_reasoning_trace(trace: list[ReActStep]) -> str:
    """Human-readable rendering of the trace."""
    # Returns formatted multi-line string showing each step

Metrics

@dataclass
class TraceMetrics:
    total_steps: int
    tool_calls: int
    successful_tool_calls: int
    failed_tool_calls: int
    total_duration_ms: float
    average_duration_ms: float
    tool_names_used: list[str]

def calculate_trace_metrics(trace: list[ReActStep]) -> TraceMetrics:
    """Calculate aggregate metrics from a trace."""

Analysis

def get_tool_usage_summary(trace: list[ReActStep]) -> dict[str, dict]:
    """Per-tool usage statistics.

    Returns:
        {
            "tool_name": {
                "calls": int,
                "failures": int,
                "avg_duration_ms": float,
                "total_duration_ms": float
            }
        }
    """

Backwards Compatibility

100% backwards compatible:

Aspect Impact
Existing code Works unchanged - reasoning=False by default
State schema Only extended when reasoning=True
Performance Zero overhead when disabled
API surface No breaking changes
Return type Same format, optional new field

Examples

# Existing code - works exactly as before
agent = create_react_agent(model, tools)
result = agent.invoke({"messages": [...]})
# result["messages"] exists
# result["reasoning_trace"] does NOT exist

# Opt-in - get new functionality
agent = create_react_agent(model, tools, reasoning=True)
result = agent.invoke({"messages": [...]})
# result["messages"] exists (same as before)
# result["reasoning_trace"] exists (new!)

Real-World Use Cases

1. Production Monitoring

# Track agent performance in production
agent = create_react_agent(model, tools, reasoning=True)

for user_request in incoming_requests:
    result = agent.invoke({"messages": [user_request]})
    trace = result["reasoning_trace"]

    # Log to monitoring system
    metrics = calculate_trace_metrics(trace)
    monitor.record("agent.steps", metrics.total_steps)
    monitor.record("agent.success_rate",
                   metrics.successful_tool_calls / metrics.tool_calls)
    monitor.record("agent.latency_p50", metrics.average_duration_ms)

    # Alert on failures
    failed = find_failed_steps(trace)
    if failed:
        alert.send(f"Agent had {len(failed)} failed tool calls")

2. User Transparency

# Show users what's happening in real-time
agent = create_react_agent(model, tools, reasoning=True)

async for event in agent.astream_events({"messages": [user_msg]}):
    if "reasoning_trace" in event:
        steps = event["reasoning_trace"]
        if steps:
            step = steps[-1]
            if step.thought:
                await websocket.send(f"💭 {step.thought.content}")
            if step.observation:
                await websocket.send(f"✓ {step.observation.tool_name} completed")

3. Quality Evaluation

# Evaluate agent quality on test set
test_cases = load_test_cases()
results = []

for test in test_cases:
    result = agent.invoke({"messages": [test.input]})
    trace = result["reasoning_trace"]
    metrics = calculate_trace_metrics(trace)

    results.append({
        "test_id": test.id,
        "steps": metrics.total_steps,
        "success": all(s.observation.success for s in trace
                      if s.observation),
        "tools_used": metrics.tool_names_used,
        "total_time_ms": metrics.total_duration_ms
    })

# Analyze patterns
avg_steps = mean(r["steps"] for r in results)
success_rate = mean(r["success"] for r in results)

4. Debugging Failed Runs

# Replay and debug a failed production run
def debug_failure(conversation_id: str):
    # Load the trace from storage
    trace = storage.load_trace(conversation_id)

    # Find what went wrong
    failed = find_failed_steps(trace)
    for step in failed:
        print(f"\n❌ Step {step.step_number} failed:")
        print(f"   Tool: {step.observation.tool_name}")
        print(f"   Input: {step.observation.tool_input}")
        print(f"   Error: {step.observation.error}")
        print(f"   Duration: {step.observation.duration_ms}ms")

    # Render full trace for context
    print("\n" + render_reasoning_trace(trace))

Implementation Details

Files Modified

  1. langgraph/prebuilt/reasoning.py (NEW)

    • Core primitives: Thought, Observation, ReActStep
    • State reducer: add_reasoning_steps

  2. langgraph/prebuilt/reasoning_helpers.py (NEW)

    • ReasoningCapture helper class
    • Utility functions: metrics, rendering, debugging

  3. langgraph/prebuilt/chat_agent_executor.py (MODIFIED)

    • Add reasoning: bool = False parameter
    • Add new state schemas with reasoning_trace
    • Integrate ReasoningCapture when reasoning=True

  4. langgraph/prebuilt/__init__.py (MODIFIED)

    • Export new primitives and utilities

Testing

59 comprehensive tests covering:

  • Core primitives (15 tests)
  • State management (5 tests)
  • Helper utilities (13 tests)
  • LangGraph integration (3 tests)
  • Serialization (3 tests)
  • Edge cases (7 tests)
  • ReasoningCapture (8 tests)
  • create_react_agent integration (5 tests)

All tests use FakeToolCallingModel following repository conventions.

Alternatives Considered

1. Separate Package (langgraph-observability)

Rejected: Adds dependency management burden, fragments ecosystem
Chosen: Built into langgraph.prebuilt

2. Always-On Capture

Rejected: Performance overhead for users who don't need it
Chosen: Opt-in via reasoning=True

3. Manual Helper Class Only

Rejected: Every user would need to wire it up themselves
Chosen: One-line parameter integration + helpers for advanced use

4. Different Parameter Name (trace=True, debug=True)

Rejected: trace is too generic, debug implies development-only
Chosen: reasoning=True clearly describes what's being captured

Success Metrics

This feature succeeds if:

  1. Adoption: > 20% of create_react_agent users enable reasoning=True
  2. Debugging Time: Reduces average debugging time by 50%
  3. Production Usage: Used in production monitoring (not just development)
  4. Documentation: Becomes standard in agent debugging tutorials

Future Enhancements

Not in this RFC, but natural extensions:

  1. Streaming Support: Stream reasoning steps as they happen
  2. Persistence: Save traces to database for replay
  3. Visualization: Web UI for exploring traces
  4. Extended Thought: Capture model's thinking tokens (when available)
  5. Custom Metrics: User-defined metric calculation

References

Conclusion

Adding reasoning=True to create_react_agent makes agent behavior visible, debuggable, and measurable with a single parameter. This is essential infrastructure for production agents that developers have been building manually. By making it built-in, we:

  • Reduce debugging time from hours to minutes
  • Enable production monitoring out of the box
  • Improve user experience with transparency
  • Maintain 100% backwards compatibility

The implementation is complete, tested, and ready for production use.

Originally created by @fede-kamel on GitHub (Dec 22, 2025). # RFC: True ReAct Primitives for LangGraph ## Summary Add explicit reasoning primitives (`Thought`, `Observation`, `ReActStep`) to LangGraph and integrate them into `create_react_agent` via a simple `reasoning=True` parameter. This makes agent decision-making **visible, debuggable, and measurable** without changing any existing APIs. ## The Problem ### Agents Are Black Boxes When you run a LangGraph agent today, you only see the message history: ```python result = agent.invoke({"messages": [HumanMessage("What's the weather in Paris?")]}) print(result["messages"]) # [HumanMessage("What's the weather in Paris?"), # AIMessage(tool_calls=[{"name": "get_weather", "args": {"city": "Paris"}}]), # ToolMessage("22°C, sunny"), # AIMessage("The weather in Paris is 22°C and sunny!")] ``` **What you CAN'T see:** - ❌ WHY did the agent decide to call `get_weather`? (reasoning is hidden in the LLM) - ❌ How long did the tool take to execute? (no timing captured) - ❌ Did any tools fail before succeeding? (no error history) - ❌ What was the agent's confidence in its decision? (not captured) - ❌ How can I debug when something goes wrong? (read through message logs manually) ### Real-World Pain Points **1. Production Debugging is Painful** ```python # Agent failed - what happened? # - Read through 50+ messages manually # - Guess which tool call failed # - No structured error information # - Can't measure failure rates ``` **2. No Observability** ```python # Questions you can't answer today: # - Which tools are called most often? # - What's the average tool execution time? # - What's the tool success rate? # - Where do agents get stuck? ``` **3. Quality Evaluation is Manual** ```python # To evaluate agent quality, you must: # - Manually review chat logs # - Count tool calls by hand # - Estimate reasoning quality subjectively # - No structured data for analysis ``` **4. Users Can't See Agent Thinking** ```python # Users see magic: # "Let me check that for you..." → [10 seconds] → "Here's the answer" # # They don't know: # - What the agent is doing # - Why it's taking time # - What steps it's following ``` ## The Solution ### Explicit ReAct Primitives Make the **Thought → Action → Observation** cycle first-class and observable: ```python from langgraph.prebuilt import create_react_agent # ONE PARAMETER: reasoning=True agent = create_react_agent( model=model, tools=[get_weather, search], reasoning=True # That's it! ) result = agent.invoke({"messages": [HumanMessage("What's the weather in Paris?")]}) # NOW YOU CAN SEE EVERYTHING trace = result["reasoning_trace"] print(trace) # [ReActStep( # thought=Thought( # content="User wants weather info. I'll use the get_weather tool.", # confidence=0.95 # ), # action={"name": "get_weather", "args": {"city": "Paris"}}, # observation=Observation( # tool_name="get_weather", # tool_output="22°C, sunny", # success=True, # duration_ms=145.3 # ), # step_number=0, # timestamp=datetime(2025, 1, 15, 10, 30, 0) # )] ``` ### What You Gain **1. Instant Debugging** ```python from langgraph.prebuilt import find_failed_steps # Find exactly what went wrong failed = find_failed_steps(trace) for step in failed: print(f"Tool {step.observation.tool_name} failed:") print(f" Error: {step.observation.error}") print(f" After {step.observation.duration_ms}ms") ``` **2. Production Metrics** ```python from langgraph.prebuilt import calculate_trace_metrics metrics = calculate_trace_metrics(trace) print(f"Total steps: {metrics.total_steps}") print(f"Success rate: {metrics.successful_tool_calls / metrics.tool_calls:.1%}") print(f"Avg duration: {metrics.average_duration_ms:.1f}ms") print(f"Tools used: {metrics.tool_names_used}") ``` **3. User Transparency** ```python from langgraph.prebuilt import render_reasoning_trace # Show users what the agent is doing for step in trace: if step.thought: print(f"💭 Thinking: {step.thought.content}") if step.action: print(f"🔧 Using tool: {step.action['name']}") if step.observation: print(f"📊 Result: {step.observation.tool_output[:50]}...") ``` **4. Quality Analysis** ```python from langgraph.prebuilt import get_tool_usage_summary # Understand agent behavior patterns summary = get_tool_usage_summary(trace) for tool_name, stats in summary.items(): print(f"{tool_name}:") print(f" Calls: {stats['count']}") print(f" Failures: {stats['failures']}") print(f" Avg time: {stats['avg_duration_ms']:.1f}ms") ``` ## Core Primitives ### 1. `Thought` - Agent Reasoning ```python @dataclass class Thought: """Captures agent's reasoning before taking action.""" content: str # The reasoning text confidence: float | None = None # 0.0-1.0 if model provides it metadata: dict[str, Any] = field(default_factory=dict) ``` **Why it matters:** - See WHY the agent made decisions - Detect reasoning patterns - Identify when agents are uncertain (low confidence) ### 2. `Observation` - Tool Execution Results ```python @dataclass class Observation: """Captures what happened when a tool was executed.""" tool_name: str tool_input: dict[str, Any] tool_output: Any success: bool error: str | None = None duration_ms: float | None = None ``` **Why it matters:** - Track tool performance (latency, failure rates) - Debug tool errors with full context - Measure production SLAs ### 3. `ReActStep` - Complete Reasoning Cycle ```python @dataclass class ReActStep: """One complete Thought → Action → Observation cycle.""" thought: Thought | None action: dict[str, Any] | None # Tool call details observation: Observation | None step_number: int timestamp: datetime def is_tool_call(self) -> bool: """True if this step called a tool.""" return self.action is not None def is_final_response(self) -> bool: """True if this is the agent's final answer.""" return self.action is None and self.thought is not None ``` **Why it matters:** - Complete audit trail of agent behavior - Reconstruct exactly what happened - Identify bottlenecks and failure points ### 4. `add_reasoning_steps` - State Accumulation ```python def add_reasoning_steps( existing: list[ReActStep] | None, new: list[ReActStep] | None, ) -> list[ReActStep]: """LangGraph reducer for accumulating reasoning trace.""" return (existing or []) + (new or []) ``` **Why it matters:** - Automatically accumulates trace across graph iterations - Works seamlessly with LangGraph's state management - No manual tracking required ## Integration with `create_react_agent` ### Simple API: One Parameter ```python def create_react_agent( model: LanguageModelLike, tools: Sequence[BaseTool] | ToolNode, *, # ... existing parameters ... reasoning: bool = False, # NEW: Enable reasoning capture ) -> CompiledStateGraph: """Create a ReAct agent with optional reasoning trace capture.""" ``` ### Automatic State Schema Selection When `reasoning=True`, the state schema automatically includes `reasoning_trace`: ```python # Without reasoning (default) class AgentState(TypedDict): messages: Annotated[list, add_messages] # With reasoning=True class AgentStateWithReasoning(TypedDict): messages: Annotated[list, add_messages] reasoning_trace: Annotated[list[ReActStep], add_reasoning_steps] # With reasoning=True AND response_format class AgentStateWithStructuredResponseAndReasoning(TypedDict): messages: Annotated[list, add_messages] structured_response: dict[str, Any] reasoning_trace: Annotated[list[ReActStep], add_reasoning_steps] ``` ### How It Works When `reasoning=True`: 1. **Tool Node Wrapping**: Tools are automatically wrapped to capture execution ```python capture = ReasoningCapture() tool_node = ToolNode(tools=tools, wrap_tool_call=capture.wrap_tool_call) ``` 2. **Automatic Capture**: Each tool call creates an `Observation` ```python observation = Observation( tool_name=tool.name, tool_input=tool_call.args, tool_output=result, success=not isinstance(result, Exception), duration_ms=elapsed_ms ) ``` 3. **State Updates**: Steps are automatically added to `reasoning_trace` ```python return { "messages": [result_message], "reasoning_trace": [step] # Accumulated by add_reasoning_steps } ``` ## Helper Utilities ### Debugging ```python def find_failed_steps(trace: list[ReActStep]) -> list[ReActStep]: """Find all steps where tools failed.""" return [s for s in trace if s.observation and not s.observation.success] def render_reasoning_trace(trace: list[ReActStep]) -> str: """Human-readable rendering of the trace.""" # Returns formatted multi-line string showing each step ``` ### Metrics ```python @dataclass class TraceMetrics: total_steps: int tool_calls: int successful_tool_calls: int failed_tool_calls: int total_duration_ms: float average_duration_ms: float tool_names_used: list[str] def calculate_trace_metrics(trace: list[ReActStep]) -> TraceMetrics: """Calculate aggregate metrics from a trace.""" ``` ### Analysis ```python def get_tool_usage_summary(trace: list[ReActStep]) -> dict[str, dict]: """Per-tool usage statistics. Returns: { "tool_name": { "calls": int, "failures": int, "avg_duration_ms": float, "total_duration_ms": float } } """ ``` ## Backwards Compatibility **100% backwards compatible:** | Aspect | Impact | |--------|--------| | **Existing code** | Works unchanged - `reasoning=False` by default | | **State schema** | Only extended when `reasoning=True` | | **Performance** | Zero overhead when disabled | | **API surface** | No breaking changes | | **Return type** | Same format, optional new field | ### Examples ```python # Existing code - works exactly as before agent = create_react_agent(model, tools) result = agent.invoke({"messages": [...]}) # result["messages"] exists # result["reasoning_trace"] does NOT exist # Opt-in - get new functionality agent = create_react_agent(model, tools, reasoning=True) result = agent.invoke({"messages": [...]}) # result["messages"] exists (same as before) # result["reasoning_trace"] exists (new!) ``` ## Real-World Use Cases ### 1. Production Monitoring ```python # Track agent performance in production agent = create_react_agent(model, tools, reasoning=True) for user_request in incoming_requests: result = agent.invoke({"messages": [user_request]}) trace = result["reasoning_trace"] # Log to monitoring system metrics = calculate_trace_metrics(trace) monitor.record("agent.steps", metrics.total_steps) monitor.record("agent.success_rate", metrics.successful_tool_calls / metrics.tool_calls) monitor.record("agent.latency_p50", metrics.average_duration_ms) # Alert on failures failed = find_failed_steps(trace) if failed: alert.send(f"Agent had {len(failed)} failed tool calls") ``` ### 2. User Transparency ```python # Show users what's happening in real-time agent = create_react_agent(model, tools, reasoning=True) async for event in agent.astream_events({"messages": [user_msg]}): if "reasoning_trace" in event: steps = event["reasoning_trace"] if steps: step = steps[-1] if step.thought: await websocket.send(f"💭 {step.thought.content}") if step.observation: await websocket.send(f"✓ {step.observation.tool_name} completed") ``` ### 3. Quality Evaluation ```python # Evaluate agent quality on test set test_cases = load_test_cases() results = [] for test in test_cases: result = agent.invoke({"messages": [test.input]}) trace = result["reasoning_trace"] metrics = calculate_trace_metrics(trace) results.append({ "test_id": test.id, "steps": metrics.total_steps, "success": all(s.observation.success for s in trace if s.observation), "tools_used": metrics.tool_names_used, "total_time_ms": metrics.total_duration_ms }) # Analyze patterns avg_steps = mean(r["steps"] for r in results) success_rate = mean(r["success"] for r in results) ``` ### 4. Debugging Failed Runs ```python # Replay and debug a failed production run def debug_failure(conversation_id: str): # Load the trace from storage trace = storage.load_trace(conversation_id) # Find what went wrong failed = find_failed_steps(trace) for step in failed: print(f"\n❌ Step {step.step_number} failed:") print(f" Tool: {step.observation.tool_name}") print(f" Input: {step.observation.tool_input}") print(f" Error: {step.observation.error}") print(f" Duration: {step.observation.duration_ms}ms") # Render full trace for context print("\n" + render_reasoning_trace(trace)) ``` ## Implementation Details ### Files Modified 1. **`langgraph/prebuilt/reasoning.py`** (NEW) - Core primitives: `Thought`, `Observation`, `ReActStep` - State reducer: `add_reasoning_steps` 2. **`langgraph/prebuilt/reasoning_helpers.py`** (NEW) - `ReasoningCapture` helper class - Utility functions: metrics, rendering, debugging 3. **`langgraph/prebuilt/chat_agent_executor.py`** (MODIFIED) - Add `reasoning: bool = False` parameter - Add new state schemas with `reasoning_trace` - Integrate `ReasoningCapture` when `reasoning=True` 4. **`langgraph/prebuilt/__init__.py`** (MODIFIED) - Export new primitives and utilities ### Testing **59 comprehensive tests** covering: - Core primitives (15 tests) - State management (5 tests) - Helper utilities (13 tests) - LangGraph integration (3 tests) - Serialization (3 tests) - Edge cases (7 tests) - `ReasoningCapture` (8 tests) - `create_react_agent` integration (5 tests) All tests use `FakeToolCallingModel` following repository conventions. ## Alternatives Considered ### 1. Separate Package (`langgraph-observability`) **Rejected**: Adds dependency management burden, fragments ecosystem **Chosen**: Built into `langgraph.prebuilt` ### 2. Always-On Capture **Rejected**: Performance overhead for users who don't need it **Chosen**: Opt-in via `reasoning=True` ### 3. Manual Helper Class Only **Rejected**: Every user would need to wire it up themselves **Chosen**: One-line parameter integration + helpers for advanced use ### 4. Different Parameter Name (`trace=True`, `debug=True`) **Rejected**: `trace` is too generic, `debug` implies development-only **Chosen**: `reasoning=True` clearly describes what's being captured ## Success Metrics This feature succeeds if: 1. **Adoption**: > 20% of `create_react_agent` users enable `reasoning=True` 2. **Debugging Time**: Reduces average debugging time by 50% 3. **Production Usage**: Used in production monitoring (not just development) 4. **Documentation**: Becomes standard in agent debugging tutorials ## Future Enhancements Not in this RFC, but natural extensions: 1. **Streaming Support**: Stream reasoning steps as they happen 2. **Persistence**: Save traces to database for replay 3. **Visualization**: Web UI for exploring traces 4. **Extended Thought**: Capture model's thinking tokens (when available) 5. **Custom Metrics**: User-defined metric calculation ## References - [ReAct Paper](https://arxiv.org/abs/2210.03629) (Yao et al. 2022) - [Chain of Thought Prompting](https://arxiv.org/abs/2201.11903) (Wei et al. 2022) - LangGraph Create React Agent: `langgraph/prebuilt/chat_agent_executor.py` ## Conclusion Adding `reasoning=True` to `create_react_agent` makes agent behavior **visible, debuggable, and measurable** with a single parameter. This is essential infrastructure for production agents that developers have been building manually. By making it built-in, we: - **Reduce debugging time** from hours to minutes - **Enable production monitoring** out of the box - **Improve user experience** with transparency - **Maintain 100% backwards compatibility** The implementation is complete, tested, and ready for production use.
yindo closed this issue 2026-02-20 17:43:05 -05:00
yindo commented 2026-02-20 17:43:05 -05:00
Author
Owner
Copy Link
Copy Source

@hinthornw commented on GitHub (Dec 23, 2025):

Hi there - this is a lot of auto-generated code that I don't think will be fun to maintain. You can already configure the model for the agent to add the reasoning effort, etc. to the underlying model.

@hinthornw commented on GitHub (Dec 23, 2025): Hi there - this is a lot of auto-generated code that I don't think will be fun to maintain. You can already configure the model for the agent to add the reasoning effort, etc. to the underlying model.
yindo commented 2026-02-20 17:43:05 -05:00
Author
Owner
Copy Link
Copy Source

@fede-kamel commented on GitHub (Dec 23, 2025):

Thanks for the feedback @hinthornw! You're absolutely right that the automatic thought extraction adds unnecessary complexity and maintenance burden. Users can already achieve this by configuring their models with extended thinking/reasoning modes (e.g., Claude with extended thinking, OpenAI reasoning mode).

Closing this RFC as the proposed approach doesn't align with LangGraph's design philosophy of keeping the framework lean and delegating such capabilities to model configuration.

Appreciate you taking the time to review!

@fede-kamel commented on GitHub (Dec 23, 2025): Thanks for the feedback @hinthornw! You're absolutely right that the automatic thought extraction adds unnecessary complexity and maintenance burden. Users can already achieve this by configuring their models with extended thinking/reasoning modes (e.g., Claude with extended thinking, OpenAI reasoning mode). Closing this RFC as the proposed approach doesn't align with LangGraph's design philosophy of keeping the framework lean and delegating such capabilities to model configuration. Appreciate you taking the time to review!
yindo commented 2026-02-20 17:43:05 -05:00
Author
Owner
Copy Link
Copy Source

@fede-kamel commented on GitHub (Dec 23, 2025):

Closing based on maintainer feedback.

@fede-kamel commented on GitHub (Dec 23, 2025): Closing based on maintainer feedback.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
agents
awaiting author revision
bug
change
chat models
checkpointer
ci
code quality
code quality
dependencies
documentation
drawing
duplicate
enhancement
good first issue
help wanted
invalid
maintainer
meta
missing information
not maintained by langchain
open-swe-auto
open-swe-max
open-swe-max-auto
pending
performance
pull-request
Mirrored from GitHub Pull Request
 question
requires design
sdk
server
stale
studio
tests
type checking
usage question
v0.5
v1.1
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: langchain-ai/langgraph#1103
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?