langchain-ai/langgraph
1
0
Fork 0
mirror of https://github.com/langchain-ai/langgraph.git synced 2026-07-15 12:35:43 -04:00
Code Issues 314 Packages Projects Releases 17 Wiki Activity

LangGraph Caching Issue with Pydantic Models #865

New Issue
Closed
opened 2026-02-20 17:42:10 -05:00 by yindo · 2 comments
yindo commented 2026-02-20 17:42:10 -05:00
Owner
Copy Link
Copy Source

Originally created by @pascalwhoop on GitHub (Jul 30, 2025).

Checked other resources

  • This is a bug, not a usage question. For questions, please use the LangChain Forum (https://forum.langchain.com/).
  • I added a clear and detailed title that summarizes the issue.
  • I read what a minimal reproducible example is (https://stackoverflow.com/help/minimal-reproducible-example).
  • I included a self-contained, minimal example that demonstrates the issue INCLUDING all the relevant imports. The code run AS IS to reproduce the issue.

Error Message and Stack Trace (if applicable)

Description

LangGraph Caching Issue with Pydantic Models

Summary

LangGraph caching does not work consistently with Pydantic models as state objects due to non-deterministic serialization, despite documentation claiming support for both Pydantic models and TypedDict.

Environment

  • LangGraph version: [latest]
  • Python version: 3.11+
  • Pydantic version: 2.x

Problem Description

When using Pydantic models as state objects in LangGraph workflows with caching enabled (CachePolicy()), the same logical state produces different cache keys on subsequent runs, leading to cache misses and redundant expensive operations.

Root Cause

The issue stems from how Python's pickle module (used internally by LangGraph for state serialization) handles Pydantic models:

  1. Memory addresses in objects: Pydantic models contain internal references that include memory addresses
  2. Non-deterministic internal metadata: Pydantic stores internal metadata that varies between instantiations
  3. Optional field handling: Different serialization behavior for None vs unset optional fields

Reproducible Example

Pydantic Model (Inconsistent Hashing)

from pydantic import BaseModel, Field
import hashlib
import pickle

class MyState(BaseModel):
    drug_name: str = Field(..., description="Name of drug")
    model: str
    result: str | None = None

# Same logical data, different hashes
state1 = MyState(drug_name="aspirin", model="gpt-4", result=None)
state2 = MyState(drug_name="aspirin", model="gpt-4", result=None)

hash1 = hashlib.sha256(pickle.dumps(state1)).hexdigest()
hash2 = hashlib.sha256(pickle.dumps(state2)).hexdigest()

print(f"Hash 1: {hash1}")
print(f"Hash 2: {hash2}")
print(f"Equal: {hash1 == hash2}")  # Often False!

TypedDict (Consistent Hashing)

from typing import TypedDict, NotRequired
import hashlib
import json

class MyState(TypedDict):
    drug_name: str
    model: str
    result: NotRequired[str | None]

# Same logical data, same hashes
state1: MyState = {"drug_name": "aspirin", "model": "gpt-4", "result": None}
state2: MyState = {"drug_name": "aspirin", "model": "gpt-4", "result": None}

hash1 = hashlib.sha256(json.dumps(state1, sort_keys=True, default=str).encode()).hexdigest()
hash2 = hashlib.sha256(json.dumps(state2, sort_keys=True, default=str).encode()).hexdigest()

print(f"Hash 1: {hash1}")
print(f"Hash 2: {hash2}")
print(f"Equal: {hash1 == hash2}")  # Always True

Impact

  • Cache misses lead to redundant expensive LLM calls
  • Inconsistent performance in production
  • Wasted computational resources
  • Misleading cache hit rates

Workaround

Convert Pydantic models to TypedDict for state objects:

# Before (Pydantic - inconsistent caching)
class WorkflowState(BaseModel):
    drug_request: DrugRequest
    model: str
    result: MyResult | None = None

# After (TypedDict - consistent caching)  
class WorkflowState(TypedDict):
    drug_request: DrugRequest
    model: str
    result: NotRequired[MyResult | None]

Proposed Solutions

Option 1: Documentation Update

Update documentation to clarify that while Pydantic models are supported as state objects, TypedDict is recommended for optimal caching performance.

Option 2: Improve Caching Logic

Implement a more deterministic serialization strategy for Pydantic models in the caching layer:

  • Use model.model_dump() with sort_keys=True instead of pickle
  • Add custom serialization logic for Pydantic models

Option 3: Cache Key Generation

Allow users to provide custom cache key generation functions for state objects.

Expected Behavior

Identical state objects (same logical data) should produce identical cache keys regardless of whether they're Pydantic models or TypedDict objects.

Current Behavior

Pydantic models with identical logical data produce different cache keys, leading to cache misses.

Additional Context

This issue affects any workflow that:

  • Uses expensive operations (LLM calls, API requests)
  • Relies on caching for performance
  • Uses Pydantic models for type safety and validation

The workaround (TypedDict) works perfectly but requires sacrificing some of Pydantic's validation and serialization benefits.

Originally created by @pascalwhoop on GitHub (Jul 30, 2025). ### Checked other resources - [x] This is a bug, not a usage question. For questions, please use the LangChain Forum (https://forum.langchain.com/). - [x] I added a clear and detailed title that summarizes the issue. - [x] I read what a minimal reproducible example is (https://stackoverflow.com/help/minimal-reproducible-example). - [x] I included a self-contained, minimal example that demonstrates the issue INCLUDING all the relevant imports. The code run AS IS to reproduce the issue. ### Error Message and Stack Trace (if applicable) ```shell ``` ### Description # LangGraph Caching Issue with Pydantic Models ## Summary LangGraph caching does not work consistently with Pydantic models as state objects due to non-deterministic serialization, despite documentation claiming support for both Pydantic models and TypedDict. ## Environment - LangGraph version: [latest] - Python version: 3.11+ - Pydantic version: 2.x ## Problem Description When using Pydantic models as state objects in LangGraph workflows with caching enabled (`CachePolicy()`), the same logical state produces different cache keys on subsequent runs, leading to cache misses and redundant expensive operations. ## Root Cause The issue stems from how Python's `pickle` module (used internally by LangGraph for state serialization) handles Pydantic models: 1. **Memory addresses in objects**: Pydantic models contain internal references that include memory addresses 2. **Non-deterministic internal metadata**: Pydantic stores internal metadata that varies between instantiations 3. **Optional field handling**: Different serialization behavior for `None` vs unset optional fields ## Reproducible Example ### Pydantic Model (Inconsistent Hashing) ```python from pydantic import BaseModel, Field import hashlib import pickle class MyState(BaseModel): drug_name: str = Field(..., description="Name of drug") model: str result: str | None = None # Same logical data, different hashes state1 = MyState(drug_name="aspirin", model="gpt-4", result=None) state2 = MyState(drug_name="aspirin", model="gpt-4", result=None) hash1 = hashlib.sha256(pickle.dumps(state1)).hexdigest() hash2 = hashlib.sha256(pickle.dumps(state2)).hexdigest() print(f"Hash 1: {hash1}") print(f"Hash 2: {hash2}") print(f"Equal: {hash1 == hash2}") # Often False! ``` ### TypedDict (Consistent Hashing) ```python from typing import TypedDict, NotRequired import hashlib import json class MyState(TypedDict): drug_name: str model: str result: NotRequired[str | None] # Same logical data, same hashes state1: MyState = {"drug_name": "aspirin", "model": "gpt-4", "result": None} state2: MyState = {"drug_name": "aspirin", "model": "gpt-4", "result": None} hash1 = hashlib.sha256(json.dumps(state1, sort_keys=True, default=str).encode()).hexdigest() hash2 = hashlib.sha256(json.dumps(state2, sort_keys=True, default=str).encode()).hexdigest() print(f"Hash 1: {hash1}") print(f"Hash 2: {hash2}") print(f"Equal: {hash1 == hash2}") # Always True ``` ## Impact - Cache misses lead to redundant expensive LLM calls - Inconsistent performance in production - Wasted computational resources - Misleading cache hit rates ## Workaround Convert Pydantic models to TypedDict for state objects: ```python # Before (Pydantic - inconsistent caching) class WorkflowState(BaseModel): drug_request: DrugRequest model: str result: MyResult | None = None # After (TypedDict - consistent caching) class WorkflowState(TypedDict): drug_request: DrugRequest model: str result: NotRequired[MyResult | None] ``` ## Proposed Solutions ### Option 1: Documentation Update Update documentation to clarify that while Pydantic models are *supported* as state objects, TypedDict is *recommended* for optimal caching performance. ### Option 2: Improve Caching Logic Implement a more deterministic serialization strategy for Pydantic models in the caching layer: - Use `model.model_dump()` with `sort_keys=True` instead of pickle - Add custom serialization logic for Pydantic models ### Option 3: Cache Key Generation Allow users to provide custom cache key generation functions for state objects. ## Expected Behavior Identical state objects (same logical data) should produce identical cache keys regardless of whether they're Pydantic models or TypedDict objects. ## Current Behavior Pydantic models with identical logical data produce different cache keys, leading to cache misses. ## Additional Context This issue affects any workflow that: - Uses expensive operations (LLM calls, API requests) - Relies on caching for performance - Uses Pydantic models for type safety and validation The workaround (TypedDict) works perfectly but requires sacrificing some of Pydantic's validation and serialization benefits.
yindo added the bugpending labels 2026-02-20 17:42:10 -05:00
yindo closed this issue 2026-02-20 17:42:10 -05:00
yindo commented 2026-02-20 17:42:10 -05:00
Author
Owner
Copy Link
Copy Source

@chitralputhran commented on GitHub (Aug 2, 2025):

Hey! Submitted a PR that fixes the Pydantic caching issue described here. The solution uses model_dump(mode='python') for deterministic serialization, which resolves the cache key inconsistency problem.

Also, included tests and documentation, but I would appreciate it if you could test it with your workflows to ensure everything works as expected. Let me know if you have any feedback.

Thanks for reviewing!

@chitralputhran commented on GitHub (Aug 2, 2025): Hey! Submitted a PR that fixes the Pydantic caching issue described here. The solution uses `model_dump(mode='python')` for deterministic serialization, which resolves the cache key inconsistency problem. Also, included tests and documentation, but I would appreciate it if you could test it with your workflows to ensure everything works as expected. Let me know if you have any feedback. Thanks for reviewing!
yindo commented 2026-02-20 17:42:10 -05:00
Author
Owner
Copy Link
Copy Source

@casparb commented on GitHub (Sep 17, 2025):

Hi @pascalwhoop, @chitralputhran. I'm closing this issue and #5811 as I cannot reproduce. Tested with Python 3.11.13 and Pydantic 2.11.7 - in your minimal case the pickled bytes are identical.

If you can still repro, please share a self-contained snippet, your specific Python version, and the output of pip freeze, and I'll be happy to take a look.

@casparb commented on GitHub (Sep 17, 2025): Hi @pascalwhoop, @chitralputhran. I'm closing this issue and #5811 as I cannot reproduce. Tested with Python 3.11.13 and Pydantic 2.11.7 - in your minimal case the pickled bytes are identical. If you can still repro, please share a self-contained snippet, your specific Python version, and the output of `pip freeze`, and I'll be happy to take a look.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
agents
awaiting author revision
bug
change
chat models
checkpointer
ci
code quality
code quality
dependencies
documentation
drawing
duplicate
enhancement
good first issue
help wanted
invalid
maintainer
meta
missing information
not maintained by langchain
open-swe-auto
open-swe-max
open-swe-max-auto
pending
performance
pull-request
Mirrored from GitHub Pull Request
 question
requires design
sdk
server
stale
studio
tests
type checking
usage question
v0.5
v1.1
No Label bug pending
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: langchain-ai/langgraph#865
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?