langchain-ai/langgraph
1
0
Fork 0
mirror of https://github.com/langchain-ai/langgraph.git synced 2026-07-15 12:35:43 -04:00
Code Issues 314 Packages Projects Releases 17 Wiki Activity

In memory cache doesn't work with in memory checkpointer #933

New Issue
Closed
opened 2026-02-20 17:42:25 -05:00 by yindo · 1 comment
yindo commented 2026-02-20 17:42:25 -05:00
Owner
Copy Link
Copy Source

Originally created by @rahul-langchain on GitHub (Aug 21, 2025).

Checked other resources

  • This is a bug, not a usage question. For questions, please use the LangChain Forum (https://forum.langchain.com/).
  • I added a clear and detailed title that summarizes the issue.
  • I read what a minimal reproducible example is (https://stackoverflow.com/help/minimal-reproducible-example).
  • I included a self-contained, minimal example that demonstrates the issue INCLUDING all the relevant imports. The code run AS IS to reproduce the issue.

Example Code

from typing import TypedDict

from langgraph.cache.memory import InMemoryCache
from langgraph.checkpoint.memory import InMemorySaver
from langgraph.graph import StateGraph
from langgraph.types import CachePolicy


class State(TypedDict):
    x: int
    result: int

def expensive_node(state: State) -> State:
    print('NOT CACHED')
    return {"result": state["x"] * 2}

graph = (
    StateGraph(State)
    .add_node(expensive_node, cache_policy=CachePolicy())
    .set_entry_point("expensive_node")
    .set_finish_point("expensive_node")
    .compile(cache=InMemoryCache(), checkpointer=InMemorySaver())
)

CONFIG = {'configurable': {'thread_id':1}}
print(graph.invoke({"x": 5}, config=CONFIG, stream_mode="updates"))
# This should hit cache, but doesn't if we use InMemorySaver
print(graph.invoke({"x": 5}, config=CONFIG, stream_mode="updates"))

Error Message and Stack Trace (if applicable)

I'd expect 'NOT CACHED' to only print once, but it prints twice. If you remove the `InMemorySaver`, the string only prints once because the cache is hit.

Description

If you add an InMemorySaver to a LangGraph that uses InMemoryCache, the cache no longer works, as seen in the example above.

System Info

System Information

OS: Darwin
OS Version: Darwin Kernel Version 24.6.0: Mon Jul 14 11:30:40 PDT 2025; root:xnu-11417.140.69~1/RELEASE_ARM64_T6041
Python Version: 3.13.6 (main, Aug 12 2025, 10:54:49) [Clang 17.0.0 (clang-1700.0.13.5)]

Package Information

langchain_core: 0.3.74
langsmith: 0.4.15
langgraph_sdk: 0.2.2

Optional packages not installed

langserve

Other Dependencies

httpx<1,>=0.23.0: Installed. No version info available.
httpx>=0.25.2: Installed. No version info available.
jsonpatch<2.0,>=1.33: Installed. No version info available.
langsmith-pyo3>=0.1.0rc2;: Installed. No version info available.
langsmith>=0.3.45: Installed. No version info available.
openai-agents>=0.0.3;: Installed. No version info available.
opentelemetry-api>=1.30.0;: Installed. No version info available.
opentelemetry-exporter-otlp-proto-http>=1.30.0;: Installed. No version info available.
opentelemetry-sdk>=1.30.0;: Installed. No version info available.
orjson>=3.10.1: Installed. No version info available.
orjson>=3.9.14;: Installed. No version info available.
packaging>=23.2: Installed. No version info available.
pydantic<3,>=1: Installed. No version info available.
pydantic>=2.7.4: Installed. No version info available.
pytest>=7.0.0;: Installed. No version info available.
PyYAML>=5.3: Installed. No version info available.
requests-toolbelt>=1.0.0: Installed. No version info available.
requests>=2.0.0: Installed. No version info available.
rich>=13.9.4;: Installed. No version info available.
tenacity!=8.4.0,<10.0.0,>=8.1.0: Installed. No version info available.
typing-extensions>=4.7: Installed. No version info available.
vcrpy>=7.0.0;: Installed. No version info available.
zstandard>=0.23.0: Installed. No version info available.

Originally created by @rahul-langchain on GitHub (Aug 21, 2025). ### Checked other resources - [x] This is a bug, not a usage question. For questions, please use the LangChain Forum (https://forum.langchain.com/). - [x] I added a clear and detailed title that summarizes the issue. - [x] I read what a minimal reproducible example is (https://stackoverflow.com/help/minimal-reproducible-example). - [x] I included a self-contained, minimal example that demonstrates the issue INCLUDING all the relevant imports. The code run AS IS to reproduce the issue. ### Example Code ```python from typing import TypedDict from langgraph.cache.memory import InMemoryCache from langgraph.checkpoint.memory import InMemorySaver from langgraph.graph import StateGraph from langgraph.types import CachePolicy class State(TypedDict): x: int result: int def expensive_node(state: State) -> State: print('NOT CACHED') return {"result": state["x"] * 2} graph = ( StateGraph(State) .add_node(expensive_node, cache_policy=CachePolicy()) .set_entry_point("expensive_node") .set_finish_point("expensive_node") .compile(cache=InMemoryCache(), checkpointer=InMemorySaver()) ) CONFIG = {'configurable': {'thread_id':1}} print(graph.invoke({"x": 5}, config=CONFIG, stream_mode="updates")) # This should hit cache, but doesn't if we use InMemorySaver print(graph.invoke({"x": 5}, config=CONFIG, stream_mode="updates")) ``` ### Error Message and Stack Trace (if applicable) ```shell I'd expect 'NOT CACHED' to only print once, but it prints twice. If you remove the `InMemorySaver`, the string only prints once because the cache is hit. ``` ### Description If you add an `InMemorySaver` to a LangGraph that uses `InMemoryCache`, the cache no longer works, as seen in the example above. ### System Info System Information ------------------ > OS: Darwin > OS Version: Darwin Kernel Version 24.6.0: Mon Jul 14 11:30:40 PDT 2025; root:xnu-11417.140.69~1/RELEASE_ARM64_T6041 > Python Version: 3.13.6 (main, Aug 12 2025, 10:54:49) [Clang 17.0.0 (clang-1700.0.13.5)] Package Information ------------------- > langchain_core: 0.3.74 > langsmith: 0.4.15 > langgraph_sdk: 0.2.2 Optional packages not installed ------------------------------- > langserve Other Dependencies ------------------ > httpx<1,>=0.23.0: Installed. No version info available. > httpx>=0.25.2: Installed. No version info available. > jsonpatch<2.0,>=1.33: Installed. No version info available. > langsmith-pyo3>=0.1.0rc2;: Installed. No version info available. > langsmith>=0.3.45: Installed. No version info available. > openai-agents>=0.0.3;: Installed. No version info available. > opentelemetry-api>=1.30.0;: Installed. No version info available. > opentelemetry-exporter-otlp-proto-http>=1.30.0;: Installed. No version info available. > opentelemetry-sdk>=1.30.0;: Installed. No version info available. > orjson>=3.10.1: Installed. No version info available. > orjson>=3.9.14;: Installed. No version info available. > packaging>=23.2: Installed. No version info available. > pydantic<3,>=1: Installed. No version info available. > pydantic>=2.7.4: Installed. No version info available. > pytest>=7.0.0;: Installed. No version info available. > PyYAML>=5.3: Installed. No version info available. > requests-toolbelt>=1.0.0: Installed. No version info available. > requests>=2.0.0: Installed. No version info available. > rich>=13.9.4;: Installed. No version info available. > tenacity!=8.4.0,<10.0.0,>=8.1.0: Installed. No version info available. > typing-extensions>=4.7: Installed. No version info available. > vcrpy>=7.0.0;: Installed. No version info available. > zstandard>=0.23.0: Installed. No version info available.
yindo added the bugpending labels 2026-02-20 17:42:25 -05:00
yindo closed this issue 2026-02-20 17:42:25 -05:00
yindo commented 2026-02-20 17:42:25 -05:00
Author
Owner
Copy Link
Copy Source

@casparb commented on GitHub (Sep 15, 2025):

Hey @rahul-langchain. The thread's state accumulates across runs with a checkpointer enabled. The default cache key is computed from the entire node input, and since the input differs between run #1 and run #2, you get a cache miss (the node input is no longer just {"x": 5}).

The good news is that you can configure CachePolicy.key_func:

from typing import TypedDict

from langgraph.cache.memory import InMemoryCache
from langgraph.checkpoint.memory import InMemorySaver
from langgraph.graph import StateGraph
from langgraph.types import CachePolicy


class State(TypedDict):
    x: int
    result: int

def expensive_node(state: State) -> State:
    print('NOT CACHED')
    return {"result": state["x"] * 2}

# let the cache know that the result is determined by state["x"]
def cache_key(state: State):
    return str(state["x"])

graph = (
    StateGraph(State)
    .add_node(expensive_node, cache_policy=CachePolicy(key_func=cache_key))
    .set_entry_point("expensive_node")
    .set_finish_point("expensive_node")
    .compile(cache=InMemoryCache(), checkpointer=InMemorySaver())
)

CONFIG = {'configurable': {'thread_id':1}}
print(graph.invoke({"x": 5}, config=CONFIG, stream_mode="updates"))
print(graph.invoke({"x": 5}, config=CONFIG, stream_mode="updates"))

You'll see the cache was used:

NOT CACHED
[{'expensive_node': {'result': 10}}]
[{'expensive_node': {'result': 10}, '__metadata__': {'cached': True}}]
@casparb commented on GitHub (Sep 15, 2025): Hey @rahul-langchain. The thread's state accumulates across runs with a checkpointer enabled. The default cache key is computed from the entire node input, and since the input differs between run #1 and run #2, you get a cache miss (the node input is no longer just `{"x": 5}`). The good news is that you can configure [`CachePolicy.key_func`](https://langchain-ai.github.io/langgraph/reference/types/?h=key_func#langgraph.types.CachePolicy.key_func): ```python from typing import TypedDict from langgraph.cache.memory import InMemoryCache from langgraph.checkpoint.memory import InMemorySaver from langgraph.graph import StateGraph from langgraph.types import CachePolicy class State(TypedDict): x: int result: int def expensive_node(state: State) -> State: print('NOT CACHED') return {"result": state["x"] * 2} # let the cache know that the result is determined by state["x"] def cache_key(state: State): return str(state["x"]) graph = ( StateGraph(State) .add_node(expensive_node, cache_policy=CachePolicy(key_func=cache_key)) .set_entry_point("expensive_node") .set_finish_point("expensive_node") .compile(cache=InMemoryCache(), checkpointer=InMemorySaver()) ) CONFIG = {'configurable': {'thread_id':1}} print(graph.invoke({"x": 5}, config=CONFIG, stream_mode="updates")) print(graph.invoke({"x": 5}, config=CONFIG, stream_mode="updates")) ``` You'll see the cache was used: ``` NOT CACHED [{'expensive_node': {'result': 10}}] [{'expensive_node': {'result': 10}, '__metadata__': {'cached': True}}] ```
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
agents
awaiting author revision
bug
change
chat models
checkpointer
ci
code quality
code quality
dependencies
documentation
drawing
duplicate
enhancement
good first issue
help wanted
invalid
maintainer
meta
missing information
not maintained by langchain
open-swe-auto
open-swe-max
open-swe-max-auto
pending
performance
pull-request
Mirrored from GitHub Pull Request
 question
requires design
sdk
server
stale
studio
tests
type checking
usage question
v0.5
v1.1
No Label bug pending
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: langchain-ai/langgraph#933
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?