langchain-ai/langgraph
1
0
Fork 0
mirror of https://github.com/langchain-ai/langgraph.git synced 2026-07-15 12:35:43 -04:00
Code Issues 314 Packages Projects Releases 17 Wiki Activity

langgraph "create run/batch" issue #744

New Issue
Closed
opened 2026-02-20 17:41:32 -05:00 by yindo · 3 comments
yindo commented 2026-02-20 17:41:32 -05:00
Owner
Copy Link
Copy Source

Originally created by @Sriketk on GitHub (Jun 25, 2025).

Originally assigned to: @hinthornw on GitHub.

Checked other resources

  • This is a bug, not a usage question. For questions, please use GitHub Discussions.
  • I added a clear and detailed title that summarizes the issue.
  • I read what a minimal reproducible example is (https://stackoverflow.com/help/minimal-reproducible-example).
  • I included a self-contained, minimal example that demonstrates the issue INCLUDING all the relevant imports. The code run AS IS to reproduce the issue.

Example Code

curl https://[our agent deployment]/runs/batch \
  --request POST \
  --header 'Content-Type: application/json' \
  --data '[
  {
    "assistant_id": "",
    "input": {},
    "stream_mode": ["values"], "on_completion": "keep"}
    },
    {
    "assistant_id": "",
    "input": {},
    "stream_mode": ["values"], "on_completion": "keep"}
    },
    {
    "assistant_id": "",
    "input": {},
    "stream_mode": ["values"], "on_completion": "keep"}
    },
    {
    "assistant_id": "",
    "input": {},
    "stream_mode": ["values"], "on_completion": "keep"}
    }]

Error Message and Stack Trace (if applicable)

Reponse: {
  "thread_id": "28eacf96-a04d-4cbb-8a72-9297c28c092b",
  "created_at": "2025-06-25T14:55:58.492886+00:00",
  "updated_at": "2025-06-25T14:56:03.941798+00:00",
  "metadata": {
    "graph_id": "****",
    "assistant_id": "82977ec6-b940-5330-bba4-a40170abe7ee"
  },
  "status": "error",
  "config": {
    "configurable": {
      "user-agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/137.0.0.0 Safari/537.36",
      "x-amzn-trace-id": "Root=1-685c0dfe-147cb5c071395b6a32216a66",
      "x-forwarded-for": "10.29.26.187",
      "x-forwarded-port": "443",
      "__after_seconds__": 0,
      "x-forwarded-proto": "https",
      "langgraph_auth_user": null,
      "langgraph_request_id": null,
      "langgraph_auth_user_id": "",
      "langgraph_auth_permissions": []
    }
  },
  "values": null,
  "interrupts": {},
  "error": "eyJlcnJvciI6IkhUVFBFeGNlcHRpb24iLCJtZXNzYWdlIjoiNDA0OiBHcmFwaCAndGVjaF9sYWJzX2FjdF9jYWInIG5vdCBmb3VuZC4gRXhwZWN0ZWQgb25lIG9mOiBbJ3RlY2hfbGFic19kcHhfcGVyc29uYV9mZWVkYmFjaycsICd0ZWNoX2xhYnNfaHJfZW1wbG95ZWVfZmVlZGJhY2snXSJ9"
}

Description

We are doing a self-hosted deployment of langgraph. We have multiple agents in their own containers all within the same ECS cluster.

We are invoking a custom cron job(not the langgraph api method) where we run our agents in the background every 2 hours.

When invoking each thread independently, we see no issue even for a large amount of requests. But when we do the run/batch command, we understand that it fires a bunch of the agents parallely as it returns the thread ids immediately and we assume that there some queueing job for the actual run. But the issue we are getting is that for large batches, this api command is looking for agents in another container. The first few invocations of the batch give success responses, but then we get error responses then on out. Our container from where we are invoking the batch job remains healthy even after the errors, so we don't assume its cause our container broke.

The error message above appears when we are trying to get the status of the thread. We are hitting this endpoint to get the error above. End point: /threads/get/threads/{thread_id}. We are expecting the status to be pending or idle, but we see error. When we decode the base64 error, we see this response, {"error":"HTTPException","message":"404: Graph 'tech_labs_act_cab' not found. Expected one of: ['tech_labs_dpx_persona_feedback', 'tech_labs_hr_employee_feedback']"}.

We have multiple containers that we set up through different dockerfiles for out different agents. When running the batch job in tech_labs_act_cab, it seems to want be executing some of the threads in a different container.

Would love to provide more details!

System Info

OS: Darwin
OS Version: Darwin Kernel Version 24.5.0: Tue Apr 22 19:54:29 PDT 2025; root:xnu-11417.121.6~2/RELEASE_ARM64_T6030
Python Version: 3.12.5 (v3.12.5:ff3bc82f7c9, Aug 7 2024, 05:32:06) [Clang 13.0.0 (clang-1300.0.29.30)]

Package Information
langchain_core: 0.3.65
langsmith: 0.3.45
langchain_postgres: 0.0.14
langgraph_api: 0.2.54
langgraph_cli: 0.2.12
langgraph_license: Installed. No version info available.
langgraph_runtime: Installed. No version info available.
langgraph_runtime_inmem: 0.3.0
langgraph_sdk: 0.1.70

Optional packages not installed
langserve

Other Dependencies
asyncpg: 0.30.0
blockbuster<2.0.0,>=1.5.24: Installed. No version info available.
click>=8.1.7: Installed. No version info available.
cloudpickle>=3.0.0: Installed. No version info available.
cryptography<45.0,>=42.0.0: Installed. No version info available.
httpx: 0.28.1
httpx>=0.25.0: Installed. No version info available.
httpx>=0.25.2: Installed. No version info available.
jsonpatch<2.0,>=1.33: Installed. No version info available.
jsonschema-rs<0.30,>=0.20.0: Installed. No version info available.
langchain-core>=0.3.64: Installed. No version info available.
langgraph-api>=0.1.20;: Installed. No version info available.
langgraph-checkpoint>=2.0.23: Installed. No version info available.
langgraph-checkpoint>=2.0.25: Installed. No version info available.
langgraph-runtime-inmem<0.4,>=0.3.0: Installed. No version info available.
langgraph-runtime-inmem>=0.0.8;: Installed. No version info available.
langgraph-sdk>=0.1.0;: Installed. No version info available.
langgraph-sdk>=0.1.66: Installed. No version info available.
langgraph>=0.2: Installed. No version info available.
langgraph>=0.3.27: Installed. No version info available.
langsmith-pyo3: Installed. No version info available.
langsmith<0.4,>=0.3.45: Installed. No version info available.
langsmith>=0.3.45: Installed. No version info available.
numpy: 1.26.4
openai-agents: Installed. No version info available.
opentelemetry-api: Installed. No version info available.
opentelemetry-exporter-otlp-proto-http: Installed. No version info available.
opentelemetry-sdk: Installed. No version info available.
orjson: 3.10.18
orjson>=3.10.1: Installed. No version info available.
orjson>=3.9.7: Installed. No version info available.
packaging: 24.2
packaging<25,>=23.2: Installed. No version info available.
pgvector: 0.3.6
psycopg: 3.2.9
psycopg-pool: 3.2.6
pydantic: 2.11.7
pydantic>=2.7.4: Installed. No version info available.
pyjwt>=2.9.0: Installed. No version info available.
pytest: Installed. No version info available.
python-dotenv>=0.8.0;: Installed. No version info available.
PyYAML>=5.3: Installed. No version info available.
requests: 2.32.4
requests-toolbelt: 1.0.0
rich: Installed. No version info available.
sqlalchemy: 2.0.41
sse-starlette<2.2.0,>=2.1.0: Installed. No version info available.
sse-starlette>=2: Installed. No version info available.
starlette>=0.37: Installed. No version info available.
starlette>=0.38.6: Installed. No version info available.
structlog<26,>=24.1.0: Installed. No version info available.
structlog>23: Installed. No version info available.
tenacity!=8.4.0,<10.0.0,>=8.1.0: Installed. No version info available.
tenacity>=8.0.0: Installed. No version info available.
truststore>=0.1: Installed. No version info available.
typing-extensions>=4.7: Installed. No version info available.
uvicorn>=0.26.0: Installed. No version info available.
watchfiles>=0.13: Installed. No version info available.
zstandard: 0.23.0

Originally created by @Sriketk on GitHub (Jun 25, 2025). Originally assigned to: @hinthornw on GitHub. ### Checked other resources - [x] This is a bug, not a usage question. For questions, please use GitHub Discussions. - [x] I added a clear and detailed title that summarizes the issue. - [x] I read what a minimal reproducible example is (https://stackoverflow.com/help/minimal-reproducible-example). - [x] I included a self-contained, minimal example that demonstrates the issue INCLUDING all the relevant imports. The code run AS IS to reproduce the issue. ### Example Code ```python curl https://[our agent deployment]/runs/batch \ --request POST \ --header 'Content-Type: application/json' \ --data '[ { "assistant_id": "", "input": {}, "stream_mode": ["values"], "on_completion": "keep"} }, { "assistant_id": "", "input": {}, "stream_mode": ["values"], "on_completion": "keep"} }, { "assistant_id": "", "input": {}, "stream_mode": ["values"], "on_completion": "keep"} }, { "assistant_id": "", "input": {}, "stream_mode": ["values"], "on_completion": "keep"} }] ``` ### Error Message and Stack Trace (if applicable) ```shell Reponse: { "thread_id": "28eacf96-a04d-4cbb-8a72-9297c28c092b", "created_at": "2025-06-25T14:55:58.492886+00:00", "updated_at": "2025-06-25T14:56:03.941798+00:00", "metadata": { "graph_id": "****", "assistant_id": "82977ec6-b940-5330-bba4-a40170abe7ee" }, "status": "error", "config": { "configurable": { "user-agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/137.0.0.0 Safari/537.36", "x-amzn-trace-id": "Root=1-685c0dfe-147cb5c071395b6a32216a66", "x-forwarded-for": "10.29.26.187", "x-forwarded-port": "443", "__after_seconds__": 0, "x-forwarded-proto": "https", "langgraph_auth_user": null, "langgraph_request_id": null, "langgraph_auth_user_id": "", "langgraph_auth_permissions": [] } }, "values": null, "interrupts": {}, "error": "eyJlcnJvciI6IkhUVFBFeGNlcHRpb24iLCJtZXNzYWdlIjoiNDA0OiBHcmFwaCAndGVjaF9sYWJzX2FjdF9jYWInIG5vdCBmb3VuZC4gRXhwZWN0ZWQgb25lIG9mOiBbJ3RlY2hfbGFic19kcHhfcGVyc29uYV9mZWVkYmFjaycsICd0ZWNoX2xhYnNfaHJfZW1wbG95ZWVfZmVlZGJhY2snXSJ9" } ``` ### Description We are doing a self-hosted deployment of langgraph. We have multiple agents in their own containers all within the same ECS cluster. We are invoking a custom cron job(not the langgraph api method) where we run our agents in the background every 2 hours. When invoking each thread independently, we see no issue even for a large amount of requests. But when we do the run/batch command, we understand that it fires a bunch of the agents parallely as it returns the thread ids immediately and we assume that there some queueing job for the actual run. But the issue we are getting is that for large batches, this api command is looking for agents in another container. The first few invocations of the batch give success responses, but then we get error responses then on out. Our container from where we are invoking the batch job remains healthy even after the errors, so we don't assume its cause our container broke. The error message above appears when we are trying to get the status of the thread. We are hitting this endpoint to get the error above. End point: /threads/get/threads/{thread_id}. We are expecting the status to be pending or idle, but we see error. When we decode the base64 error, we see this response, {"error":"HTTPException","message":"404: Graph 'tech_labs_act_cab' not found. Expected one of: ['tech_labs_dpx_persona_feedback', 'tech_labs_hr_employee_feedback']"}. We have multiple containers that we set up through different dockerfiles for out different agents. When running the batch job in tech_labs_act_cab, it seems to want be executing some of the threads in a different container. Would love to provide more details! ### System Info OS: Darwin OS Version: Darwin Kernel Version 24.5.0: Tue Apr 22 19:54:29 PDT 2025; root:xnu-11417.121.6~2/RELEASE_ARM64_T6030 Python Version: 3.12.5 (v3.12.5:ff3bc82f7c9, Aug 7 2024, 05:32:06) [Clang 13.0.0 (clang-1300.0.29.30)] Package Information langchain_core: 0.3.65 langsmith: 0.3.45 langchain_postgres: 0.0.14 langgraph_api: 0.2.54 langgraph_cli: 0.2.12 langgraph_license: Installed. No version info available. langgraph_runtime: Installed. No version info available. langgraph_runtime_inmem: 0.3.0 langgraph_sdk: 0.1.70 Optional packages not installed langserve Other Dependencies asyncpg: 0.30.0 blockbuster<2.0.0,>=1.5.24: Installed. No version info available. click>=8.1.7: Installed. No version info available. cloudpickle>=3.0.0: Installed. No version info available. cryptography<45.0,>=42.0.0: Installed. No version info available. httpx: 0.28.1 httpx>=0.25.0: Installed. No version info available. httpx>=0.25.2: Installed. No version info available. jsonpatch<2.0,>=1.33: Installed. No version info available. jsonschema-rs<0.30,>=0.20.0: Installed. No version info available. langchain-core>=0.3.64: Installed. No version info available. langgraph-api>=0.1.20;: Installed. No version info available. langgraph-checkpoint>=2.0.23: Installed. No version info available. langgraph-checkpoint>=2.0.25: Installed. No version info available. langgraph-runtime-inmem<0.4,>=0.3.0: Installed. No version info available. langgraph-runtime-inmem>=0.0.8;: Installed. No version info available. langgraph-sdk>=0.1.0;: Installed. No version info available. langgraph-sdk>=0.1.66: Installed. No version info available. langgraph>=0.2: Installed. No version info available. langgraph>=0.3.27: Installed. No version info available. langsmith-pyo3: Installed. No version info available. langsmith<0.4,>=0.3.45: Installed. No version info available. langsmith>=0.3.45: Installed. No version info available. numpy: 1.26.4 openai-agents: Installed. No version info available. opentelemetry-api: Installed. No version info available. opentelemetry-exporter-otlp-proto-http: Installed. No version info available. opentelemetry-sdk: Installed. No version info available. orjson: 3.10.18 orjson>=3.10.1: Installed. No version info available. orjson>=3.9.7: Installed. No version info available. packaging: 24.2 packaging<25,>=23.2: Installed. No version info available. pgvector: 0.3.6 psycopg: 3.2.9 psycopg-pool: 3.2.6 pydantic: 2.11.7 pydantic>=2.7.4: Installed. No version info available. pyjwt>=2.9.0: Installed. No version info available. pytest: Installed. No version info available. python-dotenv>=0.8.0;: Installed. No version info available. PyYAML>=5.3: Installed. No version info available. requests: 2.32.4 requests-toolbelt: 1.0.0 rich: Installed. No version info available. sqlalchemy: 2.0.41 sse-starlette<2.2.0,>=2.1.0: Installed. No version info available. sse-starlette>=2: Installed. No version info available. starlette>=0.37: Installed. No version info available. starlette>=0.38.6: Installed. No version info available. structlog<26,>=24.1.0: Installed. No version info available. structlog>23: Installed. No version info available. tenacity!=8.4.0,<10.0.0,>=8.1.0: Installed. No version info available. tenacity>=8.0.0: Installed. No version info available. truststore>=0.1: Installed. No version info available. typing-extensions>=4.7: Installed. No version info available. uvicorn>=0.26.0: Installed. No version info available. watchfiles>=0.13: Installed. No version info available. zstandard: 0.23.0
yindo added the question label 2026-02-20 17:41:32 -05:00
yindo closed this issue 2026-02-20 17:41:32 -05:00
yindo commented 2026-02-20 17:41:33 -05:00
Author
Owner
Copy Link
Copy Source

@hinthornw commented on GitHub (Jun 25, 2025):

Where is this deployed?

@hinthornw commented on GitHub (Jun 25, 2025): Where is this deployed?
yindo commented 2026-02-20 17:41:33 -05:00
Author
Owner
Copy Link
Copy Source

@Sriketk commented on GitHub (Jun 25, 2025):

ecs fargate

@Sriketk commented on GitHub (Jun 25, 2025): ecs fargate
yindo commented 2026-02-20 17:41:33 -05:00
Author
Owner
Copy Link
Copy Source

@hinthornw commented on GitHub (Jun 26, 2025):

Serverless? LGP is designed around worker queues. It's a persistent server. Incompatible with serverless runtimes.

@hinthornw commented on GitHub (Jun 26, 2025): Serverless? LGP is designed around worker queues. It's a persistent server. Incompatible with serverless runtimes.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
agents
awaiting author revision
bug
change
chat models
checkpointer
ci
code quality
code quality
dependencies
documentation
drawing
duplicate
enhancement
good first issue
help wanted
invalid
maintainer
meta
missing information
not maintained by langchain
open-swe-auto
open-swe-max
open-swe-max-auto
pending
performance
pull-request
Mirrored from GitHub Pull Request
 question
requires design
sdk
server
stale
studio
tests
type checking
usage question
v0.5
v1.1
No Label question
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: langchain-ai/langgraph#744
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?