langchain-ai/langgraph
1
0
Fork 0
mirror of https://github.com/langchain-ai/langgraph.git synced 2026-07-15 12:35:43 -04:00
Code Issues 314 Packages Projects Releases 17 Wiki Activity

[PR #1630] [MERGED] Implement LangGraph Scheduler for Kafka #2193

New Issue
Closed
opened 2026-02-20 17:46:30 -05:00 by yindo · 0 comments
yindo commented 2026-02-20 17:46:30 -05:00
Owner
Copy Link
Copy Source

📋 Pull Request Information

Original PR: https://github.com/langchain-ai/langgraph/pull/1630
Author: @nfcampos
Created: 9/5/2024
Status: ✅ Merged
Merged: 9/10/2024
Merged by: @nfcampos

Base: main ← Head: nc/5sep/scheduler-kafka

📝 Commits (10+)

  • 841aabf Implement LangGraph Scheduler for Kafka
  • 381ea9c Rename
  • dc5be3f Rename
  • e6d6f56 Improvements to consumer and producer patterns
  • f09b130 Use pool for postgres checkpointer
  • 5b3bd92 Catch errors thrown by nodes run by Executor
  • 2d02e5e Add error topic, add retries (eg for checkpointer database exceptions)
  • ad463ec Add more test assertions
  • 8e6caab Add test for interrupts
  • d206621 Rename test

📊 Changes

41 files changed (+3713 additions, -334 deletions)

View changed files

📝 .github/workflows/_lint.yml (+1 -1)
📝 .github/workflows/_test.yml (+1 -1)
📝 .github/workflows/ci.yml (+95 -105)
📝 libs/checkpoint-postgres/langgraph/checkpoint/postgres/__init__.py (+7 -1)
📝 libs/checkpoint-postgres/langgraph/checkpoint/postgres/aio.py (+15 -11)
📝 libs/checkpoint-postgres/langgraph/checkpoint/postgres/base.py (+10 -0)
📝 libs/checkpoint-sqlite/langgraph/checkpoint/sqlite/__init__.py (+6 -1)
📝 libs/checkpoint-sqlite/langgraph/checkpoint/sqlite/aio.py (+6 -1)
📝 libs/langgraph/langgraph/constants.py (+13 -3)
📝 libs/langgraph/langgraph/errors.py (+13 -0)
📝 libs/langgraph/langgraph/managed/is_last_step.py (+1 -1)
📝 libs/langgraph/langgraph/pregel/__init__.py (+4 -3)
📝 libs/langgraph/langgraph/pregel/algo.py (+20 -20)
📝 libs/langgraph/langgraph/pregel/debug.py (+1 -0)
📝 libs/langgraph/langgraph/pregel/executor.py (+16 -21)
📝 libs/langgraph/langgraph/pregel/loop.py (+93 -19)
📝 libs/langgraph/langgraph/pregel/runner.py (+30 -8)
📝 libs/langgraph/langgraph/pregel/types.py (+1 -0)
📝 libs/langgraph/tests/compose-postgres.yml (+1 -0)
📝 libs/langgraph/tests/test_pregel.py (+130 -81)

...and 21 more files

📄 Description

  • Orchestrator and Executor classes to run LangGraph in a distributed fashion using Kafka as a message bus for communication
  • Orchestrator and Executor run on-demand when a new message is published to the topic they listen to
  • Orchestrator is responsible for running the Pregel algorithm (deciding next tasks to run) and sending messages to the executor topic
  • Executor is responsible for executing each task (node), and sending messages to the orchestrator topic when done

🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.

## 📋 Pull Request Information **Original PR:** https://github.com/langchain-ai/langgraph/pull/1630 **Author:** [@nfcampos](https://github.com/nfcampos) **Created:** 9/5/2024 **Status:** ✅ Merged **Merged:** 9/10/2024 **Merged by:** [@nfcampos](https://github.com/nfcampos) **Base:** `main` ← **Head:** `nc/5sep/scheduler-kafka` --- ### 📝 Commits (10+) - [`841aabf`](https://github.com/langchain-ai/langgraph/commit/841aabf5b383bf9da84b3c3f8ef724c9f23f2d52) Implement LangGraph Scheduler for Kafka - [`381ea9c`](https://github.com/langchain-ai/langgraph/commit/381ea9cec8dfbc7f47390ebf0c2bdd5269dd4f64) Rename - [`dc5be3f`](https://github.com/langchain-ai/langgraph/commit/dc5be3fd58c5d0677ef5acdb20e26c97c2573b8a) Rename - [`e6d6f56`](https://github.com/langchain-ai/langgraph/commit/e6d6f565a52f5bf981f317db7f729c330cd9735c) Improvements to consumer and producer patterns - [`f09b130`](https://github.com/langchain-ai/langgraph/commit/f09b1308370b9673d1c95bd6b3738639bb5a95c3) Use pool for postgres checkpointer - [`5b3bd92`](https://github.com/langchain-ai/langgraph/commit/5b3bd920d8d662251c719c711eacefba384ec01d) Catch errors thrown by nodes run by Executor - [`2d02e5e`](https://github.com/langchain-ai/langgraph/commit/2d02e5e0c5d2f5ef286b4ee90d1c02b4f9cdc0c5) Add error topic, add retries (eg for checkpointer database exceptions) - [`ad463ec`](https://github.com/langchain-ai/langgraph/commit/ad463ec90a50836b800c58cafffb46208a21ffc2) Add more test assertions - [`8e6caab`](https://github.com/langchain-ai/langgraph/commit/8e6caab78c739030ac2bc08e52a59d195e79d7b7) Add test for interrupts - [`d206621`](https://github.com/langchain-ai/langgraph/commit/d20662123002592b76cb9e936e7a75550d44dd33) Rename test ### 📊 Changes **41 files changed** (+3713 additions, -334 deletions) <details> <summary>View changed files</summary> 📝 `.github/workflows/_lint.yml` (+1 -1) 📝 `.github/workflows/_test.yml` (+1 -1) 📝 `.github/workflows/ci.yml` (+95 -105) 📝 `libs/checkpoint-postgres/langgraph/checkpoint/postgres/__init__.py` (+7 -1) 📝 `libs/checkpoint-postgres/langgraph/checkpoint/postgres/aio.py` (+15 -11) 📝 `libs/checkpoint-postgres/langgraph/checkpoint/postgres/base.py` (+10 -0) 📝 `libs/checkpoint-sqlite/langgraph/checkpoint/sqlite/__init__.py` (+6 -1) 📝 `libs/checkpoint-sqlite/langgraph/checkpoint/sqlite/aio.py` (+6 -1) 📝 `libs/langgraph/langgraph/constants.py` (+13 -3) 📝 `libs/langgraph/langgraph/errors.py` (+13 -0) 📝 `libs/langgraph/langgraph/managed/is_last_step.py` (+1 -1) 📝 `libs/langgraph/langgraph/pregel/__init__.py` (+4 -3) 📝 `libs/langgraph/langgraph/pregel/algo.py` (+20 -20) 📝 `libs/langgraph/langgraph/pregel/debug.py` (+1 -0) 📝 `libs/langgraph/langgraph/pregel/executor.py` (+16 -21) 📝 `libs/langgraph/langgraph/pregel/loop.py` (+93 -19) 📝 `libs/langgraph/langgraph/pregel/runner.py` (+30 -8) 📝 `libs/langgraph/langgraph/pregel/types.py` (+1 -0) 📝 `libs/langgraph/tests/compose-postgres.yml` (+1 -0) 📝 `libs/langgraph/tests/test_pregel.py` (+130 -81) _...and 21 more files_ </details> ### 📄 Description - Orchestrator and Executor classes to run LangGraph in a distributed fashion using Kafka as a message bus for communication - Orchestrator and Executor run on-demand when a new message is published to the topic they listen to - Orchestrator is responsible for running the Pregel algorithm (deciding next tasks to run) and sending messages to the executor topic - Executor is responsible for executing each task (node), and sending messages to the orchestrator topic when done --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>
yindo added the pull-request label 2026-02-20 17:46:30 -05:00
yindo closed this issue 2026-02-20 17:46:30 -05:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
agents
awaiting author revision
bug
change
chat models
checkpointer
ci
code quality
code quality
dependencies
documentation
drawing
duplicate
enhancement
good first issue
help wanted
invalid
maintainer
meta
missing information
not maintained by langchain
open-swe-auto
open-swe-max
open-swe-max-auto
pending
performance
pull-request
Mirrored from GitHub Pull Request
 question
requires design
sdk
server
stale
studio
tests
type checking
usage question
v0.5
v1.1
No Label pull-request
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: langchain-ai/langgraph#2193
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?