[PR #549] [CLOSED] Add utilities to extract node and graph test cases from a thread #1532

Closed
opened 2026-02-20 17:44:58 -05:00 by yindo · 0 comments
Owner

📋 Pull Request Information

Original PR: https://github.com/langchain-ai/langgraph/pull/549
Author: @nfcampos
Created: 5/29/2024
Status: Closed

Base: mainHead: nc/29may/eval-utils


📝 Commits (1)

  • 448dad9 Add utilities to extract node and graph test cases from a thread

📊 Changes

7 files changed (+572 additions, -64 deletions)

View changed files

📝 langgraph/checkpoint/memory.py (+36 -5)
langgraph/eval/__init__.py (+113 -0)
📝 langgraph/pregel/__init__.py (+10 -2)
📝 langgraph/utils.py (+3 -0)
📝 tests/memory_assert.py (+1 -0)
📝 tests/test_pregel.py (+329 -29)
📝 tests/test_pregel_async.py (+80 -28)

📄 Description

  • node test cases are input/output pairs for each node in the graph
  • graph test cases are multi-turn test cases with sequences of inputs, outputs and trajectories

These can eg. be saved to a LangSmith dataset, or used for evals directly


🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.

## 📋 Pull Request Information **Original PR:** https://github.com/langchain-ai/langgraph/pull/549 **Author:** [@nfcampos](https://github.com/nfcampos) **Created:** 5/29/2024 **Status:** ❌ Closed **Base:** `main` ← **Head:** `nc/29may/eval-utils` --- ### 📝 Commits (1) - [`448dad9`](https://github.com/langchain-ai/langgraph/commit/448dad946c8b70b61e2058917e98b5b31c2fe67e) Add utilities to extract node and graph test cases from a thread ### 📊 Changes **7 files changed** (+572 additions, -64 deletions) <details> <summary>View changed files</summary> 📝 `langgraph/checkpoint/memory.py` (+36 -5) ➕ `langgraph/eval/__init__.py` (+113 -0) 📝 `langgraph/pregel/__init__.py` (+10 -2) 📝 `langgraph/utils.py` (+3 -0) 📝 `tests/memory_assert.py` (+1 -0) 📝 `tests/test_pregel.py` (+329 -29) 📝 `tests/test_pregel_async.py` (+80 -28) </details> ### 📄 Description - node test cases are input/output pairs for each node in the graph - graph test cases are multi-turn test cases with sequences of inputs, outputs and trajectories These can eg. be saved to a LangSmith dataset, or used for evals directly --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>
yindo added the pull-request label 2026-02-20 17:44:58 -05:00
yindo closed this issue 2026-02-20 17:44:58 -05:00
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: langchain-ai/langgraph#1532