langchain-ai/langsmith-sdk
1
0
Fork 0
mirror of https://github.com/langchain-ai/langsmith-sdk.git synced 2026-07-19 22:33:51 -04:00
Code Issues 125 Packages Projects Releases 16 Wiki Activity

[PR #2331] perf(evaluation): replace polling with asyncio.Event in AsyncExperime… #2247

New Issue
Open
opened 2026-02-15 23:17:05 -05:00 by yindo · 0 comments
yindo commented 2026-02-15 23:17:05 -05:00
Owner
Copy Link
Copy Source

📋 Pull Request Information

Original PR: https://github.com/langchain-ai/langsmith-sdk/pull/2331
Author: @baskaryan
Created: 2/1/2026
Status: 🔄 Open

Base: main ← Head: fix/async-results-polling

📝 Commits (1)

  • b026aff perf(evaluation): replace polling with asyncio.Event in AsyncExperimentResults

📊 Changes

1 file changed (+9 additions, -7 deletions)

View changed files

📝 python/langsmith/evaluation/_arunner.py (+9 -7)

📄 Description

…ntResults

Problem

AsyncExperimentResults.__anext__ used a polling loop with 50ms sleep to wait for new results:

async def _wait_until_index(index: int) -> None:
    while self._processed_count < index:
        await asyncio.sleep(0.05)  # 50ms delay per check

This adds unnecessary latency - for 100 examples, potentially ~5 seconds of cumulative wait time doing nothing but sleeping and checking.

Solution

Replace polling with asyncio.Event for instant notification:

  1. Add self._new_result = asyncio.Event() in __init__
  2. Signal self._new_result.set() in _process_data when results are added
  3. Wait on self._new_result.wait() in __anext__ instead of polling

This eliminates the polling delay entirely - results are now available immediately when ready.

Changes

  • __init__: Added _new_result Event
  • __anext__: Replaced polling loop with await self._new_result.wait()
  • _process_data: Signal event after appending each result and on completion

🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.

## 📋 Pull Request Information **Original PR:** https://github.com/langchain-ai/langsmith-sdk/pull/2331 **Author:** [@baskaryan](https://github.com/baskaryan) **Created:** 2/1/2026 **Status:** 🔄 Open **Base:** `main` ← **Head:** `fix/async-results-polling` --- ### 📝 Commits (1) - [`b026aff`](https://github.com/langchain-ai/langsmith-sdk/commit/b026aff9b50d1976089a2f6d06264fdf017785b5) perf(evaluation): replace polling with asyncio.Event in AsyncExperimentResults ### 📊 Changes **1 file changed** (+9 additions, -7 deletions) <details> <summary>View changed files</summary> 📝 `python/langsmith/evaluation/_arunner.py` (+9 -7) </details> ### 📄 Description …ntResults ## Problem `AsyncExperimentResults.__anext__` used a polling loop with 50ms sleep to wait for new results: ```python async def _wait_until_index(index: int) -> None: while self._processed_count < index: await asyncio.sleep(0.05) # 50ms delay per check ``` This adds unnecessary latency - for 100 examples, potentially ~5 seconds of cumulative wait time doing nothing but sleeping and checking. ## Solution Replace polling with `asyncio.Event` for instant notification: 1. Add `self._new_result = asyncio.Event()` in `__init__` 2. Signal `self._new_result.set()` in `_process_data` when results are added 3. Wait on `self._new_result.wait()` in `__anext__` instead of polling This eliminates the polling delay entirely - results are now available immediately when ready. ## Changes - `__init__`: Added `_new_result` Event - `__anext__`: Replaced polling loop with `await self._new_result.wait()` - `_process_data`: Signal event after appending each result and on completion --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>
yindo added the pull-request label 2026-02-15 23:17:05 -05:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Annotation Queue
Bulk Export
Datasets
Evaluators
Experiments
Feature
Migrated
Perf
Playground
Polish
Prompt Hub
TypeScript
User Feature Request
backend
bug
documentation
enhancement
evaluation
frontend
goalie
insights
integrations
integrations
javascript
multi-modal
needs-design
needs-investigation
needs-scoping
nit
opentelemetry
oss
otel
pull-request
Mirrored from GitHub Pull Request
 python
sdk
token tracking
tracing + observability
ui
No Label pull-request
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: langchain-ai/langsmith-sdk#2247
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?