Commit Graph

8 Commits

Author SHA1 Message Date
dependabot[bot] 3175ccca1b build(deps): bump langsmith from 0.5.20 to 0.6.0 (#512)
[//]: # (dependabot-start)
⚠️  **Dependabot is rebasing this PR** ⚠️ 

Rebasing might not happen immediately, so don't worry if this takes some
time.

Note: if you make any changes to this PR yourself, they will take
precedence over the rebase.

---

[//]: # (dependabot-end)

Bumps [langsmith](https://github.com/langchain-ai/langsmith-sdk) from
0.5.20 to 0.6.0.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/langchain-ai/langsmith-sdk/releases">langsmith's
releases</a>.</em></p>
<blockquote>
<h2>v0.6.0</h2>
<h2>What's Changed</h2>
<ul>
<li>chore(js): bump JS to 0.4.3 by <a
href="https://github.com/dqbd"><code>@​dqbd</code></a> in <a
href="https://redirect.github.com/langchain-ai/langsmith-sdk/pull/2253">langchain-ai/langsmith-sdk#2253</a></li>
<li>Revert &quot;feat: add js prompt caching&quot; by <a
href="https://github.com/angus-langchain"><code>@​angus-langchain</code></a>
in <a
href="https://redirect.github.com/langchain-ai/langsmith-sdk/pull/2258">langchain-ai/langsmith-sdk#2258</a></li>
<li>Revert &quot;feat: Replace UUID5 with deterministic UUID7 for
replicas&quot; by <a
href="https://github.com/angus-langchain"><code>@​angus-langchain</code></a>
in <a
href="https://redirect.github.com/langchain-ai/langsmith-sdk/pull/2257">langchain-ai/langsmith-sdk#2257</a></li>
<li>release(js): bump to 0.4.4 by <a
href="https://github.com/dqbd"><code>@​dqbd</code></a> in <a
href="https://redirect.github.com/langchain-ai/langsmith-sdk/pull/2259">langchain-ai/langsmith-sdk#2259</a></li>
<li>feat: add prompt cache back and setup environment tests by <a
href="https://github.com/langchain-infra"><code>@​langchain-infra</code></a>
in <a
href="https://redirect.github.com/langchain-ai/langsmith-sdk/pull/2260">langchain-ai/langsmith-sdk#2260</a></li>
<li>feat(python): Bump pydantic to v2 by <a
href="https://github.com/angus-langchain"><code>@​angus-langchain</code></a>
in <a
href="https://redirect.github.com/langchain-ai/langsmith-sdk/pull/2248">langchain-ai/langsmith-sdk#2248</a></li>
</ul>
<p><strong>Full Changelog</strong>: <a
href="https://github.com/langchain-ai/langsmith-sdk/compare/v0.5.2...v0.6.0">https://github.com/langchain-ai/langsmith-sdk/compare/v0.5.2...v0.6.0</a></p>
<h2>v0.6.0rc0</h2>
<h2>What's Changed</h2>
<ul>
<li>feat(js): Add support for tracing AI SDK 6 by <a
href="https://github.com/jacoblee93"><code>@​jacoblee93</code></a> in <a
href="https://redirect.github.com/langchain-ai/langsmith-sdk/pull/2237">langchain-ai/langsmith-sdk#2237</a></li>
<li>fix(js): Remove default Jestlike timeout by <a
href="https://github.com/jacoblee93"><code>@​jacoblee93</code></a> in <a
href="https://redirect.github.com/langchain-ai/langsmith-sdk/pull/2243">langchain-ai/langsmith-sdk#2243</a></li>
<li>feat(js): Add support for tracing tool loop agent by <a
href="https://github.com/jacoblee93"><code>@​jacoblee93</code></a> in <a
href="https://redirect.github.com/langchain-ai/langsmith-sdk/pull/2244">langchain-ai/langsmith-sdk#2244</a></li>
<li>feat: Replace UUID5 with deterministic UUID7 for replicas by <a
href="https://github.com/angus-langchain"><code>@​angus-langchain</code></a>
in <a
href="https://redirect.github.com/langchain-ai/langsmith-sdk/pull/2249">langchain-ai/langsmith-sdk#2249</a></li>
<li>feat: add prompt caching to python sdk by <a
href="https://github.com/langchain-infra"><code>@​langchain-infra</code></a>
in <a
href="https://redirect.github.com/langchain-ai/langsmith-sdk/pull/2246">langchain-ai/langsmith-sdk#2246</a></li>
<li>feat: add js prompt caching by <a
href="https://github.com/langchain-infra"><code>@​langchain-infra</code></a>
in <a
href="https://redirect.github.com/langchain-ai/langsmith-sdk/pull/2251">langchain-ai/langsmith-sdk#2251</a></li>
<li>fix(claude): correctly parse llm and tool inputs in claude agent sdk
by <a
href="https://github.com/angus-langchain"><code>@​angus-langchain</code></a>
in <a
href="https://redirect.github.com/langchain-ai/langsmith-sdk/pull/2255">langchain-ai/langsmith-sdk#2255</a></li>
<li>bump(python): 0.5.2 by <a
href="https://github.com/angus-langchain"><code>@​angus-langchain</code></a>
in <a
href="https://redirect.github.com/langchain-ai/langsmith-sdk/pull/2256">langchain-ai/langsmith-sdk#2256</a></li>
</ul>
<p><strong>Full Changelog</strong>: <a
href="https://github.com/langchain-ai/langsmith-sdk/compare/v0.5.1...v0.6.0rc0">https://github.com/langchain-ai/langsmith-sdk/compare/v0.5.1...v0.6.0rc0</a></p>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li>See full diff in <a
href="https://github.com/langchain-ai/langsmith-sdk/commits/v0.6.0">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=langsmith&package-manager=npm_and_yarn&previous-version=0.5.20&new-version=0.6.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)


</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-05-05 05:41:13 +00:00
Hunter Lovell d49b175dbd feat(evals): add aggregate all-evals entrypoint and suite-builder structure (#457)
## Summary

This PR introduces a first-class aggregate eval entrypoint (`evals/all`)
that runs every existing eval suite through one top-level
LangSmith/Vitest describe block.

To support that cleanly, eval suites were refactored into a reusable
two-file shape: `index.ts` defines suite registration logic
(`define...Suite(runner)`), and `eval.test.ts` owns LangSmith
attribution (`ls.describe(...)`). This removes reliance on ad-hoc global
override wiring and gives the all-evals package a single explicit
integration surface.

## Changes

### Add aggregate all-evals package
- Added `@deepagents/eval-all` package:
- `evals/all/eval.test.ts` imports and invokes every suite registration
function.
- `evals/all/vitest.config.ts` runs only the aggregate `eval.test.ts`
entrypoint.
  - Added package README and package metadata.

### Refactor all eval suites to reusable registration modules
- Standardized suite implementation files to `index.ts`.
- Standardized eval wrappers to `eval.test.ts`.
- For Oolong datasets, standardized to `<dataset>.ts` +
`<dataset>.eval.test.ts`.
- Moved `ls.describe` dataset/project attribution into wrapper test
files so registration modules focus on test definitions only.
- Ensured suite registration functions accept `runner` as an argument
across suites.

### Ensure all existing evals are included in aggregate execution
- `evals/all/eval.test.ts` now imports and calls all suite registration
functions for:
  - core eval suites under `evals/*`
  - oolong dataset suites under `evals/oolong/datasets/*`

### Documentation updates
- Updated `evals/README.md` examples and structure references for
`index.ts` + `eval.test.ts` layout.
- Updated `evals/all/README.md` to document aggregate-run architecture.

### Lockfile updates
- Updated `pnpm-lock.yaml` to include the new `evals/all` workspace
package and dependencies.
2026-04-14 00:28:06 -07:00
Hunter Lovell 89ee206ba6 chore: upgrade langgraph + langchain (#434) 2026-04-07 08:41:32 -07:00
Hunter Lovell ca5cc0acfb chore: merge alpha into main (#404)
* Revert "revert: "feat(deepagents): support multimodal files for backends (#298)" (#352)" (#353)

This reverts commit 03ea1c9729.

* revert: "revert: "feat(sdk): add async subagent middleware for remote LangGraph servers  (#323)" (#351)" (#354)

* Revert "revert: "feat(sdk): add async subagent middleware for remote LangGraph servers  (#323)" (#351)"

This reverts commit 367e43a5ed.

* use any backend protocol

* Reapply "chore(deepagents): refactor backend method names - `lsInfo` -> `ls`, …" (#349) (#356)

This reverts commit 573479d743.

* Reapply "chore(sdk): unify sync subagents and async subagents into a single pr…" (#348) (#355)

This reverts commit 96dc34ccb6.

* chore: align alpha with main (#358)

* fix(deepagents): remove orphaned ToolMessages for Gemini compatibility (#335)

* fix(deepagents): remove orphaned ToolMessages for Gemini compatibility

* Fix ToolMessages for Gemini compatibility

---------

Co-authored-by: Christian Bromann <git@bromann.dev>

* fix(deepagents): throw on built-in tool collision (#330)

* add error

* Create big-horses-fail.md

* add config error class

* cr

---------

Co-authored-by: Christian Bromann <git@bromann.dev>

* fix(deepagents): use `crypto.randomUUID()` instead of uuid (#336)

* fix(deepagents): use crypto.randomUUID() instead of uuid

* update pnpm-lock

* Create grumpy-weeks-wave.md

* Update libs/deepagents/src/middleware/fs.int.test.ts

* feat(deepagent): add LangSmithSandbox (#324)

* feat(deepagent): add LangSmithSandbox

* Change deepagents version from patch to minor

* format

* fix tests

* format

* make it a patch

* cr

* cr

* fix

* cr

* chore: version packages (#321)

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

* regen lockfile

* fix langsmith tests so that they use backend protocol v2 methods

* format

---------

Co-authored-by: pawel-twardziak <pawel.twardziak.dev@gmail.com>
Co-authored-by: Christian Bromann <git@bromann.dev>
Co-authored-by: Maahir Sachdev <maahir.sachdev@langchain.dev>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

* feat(deepagents): add completion notifier middleware for async subagents (#334)

Port of langchain-ai/deepagents#2119 to TypeScript. Adds a
createCompletionNotifierMiddleware that async subagents can use to
proactively notify their supervisor when they complete or error,
closing the gap where the supervisor only learns about completion
when someone calls check_async_task.

- New createCompletionNotifierMiddleware with afterAgent and
  wrapModelCall hooks
- Uses @langchain/langgraph-sdk Client to send runs.create() to the
  supervisor's thread
- Reads parent_thread_id from subagent state (injected by
  start_async_task)
- Derives task_id from runtime.configurable.thread_id
- Silently no-ops if parent context is missing
- Guards against duplicate notifications
- 22 unit tests covering all hooks, edge cases, and error paths

fix(deepagents): make url required in completion notifier (no ASGI in JS)

JS does not have ASGI transport like Python, so the url parameter
must be provided explicitly. Removed all ASGI references from docs
and the localhost fallback default.

fix(deepagents): throw on built-in tool collision (#330)

* add error

* Create big-horses-fail.md

* add config error class

* cr

---------

Co-authored-by: Christian Bromann <git@bromann.dev>

fix(deepagents): use `crypto.randomUUID()` instead of uuid (#336)

* fix(deepagents): use crypto.randomUUID() instead of uuid

* update pnpm-lock

* Create grumpy-weeks-wave.md

* Update libs/deepagents/src/middleware/fs.int.test.ts

feat(deepagent): add LangSmithSandbox (#324)

* feat(deepagent): add LangSmithSandbox

* Change deepagents version from patch to minor

* format

* fix tests

* format

* make it a patch

* cr

* cr

* fix

* cr

regen lockfile

linting

linting

add missing url property

chore: version packages (#321)

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

changeset

regen lockfil

* chore: enter alpha pre-release

* chore: target alpha for releases

* chore: version packages (alpha) (#359)

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

* chore(deepagents): extend supported backend file types (#363)

* extend supported file types

* Create strong-tigers-share.md

---------

Co-authored-by: Hunter Lovell <40191806+hntrl@users.noreply.github.com>

* chore(deepagents): implement async subagents + use stream example (#360)

* async subagents + use stream example

* fix lockfile

* format

* linting

* readme and linting

* format

* proactively send responses when subagents complete

* better examples

* feat(deepagents): rename completion notifier to completion callback and align with Python (#361)

* feat(deepagents): rename completion notifier to completion callback and align with Python PR

- Rename completion_notifier.ts -> completion_callback.ts to match Python's
  completion_callback.py naming
- Rename exports: createCompletionNotifierMiddleware -> createCompletionCallbackMiddleware,
  CompletionNotifierOptions -> CompletionCallbackOptions
- Rename state key: parent_thread_id -> callbackThreadId, option: parentGraphId -> callbackGraphId
- Make url optional (Python allows omitting for ASGI transport)
- Match Python's strict error behavior: throw on empty messages, non-AIMessage types,
  and missing callbackThreadId
- Add truncation suffix with task_id hint for long messages
- Use generic error message in wrapModelCall (don't leak error details)
- Remove duplicate notification guard (Python notifies on every error)
- Add extractCallbackContext to async_subagents.ts: injects callbackThreadId
  into subagent input state when launching via start_async_task
- Add tests for extractCallbackContext and callback context injection

* cr

* Rename completion notifier to completion callback

Renamed completion notifier to completion callback for consistency with Python.

* fix(sdk): `AsyncTask` `updatedAt` field doesn't update on task status changes (#400)

* update updatedAt field to change on any task update

* added changeset

* chore: set up self hosted async subagent example (#399)

* self hosted async subagent example

* with postgres

* formatting

* eslint disable no console

* fix dockerfile and readme

* Update examples/async-subagent-server/server.ts

Co-authored-by: Christian Bromann <git@bromann.dev>

---------

Co-authored-by: Christian Bromann <git@bromann.dev>

* chore(sdk): update async subagent middleware for agent protocol (#394)

* update async subagent middleware for agent protocol

* add changeset

* Update libs/deepagents/src/middleware/async_subagents.ts

Co-authored-by: Hunter Lovell <40191806+hntrl@users.noreply.github.com>

* Update libs/deepagents/src/middleware/async_subagents.ts

Co-authored-by: Hunter Lovell <40191806+hntrl@users.noreply.github.com>

* Update libs/deepagents/src/middleware/async_subagents.ts

Co-authored-by: Hunter Lovell <40191806+hntrl@users.noreply.github.com>

* differentiate agent protocol

---------

Co-authored-by: Hunter Lovell <40191806+hntrl@users.noreply.github.com>

* chore(repo): migrate linting and formatting to oxc tooling (#391)

* chore(repo): migrate linting and formatting to oxc tooling

* cr

* cr

* chore(lint): clean up console disables for oxlint

* cr

* Apply suggestions from code review

Co-authored-by: Christian Bromann <git@bromann.dev>

---------

Co-authored-by: Christian Bromann <git@bromann.dev>

* refactor(deepagents): clean up createDeepAgent middleware wiring (#392)

* refactor(deepagents): clean up createDeepAgent middleware wiring

* fix(deepagents): avoid duplicate HITL middleware on subagents

* add comments, remove iife

* Create ten-masks-flow.md

* fix(deepagents): align prompt templates with runtime behavior (#393)

* fix(deepagents): align prompt templates with runtime behavior

* chore: add changeset for prompt alignment fixes

* cr

* cr

* fix store backend and tests

* lint

* fix rests and resolveBackend

* lint

* fix failing tests

* revert adapt resolve backend

* fix resolve backend

* better variable name

* fix backend factory to return a maybe promise

* mark resolve backend as internal

* format

---------

Co-authored-by: Colin Francis <131073567+colifran@users.noreply.github.com>
Co-authored-by: pawel-twardziak <pawel.twardziak.dev@gmail.com>
Co-authored-by: Christian Bromann <git@bromann.dev>
Co-authored-by: Maahir Sachdev <maahir.sachdev@langchain.dev>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Colin Francis <colin.francis@langchain.dev>
2026-04-01 17:10:16 -07:00
Christian Bromann 92b26577b8 fix(deepagents): bump langchain deps (#395)
* fix(deepagents): bump langchain deps

* formay

* Update changeset by removing deprecated packages

Removed several packages from the changeset.

* fix build
2026-03-30 21:24:30 -07:00
Christian Bromann 9301a9efcc fix(deepagents): update langchain packages (#390)
* fix(deepagents): update langchain packages

* Create warm-parents-prove.md

* format
2026-03-30 12:59:24 -07:00
Hunter Lovell 3394e22016 chore: bump langchain version (#319)
* chore: bump langchain version

* cr

* cr
2026-03-17 17:05:57 -07:00
Hunter Lovell b30cdf9064 feat: add eval harness and initial eval suites (#309)
* Squashed '.agents/skills/langsmith-skills/' content from commit aec5e8f

git-subtree-dir: .agents/skills/langsmith-skills
git-subtree-split: aec5e8f059ff50b1cde655e50b27a25da7981e60

* feat: add eval harness and initial eval suites

* cr

* cr

* cr

* cr

* cr

* cr

* cr
2026-03-17 10:06:55 -07:00