Commit Graph

166 Commits

Author SHA1 Message Date
Mason Daugherty 34aaab8682 chore(ci): make talon check failures advisory on PRs that don't touch libs/talon (#5808)
## Problem

`talon` editable-installs `deepagents-code`
(`libs/talon/pyproject.toml`, `[tool.uv.sources]`), so the `talon`
filter in `ci.yml` includes `libs/code/**` and every libs/code PR runs
`lint-talon`/`test-talon`. When a libs/code change breaks talon,
`ci_success` fails, and because branch protection requires ` CI
Success`, the libs/code PR cannot merge until the author also fixes the
talon regression — inside the same PR. That combined PR then makes
release-please fan out changelog entries across both packages.

It is acceptable for PRs that don't touch `libs/talon` to merge with
talon broken at HEAD; the breakage gets fixed in a follow-up
`fix(talon): ...` PR (ideally in a stack). talon is the only consumer of
the libs/code path source, so no other package's CI is affected by this
policy.

## Approach

Advisory, not skipped: talon CI still *runs* on libs/code PRs (the
signal is valuable) — only its blocking effect changes.

- The waiver decision lives in a new helper script,
`.github/scripts/checks/ci_gate.py`, which `ci_success` calls instead of
grepping the `toJSON(needs.*.result)` blob. On `pull_request` runs where
the `talon` filter output is not `'true'`, `failure` results from
`lint-talon`/`test-talon` are waived with a loud `::warning::` per
waived job; every other job still blocks. `cancelled` results are never
waived, genuine talon PRs (`talon == 'true'`) stay strict, and
`push`/`merge_group` runs stay fully strict — main runs every job
unconditionally, so a broken talon is still visible there and cannot
silently go red.
- A new `talon-failure-advisory` job posts/updates a sticky PR comment
(`<!-- talon-ci-advisory -->`) when the waiver fires, naming the failed
jobs, linking the workflow run, and calling out the required
`fix(talon)` follow-up; it deletes the sticky once talon is green again.
It mirrors the sticky-comment pattern in
`release_fanout_bypass_warn.yml`, including the `core.warning` +
job-summary fallback when commenting fails (fork tokens, rate limits).
The job is advisory-only and is deliberately not in `ci_success.needs`,
so it can never gate.
- Unit tests in `.github/scripts/tests/checks/test_ci_gate.py` cover the
waiver matrix (waived lint/test failures, non-talon failures still
blocking, talon PRs and push/merge_group staying strict, cancelled never
waived). They run in the existing `check-release-options` job, which is
unchanged.

No branch-protection settings change is needed: the repo already
requires only ` CI Success` and `validate release dependencies against
PyPI`, both compatible with this change.

Note on scope: the waiver keys off `talon != 'true'`. SDK-only PRs
(`libs/deepagents/**`) also flip the `talon` filter on via fan-out, so
an SDK PR that breaks talon still blocks — only PRs where talon CI runs
*solely* because of the libs/code path source get the waiver.
2026-08-25 11:36:55 -04:00
Mason Daugherty da67c86e52 chore(repo): remove deprecated libs/cli package (#5791)
`deepagents-cli` was deprecated in #5780 in favor of
`managed-deepagents` (`mda`), and 0.3.0 (#5782) was its final release.
The package has been warning at import time since; this PR deletes the
source tree and unwinds it from repo infrastructure.

What changes:

- Deletes `libs/cli/` (source, tests, lockfile, docs).
- Removes `libs/cli` from `release-please-config.json` /
`.release-please-manifest.json` and from the release detection/dispatch
logic in `release-please.yml` and `release.yml`, so release-please stops
managing the package.
- Removes the `cli`/`deepagents-cli` scopes from PR title lint,
branch-name check (workflow + `.githooks/pre-push`), the PR labeler
config and scope-rename aliases, the `cli` file rules, CI change
filters, lint/test jobs, integration-test options, dependabot,
pre-commit hooks, CODEOWNERS, and the bug-report template.
- Updates docs (`AGENTS.md`, `libs/README.md`, `libs/DEVELOPMENT.md`,
`RELEASING.md`, threat models, openwiki briefs) and release-infra script
tests that used `libs/cli` as a fixture (swapped to `libs/acp` /
`libs/evals` equivalents).

Intentionally left in place:

- `examples/ralph_mode/` still imports `deepagents_cli` — tracked
separately; the example needs migration onto `libs/code`'s
non-interactive runner.
- Historical references: the `libs/code/CHANGELOG.md` fork note, and the
`SNAPSHOT_NAME = "deepagents-cli"` constant in the LangSmith sandbox
integration test (it names a live LangSmith snapshot, not the deleted
package).
- `CLI_MAX_RETRIES_KEY = "__deepagents_cli_max_retries__"` in
`libs/code` — a live internal carrier key; renaming is out of scope.

Published `deepagents-cli` releases on PyPI are unaffected; the repo
simply stops cutting new ones.

---------

Signed-off-by: Mason Daugherty <github@mdrxy.com>
2026-08-24 15:26:38 -04:00
Mason Daugherty c9c9fce4ee ci(infra): test release wheels across supported Python versions (#5771)
Release checks now validate built wheels across every Python version
allowed by each package's `requires-python` range.

---

This closes the gap where PR CI covered the full interpreter range while
publication tested only one version. The packaging stages stay on the
package's minimum supported interpreter, and `pre-release-checks` fans
out with `fail-fast: false`.

<details>
<summary>Test plan</summary>

- `python3 -m pytest .github/scripts/tests -q --no-header -p
no:cacheprovider` (1144 passed)
- Ruff check and format-check the changed Python helpers and tests

</details>

Made by [Open
SWE](https://openswe.vercel.app/agents/75cfe962-24e7-71d3-534b-ad27c0d7bc0e)

## References
- Plan:
https://openswe.vercel.app/agents/75cfe962-24e7-71d3-534b-ad27c0d7bc0e/plan

Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
2026-08-24 10:18:06 -04:00
Mason Daugherty d5c12841b0 fix(infra): warn instead of failing when the ripgrep timeout PR is unresolvable (#5667)
The ripgrep timeout warning workflow no longer fails when the CI run's
pull request cannot be resolved. It now logs a warning naming the
timeout artifacts and exits cleanly.

---

The `🔍 Ripgrep timeout warning` workflow fires on `workflow_run` after
CI completes and posts a sticky comment on the PR when a ripgrep install
gave up. When it cannot resolve a PR for the completed run, it currently
hard-fails with `CI run <id> has no resolvable pull request. A ripgrep
timeout went unreported.` — as seen on [this
run](https://github.com/langchain-ai/deepagents/actions/runs/32306207706).

The only reachable way to hit that branch is a race: the CI run was a
`pull_request` event (the job's `if:` filters everything else out), but
the PR was closed or its head branch deleted between CI completing and
the dispatcher firing. Non-PR events can't produce the
`ripgrep-timeout-*` artifacts at all — the strict install's bypass path
that writes them only activates when a `bypass-ripgrep-check` label
resolves, which requires a `pull_request` context.

So the failure was the alarm going off in a hallway with no PR in it:
there is no conversation left to warn, and the CI run's own
`::warning::` annotation is already the durable record of the timeout.
The unresolvable-PR path now uses `core.warning` (naming the artifacts)
instead of `core.setFailed`.

<details>
<summary>Test plan</summary>

- `uv run --no-project --with pytest --with pyyaml python -m pytest
.github/scripts/tests/workflows/test_ci_workflow.py -q` — 44 passed.

</details>
2026-08-19 18:59:24 -04:00
Mason Daugherty a0990ff0d9 feat(infra): bypass-ripgrep-check label for release PR strict install (#5657)
A release PR whose strict ripgrep install hits a transient apt or mirror
failure currently fails CI with no recourse but a re-run. Maintainers
can now add the `bypass-ripgrep-check` label to the release PR to
tolerate that failure: the affected legs skip the ripgrep-gated tests, a
sticky comment on the PR records which legs ran without ripgrep, and
when the PR merges the publish run is dispatched with
`dangerous-skip-ripgrep-check=true` so its own ripgrep install tolerates
the same flake instead of re-failing.

---

## Why

The strict install in `_test.yml` (and the matching `Install ripgrep`
step in `release.yml`) runs with no timeout on release-sensitive paths
on purpose: the rg-gated tests, including the symlink-containment check,
must exercise the real binary rather than the Python fallback, and a
silent skip at that gate is worse than a red job. The cost is that pure
infrastructure flake — an apt mirror timeout, a wedged dpkg — blocks an
otherwise-good release.

This mirrors the existing `bypass-warnings-check` and `release: skip sdk
pin check` patterns rather than inventing a new mechanism:

- **`_test.yml`**: a new `🏷️ Resolve ripgrep bypass` step reads the live
PR labels via `gh api` (not the event payload, so a label added before a
re-run takes effect) and fails closed on an API error. The strict step
keeps trying to install, but on a labeled PR an apt failure exits 0
instead of failing. `DEEPAGENTS_RIPGREP_EXPECTED` is only set when `rg`
is actually usable, so a genuinely missing binary lets
`require_ripgrep()` skip the gated tests on that leg rather than fail
them.
- **`ripgrep_timeout_comment.yml`**: the bypassed strict leg emits the
same `ripgrep-timeout-*` marker artifact as the soft path, so the
existing sticky-comment workflow reports it. The comment now reads the
PR's labels (fail-closed) and branches its wording — a bypassed release
PR says so explicitly and notes that the publish run was dispatched with
the override, instead of the old (now-inaccurate) "release PRs fail if
ripgrep does not install" line.
- **`release-please.yml`**: after the SDK-pin block, it checks the
merged release PR for `bypass-ripgrep-check` via the existing
fail-closed `release_pr_has_label` helper and, when present, passes `-f
dangerous-skip-ripgrep-check=true` to every dispatched package. Unlike
the SDK-pin check this is not package-gated, because every package's
publish run shares the same ripgrep step. The dispatch log, step
summary, and recovery commands all surface the override.
- **`release.yml`**: a new `dangerous-skip-ripgrep-check` boolean input
is logged in the dispatch-inputs summary and softens the `Install
ripgrep` step — a failure is tolerated and `DEEPAGENTS_RIPGREP_EXPECTED`
left unset only when the input is `true`.

Scope: the label is honored only on release PRs. `push`-to-`main` and
merge-queue runs have no PR label to read, so they always enforce the
strict install.

The label must be created in repo settings (`bypass-ripgrep-check`)
before this can take effect; documented under "Release Failed: Ripgrep
Install" in `RELEASING.md`.

<details>
<summary>Test plan</summary>

- Extended `test_ci_workflow.py` with an executable
`test_strict_ripgrep_install_bypass` that runs the strict step's real
`run:` body against stubbed `apt-get`/`rg`/`dpkg`: unlabeled apt
failures fail with the original exit code; labeled failures are
tolerated; ripgrep is only promised to the tests when `rg` is actually
usable; a bypassed leg with no usable `rg` emits the marker artifact.
- Updated the strict-step and upload-step drift assertions to match the
new condition and artifact selection, and added comment-workflow
assertions for the label read and bypass wording.
- All 1108 workflow-helper tests pass; the touched YAML files parse; the
new bash bodies pass `bash -n`; the comment workflow's JS passes `node
--check`; ruff check/format clean on the test file.

</details>
2026-08-19 17:44:15 -04:00
Mason Daugherty 81c6e29577 fix(infra): unset UV_PYTHON when installing release wheels with uv pip (#5655)
Release dependency validation and release import steps no longer fail
with `No virtual environment found for Python 3.14` when the release
package targets a different Python than the one setup-uv installed.

---

The `uv_setup` composite action installs Python 3.14 for running helper
scripts, and `setup-uv` exports `UV_PYTHON=3.14` job-wide. `UV_PYTHON`
takes precedence over `VIRTUAL_ENV` for `uv pip install`, so when the
freshness gate (or release import test) created a venv from the release
package's own `python_version` (e.g. 3.12) and then ran `VIRTUAL_ENV=...
uv pip install`, uv refused to use it:

```
error: No virtual environment found for Python 3.14; run `uv venv` to create an environment, or pass `--system` to install into a non-virtual environment
```

Seen on the `deepagents-code` 0.1.58 release PR:
https://github.com/langchain-ai/deepagents/actions/runs/32297131672/job/96211155972

The fix prefixes each `uv pip install` in these steps with `env -u
UV_PYTHON` so the install targets the venv the step just created. This
affected three call sites: the "Install wheel with dependencies from
PyPI" step in `check_dep_freshness.yml`, and the "Import dist package" /
"Import published package (again)" steps in `release.yml` — the latter
two haven't failed only because the package Python happened to match the
setup version recently. The workflow-mirror test asserting the shared
install invocation is updated to match.
2026-08-19 16:27:36 -04:00
Mason Daugherty 28d44d49c8 feat(infra): name the package in curated release-notes check output (#5656)
The curated release-notes refresh runs share one package-agnostic job
name ("Refresh curated release notes check") across every open release
PR, and neither the job output nor the refreshed check named the package
or version being validated. A failure run surfaced only a bare list of
stale-state reasons ("applied metadata references an older override
comment; …"), leaving no indication of which component was gated or
whether the failure was a real verdict versus a validator error.

`checkCuratedState` now returns the resolved `component` and `version`
on every status it reaches after resolving them, and the workflow
threads them through the output:

- The job log line reads `Curated release-note state for the <component>
<version> release PR: <status>.`
- The refreshed check's title and summary name the target, e.g. `Curated
release notes are valid for the deepagents-code 0.1.58 release PR`.
- The `setFailed` annotation behind a `failed` result is prefixed with
`for <component> <version>:` so the annotation GitHub surfaces on the
run identifies the package on its own.

The failure conclusion itself is unchanged and is by design: dispatch
runs execute from `main`, so the run's native job conclusion is how the
gate reports the release PR's state — `setFailed` marks the run failed
whenever the curated notes are stale, which is distinct from a validator
error (that path gets its own "hit an error talking to GitHub" check
summary).
2026-08-19 16:24:37 -04:00
Mason Daugherty 2c6502fe30 fix(infra): retry the new-entries warning comment and grant pull-requests: write (#5654)
The curated release notes check no longer logs a `Resource not
accessible by integration` warning when posting its courtesy comment
about newly generated changelog entries on a release PR.

---

The check posts a best-effort comment when the generated changelog
drifts from the curated draft. On [this
run](https://github.com/langchain-ai/deepagents/actions/runs/32293839088/job/96201984315)
on #5591, that `POST
/repos/langchain-ai/deepagents/issues/5591/comments` returned 403 even
though the job token had `issues: write` — while an identical run 30
minutes earlier on the same PR posted comments fine. The release PR
branch is owned by the `release-bot` app installation
(`langchain-oss-automated-triage[bot]`), and the token posting here is
`github-actions[bot]`; every other workflow in this repo that comments
on PRs grants `pull-requests: write`, while this one granted only
`issues: write` + `pull-requests: read`.

Two changes:

- Grant `pull-requests: write` in `release_notes_check.yml`, matching
the other PR-commenting workflows.
- Retry the courtesy comment up to 2 times (2s apart) before degrading
to a log warning. The comment is intentionally non-blocking — the gate
still fails closed with its specific, actionable reason — but a
transient 403 shouldn't surface as a confusing warning, and in
`issue_comment`/`workflow_dispatch` refresh runs it would otherwise
leave the refreshed required check red over a courtesy comment.

The gate failure on that run was itself real
(changelog/body/applied-commit drift) and is unaffected by this change;
this only fixes the courtesy-comment path.
2026-08-19 16:08:03 -04:00
Mason Daugherty 0399dea7d9 ci(infra): warn instead of fail on unreleased-Python caps in third-party deps (#5653)
The release dependency gate (`check_dep_freshness.yml` →
`check_wheel_dep_freshness.py`) currently hard-fails a release PR when a
third-party dependency's newest published release declares a
`requires-python` upper bound that excludes a CPython minor the wheel
claims but that has not shipped yet. This is blocking the
`deepagents-code` 0.1.58 release PR (#5591): the wheel claims
`>=3.12,<4.0`, and the optional `ibm` extra's
`langchain-ibm>=1.1.0,<2.0.0` only publishes `<3.15,>=3.10`. pip already
refuses that extra on the unreleased interpreter and installs it fine on
3.12–3.14, so the release is blocked purely on another project widening
its bound.

This change downgrades that specific case from a hard `::error::` to a
`::warning::` plus a step-summary note. The warning applies only when
all of these hold:

- the dependency is **third-party** — repo-managed siblings keep the
strict freshest-metadata assertion that caught the coordinated
Code/Talon Python-floor bump;
- a published release **satisfies the version constraint** (a missing
release still fails);
- the Python coverage gap is **exactly the next unreleased CPython minor
through the wheel's ceiling** (e.g. `<3.15` while 3.14 is current).

Gaps at real installation targets still fail: the wheel's minimum
(dependency rejects the install floor, e.g. dep requires `>=3.13`
against the wheel's `>=3.12`), any middle minor (e.g. dep ships only
`>=3.13,<3.14`), and patch-level exclusions on released lines.

The gate's "latest existing minor" anchor is the interpreter running the
check; `check_dep_freshness.yml` pins that job to Python 3.14, so the
verdict is deterministic per workflow, and the new unit tests pin
`sys.version_info` to stay runner-independent.
2026-08-19 15:53:21 -04:00
Mason Daugherty 240fdbf364 ci(ci): bound ripgrep install on ordinary PRs (#5642)
Slow apt metadata refreshes should not block ordinary pull-request
validation, so non-release PR jobs now continue after a two-minute
ripgrep install timeout and publish a sticky warning after CI. Release
PRs, merge-queue/main runs, and release artifact tests keep the strict
install path.

<details>
<summary>Test plan</summary>

- 90 workflow contract tests pass; changed workflows also parse as valid
YAML.

</details>

Made by [Open
SWE](https://openswe.vercel.app/agents/1a61b60c-cf77-5b18-83d9-3a2028f55fb6)

---------

Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
2026-08-19 15:31:54 -04:00
Mason Daugherty 20a5b5c1be ci(infra): validate release dependencies against PyPI (#5612)
Release PRs now fail before merge when their built wheel needs
dependency metadata that has not reached PyPI, or when the wheel cannot
be installed from the public index the way `release.yml` installs it.

---

Source CI installs sibling packages from local editable paths, which can
hide a required release order: a sibling package's source may declare a
new Python floor or dependency floor that PyPI's latest published wheel
does not yet have. Merging in that state would break the post-merge
release install.

The `📦 Check Dependency Freshness` workflow now, for release PRs only:

1. Builds the release wheel and checks every `Requires-Dist` constraint
against PyPI. For repo-managed dependencies it also verifies the newest
eligible PyPI files satisfy the sibling's current Python metadata.
Direct URL dependencies fail, since this path verifies PyPI.
2. Installs the wheel into a clean environment on the package's release
Python with `--index-url https://pypi.org/simple`, with Talon getting
the same `--prerelease allow` exception as the release workflow.

Non-release PRs get a passing no-op, so the job is safe to require
globally.

## Demonstration

The check passes on this normal (non-release) PR: [workflow run
32184070069](https://github.com/langchain-ai/deepagents/actions/runs/32184070069).

A live GitHub-hosted failure run is not feasible before merge:
pull-request workflows execute from the head branch, so the failing
Talon 0.0.4 release PR (#5605) does not contain this code, and a new
`workflow_dispatch` trigger is not recognized until it exists on the
default branch. Locally, the same check against the built Talon wheel
and live PyPI JSON fails with:

> `deepagents-code 0.1.57 (latest on PyPI) declares
requires-python<4.0,>=3.11, but the current deepagents-code metadata
requires >=3.12,<4.0. Constraint: <1.0.0,>=0.1.30. Release
deepagents-code first, then re-run this check.`

The clean Python 3.12 install step was exercised separately and resolved
138 packages from PyPI, including `deepagents-code==0.1.57` — expected,
since the install step proves the public graph resolves while the
metadata check catches the unpublished sibling change.

## Required status

After this lands, a maintainer must add the literal `validate release
dependencies against PyPI` job name to the `main` ruleset's required
status checks (currently only ` CI Success` is required). The job
passes on non-release PRs, so requiring it will not block unrelated PRs.
2026-08-19 11:13:15 -04:00
Mason Daugherty 04de43e05a feat(code,talon): require Python 3.12 or greater (#5603)
`dcode` now requires Python 3.12 or newer (previously 3.11). Users on
3.11 will need to upgrade their interpreter to install or update to this
release.

---

The floor bump lets the codebase use 3.12-only syntax and drop 3.11
compatibility shims:

- PEP 695 `type` statements replace `TypeAlias` annotations, and generic
functions/classes use type-parameter syntax instead of
`TypeVar`/`Generic[...]`
- `Protocol` / `override` import from `typing` instead of
`typing_extensions`
- The pre-PEP-706 `tarfile` extraction fallback (needed only for
3.11.0–3.11.3) is gone; extraction always uses `filter="data"`
- `uv.lock` re-locks without 3.11-only backports (`tomli`,
`backports-tarfile`)
- CI matrices, `ty`, and ruff `target-version` all move to 3.12;
integration test defaults and release builds are updated to match

`deepagents-talon` depends on `deepagents-code`, so its floor moves in
the same commit — it cannot bump independently without a broken window
where talon allows an interpreter its dependency rejects. This is the
deliberate coordinated multi-package bump called out in RELEASING.md, so
the `allow-lockfile-release` label is applied to acknowledge the talon
release-please fan-out.

<details>
<summary>Test plan</summary>

- `ruff check` / `ruff format --check` clean
- `ty check` clean
- Full `libs/code` unit suite passes (two pre-existing failures in
`test_app.py` / `test_server_graph.py` reproduce on the base commit and
are unrelated)
- COMMANDS.md catalog check and lockfile freshness hooks pass

</details>
2026-08-18 14:53:06 -04:00
Mason Daugherty 3dd4fe5cdc feat(code): show conversation turns alongside message counts (#5571)
`/offload` feedback and the Debug Console now show conversation turns
together with raw message counts.

---

Message counts are hard to interpret. One user prompt can become several
provider messages (a tool call, a tool result, and a follow-up model
turn), and the count changes from provider to provider. Conversation
turns — the user prompts the model actually saw — do not change between
providers. This makes them a better unit for comparing offloads.

Changes:

- The offload report now shows both units, for example `Offloaded 4
older messages (2 conversation turns)` and `6 messages (3 conversation
turns) kept`. The turn count excludes internal human messages
(goal-state notices and `[SYSTEM]` interruptions) that collect below the
cutoff.
- The Debug Console `Messages` field shows `N messages (M rendered), K
turns`, with correct singular and plural labels.
- The new `MessageStore.turn_count` property counts `USER` and `SKILL`
rows. A `/skill` invocation creates one `SKILL` row instead of a `USER`
row, so each skill use counts as one turn. It counts all rendered rows,
including local-only flows such as `!shell` and most slash commands.
That is a broader set than the offload report's turn count, which comes
from graph state; the property's docstring describes the difference.

---------

Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
2026-08-17 23:17:41 -04:00
Mason Daugherty 064a75b311 feat(infra): credit issue reporters in release notes (#5565)
GitHub release notes now include a **Special thanks** section. It
credits the users who reported the issues that this release's pull
requests closed, with links to those issues. The existing
community-contributor shoutouts credit pull request authors; this
section credits issue reporters. Users who are internal maintainers on
this release are not included.

```md
**Special thanks** to everyone who reported the issues addressed in this release: @carol ([#5310](https://github.com/langchain-ai/deepagents/issues/5310)), @dave ([#5201](https://github.com/langchain-ai/deepagents/issues/5201), [#5202](https://github.com/langchain-ai/deepagents/issues/5202))
```

---

The existing `collect_contributors` walk does the lookup. Its per-PR `gh
pr view` call now also requests `closingIssuesReferences`, so both
attribution lists use the same range, retry, and failure handling.

For each merged PR, the issue author is credited — not the PR author.
Entries are grouped by login with deduplicated, ascending issue numbers,
and rendered as a comma-separated list like the other attribution lines.
Bot accounts (`[bot]` suffix), empty logins, and unusable issue numbers
are skipped with a warning. A missing `closingIssuesReferences` field
also warns instead of silently producing an empty section. If no closed
issues are found, the section is omitted. Lookup failures never block a
release.
2026-08-17 21:58:57 -04:00
Mason Daugherty 4544fac203 feat(infra): warn and close draft PRs on the same age-based schedule (#5550)
Draft PRs now get the same treatment as all other PRs: a warning at 14
days, closure at 30 days. Only `do-not-close` (or a `!keep-open`
comment) and release-please PRs are exempt.
2026-08-17 14:42:40 -04:00
Mason Daugherty a84c557e2b ci(infra): render !keep-open as a fenced block in the stale-PR warning (#5538)
The stale-PR warning comment now renders the keep-open command as its
own fenced code block, so GitHub shows a copy button next to it — a
maintainer can copy the exact phrase instead of selecting it out of a
sentence.

---

The warning posted by `close-old-prs.js` previously ended with "...or
comments `!keep-open`." inline. It now ends with "...or comments:"
followed by:

```
!keep-open
```

Only the warning body changes; the post-close message keeps its inline
mention (no copy affordance needed once the PR is already closed), and
the `keep_open_on_comment.yml` trigger still matches the phrase anywhere
in a comment body, so comments posted via the copy button work the same
as before.
2026-08-17 01:48:03 -04:00
Mason Daugherty 0f75e2ec40 ci(infra): exempt PRs from auto-close via !keep-open comment (#5517)
Commenting `!keep-open` on a PR now exempts it from the auto-close
sweep, same as adding the `do-not-close` label — easier to fire off a
quick comment than to open the label picker.

---

The new `keep_open_on_comment.yml` workflow listens for `issue_comment:
created` on PRs. When a MEMBER/OWNER/COLLABORATOR comments anything
containing `!keep-open`, it reacts 👀, applies `do-not-close` via a
shared `applyBypassLabel` helper, then reacts 🚀. Applying the label
fires the `labeled` event, so the existing `clear_pending_deletion.yml`
workflow removes `pending-deletion` and minimizes the stale warning
comment — the end state is identical to adding the label by hand.

- The actor gate is `author_association` rather than a user list, so
anyone with triage access can use it and no one gains a power they
didn't already have. Bots and CONTRIBUTORs are excluded.
- The match is a substring, not exact equality, so `still relevant —
!keep-open` works. The `!` prefix keeps ordinary prose from accidentally
exempting a PR.
- Only `created` is handled: editing or deleting the comment later does
not remove the label, matching the label's own manual semantics.
- The warning and close notices posted by the sweep now mention
`!keep-open` alongside the label.
2026-08-14 13:38:20 -07:00
Mason Daugherty 30f4845af1 chore(ci): collapse draft instructions into a details toggle (#5490)
The maintainer-instructions echo that `@release-bot draft` appends to
its draft comment now renders inside a collapsed `<details>` block, with
a 📝 emoji and bold summary line. This keeps the draft comment cleaner
while still making the steering text auditable on the PR.

---

The draft comment previously ended with a plain `Drafted with maintainer
instructions: ...` line, which could be long and visually noisy.
Wrapping it in a toggle preserves the audit trail without cluttering the
comment body.
2026-08-13 12:30:46 -07:00
Mason Daugherty f7cd46c92c ci(infra): backtick package name and tabulate minimums PR body (#5485)
PRs opened by `raise_langchain_minimums.yml` get titles like
``chore(deps): raise dependency minimums for `deepagents-code` ``
instead of the package name in plain text, and the list of raised bounds
in the body becomes a Manifest / Dependency / Change table instead of
bullets.

---

Two readability fixes to the automated dependency-minimums PRs, e.g.
#5481:

- The package name in the title (and the workflow's commit message) was
the only identifier not wrapped in backticks — the PR body, workflow
comments, and our title conventions all put named entities in backticks.
The title now reads ``... for `deepagents-code` ``.
- With several raised bounds in one run, the bullet format buried the
actual old → new specifier change at the end of a long line led by the
manifest path. Rendered as a table:

  | Manifest | Dependency | Change |
  |---|---|---|
| `libs/code/pyproject.toml` | `langchain` | `langchain>=1.3.14,<2.0.0`
→ `langchain>=1.3.15,<2.0.0` |
| `libs/code/pyproject.toml` | `langsmith` |
`langsmith[sandbox]>=0.10.10` → `langsmith[sandbox]>=0.10.18` |

The title change is cosmetic only — `chore(deps):` parsing by
release-please and the PR title lint are unaffected. #5481's title was
already edited in place; the table format applies to the next scheduled
run.
2026-08-13 11:39:35 -07:00
Nick Hollon 8d4cabc235 chore(evals): track the unversioned harbor-index dataset (#5467)
The autonomous category and the `harbor.yml` dataset dropdown pin
`harbor-index/harbor-index-1.0`, a frozen one-revision package, so runs
never pick up index updates.
2026-08-13 09:11:54 -04:00
Mason Daugherty b630fb32e0 feat(infra): post a timeline comment when @release-bot draft regenerates notes (#5456)
Re-running `@release-bot draft` on a release PR now posts a short
timeline comment linking to the regenerated notes, so the refresh is
visible without re-opening the original comment.

---

Re-drafting edits the bot's curated-notes comment in place, and GitHub
surfaces comment edits quietly: no notification, no timeline entry. From
the timeline the PR looked unchanged, so a maintainer who re-ran `draft`
with steering instructions had no signal that the notes actually
regenerated.

The upsert helper now reports whether it created or edited the draft
comment, and the draft job posts the pointer only after an in-place
edit. First-time drafts (including the automatic `ready_for_review`
draft) post nothing extra — a brand-new comment already shows up in the
timeline. The notice is best-effort: if posting it fails, the draft
still succeeds and the failure lands as a workflow warning. The notice
body never mentions `@release-bot`, so it cannot re-trigger the command
flow.

Example — maintainer re-runs `draft` with steering instructions:

1. Bot edits its existing curated-notes comment in place (no timeline
entry).
2. Bot posts a new timeline comment:

> The curated release-notes comment on this PR was regenerated in place;
review the latest draft in [the original comment](#issuecomment-…).
2026-08-12 14:12:17 -07:00
Mason Daugherty 524af48ca2 ci(infra): clear pending-deletion when a PR gains do-not-close (#5411)
Adding `do-not-close` to a PR with `pending-deletion` now removes the
stale auto-close warning at once.

---

The daily cleanup job already removes `pending-deletion` when it finds
`do-not-close`. This new workflow does the same work when a maintainer
adds `do-not-close`. It also minimizes the warning comment.

`close-old-prs.js` checks for `do-not-close` before it adds
`pending-deletion` and before it closes a PR. If the label appears
during a run, the script removes `pending-deletion`, minimizes the
warning comment, and skips the PR.

The workflow uses `pull_request_target` so it can update labels on PRs
from forks. It does not check out or run PR code.
2026-08-11 09:50:45 -07:00
Shrikar Seshadri d60560d695 fix(evals): use gpt-5.6-luna for research grading (#5337)
A single `judge_models` setting now controls both tau3 and DRBench
grading, with `gpt-5.6-luna` as the default for both when the input is
empty.

---

Pinned upstream DRBench hardcodes three model-routing allowlists and
therefore rejected the suite default before reaching OpenAI. The
verifier now registers exactly `gpt-5.6-luna` in those allowlists at
runtime, allowing it to use the existing `OPENAI_API_KEY` path without
broadening support to arbitrary model identifiers.

The separate `research_judge_models` input is removed. Native DRBench
judges and suite-wide `openrouter/<vendor>/<model>` judges remain
supported; an unsupported bare research judge falls back to
`gpt-5.6-luna` with a visible workflow warning while other categories
retain the requested suite-wide judge.

<details>
<summary>Test plan</summary>

- 75 focused DRBench verifier tests passed, with 1 skipped
- 43 focused unified-workflow tests passed
- 471 eval unit tests passed, with 1 skipped
- 980 GitHub-script tests passed
- Canonical eval lint, type checking, and catalog validation passed
- The 226-package eval lock resolved successfully offline
- `git diff --check` passed
- [Combined smoke
dispatch](https://github.com/langchain-ai/deepagents/actions/runs/31132860866)
passed: tau3 earned `1.0` with `JUDGE_MODELS=gpt-5.6-luna`; DR0001
completed with no verifier error and recorded both its requested and
actual judge as `gpt-5.6-luna`

</details>
2026-08-07 12:33:39 -07:00
Mason Daugherty a97bdb6577 ci(infra): fail on unexpected test warnings (#5346)
Every package under `libs/` now fails its test run on any warning the
repo has not explicitly accepted. Each package's pytest `filterwarnings`
opens with `"error"`, followed by a short reviewed allowlist of
categorical third-party warnings (each with a justification comment).
One test that deliberately emits a warning
(`test_metadata_less_dist_info_does_not_raise`) scopes its exception
with `pytest.mark.filterwarnings` instead of a package-level entry. A
static contract test under `.github/scripts/tests/workflows/` asserts
every package opts in and the workflow bypass wiring stays intact.

Applying the policy surfaced two real bugs, fixed at the source rather
than allowlisted: a `SyntaxWarning` from a non-raw docstring in the ACP
`GenericFakeChatModel`, and a `ResourceWarning` from an unclosed
`task.toml` handle in the DRBench adapter test.

Maintainers can bypass the policy on a PR by applying the
`bypass-warnings-check` label and re-running failed jobs: `_test.yml`
reads live PR labels from the GitHub API (re-runs replay the original
event payload, so `github.event.pull_request.labels` can be stale) and
passes `-W default` on the pytest command line, which outranks every ini
filter for that run. The step is fail-closed — an API error enforces
rather than bypasses — and `push`/`merge_group` runs always enforce, so
the label cannot smuggle warnings into the merge queue.

---------

Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
2026-08-06 22:16:52 -04:00
Mason Daugherty be91094199 feat(infra): acknowledge accepted @release-bot commands with a reply comment (#5345)
Running `@release-bot draft` or `@release-bot apply` on a release-please
PR now gets an immediate reply confirming the command was accepted,
instead of silence until the drafted or applied comment appears minutes
later.

---

Manual release-bot commands go through several validation gates in
`validateTrigger` — the PR must be a release-please branch, it must be
ready for review, and the commenter must have write access. Every
rejection path already posted an explanatory comment, but the success
path returned `shouldRun: true` without saying anything, leaving the
maintainer staring at an unchanged PR wondering whether the command
registered, whether the workflow is queued, or whether they mistyped the
mention.

The acknowledgment comment is posted only after all gates pass and only
for manual `issue_comment` commands; the automatic `ready_for_review`
draft trigger stays silent to avoid adding noise to every release PR.
The reply names the command and component (e.g. "Running `draft` for the
`deepagents-code` release PR...") so it's clear what was accepted, and
notes that the result arrives as a follow-up comment.
2026-08-05 23:50:36 -04:00
Mason Daugherty ed0b6785f8 feat(ci): raise langchain minimums workflow (#5330)
Adds a workflow that raises LangChain-ecosystem dependency lower bounds
to the latest stable PyPI release and opens a `chore(deps):` PR with the
result. Runs weekly (Monday 09:00 UTC) across every release package, and
can be dispatched manually against a single package or `all`.

Keeping integration minimums current is a recurring manual chore. This
automates the "bump the floor" half: for each in-scope requirement
(`langchain*`, `langgraph*`, `langsmith*`, `deepagents*`) declaring a
`>=`/`~=` floor in `[project.dependencies]`,
`[project.optional-dependencies]`, or `[dependency-groups]`, the floor
is rewritten in place and one PR is opened.

- **Raises within the existing range, never past an upper bound.**
`deepagents>=0.7.0,<0.8.0` becomes `>=0.7.9,<0.8.0`, never the
unsatisfiable `>=0.8.0,<0.8.0`. Pre-releases are never selected, and a
floor already ahead of the latest stable release (intentional prerelease
coordination) is left alone. Each rewrite is checked against the
versions PyPI actually offers, so a bump can't quietly exclude a version
the old range allowed.
- **Preserves everything else in the specifier.** Upper bounds, extras,
and markers survive verbatim, and `~=` ceilings are held by keeping the
original component count — `~=1.2` becomes `~=1.9`, not `~=1.9.4`.
- **Skips what it shouldn't touch.** Exact `==` pins, specs with no
floor, URL requirements, workspace-local `[tool.uv.sources]` deps, and
the manifest's own package name.
- **Regenerates every `uv.lock` the edits invalidate** — not only the
edited packages: a lockfile embeds the specifiers of anything it
resolves from a local path source, so raising a floor in
`libs/deepagents` stales `libs/evals` and friends. Interpreter versions
come from `check_lockfiles_pre_commit` so they match what
`check_lockfiles.yml` verifies.
- **Reports failure instead of silence.** An unresolvable PyPI lookup or
an unrewritable manifest is listed in the PR body under "Not raised" and
fails the run, so an unattended cron never renders an outage as
"everything is already up to date".
- **Idempotent per package.** If a PR is already open for the selected
package, or nothing needs raising, the run exits without creating a
duplicate. PRs are created with the Org Membership App token so required
`pull_request` checks trigger, and titled `chore(deps):` so
release-please doesn't fan out a separate release PR.
2026-08-05 19:06:32 -04:00
Shrikar Seshadri af58af2cc4 feat(evals): add DRBench as a research eval category (#5320)
Adds ServiceNow's [DRBench](https://github.com/ServiceNow/drbench) as a
`research` category in `unified_evals.yml`: enterprise deep-research
tasks whose documents are served by a containerized app stack
(Nextcloud, Mattermost, Roundcube IMAP, filebrowser), with the agent's
report graded by upstream's own metric suite.

**Full walkthrough of how the benchmark and its grading work:**
[DRBench: Architecture, Grading and
Results](https://app.notion.com/p/DRBench-Architecture-Grading-and-Results-3b3808527b178101a3a6c43c5eb28872?source=copy_link)

## What's here:
- **100 Harbor task directories**: **not committed** — generated by
`make dataset` from upstream's configs at the pinned commit
`UPSTREAM_SHA`, with a blobless sparse checkout that takes ~2.4 MiB of
config JSON and skips the ~69 MiB document corpus (app mode's per-task
image serves the documents). A git commit hash is a hash of its content,
so the fetch is pinned as firmly as a copy would be, without carrying
1,105 generated and vendored files — including 100 gold
`solution/solve.sh` answer keys — in a public repo. A `drbench-dataset`
CI job builds the dataset twice and diffs, since the generated tree is
no longer reviewable in a diff.
- **App mode, arm64 only** — upstream publishes the task images for
arm64, so `research` needs `sandbox_env: docker` with `runner_label:
ubuntu-24.04-arm`. It pins these itself, so no other input needs
changing to include it.
- **Task profiles**: `lite` is upstream's own 15-task MinEval subset;
`full` is a 30-task set matching the benchmark's easy/medium/hard
proportions (representative, *not* comparable to the paper's
FullBenchmark number).
- **Per-suite configurability in `unified_evals.yml`**: sandbox
environment (local Docker vs. LangSmith Sandboxes), runner CPU
architecture, and concurrency, each overridable per category (`research`
defaults to concurrency=1).

## ⚠️ The default category set changes
`categories` defaults from `autonomous,conversation,context` to
**`autonomous,conversation,research`**. This is deliberate — `research`
replaces `context` rather than being added, to keep the default
dispatch's cost and wall-clock roughly flat — but it means **the default
scorecard no longer measures Context-Bench, and its trend line breaks at
this commit**. `context` still runs when named explicitly (`categories:
autonomous,conversation,context,research` runs all four). Whoever owns
the nightly baseline should confirm this before merge.

## Grading
The DRBench judge is a separate input. Upstream routes on the model
name, so its verifier drives only `gpt-4o` / `gpt-4o-mini` directly;
anything else must be a single `openrouter/<vendor>/<model>` slug. The
default is `gpt-4o`. A suite-wide `judge_models` that DRBench cannot
drive does **not** fail the research leaf — it falls back to `gpt-4o`
and says so in the step summary — so the ordinary independent-grader
flow still works on a mixed dispatch. Use `research_judge_models` to pin
DRBench's grader deliberately. Any grader other than `gpt-4o` makes the
numbers incomparable to earlier runs and to the paper, so re-baseline
rather than reading a delta.

Because `research` is scored on a graded reward, its `pass@K` is 0.000
by construction. The radar chart plots `avg@K` for such categories, and
the cross-model ranking is on macro `avg@K` — on a `pass@K` ranking, a
row whose research leaf succeeded was averaged against that structural 0
while a row whose leaf failed was averaged without it, so failing
research paid better than completing it.

## Example runs
-
[30960340040](https://github.com/langchain-ai/deepagents/actions/runs/30960340040)
— full `research` suite
-
[30985445289](https://github.com/langchain-ai/deepagents/actions/runs/30985445289)
— `lite` suite (autonomous + conversation + research)
-
[31033935295](https://github.com/langchain-ai/deepagents/actions/runs/31033935295)
— DR0001 + DR0002 with no task files in the repo: avg@1 0.520, 0 errored
2026-08-05 14:09:04 -07:00
Mason Daugherty 124704ac9b feat(infra): branch-name pre-push hook and advisory CI check (#5297)
Internal branches are expected to follow
`<github-username>/<scope>/<short-description>` per AGENTS.md, but
nothing enforces it today. This adds two layers of enforcement: a
pre-commit-managed pre-push hook that blocks pushing non-conformant
branch names locally, and an advisory CI check that posts a sticky PR
comment when a PR head branch does not follow the convention.

---

## Why

Branch names in this repo are a mix of conformant (`mdrxy/code/...`) and
ad-hoc (`abalone-ricotta`, `fix/vscode-root-interpreter-path`) names.
The convention is documented but unenforced, so drift is the default.

## What changed

- **`.githooks/pre-push`** (new, committed) — the branch-name check
invoked by pre-commit at the `pre-push` stage. It blocks branches that
do not match `<github-username>/<scope>/<short-description>`. The
username resolves from `git config github.user`, falling back to `gh api
user` and then the local part of `user.email`. The scope segment is
validated against the `pr_lint.yml` scope list (plus `docs`).
`pre-commit install --install-hooks` installs it alongside the existing
pre-commit and commit-message hooks; bypassable with `--no-verify` or
`SKIP=branch-name`.
- **`.github/workflows/branch_name_check.yml`** (new) — runs on
same-repo PRs (forks are exempt). Posts or updates a sticky warning
comment on non-conformant head branches and deletes it once renamed.
Always exits green — it is a reminder, not a gate.
- **`libs/DEVELOPMENT.md`** — setup docs for the hook.

Protected branches (`main`, `vX.Y`) and automation branches
(`release-please--*`, `dependabot/*`, `copilot/*`) are exempt in both
layers.

## Points for review

- The scope list is duplicated across `pr_lint.yml`, the hook, and the
workflow (each notes it must stay in sync). A shared source would be
nicer but adds indirection for three small consumers.
- The hook enforces the *resolved local user* login as the prefix, so
pushing a branch prefixed with someone else's username from your machine
is blocked locally. CI does not compare the username segment to the PR
author, since automation and maintainers legitimately push branches on
others' behalf.
2026-08-04 11:57:55 -04:00
Mason Daugherty 46e006757b feat(infra): accept maintainer instructions on @release-bot draft (#5299)
Maintainers can now steer a release-notes draft in one shot:
`@release-bot draft <instructions>` feeds the text after the command
into the drafting model as guidance, so a draft can be re-generated with
direction (e.g. `@release-bot draft emphasize the breaking SDK change
and lead with it`) instead of drafting blind and hand-editing after.

---

The release-notes bot's drafting step previously took only the generated
changelog — the only way to shape the result was to edit the posted
draft by hand. This adds a lightweight way to steer the model at draft
time.

- `draft` accepts optional instructions on the same line; `apply` is
unchanged and ignores trailing text, since it republishes the stored
draft verbatim.
- Instructions are capped at 500 characters and truncated at any `@`, so
trailing text can't smuggle a second `@release-bot` mention into the
prompt.
- The instruction is passed to the model in the user message as guidance
explicitly subordinate to the fixed editing rules, never into the system
prompt — the "do not invent behavior / no version heading" constraints
stay authoritative.
- The posted draft comment echoes `Drafted with maintainer instructions:
…` (outside the metadata block and the editable content markers, so both
parsers are unaffected), keeping the prompt that produced a draft
auditable on the PR.
- `prepareDraft` re-sanitizes instructions independently of the comment
parser, so the guarantee holds at the process boundary into the model
request regardless of caller.

The drafting model still runs with no filesystem, shell, or network
tools, and its output is re-validated by `validateDraftOutput` before
publishing — this change only widens what a maintainer can ask for, not
what the model can do.
2026-08-04 10:30:34 -04:00
Mason Daugherty bebafaa0c8 fix(infra): tolerate unpublished tags in fan-out watch (#5216)
The release-please fan-out watch no longer crashes while a merged
release is waiting to publish its tag. It skips that component with a
visible warning and continues checking the remaining packages.

---

Release-please updates the manifest when its release PR merges, but the
package tag is created later, after pre-release checks pass. If those
checks fail, as they did for `deepagents-code` 0.1.51, the manifest
points to a tag that does not yet exist. The fan-out detector currently
passes that tag directly to `git diff`, which exits with `fatal: bad
revision` and prevents the watch from examining any other package.

The detector now verifies each baseline ref before diffing. `git
rev-parse` return code 1 means the release tag is absent, so the
component is skipped with a warning and checked again on the next
release-please run or hourly backstop. Any other git failure raises and
makes `main()` exit 2, preserving the advisory's fail-closed behavior.
Diff failures for an existing tag also remain fatal.

<details>
<summary>Test plan</summary>

- `.github/scripts/tests/release/test_check_open_release_fanout.py` — 9
passed, covering ref found, ref absent, operational lookup failure, and
existing-ref diff failure.
- Full workflow-helper suite — 891 passed.
- Ruff checks pass for the detector and its tests.

</details>
2026-07-31 02:01:01 -04:00
Mason Daugherty 9af437b2f6 test(infra): de-flake release-notes git history helpers (#5212)
CI hit `subprocess.CalledProcessError ... 'commit', '-m', 'fix(example):
generated 77'] returned non-zero exit status 128` in
`test_git_log_limits_large_history`, with no way to tell *why* git
failed.

---

The release-notes helper tests build synthetic histories by shelling out
to git three times per commit. The truncation test created 101 commits
that way, so one transient git failure anywhere in the loop fails the
whole job. `_commit_many` now writes a batch of commits with a single
`git fast-import` process, removing ~300 subprocess spawns from that
test alone (plus ~150 across the byte-budget and contributor-cap tests)
and cutting its runtime from ~2s to ~0.6s.

Two supporting changes make the remaining git usage safer to debug and
less racy:

- `_git` no longer relies on `check=True`. `str(CalledProcessError)`
reports only argv and the exit status, which is precisely why the
original failure was undiagnosable; the helper now raises with git's own
`fatal:` line and both output streams, matching the pattern already used
for the Node helper tests.
- Temporary repositories set `gc.auto=0` and `maintenance.auto=false`,
so `git commit` never forks background maintenance that can hold the
repository locks while the next command runs.

The underlying `fatal:` line was never captured, so this reduces the
exposed surface by two orders of magnitude *and* guarantees the next
occurrence names its own cause instead of a bare exit code.

Made by [Open
SWE](https://openswe.vercel.app/agents/598a0622-925d-f68a-006d-66cf035c9cdb)

Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com>
2026-07-30 19:52:54 -04:00
Mason Daugherty 26557c381b feat(infra): local rebuild path for GitHub release notes (#5187)
Extract the release-notes job's inline bash body-construction steps into
a shared Python script (.github/scripts/release/build_release_notes.py)
so maintainers can rebuild the production-shaped GitHub release body
locally when the CI notes job fails or produces an empty body after
publish.

The script mirrors the CI behavior exactly: changelog extraction,
contributor shoutouts, collapsible package-scoped git log, size budget
and truncation, pre-release banner, and branch provenance annotation. It
supports --offline mode for degraded environments and --github-output
for CI use.

---

Wire release.yml to call the shared script from the release-notes job,
replacing the five inline bash steps (resolve-refs, generate-git-log,
resolve-releaser, generate-release-body, finalize-release-body) with a
single step. The release-body output still flows to mark-release, and
publish remains independent of notes success.

Update tests: add comprehensive unit tests for the shared script in
test_build_release_notes.py (46 tests covering version detection,
changelog extraction, git log generation, tag resolution, body assembly,
and size budgets), and slim test_release_git_log.py to cover only the
steps that remain inline (setup job's resolve-sha).

Document the recovery path in RELEASING.md: rebuild body locally with
the script, apply with gh release edit --notes-file, verify with gh
release view. Cross-link the existing pre-release enrichment guidance.
2026-07-30 17:04:46 -04:00
Mason Daugherty 875836547f fix(infra): run update-lockfiles after skipped trigger-releases (#5206)
Release PRs opened or updated by release-please get their `uv.lock`
files regenerated again. The `update-lockfiles` job was silently
skipping on every push to `main` — the workflow showed green while
release PR lockfiles went stale and the "🔒 Check Lockfiles" workflow
failed on them — because #5161 put the legitimately-skipped
`trigger-releases` job in its upstream `needs` chain and the job's bare
`if:` kept GitHub Actions' implicit `success()` gate. #5169 fixed this
for the `release-please` job; this applies the same gating to
`update-lockfiles` (and audits the remaining downstream jobs).

---

## Why

`update-lockfiles` needs `release-please`, which (after #5161)
transitively needs `trigger-releases` via `guard-pending-release`. On
ordinary pushes to `main`, `trigger-releases` is skipped — and without a
status-check function in the job's `if:`, Actions' implicit `success()`
gate on the whole `needs` chain fails closed. The job was skipped with
an all-green workflow; observed on release PR #5170, where lockfiles had
to be regenerated by hand.

## Fix

- `update-lockfiles`: `!cancelled()` plus explicit `result == 'success'`
checks on every dependency in the chain (`guard-empty-commit`,
`detect-release-commit`, `guard-pending-release`, `release-please`),
keeping the existing `prs != '[]' && prs != ''` requirement. Fail-closed
semantics preserved: a crashed guard or unset `prs` still blocks the
job. `trigger-releases` is deliberately not re-checked — on release
commits, `release-please` succeeding already implies the guard saw the
dispatch succeed.
- `dispatch-release-notes-check`: audit found its `always()` gate
already survived the skipped-`trigger-releases` case, but would also run
after a cancelled workflow or red guardians. Added `!cancelled()` and
the same hand-checked ancestor results; `update-lockfiles` stays a
sequencing-only need whose own failure does not block the dispatch.
- `guard-empty-commit`, `detect-release-commit`, and `trigger-releases`
need no changes — their `needs` chains never contain a
legitimately-skipped job.

Tests pin the new truth tables for both jobs, mirroring the style #5169
used for `release-please`, and `_evaluate` now handles `always()`.
2026-07-30 16:52:49 -04:00
Mason Daugherty bfd9f1fe6d chore(infra): clarify that release-notes edits are optional (#5193)
Clarify in the curated release-notes draft comment that editing the
notes is optional.

---

The draft comment posted on release PRs previously read "Review and edit
the release notes between the content markers below." Adding "as needed"
makes it clear maintainers don't have to change anything when the
generated notes are already fine, rather than implying an edit is always
expected.
2026-07-30 13:15:29 -04:00
Mason Daugherty 3ba731fb1c ci(infra): soft-run release-deps check under acknowledged bypass (#5168)
`release-deps: acknowledged` no longer skips any job. Every check runs
and reports; the label only stops one of them from blocking.

| Check | Effect of the label |
| --- | --- |
| `📦 Check Release Dependencies` | Resolves in report-only mode; check
goes green, sticky comment still lists the packages that owe releases. |
| `📦 Check Dependency Freshness` | None. Advisory in all cases; its
comment stays on the PR. |
| `🔗 Check SDK Pin` | Clears the hard failure on a **prerelease** pin.
Stale-pin behaviour unchanged. |

A green check under the label now means "the listed packages still owe
releases," never "deps resolved." Skipping the resolve job instead
produced a green check that read as an all-clear while PyPI installs
were still broken — and deleted the record of the remaining release
debt.

## Follow-up releases are reported on every path

The check lists sibling release-please packages whose **published** PyPI
`requires_dist` cannot satisfy a head requirement on the branch — e.g.
published `0.1.49` constrains `deepagents` to `==0.7.0b2`; head requires
`==0.7.0`.

This is gated on neither the label nor the resolver verdict. Resolution
passing only proves the *changed* package installs today; a
reverse-dependent whose published metadata caps the new line still
breaks `pip install` for its own users. So a clean resolve with
outstanding follow-ups posts the sticky and stays green, and the sticky
clears itself only once nothing is left to report.

The analysis is advisory and can never fail the gate — a crash in it
degrades to a warning on the report. Packages whose PyPI metadata could
not be determined are reported as indeterminate and never produce a
"needs release" claim.

Overlap between a published constraint and a head requirement is decided
by a finite probe sample, which can prove overlap but never
disjointness. The error directions are asymmetric — falsely reporting
overlap hides a real conflict, while falsely reporting disjoint only
adds a spurious row — so every inconclusive case reports
overlap-unknown.

`RELEASING.md` gains a "Releasing a new line ahead of its dependents"
section covering when the label is appropriate (coordinated release
order) versus when the metadata is simply wrong, plus the table above.

<details>
<summary>Test plan</summary>

- 717 tests pass across the helper suites (`pytest
.github/scripts/tests`, which is what `ci.yml` runs); `ty check
.github/scripts/release/` is clean.
- New unit tests cover: follow-ups rendered on failure and on clean
resolves; every conflicting package listed even where the resolver would
prove only one; acknowledged soft-run exits 0 and writes the sticky; an
acknowledged clean run clears it; indeterminate metadata never claims
"needs release"; an unpublished package is not reported as
indeterminate; a crash in the advisory analysis does not fail the gate.
- The overlap heuristic has a boundary matrix (`==` vs `>=,<`, open vs
closed edges, `~=`, `!=`, wildcard pins, single-component versions)
asserting disjointness is only reported when provable.
- Verified end to end against live PyPI: a simulated
`release(deepagents)` flags `deepagents-code` (published `0.1.49` pins
`deepagents==0.7.0b2`, head requires `==0.7.0`); a simulated
`release(code)` reports no conflicts.

</details>
2026-07-29 17:25:38 -04:00
Mason Daugherty e47f9f9439 fix(infra): run release-please after skipped trigger-releases (#5169)
After #5161, ordinary pushes to `main` skipped the `release-please`
maintenance job even when the in-flight-publish guard had already
written `skip=false`, so release PRs stopped refreshing.

---

## Why

#5161 made `guard-pending-release` depend on `trigger-releases` so a
merged release commit can dispatch publish first, then refresh sibling
release PRs. On a normal non-release push, `trigger-releases` is
**skipped**.

The guard itself already overrode Actions' default `success()` gate with
`!cancelled()` and explicit `needs.*.result` checks, so it still ran and
correctly reported `skip=false`.

`release-please` only had:

```yaml
if: needs.guard-pending-release.outputs.skip == 'false'
```

Without a status-check function, Actions still applies the implicit
`success()` gate. With a skipped ancestor behind the successful guard,
that gate failed closed and `release-please` was skipped with empty
steps — leaving the whole workflow green. Observed on:

- https://github.com/langchain-ai/deepagents/actions/runs/30487226539
(#5161 merge)
- https://github.com/langchain-ai/deepagents/actions/runs/30487483156
(`fix(sdk)` follow-up)

## Fix

Give `release-please` the same pattern as the guard:

- `!cancelled()` so the upstream-skipped dispatch job does not poison
maintenance
- explicit success checks on every direct dependency
- still require `outputs.skip == 'false'` so crash / hard-timeout /
deferred paths stay fail-closed

Tests now pin this truth table (and stop asserting the old bare `skip ==
'false'` condition, which is exactly what broke main).
2026-07-29 16:27:05 -04:00
Mason Daugherty 453838367a ci(infra): refresh open release PRs after sibling merges (#5161)
## The situation before

This repo releases ten packages independently, so it is normal to have
several open release PRs at once. Those PRs all touch shared files —
most importantly `.release-please-manifest.json`. When one of them
merges, the others are instantly stale, and release-please is what
normally rewrites them.

But release commits deliberately skipped release-please entirely, for a
good reason: running it while a publish is mid-flight is dangerous.
Merging a release PR bumps the manifest immediately, while the git tag
is only created minutes later by `release.yml`. In that window
release-please sees a version with no matching tag, concludes the
package was never released, and proposes resetting it to `0.1.0` with
the full history. That is not hypothetical — it happened once (`0.1.8` →
`0.1.0`).

So the workflow chose the safe option: on a release commit, dispatch the
publish and do nothing else.

## Why that was a problem

**Completing a publish does not push to `main`.** It creates a tag and a
GitHub release, and neither re-triggers `release-please.yml`.

So after a release merge there was no follow-up run — ever. The
remaining open release PRs stayed stale until some unrelated push
happened to land, which could be days. This made other release PRs sit
with merge conflicts "forever" on shared files.

## What this changes

Publishing still goes first and is never blocked. The difference is that
the *same run* now sticks around, waits for publishing to settle, and
then does the maintenance it used to skip:

1. `detect-release-commit`
2. `trigger-releases` — fires immediately, `needs` unchanged, still not
blocked by anything
3. `guard-pending-release` — new: also runs on release commits, after a
successful dispatch. Polls until no merged release PR is still labeled
`autorelease: pending`
4. `release-please` — runs only on an explicit `skip=false` from the
guard
5. `update-lockfiles` — unchanged

Normal (non-release) pushes are unaffected: same wait, same fail-closed
maintenance gate as before.

<details>
<summary><b>Why this ordering is safe</b></summary>

**Publishing cannot be delayed by the wait.** `trigger-releases` depends
only on `detect-release-commit`, and the guard is deliberately outside
the `release-please` concurrency group — a poll of up to 45 minutes must
not hold the serialization slot other pushes queue on. Only step 4 is
serialized, since release-please mutates shared release branches.

**Maintenance requires a positive all-clear.** `release-please` runs
only on `skip == 'false'`. A guard that crashes, times out, or is
skipped leaves that output unset, which blocks maintenance rather than
letting it recompute against unverified state. `trigger-releases` is
intentionally *not* in `release-please.needs`, so its skipped status on
normal pushes can't loosen that gate.

**The wait is repo-wide by design.** It covers every pending release PR,
not just the merged one, because release-please recomputes *all*
components on every run — any single package sitting between "version
bumped" and "tag created" is enough to trigger the downgrade above.
Scoping the wait to your own PR would not be safe.

</details>

<details>
<summary><b>Hardening added after review</b></summary>

Routing release commits through the guard exposed three paths that
previously could not be reached, plus one latent bug:

- **A release commit matching no package used to be a silent no-op.**
The package list is hardcoded in both `detect-release-commit` and
`trigger-releases`. If a future package is wired into one but not the
other, nothing publishes, nothing refreshes, and every job reports
success. The detector now fails loudly, and a config-derived test keeps
the two lists from drifting.
- **The guard treated an unreadable API response as "nothing pending."**
`format_pending` swallowed its own parse failures, so a
successful-but-unparseable `gh` response was indistinguishable from an
empty result — which would wave release-please through during exactly
the dangerous window. Parse failures now feed the existing
retry/fail-closed path.
- **A failed publish reddened two workflows.** Since the pending set is
repo-wide, a package left stuck days earlier would also fail an
unrelated release that dispatched perfectly. Release commits now defer
with `skip=true` and a step summary naming the failed run, extending the
mechanism hotfix pushes already used. Ordinary pushes still fail loudly,
where a red guard is the only signal a human would see. Maintenance is
skipped either way — this only changes red vs. green and whether the
message is actionable.
- **`git log` was unchecked**, so a failure would silently misclassify
the push and route a failed release down the wrong branch.

</details>

<details>
<summary><b>Cases that still need a later push</b></summary>

These are documented in `RELEASING.md` with the signal to look for and
what to do:

| Situation | Signal |
| --- | --- |
| Publish still in flight after 45 min | `release-please.yml` green,
`deferred` step summary |
| Publish failed (yours, or one stuck earlier) | `release.yml` red;
`release-please.yml` green with a `deferred (release commit)` summary |
| GitHub release state unreadable | `release-please.yml` **red** at
`guard-pending-release` |
| Several release PRs merged at once | Some `release-please` jobs show
**cancelled** — expected; the surviving newest run recomputes every
component |

Cross-run sibling merges can still race around the pending-label window.
This intentionally does not add a hard global cross-run barrier.

</details>

<details>
<summary><b>Testing</b></summary>

`.github/scripts/tests` passes (681 tests). The guard's `if:` condition
is now pinned by a truth table over a small expression evaluator rather
than substring checks — mutation-tested against five broken variants
(flattened parens, `||`↔`&&` swap, negated trigger check, `always()` for
`!cancelled()`), all of which are caught and four of which passed the
previous assertions.

Also verified directly: every `run:` block passes `bash -n`;
`format_pending` correctly separates "empty" from "unreadable" across
six payloads; and the guard's three exit paths behave as intended when
driven with stubbed `gh`/`git` (ordinary push → exit 1 with `skip`
unset; release commit and hotfix → exit 0 with `skip=true`).

</details>
2026-07-29 16:07:11 -04:00
Mason Daugherty c9c0c83d5e fix(infra): harden release-notes draft model configuration (#5162)
Release-note drafting now rejects OpenAI Responses-API-only models up
front and raises the shared output-token ceiling so reasoning models can
finish large changelogs.

---

`RELEASE_BOT_MODEL` is an operator-facing Actions variable, so docs and
the helper need to stay honest about which models can actually work. The
drafter only calls Chat Completions, but `RELEASING.md` told people to
pick any JSON-Schema model — including OpenAI `*-pro` / `*codex*` IDs
that only work on Responses. That used to surface as a confusing HTTP
failure on the release PR.

The previous 4096 token cap is also too tight for reasoning models:
OpenAI counts reasoning against `max_completion_tokens`, and Anthropic
extended thinking has the same shape. `responseText()` already fails
closed on non-stop finishes, so large releases would post a failure
comment instead of truncating silently — still not what we want on the
first big multi-package changelog.

This change:
- rejects known Responses-only OpenAI names (mirroring
`langchain-openai`'s prefix/`codex` check) with an error that names
`RELEASE_BOT_MODEL`
- raises one shared `MAX_OUTPUT_TOKENS` ceiling used by all three
providers
- documents the Chat Completions constraint for operators

Out of scope: implementing Responses API support.

---------

Signed-off-by: Mason Daugherty <github@mdrxy.com>
2026-07-29 15:51:18 -04:00
Mason Daugherty bba21f1eca fix(infra): exempt release PRs from stale cleanup (#5145)
Release-please's PRs are no longer warned or auto-closed by the old-PR
cleanup, and any existing `pending-deletion` label is removed from them
on the next run.

---

Release-please keeps one long-lived PR per package and updates it in
place, so "days since opened" — the only staleness signal this workflow
has — is meaningless for them. #4297 was warned by this workflow on
2026-07-29 at 33 days old and needed a manual `do-not-close` to survive.
Requiring that label duplicates what release-please already maintains.

A PR is exempt when it carries `release` or `autorelease: pending`
**and** was opened by `github-actions[bot]` on a
`release-please--branches--main--components--*` branch in this
repository. The label alone is not enough: `release` is derived from the
PR title by the unified labeler, and `release` is an allowed title type,
so a label-only check would let any contributor opt their own PR out of
cleanup indefinitely. When a release label appears without matching
provenance, the run logs a warning — so genuine drift (a renamed default
branch, a `separate-pull-requests` flip, a token change) surfaces in the
log instead of silently closing release PRs again.

The stale-label sweep uses the same classification, so already-warned
release PRs recover automatically. Draft and `do-not-close` behavior and
the age thresholds are unchanged. The run summary gains a
`skippedRelease` sub-count so an over-applying exemption is visible at a
glance.

Two pre-existing sweep bugs are fixed alongside, since this change
routes a new category of PR through it:

- The sweep swallowed non-404 errors into a warning and returned a green
run, so a sweep that died on its first PR looked identical to one with
nothing to do. It now fails the run, matching how a failed PR search is
already handled. **This can turn a currently-green run red if the sweep
has been quietly failing.**
- `staleCleared` counted PRs whose label was already gone — the label
search index lags the removals made earlier in the same run. Only real
removals are counted now, so the number may read lower than before.
2026-07-29 15:04:19 -04:00
Mason Daugherty 249121c584 ci(infra): require the release-notes helper by its current path (#5160)
Follow-up to #5159.

---

The curated release-notes check resolved its helper through a list of
candidate paths, trying the current filename first and falling back to
the pre-rename ones. That existed for a single reason: this workflow
takes its YAML from the pull request but checks the helper itself out of
`main`, so during #5159 the new filename did not exist on `main` yet —
and that PR needed this required check to pass. The fallback bridged
exactly one merge.

`main` now carries the helper at its current path and neither old path
exists anywhere, so those entries are unreachable. This replaces the
candidate list with a direct `require`, and leaves a comment explaining
why a future rename will need the same temporary fallback — the trap is
not obvious from reading the workflow.

The test that pinned the fallback ordering is replaced with a stronger
invariant: every helper script either workflow loads out of its
`trusted-source` checkout must exist in the repository. A wrong path
there is invisible to YAML linting and only surfaces when the workflow
runs, which in the check workflow means a broken required check. The new
assertion covers both the automation and check workflows, and both the
`require` and `run:` call sites.
2026-07-29 14:46:26 -04:00
Mason Daugherty 8337e33818 ci(infra): generalize curated release notes to all packages (#5159)
Curated release notes now cover every release-please package, not just
`deepagents-code`.

---

Previously the release-notes bot and its check gate were hard-coded to
dcode. Every other package either skipped curation entirely or required
hand-editing `CHANGELOG.md` plus the PR body and redoing that work after
each release-please sync.

`@release-bot` now targets whichever package a release PR is for. The
component is read from the release-please head branch and validated
against `release-please-config.json`, so a package added to that config
is covered without touching a workflow. On a fanout release each
package's PR is drafted and applied on its own.

Because the changelog path and branch ref are now derived from a head
ref instead of being constants, the component has to resolve to a real
entry in `release-please-config.json` before any path is built, the
branch and the PR title must name the same component, and the apply step
re-derives its write target from the config rather than trusting the
state it was handed. A PR retargeted to a different package part-way
through apply is rejected instead of receiving another package's notes.

### What operators will notice

- The command mention is `@release-bot draft` / `@release-bot apply`
(was `@dcode-release-bot`).
- The workflows are `release_notes.yml` and `release_notes_check.yml`.
- The `curated release notes` required check keeps its name, so branch
protection needs no edit.
- Curation now applies to the partner packages as well, including
releases that only bump a dependency. That is a real increase in
per-release work for those packages; `release: dangerously skip curated
notes` still ships a package with the generated changelog as-is.
2026-07-29 14:40:20 -04:00
Shrikar Seshadri e46b4c3884 feat(evals): per-leaf token & cost analysis for unified evals (#5002)
## What this PR does

- Adds **per-variant token & cost reporting** to the unified eval
workflow. For every variant tested — a (branch × agent config × model)
combination — it shows **total input tokens, total output tokens, and
total cost (USD)**.
- Renders these in a new **"Token usage and cost"** table in the run
summary: one row per variant, right beside the existing accuracy
leaderboard.
- Reports cost in **two ways** per variant: **completed-only** (trials
that finished cleanly) and **true spend** (every trial), plus a
**`completed / expected (N errored)`** count — so a variant that erred
out is obvious and not silently compared as if it were clean. Expected
counts come from **tasks × rollouts** (not shard count), so coverage is
reported correctly.
- Pulls the numbers from **LangSmith** (the only place token/cost data
exists — local result files only carry pass/fail), then joins them back
to each variant.
- **Keeps the API key isolated**: only the one usage-collection job
(scoped to the `evals` Environment, `contents: read` only) can read from
LangSmith; the chart-publishing job never sees the key. If LangSmith is
unavailable, the columns degrade to `—` instead of failing the run.
- **Leaves the accuracy leaderboard and all existing metrics
unchanged.**

### Built on stock Harbor (no fork, no custom plugin)

- Runs on **released Harbor `0.20.0`**
(`harbor[langsmith]>=0.20.0,<0.21.0` + `harbor-langsmith>=0.3.0,<0.4.0`)
— **no `harbor_package_override` fork**. Harbor 0.20's stock `langsmith`
plugin already logs each leaf to exactly the LangSmith experiment name
it is given, so the custom `unified_langsmith_plugin` is **deleted**.
- **LangSmith experiment names are computed up front** from a single
shared helper (`.github/scripts/experiment_name.py`), used by the run
step, the prep step, and the aggregator — so the name can never drift
between where a leaf is logged and where its cost is read back. This
replaces the previous marker-file round-trip.
- **Standard exception-based retry** (`--max-retries`) — the fork-only
`--retry-if-reward-below` reward-gate is removed.

## Tests
- `.github/scripts` helper-script suite passes (experiment-name helper,
prep, collector, shard/row aggregation, workflow wiring). `libs/evals`
unit tests pass (Harbor + LangSmith integration). `ruff format`, `ruff
check`, and `ty` type-check are clean over the eval package; workflow
YAML parses.
- Verified **end to end on a live stock-Harbor 0.20 Docker run** (no
override) comparing two branches (`main` vs
`ss/benchmark-unified-evals-no-todos`), model `gpt-5.6-terra`, 2 context
tasks × 2 rollouts. The usage job pulled real per-variant token/cost
from LangSmith and rendered the table:
  - `main / bare`: **208,526 tokens · $0.19** (2/4 completed, 2 errored)
- `ss/benchmark-unified-evals-no-todos / bare`: **345,562 tokens ·
$0.30** (2/4 completed, 2 errored)

(The errored rollouts are the eval subjects themselves failing on those
tasks — the report surfaces them via the `completed / expected (N
errored)` count rather than hiding them, which is exactly the feature.)
- Standard `--max-retries` retry verified on a separate run: a failed
trial is retried and the "actual retries" summary still reads
`result.json` `stats.n_retries`.

## Eval Run with Changes
https://github.com/langchain-ai/deepagents/actions/runs/30129067425
2026-07-29 00:30:25 -07:00
Mason Daugherty 18822b290a chore(infra): nest GitHub helper scripts by domain (#5126)
Nest GitHub helper scripts into domain folders (`checks`, `evals`,
`labeling`, `release`, `tests`) and update workflow, pre-commit,
Makefile, and doc path references.

This is a structure-only cleanup: runtime behavior is unchanged.
Helper-script CI path is now `pytest .github/scripts/tests`.

---

Workflows stay flat under `.github/workflows/` (GitHub only discovers
top-level workflow files). Production helpers are grouped by domain;
coupled modules stay together (for example `check_dep_freshness` with
`check_release_deps`).
2026-07-28 15:04:56 -04:00
Mason Daugherty fef1ff33e9 chore(deps): raise langchain-anthropic minimum to 1.5.3 (#5120)
Raise the `langchain-anthropic` lower bound to `1.5.3` across packages
and examples so installs stay on the latest stable release.

---

Also drops a hand-maintained test copy of Harbor's agent dependency list
so pin bumps don't require a second, easy-to-forget edit.

### Why the Harbor test change

Harbor installs the eval agent from a committed `langgraph.json`. That
file lists every provider package (Anthropic, OpenAI, Fireworks, …) so a
local run can pick any model. In CI, a small helper then **prunes** that
list down to the one provider the job actually uses, so each sandbox
doesn't install Wikipedia of chat SDKs.

The pruner has a unit/CI suite under `.github/scripts/`. Until this PR,
that suite kept its own constant (`SAMPLE_DEPS`) that was a **verbatim
second copy** of the real `langgraph.json` dependencies, plus a test
whose only job was “//snip// these two lists still match.”

That design meant every pin bump in Harbor had to touch two places. This
PR only updated the real file at first; CI's **Validate Release
Options** job failed because the fixture still said `1.5.2`.

Rather than keep nursing that mirror, the suite now **reads the
committed `langgraph.json` directly** for the full multi-provider
dependency list. Version pins live in one place. What still needs
deliberate sync is only the provider **name → package** map in the
pruner (which provider string corresponds to which `langchain-*`
package) — not the version lower bounds.

Lockfiles were refreshed with a targeted `langchain-anthropic` upgrade;
resolution also moves transitive `langchain-core` to `1.5.2` where locks
still recorded the previous pin.
2026-07-28 13:37:05 -04:00
Mason Daugherty 4f81081528 ci(infra): block and surface release-please fan-out (#5077)
After a bump-worthy PR touches files under several managed packages,
release-please attributes that single commit to every touched component
and opens a separate release PR for each. The existing lockfile-only
gate missed the common case where dependency lower bounds in
`pyproject.toml` were also edited, and applying a bypass label only left
a quiet ℹ️ note.

Pre-merge checks now fail bump-worthy PRs that edit real files in more
than one release-please component (or only lockfiles inside a managed
package), stickies prescribe the single-package feature/fix +
`chore(deps):` split, and bypass labels get a loud warning naming every
component still on track for a release. Docs and the PR template spell
out the one-component rule, and a post-merge watch flags open release
PRs whose only unreleased package delta is lockfile churn.

---

Related postmortem: multi-component fan-out from a feature PR that also
bumped shared dependency floors.

**Not in this PR:** restricting who can apply bypass labels / renaming
labels / requiring a written justification reply.
2026-07-27 11:25:26 -04:00
Hunter Lovell c0afec5ab7 feat(code): add Claude Opus 5 support (#5049) 2026-07-24 13:37:48 -07:00
Shrikar Seshadri a58212f1e2 feat(evals): add timeout and retry controls (#4893)
Depends on #4872

Unified evals can configure agent timeout scaling and retry incomplete
Harbor trials.

---
## Changes Made to Harbor

- Branch: `ss/langgraph/managed-python-bootstrap`, Commit: `d2e20423`
- Added an optional reward-gated retry via a new
`--retry-if-reward-below <float>` CLI flag (backed by
`RetryConfig.retry_if_reward_below`, default `None`).
- Behavior:
- When set, an errored trial is retried only if its primary reward is
below the threshold.
  - A missing reward always qualifies for retry.
- Layers on top of the existing exception include/exclude policy -
`AgentTimeoutError` and other default-excluded exceptions stay
non-retryable.
- When unset, retry behavior is unchanged (trial is retried if it
recorded an exception, regardless of reward)
- Files changed:
- `models/job/config.py` — new `RetryConfig.retry_if_reward_below`
field.
- `models/trial/result.py` — new `TrialResult.primary_reward()` helper.
- `trial/queue.py` — reward gate wired into the retry decision, after
the exception check.
  - `cli/jobs.py` — `--retry-if-reward-below` option → `config.retry`.
- `upload/uploader.py` — reused the new helper (removed duplicated
reward extraction).



## What this PR Does:

- Adds cost-safe n_retries=0 and agent_timeout_multiplier=1.0 dispatch
inputs.
- Validates both inputs during unified prep before any eval matrix jobs
launch.
- Uses Harbor retry classification plus --retry-if-reward-below 1.0 when
retries are enabled
- Reports configured controls and actual Harbor retry counts in workflow
summaries.
- Keeps comparator identities and metrics unchanged.

## Tests
- All 622 GitHub helper-script tests pass; focused Ruff checks and
workflow YAML parsing pass.
- One-task immutable-Harbor Docker run 29860384397 passed end to end
with n_retries=1 and agent_timeout_multiplier=1.5. The task scored 0
without an exception and correctly recorded 0 actual retries.


## Eval Run with Changes:
https://github.com/langchain-ai/deepagents/actions/runs/29868334615

## Harbor + Harbor LangSmith Override to Use for Evals:

`-f 'harbor_package_override=harbor @
git+https://github.com/nick-hollon-lc/harbor.git@ss/langgraph/managed-python-bootstrap
harbor-langsmith @
git+https://github.com/nick-hollon-lc/harbor.git@ss/langgraph/managed-python-bootstrap#subdirectory=packages/harbor-langsmith'`
2026-07-24 11:42:48 -07:00
Mason Daugherty 723ef8a501 ci(infra): clarify release PR bot and pin check copy (#5033)
Clarify release PR and release-bot copy so maintainers get clearer
guidance on note hygiene, prerelease SDK pins, and next steps.

---

Makes release PR surface copy more actionable:

- **Release-please PR header** — explain *why* the notes preview should
stay in sync with `CHANGELOG.md` even though publish uses the merged
changelog: the PR body remains the historical record for reviewers and
later readers.
- **Curated notes draft comment** — visible horizontal rules around the
editable content markers, prefer copyable fenced code blocks for
`@dcode-release-bot apply` / `draft`, and clearer wording for the
curated-notes gate bypass label.
- **Curated notes check** — when notes are ready for review, link the
bot draft comment and put the apply command in a code block.
- **Prerelease SDK pin warning** — drop the misleading “this is allowed”
framing; treat `release-deps: acknowledged` as an explicit
acknowledgement before merge (and note that it also covers
deps/freshness failures common for unpublished prerelease SDK pins).

Affects future release PR automation/text only; does not change an
already-open release PR body/comments until they are regenerated.
2026-07-24 00:47:01 -04:00
Mason Daugherty 861e8e78c3 test(infra): harden workflow credential contracts (#5006)
Credential-scoping regressions now fail with precise assertions instead
of passing when secret names, conditions, and references merely appear
somewhere in the same workflow step.

---

The static contracts now parse workflow YAML and compare the complete
integration and disabled-release `env` mappings. This binds each secret
to its exact package condition and credential reference while rejecting
unexpected additional entries, including an accidentally unconditional
secret.

A separate reachability contract records `libs/code` and
`libs/partners/quickjs` as deliberate free-text override targets and
verifies every credential condition still points to an existing package
path. The dormant integration workflow and its current matrix are
otherwise unchanged.

The same parser-based approach replaces whitespace-sensitive extraction
for the GitHub App, release, and OpenWiki contracts. A release comment
explains why Deep Agents Code intentionally receives no provider
credentials there, and two setup comments now refer to the plural
credentials they list.
2026-07-23 17:42:46 -04:00
Mason Daugherty a68dd068ff ci(infra): scope GitHub Actions credentials (#5004)
GitHub Actions now exposes stored credentials only to the automation and
package tests that need them, and OpenWiki is ready to read its API keys
from a dedicated environment instead of repository scope.

---

The GitHub App Client ID is not sensitive, so token-minting workflows
now read it from the repository variable while the private key remains a
secret. Integration-test credentials are selected by matrix package,
including wiring the previously unavailable Daytona key. The disabled
release integration step keeps its future credential wiring but applies
the same package-level isolation.

OpenWiki now targets an `openwiki` environment. Repository-level
fallback keeps the workflow working during migration; after environment
secrets are configured and a manual run succeeds, the repository copies
can be removed.
2026-07-23 16:30:24 -04:00