mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-07 06:16:06 -10:00
Nicky LeachandClaude Opus 5.5
79c724b292
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - The Runner support suite (`pnpm test:e2e:runner:unit`) checks the code behind paid Runner runs: workflow security policy, evidence oracles, cost receipts and controller recovery. > - Only a manual Docker Runner check ran this suite. PR CI did not run it. > - So changes on master broke 16 of its tests, and nobody saw it. #19527 and #19529 fix those tests. > - This pull request runs the suite in PR CI and makes the `verify` check require it. > - The benefit is that a change that breaks the suite fails on its own pull request, before merge. ## Linked Issues or Issue Description Refs #19527 and #19529 (both merged; they repair the suite). Also accounts for #19533, which added the sandbox provider job. **Merge order:** #19527 and #19529 have merged, so the suite now passes on master. This PR can merge. **What existing behavior does this improve?** PR CI for `pr-trusted.yml`. **Current behavior** `pnpm test:e2e:runner:unit` runs only in `docker-runner-check.yml`, by manual dispatch. Master changes broke 12–13 tests by Nov 3 and 16 by Nov 6, with no signal on any pull request. **Proposed behavior** A required "Runner support suite" job runs the suite on pull requests. **Reason and benefit** Drift fails on the pull request that causes it. Authors update deliberate pins (profile versions, suite hashes, workflow policy) in the same pull request. ## What Changed - `pr-trusted.yml`: new job `verify_runner_e2e_support` ("Runner support suite"). It uses the same steps as other trusted jobs: checkout without credentials, setup-node directly before pnpm, the npm overrides, the shared `setup-ci-dependencies` restore, and CI profiling. Then it runs `pnpm test:e2e:runner:unit`. It has no secrets. Timeout: 20 minutes. - `pr-trusted.yml`, job `verify`: the job now needs the new job and checks its result. A failed, cancelled or wrongly skipped run fails `verify`. - `turbo.json`: new routing task `ci:runner-e2e-support`. Its inputs are `server/**`, `cli/**`, `packages/**` and `ui/**`. Changes to `tests/**`, `scripts/**` and `.github/**` already force full CI, so they also run the job. - `.github/ci-task-matrix.json`: one lane for the new job. - Tests: `ci-plan.test.mjs`, `ci-profile.test.mjs`, `e2e-shard.test.mjs` and `workflow-security.test.ts` now include the new job. A UI component change now selects this one Runner task. The suite imports UI modules and checks that catalog-listed UI files stay tracked in git, so UI changes can break it. - CI profiling: when `PAPERCLIP_CI_PROFILE` is on, the suite's Vitest config adds `scripts/ci-profile/vitest-reporter.mjs`, and `run-unit.mjs` runs the Node tests through `scripts/ci-profile/node-test.mjs`. So the job uploads per-test timings, like the root and Runner package suites. The launcher loads the profiling helper only after its "Vitest is not installed" check. - `tests/runner-e2e/README.md`: says that PR CI runs the suite, and that deliberate pin changes go in the same pull request. ## Verification - `node --test .github/scripts/tests/*.test.mjs scripts/ci/*.test.mjs scripts/__tests__/e2e-shard.test.mjs scripts/__tests__/release-verify-workflow.test.mjs scripts/__tests__/runner-release-crate-cache.test.mjs`: 830 passed, 3 failed. The same 3 fail on the base branch without this change. They need local tools that are not on this Mac. - A local run with `PAPERCLIP_CI_PROFILE=true` wrote per-test and per-file timing records for Vitest and for Node. Without profiling, output and exit codes do not change. Both `unit-launcher.test.mjs` tests pass. - The aggregate test now covers the new job: `verify` rejects it when it fails, is cancelled, or is skipped while selected. - The pnpm bootstrap policy tests in `workflow-security.test.ts` pass with 10 setups in `pr-trusted.yml`. - On this PR's merged tree (`80b526d697`, master plus this PR): `pnpm test:e2e:runner:unit` passed, Vitest 320 files (6,830 tests) and node:test 373, in 58s. It also passes on master `4c1f629c31`. - Suite runtime on a 10-core Mac: 43–63s. With `--maxWorkers=4`: Vitest 118s, about 490s of CPU in total. On a 4-vCPU GitHub runner I expect about 3–4 minutes for the tests and 1–2 minutes for setup. The job runs in parallel with the longer test shards, so it should not lengthen CI. - `pr.yml` calls `pr-trusted.yml@master`. This PR's own CI does not run the new job. The first real CI timing comes from the first pull request after merge. ## Risks - More runner minutes: one extra job on most pull requests (all except documentation-only ones). - The suite has not run on Linux CI recently: old `docker-runner-check` logs have expired. A Linux-only failure would show up on the first pull requests after merge. The job can be removed from the `verify` list quickly if needed. - #19229 removes `version: 9.15.4` from the pnpm steps in this file. Whichever PR merges second needs a small rebase. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), by Anthropic, in Claude Code with tool use (shell, file edits, git, GitHub CLI). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <[email protected]>
294 lines
6.1 KiB
JSON
294 lines
6.1 KiB
JSON
{
|
|
"typecheck_release_registry": [
|
|
{
|
|
"task": "ci:typecheck"
|
|
}
|
|
],
|
|
"general_tests": [
|
|
{
|
|
"group": "general-server-without-chat",
|
|
"group_label": "server (1/12)",
|
|
"shard_index": 0,
|
|
"shard_count": 12,
|
|
"task": "ci:server"
|
|
},
|
|
{
|
|
"group": "general-server-without-chat",
|
|
"group_label": "server (2/12)",
|
|
"shard_index": 1,
|
|
"shard_count": 12,
|
|
"task": "ci:server"
|
|
},
|
|
{
|
|
"group": "general-server-without-chat",
|
|
"group_label": "server (3/12)",
|
|
"shard_index": 2,
|
|
"shard_count": 12,
|
|
"task": "ci:server"
|
|
},
|
|
{
|
|
"group": "general-server-without-chat",
|
|
"group_label": "server (4/12)",
|
|
"shard_index": 3,
|
|
"shard_count": 12,
|
|
"task": "ci:server"
|
|
},
|
|
{
|
|
"group": "general-server-without-chat",
|
|
"group_label": "server (5/12)",
|
|
"shard_index": 4,
|
|
"shard_count": 12,
|
|
"task": "ci:server"
|
|
},
|
|
{
|
|
"group": "general-server-without-chat",
|
|
"group_label": "server (6/12)",
|
|
"shard_index": 5,
|
|
"shard_count": 12,
|
|
"task": "ci:server"
|
|
},
|
|
{
|
|
"group": "general-server-without-chat",
|
|
"group_label": "server (7/12)",
|
|
"shard_index": 6,
|
|
"shard_count": 12,
|
|
"task": "ci:server"
|
|
},
|
|
{
|
|
"group": "general-server-without-chat",
|
|
"group_label": "server (8/12)",
|
|
"shard_index": 7,
|
|
"shard_count": 12,
|
|
"task": "ci:server"
|
|
},
|
|
{
|
|
"group": "general-server-without-chat",
|
|
"group_label": "server (9/12)",
|
|
"shard_index": 8,
|
|
"shard_count": 12,
|
|
"task": "ci:server"
|
|
},
|
|
{
|
|
"group": "general-server-without-chat",
|
|
"group_label": "server (10/12)",
|
|
"shard_index": 9,
|
|
"shard_count": 12,
|
|
"task": "ci:server"
|
|
},
|
|
{
|
|
"group": "general-server-without-chat",
|
|
"group_label": "server (11/12)",
|
|
"shard_index": 10,
|
|
"shard_count": 12,
|
|
"task": "ci:server"
|
|
},
|
|
{
|
|
"group": "general-server-without-chat",
|
|
"group_label": "server (12/12)",
|
|
"shard_index": 11,
|
|
"shard_count": 12,
|
|
"task": "ci:server"
|
|
},
|
|
{
|
|
"group": "general-chat",
|
|
"group_label": "chat (1/3)",
|
|
"shard_index": 0,
|
|
"shard_count": 3,
|
|
"task": "ci:chat"
|
|
},
|
|
{
|
|
"group": "general-chat",
|
|
"group_label": "chat (2/3)",
|
|
"shard_index": 1,
|
|
"shard_count": 3,
|
|
"task": "ci:chat"
|
|
},
|
|
{
|
|
"group": "general-chat",
|
|
"group_label": "chat (3/3)",
|
|
"shard_index": 2,
|
|
"shard_count": 3,
|
|
"task": "ci:chat"
|
|
},
|
|
{
|
|
"group": "general-workspaces-a",
|
|
"group_label": "workspaces-a (1/2)",
|
|
"shard_index": 0,
|
|
"shard_count": 2,
|
|
"task": "ci:workspaces-a"
|
|
},
|
|
{
|
|
"group": "general-workspaces-a",
|
|
"group_label": "workspaces-a (2/2)",
|
|
"shard_index": 1,
|
|
"shard_count": 2,
|
|
"task": "ci:workspaces-a"
|
|
},
|
|
{
|
|
"group": "general-workspaces-b",
|
|
"group_label": "workspaces-b",
|
|
"task": "ci:workspaces-b"
|
|
},
|
|
{
|
|
"group": "general-workspaces-c",
|
|
"group_label": "workspaces-c",
|
|
"task": "ci:db"
|
|
}
|
|
],
|
|
"docker_context_integrity": [
|
|
{
|
|
"task": "ci:docker"
|
|
}
|
|
],
|
|
"sandbox_provider_tests": [
|
|
{
|
|
"task": "ci:sandbox-providers"
|
|
}
|
|
],
|
|
"verify_paperclip_runner": [
|
|
{
|
|
"lane_label": "static checks",
|
|
"command": "check:static",
|
|
"task": "ci:runner-static"
|
|
},
|
|
{
|
|
"lane_label": "rust",
|
|
"command": "check:runner",
|
|
"release_crate": true,
|
|
"task": "ci:runner-rust"
|
|
},
|
|
{
|
|
"lane_label": "vitest package",
|
|
"command": "test:typescript:vitest --shard=1/2",
|
|
"task": "ci:runner-vitest"
|
|
},
|
|
{
|
|
"lane_label": "vitest native server",
|
|
"command": "test:typescript:vitest --shard=2/2",
|
|
"release_crate": true,
|
|
"task": "ci:runner-vitest"
|
|
}
|
|
],
|
|
"build": [
|
|
{
|
|
"task": "ci:build"
|
|
}
|
|
],
|
|
"verify_serialized_server": [
|
|
{
|
|
"shard_index": 0,
|
|
"shard_count": 9,
|
|
"shard_label": "1/9",
|
|
"task": "ci:serialized"
|
|
},
|
|
{
|
|
"shard_index": 1,
|
|
"shard_count": 9,
|
|
"shard_label": "2/9",
|
|
"task": "ci:serialized"
|
|
},
|
|
{
|
|
"shard_index": 2,
|
|
"shard_count": 9,
|
|
"shard_label": "3/9",
|
|
"task": "ci:serialized"
|
|
},
|
|
{
|
|
"shard_index": 3,
|
|
"shard_count": 9,
|
|
"shard_label": "4/9",
|
|
"task": "ci:serialized"
|
|
},
|
|
{
|
|
"shard_index": 4,
|
|
"shard_count": 9,
|
|
"shard_label": "5/9",
|
|
"task": "ci:serialized"
|
|
},
|
|
{
|
|
"shard_index": 5,
|
|
"shard_count": 9,
|
|
"shard_label": "6/9",
|
|
"task": "ci:serialized"
|
|
},
|
|
{
|
|
"shard_index": 6,
|
|
"shard_count": 9,
|
|
"shard_label": "7/9",
|
|
"task": "ci:serialized"
|
|
},
|
|
{
|
|
"shard_index": 7,
|
|
"shard_count": 9,
|
|
"shard_label": "8/9",
|
|
"task": "ci:serialized"
|
|
},
|
|
{
|
|
"shard_index": 8,
|
|
"shard_count": 9,
|
|
"shard_label": "9/9",
|
|
"task": "ci:serialized"
|
|
}
|
|
],
|
|
"canary_dry_run": [
|
|
{
|
|
"task": "ci:canary"
|
|
}
|
|
],
|
|
"e2e_shards": [
|
|
{
|
|
"shard_index": 0,
|
|
"shard_count": 8,
|
|
"shard_label": "1/8",
|
|
"task": "ci:e2e"
|
|
},
|
|
{
|
|
"shard_index": 1,
|
|
"shard_count": 8,
|
|
"shard_label": "2/8",
|
|
"task": "ci:e2e"
|
|
},
|
|
{
|
|
"shard_index": 2,
|
|
"shard_count": 8,
|
|
"shard_label": "3/8",
|
|
"task": "ci:e2e"
|
|
},
|
|
{
|
|
"shard_index": 3,
|
|
"shard_count": 8,
|
|
"shard_label": "4/8",
|
|
"task": "ci:e2e"
|
|
},
|
|
{
|
|
"shard_index": 4,
|
|
"shard_count": 8,
|
|
"shard_label": "5/8",
|
|
"task": "ci:e2e"
|
|
},
|
|
{
|
|
"shard_index": 5,
|
|
"shard_count": 8,
|
|
"shard_label": "6/8",
|
|
"task": "ci:e2e"
|
|
},
|
|
{
|
|
"shard_index": 6,
|
|
"shard_count": 8,
|
|
"shard_label": "7/8",
|
|
"task": "ci:e2e"
|
|
},
|
|
{
|
|
"shard_index": 7,
|
|
"shard_count": 8,
|
|
"shard_label": "8/8",
|
|
"task": "ci:e2e"
|
|
}
|
|
],
|
|
"verify_runner_e2e_support": [
|
|
{
|
|
"task": "ci:runner-e2e-support"
|
|
}
|
|
]
|
|
}
|