mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-07 06:16:06 -10:00
Nicky LeachandClaude Opus 5.5
a2aa69bc62
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Contributors wait for PR CI before merge, so CI wall-clock time sets the pace of work > - The Runner package Vitest lane is the PR CI critical path. It finishes a median 15.3 minutes after the push > - That lane runs all 311 package test files in one job with 2 workers: 1,381 seconds of serial work and 727 seconds of wall time > - Ten files account for 61% of the time. Vitest's `--shard` splits by file count, not by duration, so it cannot balance this work > - This pull request splits the package files into three lanes balanced by duration, and runs the native server group in its own lane > - The benefit is about 232 seconds of tests per package lane instead of 727 seconds ## Linked Issues or Issue Description **What existing behavior does this improve?** The time that PR CI needs to run the Runner package Vitest tests. **Subsystem affected** CI: the `Verify Paperclip Runner` Vitest lanes in `pr-trusted.yml`, and `packages/paperclip-runner/scripts/run-pr-vitest-lane.mjs`. **Current behavior** The `vitest package` lane takes a median of 13.6 minutes. It is the last job to finish in most PR runs. The lane keeps a 2-worker limit so that Runner and sidecar processes can start within their deadlines. Average CPU use is 29%. **Proposed behavior** Three package lanes each run about one third of the files, balanced by measured duration. Each lane keeps the 2-worker limit. The native server group runs in its own lane, as before. **Reason and benefit** The Runner lane stops being the critical path. More workers per lane would add CPU contention and timeouts, so this change adds lanes instead. The cost is about 3 runner-minutes per run for the extra setup. **Breaking changes** None. The `master` ruleset requires only `ci / verify` and `ci / e2e`, so the new job names do not change required checks. ## What Changed - Add `packages/paperclip-runner/scripts/vitest-package-lanes.mjs`. - It gets the file inventory from `vitest list --filesOnly --json`, so it follows `vitest.config.ts`. - It partitions the files with the `partitionGeneralServerSuites` function from `scripts/general-server-shard.mjs`. - Add `packages/paperclip-runner/scripts/vitest-lane-durations.json`. - It uses the same schema as `scripts/general-server-shard-durations.json`. - It is seeded from CI run 42290920882. It measures 310 of the 311 files. - `run-pr-vitest-lane.mjs` accepts two new arguments: - `--package-lane=K/N` runs one partition. - `--native-server` runs only the native server group. - The `--shard=1/2` and `--shard=2/2` arguments keep their exact behavior for the matrix that master still sends. - `.github/ci-task-matrix.json` and the fallback matrix in `pr-trusted.yml` have three `vitest package (K/3)` lanes and the `vitest native server` lane. The labels keep the `vitest` prefix because `scripts/ci-profile/trace.mjs` matches `Verify Paperclip Runner (vitest`. - Tests: - the lanes cover the inventory exactly once, checked with real `vitest list`; - the old `--shard` commands plan the same commands as before; - the manifest schema is correct; - bad lane arguments are refused. - Update the Runner Vitest lane paragraph in `doc/DEVELOPING.md`. ## Verification - `node --test scripts/__tests__/run-vitest-stable-shard.test.mjs`: 44/44 pass. - `node --test scripts/__tests__/runner-release-crate-cache.test.mjs .github/scripts/tests/ci-plan.test.mjs`, together with the file above: 71/71 pass. - Related workflow, cache, routing and profile tests: 446/446, 153/153 and 14/14 pass. - Local end-to-end run of `GITHUB_WORKFLOW=PR node scripts/run-pr-vitest-lane.mjs --package-lane=3/3` on macOS: - the prep steps ran, and the lane ran 104 of 311 files in 123 seconds; - 2 tests failed: a macOS `kill EPERM` and a 5-second timeout under full local load without the CI worker limit; - both files pass when run alone (35/35 tests). - This PR's own CI runs the matrix from master, so it uses the old `--shard` commands. The new lanes run in CI only after merge. Reviewers can compare the job durations of the first PR runs after the merge. | Lane | Files | Serial estimate | Wall estimate with 2 workers | | --- | --- | --- | --- | | 1/3 | 103 | 464.9 s | ~232 s | | 2/3 | 104 | 465.0 s | ~232 s | | 3/3 | 104 | 465.0 s | ~232 s | ## Risks - The weights come from one CI log and do not include collection or setup time. The balance can be uneven until a refresh measures real lane runs. - Each lane passes its file paths to Vitest as filters, and Vitest matches a filter as a substring of the path. A path that is a substring of another path would run a file in two lanes. No current path does this. If one does, the script prints a warning, and no file is dropped. - Each package lane runs `ensure:eval-build-deps` and `build:rust` again, which adds about 3 runner-minutes per run. - A comment in `scripts/run-vitest-stable.mjs` (about line 110) still says that the last Runner shard runs the native group. A follow-up will correct it. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`) in Claude Code, with tool use and code execution. A Claude sub-agent wrote the change, and the parent session reviewed it and ran the local lane check. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <[email protected]>
304 lines
6.4 KiB
JSON
304 lines
6.4 KiB
JSON
{
|
|
"typecheck_release_registry": [
|
|
{
|
|
"task": "ci:typecheck"
|
|
}
|
|
],
|
|
"general_tests": [
|
|
{
|
|
"group": "general-server-without-chat",
|
|
"group_label": "server (1/12)",
|
|
"shard_index": 0,
|
|
"shard_count": 12,
|
|
"task": "ci:server"
|
|
},
|
|
{
|
|
"group": "general-server-without-chat",
|
|
"group_label": "server (2/12)",
|
|
"shard_index": 1,
|
|
"shard_count": 12,
|
|
"task": "ci:server"
|
|
},
|
|
{
|
|
"group": "general-server-without-chat",
|
|
"group_label": "server (3/12)",
|
|
"shard_index": 2,
|
|
"shard_count": 12,
|
|
"task": "ci:server"
|
|
},
|
|
{
|
|
"group": "general-server-without-chat",
|
|
"group_label": "server (4/12)",
|
|
"shard_index": 3,
|
|
"shard_count": 12,
|
|
"task": "ci:server"
|
|
},
|
|
{
|
|
"group": "general-server-without-chat",
|
|
"group_label": "server (5/12)",
|
|
"shard_index": 4,
|
|
"shard_count": 12,
|
|
"task": "ci:server"
|
|
},
|
|
{
|
|
"group": "general-server-without-chat",
|
|
"group_label": "server (6/12)",
|
|
"shard_index": 5,
|
|
"shard_count": 12,
|
|
"task": "ci:server"
|
|
},
|
|
{
|
|
"group": "general-server-without-chat",
|
|
"group_label": "server (7/12)",
|
|
"shard_index": 6,
|
|
"shard_count": 12,
|
|
"task": "ci:server"
|
|
},
|
|
{
|
|
"group": "general-server-without-chat",
|
|
"group_label": "server (8/12)",
|
|
"shard_index": 7,
|
|
"shard_count": 12,
|
|
"task": "ci:server"
|
|
},
|
|
{
|
|
"group": "general-server-without-chat",
|
|
"group_label": "server (9/12)",
|
|
"shard_index": 8,
|
|
"shard_count": 12,
|
|
"task": "ci:server"
|
|
},
|
|
{
|
|
"group": "general-server-without-chat",
|
|
"group_label": "server (10/12)",
|
|
"shard_index": 9,
|
|
"shard_count": 12,
|
|
"task": "ci:server"
|
|
},
|
|
{
|
|
"group": "general-server-without-chat",
|
|
"group_label": "server (11/12)",
|
|
"shard_index": 10,
|
|
"shard_count": 12,
|
|
"task": "ci:server"
|
|
},
|
|
{
|
|
"group": "general-server-without-chat",
|
|
"group_label": "server (12/12)",
|
|
"shard_index": 11,
|
|
"shard_count": 12,
|
|
"task": "ci:server"
|
|
},
|
|
{
|
|
"group": "general-chat",
|
|
"group_label": "chat (1/3)",
|
|
"shard_index": 0,
|
|
"shard_count": 3,
|
|
"task": "ci:chat"
|
|
},
|
|
{
|
|
"group": "general-chat",
|
|
"group_label": "chat (2/3)",
|
|
"shard_index": 1,
|
|
"shard_count": 3,
|
|
"task": "ci:chat"
|
|
},
|
|
{
|
|
"group": "general-chat",
|
|
"group_label": "chat (3/3)",
|
|
"shard_index": 2,
|
|
"shard_count": 3,
|
|
"task": "ci:chat"
|
|
},
|
|
{
|
|
"group": "general-workspaces-a",
|
|
"group_label": "workspaces-a (1/2)",
|
|
"shard_index": 0,
|
|
"shard_count": 2,
|
|
"task": "ci:workspaces-a"
|
|
},
|
|
{
|
|
"group": "general-workspaces-a",
|
|
"group_label": "workspaces-a (2/2)",
|
|
"shard_index": 1,
|
|
"shard_count": 2,
|
|
"task": "ci:workspaces-a"
|
|
},
|
|
{
|
|
"group": "general-workspaces-b",
|
|
"group_label": "workspaces-b",
|
|
"task": "ci:workspaces-b"
|
|
},
|
|
{
|
|
"group": "general-workspaces-c",
|
|
"group_label": "workspaces-c",
|
|
"task": "ci:db"
|
|
}
|
|
],
|
|
"docker_context_integrity": [
|
|
{
|
|
"task": "ci:docker"
|
|
}
|
|
],
|
|
"sandbox_provider_tests": [
|
|
{
|
|
"task": "ci:sandbox-providers"
|
|
}
|
|
],
|
|
"verify_paperclip_runner": [
|
|
{
|
|
"lane_label": "static checks",
|
|
"command": "check:static",
|
|
"task": "ci:runner-static"
|
|
},
|
|
{
|
|
"lane_label": "rust",
|
|
"command": "check:runner",
|
|
"release_crate": true,
|
|
"task": "ci:runner-rust"
|
|
},
|
|
{
|
|
"lane_label": "vitest package (1/3)",
|
|
"command": "test:typescript:vitest --package-lane=1/3",
|
|
"task": "ci:runner-vitest"
|
|
},
|
|
{
|
|
"lane_label": "vitest package (2/3)",
|
|
"command": "test:typescript:vitest --package-lane=2/3",
|
|
"task": "ci:runner-vitest"
|
|
},
|
|
{
|
|
"lane_label": "vitest package (3/3)",
|
|
"command": "test:typescript:vitest --package-lane=3/3",
|
|
"task": "ci:runner-vitest"
|
|
},
|
|
{
|
|
"lane_label": "vitest native server",
|
|
"command": "test:typescript:vitest --native-server",
|
|
"release_crate": true,
|
|
"task": "ci:runner-vitest"
|
|
}
|
|
],
|
|
"build": [
|
|
{
|
|
"task": "ci:build"
|
|
}
|
|
],
|
|
"verify_serialized_server": [
|
|
{
|
|
"shard_index": 0,
|
|
"shard_count": 9,
|
|
"shard_label": "1/9",
|
|
"task": "ci:serialized"
|
|
},
|
|
{
|
|
"shard_index": 1,
|
|
"shard_count": 9,
|
|
"shard_label": "2/9",
|
|
"task": "ci:serialized"
|
|
},
|
|
{
|
|
"shard_index": 2,
|
|
"shard_count": 9,
|
|
"shard_label": "3/9",
|
|
"task": "ci:serialized"
|
|
},
|
|
{
|
|
"shard_index": 3,
|
|
"shard_count": 9,
|
|
"shard_label": "4/9",
|
|
"task": "ci:serialized"
|
|
},
|
|
{
|
|
"shard_index": 4,
|
|
"shard_count": 9,
|
|
"shard_label": "5/9",
|
|
"task": "ci:serialized"
|
|
},
|
|
{
|
|
"shard_index": 5,
|
|
"shard_count": 9,
|
|
"shard_label": "6/9",
|
|
"task": "ci:serialized"
|
|
},
|
|
{
|
|
"shard_index": 6,
|
|
"shard_count": 9,
|
|
"shard_label": "7/9",
|
|
"task": "ci:serialized"
|
|
},
|
|
{
|
|
"shard_index": 7,
|
|
"shard_count": 9,
|
|
"shard_label": "8/9",
|
|
"task": "ci:serialized"
|
|
},
|
|
{
|
|
"shard_index": 8,
|
|
"shard_count": 9,
|
|
"shard_label": "9/9",
|
|
"task": "ci:serialized"
|
|
}
|
|
],
|
|
"canary_dry_run": [
|
|
{
|
|
"task": "ci:canary"
|
|
}
|
|
],
|
|
"e2e_shards": [
|
|
{
|
|
"shard_index": 0,
|
|
"shard_count": 8,
|
|
"shard_label": "1/8",
|
|
"task": "ci:e2e"
|
|
},
|
|
{
|
|
"shard_index": 1,
|
|
"shard_count": 8,
|
|
"shard_label": "2/8",
|
|
"task": "ci:e2e"
|
|
},
|
|
{
|
|
"shard_index": 2,
|
|
"shard_count": 8,
|
|
"shard_label": "3/8",
|
|
"task": "ci:e2e"
|
|
},
|
|
{
|
|
"shard_index": 3,
|
|
"shard_count": 8,
|
|
"shard_label": "4/8",
|
|
"task": "ci:e2e"
|
|
},
|
|
{
|
|
"shard_index": 4,
|
|
"shard_count": 8,
|
|
"shard_label": "5/8",
|
|
"task": "ci:e2e"
|
|
},
|
|
{
|
|
"shard_index": 5,
|
|
"shard_count": 8,
|
|
"shard_label": "6/8",
|
|
"task": "ci:e2e"
|
|
},
|
|
{
|
|
"shard_index": 6,
|
|
"shard_count": 8,
|
|
"shard_label": "7/8",
|
|
"task": "ci:e2e"
|
|
},
|
|
{
|
|
"shard_index": 7,
|
|
"shard_count": 8,
|
|
"shard_label": "8/8",
|
|
"task": "ci:e2e"
|
|
}
|
|
],
|
|
"verify_runner_e2e_support": [
|
|
{
|
|
"task": "ci:runner-e2e-support"
|
|
}
|
|
]
|
|
}
|