mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-07 06:16:06 -10:00
Nicky LeachandClaude Opus 5.5
fb3cf1a072
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Contributors wait for PR CI before merge, so CI wall-clock time sets the pace of work > - PR CI runs Vitest shards with 4 workers on 4-vCPU runners, so timing-sensitive tests sometimes time out under load > - One such timeout fails the whole run, and a re-run costs another 15 minutes or more > - In one 7-hour window on 2026-11-06, about 30 failed jobs were test flakes on commits that passed on a later attempt > - This pull request retries a failed Vitest case once in PR CI only, and reports every pass that needed the retry > - The benefit is fewer whole-run re-runs, while each flaky test stays visible so that we can fix it ## Linked Issues or Issue Description **What existing behavior does this improve?** PR CI fails the whole run when one Vitest case fails once because of runner load. **Subsystem affected** CI: the PR Vitest lanes (general, serialized, workspace and Runner package lanes). **Current behavior** Vitest uses `retry: 0` everywhere. One timeout under load fails the run. A person must notice it and re-run the failed jobs. About 20% of recently merged PRs needed at least one re-run. **Proposed behavior** PR CI retries a failed Vitest case one time. A test that passes only on the retry gets a GitHub warning annotation and a row in the job summary. Local runs, release, cloud readiness and master keep `retry: 0`. **Reason and benefit** A re-run of one test case costs seconds. A re-run of the CI run costs 15 minutes or more. The report keeps flaky tests visible. Separate pull requests fix the known flaky tests. **Breaking changes** None. ## What Changed - Add `scripts/ci/vitest-retry.mjs`. It sets 1 retry when `GITHUB_ACTIONS=true` and `GITHUB_WORKFLOW=PR`, and 0 in other contexts. `PAPERCLIP_VITEST_RETRY` overrides the value. - Add `scripts/ci/retry-reporter.mjs`. It writes a `::warning` annotation and a step-summary table row for each test that passed only on a retry. - `scripts/run-vitest-stable.mjs` passes `--retry` to `vitest run`. The `vitest list` collection step does not retry. - The root `vitest.config.ts` and `packages/paperclip-runner/vitest.config.ts` add the retry reporter beside the existing reporters. The Runner config sets `retry` from the same gate. - `.github/workflows/pr-trusted.yml` reads the `CI_VITEST_RETRY` repository variable. Set it to `0` to turn retries off without a code change. - Add `scripts/ci/retry-reporter.test.mjs` to `test:release-registry`, and document the behavior in `doc/DEVELOPING.md`. ## Verification - `node --test scripts/ci/retry-reporter.test.mjs`: 11/11 pass. The tests cover the gate matrix, the reporter output, a real Vitest run with a test that fails once, the wrapper arguments, and the Runner config. - `node --test scripts/__tests__/run-vitest-stable-shard.test.mjs`: 39/39 pass. - `node --test .github/scripts/tests/*.test.mjs`: 744/744 pass. - Manual check: a temporary test that fails once ran through `run-vitest-stable.mjs` and through the Runner package config. With the PR environment it passed and wrote the annotation and the summary row. With `GITHUB_WORKFLOW=Release` it failed. The temporary test is removed. ## Risks - A retry can hide a flaky test. The annotation and the summary row reduce this risk. - A test that fails for a real reason now runs twice in PR CI, so a failed run takes slightly longer. - `pr.yml` loads `pr-trusted.yml` from master, so the `CI_VITEST_RETRY` switch works only after merge. The retry itself is in the scripts and starts with this pull request. - Low risk otherwise: release and master verification are not changed. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`) in Claude Code, with tool use and code execution. A Claude sub-agent wrote the change, and the parent session reviewed it. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 5.5 <[email protected]>
142 lines
12 KiB
JSON
142 lines
12 KiB
JSON
{
|
|
"name": "paperclip",
|
|
"private": true,
|
|
"type": "module",
|
|
"scripts": {
|
|
"perf:workspace-admission": "node --import ./server/node_modules/tsx/dist/loader.mjs server/scripts/benchmark-workspace-admission.ts",
|
|
"postinstall": "node scripts/link-plugin-dev-sdk.mjs",
|
|
"preflight:workspace-links": "node cli/node_modules/tsx/dist/cli.mjs scripts/ensure-workspace-package-links.ts",
|
|
"dev": "pnpm --filter @paperclipai/server exec tsx ../scripts/dev-runner.ts watch",
|
|
"dev:watch": "pnpm --filter @paperclipai/server exec tsx ../scripts/dev-runner.ts watch",
|
|
"dev:once": "pnpm --filter @paperclipai/server exec tsx ../scripts/dev-runner.ts dev",
|
|
"dev:list": "pnpm --filter @paperclipai/server exec tsx ../scripts/dev-service.ts list",
|
|
"dev:stop": "pnpm --filter @paperclipai/server exec tsx ../scripts/dev-service.ts stop",
|
|
"dev:server": "pnpm --filter @paperclipai/server dev",
|
|
"dev:ui": "pnpm --filter @paperclipai/ui dev",
|
|
"dev:mobile": "pnpm --filter @paperclipai/ui build && pnpm --filter @paperclipai/ui preview",
|
|
"dev:both": "node scripts/dev-both.mjs",
|
|
"prototype:dot": "pnpm --filter @paperclipai/server exec tsx scripts/dot-runner-demo.ts",
|
|
"prototype:dot:live": "pnpm --filter @paperclipai/server exec tsx scripts/dot-runner-live.ts",
|
|
"storybook": "pnpm --filter @paperclipai/ui storybook",
|
|
"build-storybook": "pnpm --filter @paperclipai/ui build-storybook",
|
|
"build": "pnpm run preflight:workspace-links && pnpm -r build",
|
|
"build:feature-catalog": "node cli/node_modules/tsx/dist/cli.mjs scripts/generate-feature-catalog.ts",
|
|
"typecheck": "pnpm run preflight:workspace-links && pnpm -r typecheck",
|
|
"typecheck:build-gaps": "pnpm run preflight:workspace-links && pnpm --filter @paperclipai/plugin-sdk ensure-build-deps && pnpm --filter @paperclipai/server build && node scripts/run-typecheck-build-gaps.mjs",
|
|
"test": "pnpm run test:run",
|
|
"test:watch": "pnpm run preflight:workspace-links && vitest",
|
|
"test:run": "pnpm run preflight:workspace-links && node scripts/run-vitest-stable.mjs",
|
|
"test:runner-acceptance": "vitest run --config tests/runner-acceptance/vitest.config.ts",
|
|
"test:runner-acceptance:typecheck": "tsc -p tests/runner-acceptance/tsconfig.json",
|
|
"test:run:general": "pnpm run preflight:workspace-links && pnpm --filter @paperclipai/plugin-sdk ensure-build-deps && node scripts/run-vitest-stable.mjs --mode general",
|
|
"test:run:serialized": "pnpm run preflight:workspace-links && pnpm --filter @paperclipai/plugin-sdk ensure-build-deps && node scripts/run-vitest-stable.mjs --mode serialized",
|
|
"db:generate": "pnpm --filter @paperclipai/db generate",
|
|
"db:migrate": "pnpm --filter @paperclipai/db migrate",
|
|
"issue-references:backfill": "pnpm run preflight:workspace-links && tsx scripts/backfill-issue-reference-mentions.ts",
|
|
"secrets:migrate-inline-env": "tsx scripts/migrate-inline-env-secrets.ts",
|
|
"db:backup": "./scripts/backup-db.sh",
|
|
"paperclipai": "node cli/node_modules/tsx/dist/cli.mjs cli/src/index.ts",
|
|
"build:npm": "./scripts/build-npm.sh",
|
|
"release": "./scripts/release.sh",
|
|
"release:canary": "./scripts/release.sh canary",
|
|
"release:stable": "./scripts/release.sh stable",
|
|
"release:github": "./scripts/create-github-release.sh",
|
|
"release:rollback": "./scripts/rollback-latest.sh",
|
|
"release:bootstrap-package": "node scripts/bootstrap-npm-package.mjs",
|
|
"check:tokens": "node scripts/check-forbidden-tokens.mjs",
|
|
"check:token-gates": "node scripts/check-token-gates.mjs && node scripts/sync-agent-palette-tokens.mjs --check && node scripts/sync-cliplab-character.mjs --check",
|
|
"check:node-version": "node scripts/check-node-version-policy.mjs",
|
|
"check:no-git-push": "node scripts/check-no-git-push.mjs",
|
|
"check:module-boundaries": "node scripts/check-module-boundaries.mjs",
|
|
"test:check-no-git-push": "node --test scripts/check-no-git-push.test.mjs",
|
|
"test:install-sh-docker": "./scripts/test-install-sh-docker.sh",
|
|
"test:hermes-gateway-smoke": "node --test scripts/smoke/hermes-gateway-smoke.test.mjs",
|
|
"docs:dev": "cd docs && npx mintlify dev",
|
|
"smoke:hermes-gateway-join": "./scripts/smoke/hermes-gateway-join.sh",
|
|
"smoke:hermes-gateway-e2e": "./scripts/smoke/hermes-gateway-e2e.sh",
|
|
"smoke:openclaw-join": "./scripts/smoke/openclaw-join.sh",
|
|
"smoke:openclaw-docker-ui": "./scripts/smoke/openclaw-docker-ui.sh",
|
|
"smoke:openclaw-sse-standalone": "./scripts/smoke/openclaw-sse-standalone.sh",
|
|
"smoke:mcp-fixtures": "node scripts/smoke/mcp-fixture-harness.mjs",
|
|
"smoke:notion-generic-live": "node scripts/smoke/notion-generic-live.mjs",
|
|
"smoke:posthog-live": "node scripts/smoke/posthog-live.mjs",
|
|
"smoke:terminal-bench-loop-skill": "node scripts/smoke/terminal-bench-loop-skill-smoke.mjs",
|
|
"test:release-registry": "node scripts/ci-profile/node-test.mjs scripts/ci-profile/reporters.test.mjs scripts/ci/retry-reporter.test.mjs scripts/__tests__/runner-vitest-package-lanes.test.mjs scripts/__tests__/canary-startup-playwright.test.mjs scripts/__tests__/cursor-public-install-sandbox.test.mjs scripts/__tests__/grok-public-install-sandbox.test.mjs scripts/__tests__/public-install-closure.test.mjs scripts/__tests__/public-install-registry.test.mjs scripts/__tests__/prefetch-child.test.mjs scripts/__tests__/stage-bundled-packages.test.mjs scripts/__tests__/pack-staged-packages.test.mjs scripts/__tests__/retain-runner-qualification-packages.test.mjs scripts/__tests__/runner-release-crate-cache.test.mjs scripts/__tests__/runner-debug-crate-cache.test.mjs scripts/__tests__/agent-build-cache.test.mjs scripts/__tests__/vitest-worker-isolation.test.mjs scripts/verify-release-registry-state.test.mjs scripts/release-package-map.test.mjs scripts/check-release-package-bootstrap.test.mjs scripts/check-no-git-push.test.mjs scripts/release-lib.test.mjs scripts/release-registry-versions.test.mjs scripts/link-plugin-dev-sdk.test.js scripts/acpx-patch-packaging.test.mjs scripts/service-onboard-smoke.test.mjs scripts/docker-onboard-smoke.test.mjs scripts/preview-artifacts.test.mjs scripts/cloud-migrator-artifacts.test.mjs scripts/typecheck-test-programs.test.mjs scripts/__tests__/local-verify.test.mjs scripts/test-sandbox-providers.test.mjs",
|
|
"storybook-visual:baseline": "node scripts/storybook-visual-baseline.mjs",
|
|
"test:storybook-visual": "node scripts/storybook-visual-baseline.mjs download && node scripts/storybook-visual-baseline.mjs verify && pnpm build-storybook && npx playwright test --config tests/storybook-visual/playwright.config.ts",
|
|
"test:storybook-visual:update": "node scripts/storybook-visual-baseline.mjs download && pnpm build-storybook && npx playwright test --config tests/storybook-visual/playwright.config.ts --update-snapshots && node scripts/storybook-visual-baseline.mjs pack",
|
|
"test:e2e": "node scripts/ci-profile/run.mjs playwright npx playwright test --config tests/e2e/playwright.config.ts",
|
|
"test:e2e:runner": "node cli/node_modules/tsx/dist/cli.mjs tests/runner-e2e/launch.ts",
|
|
"test:e2e:runner:image-id": "node cli/node_modules/tsx/dist/cli.mjs tests/runner-e2e/daytona-image-content.ts",
|
|
"test:e2e:runner:judge-first-task": "node cli/node_modules/tsx/dist/cli.mjs tests/runner-e2e/first-task-judge.ts",
|
|
"test:e2e:runner:dashboard": "node cli/node_modules/tsx/dist/cli.mjs tests/runner-e2e/dashboard-regenerate.ts",
|
|
"test:e2e:runner:models:update": "node cli/node_modules/tsx/dist/cli.mjs tests/runner-e2e/openrouter-models-update.ts",
|
|
"test:e2e:runner:history:publish": "node cli/node_modules/tsx/dist/cli.mjs tests/runner-e2e/history-publish.ts",
|
|
"test:runner-recovery": "vitest run server/src/services/native-runtime/native-replacement-evidence.test.ts server/src/services/native-runtime/stopped-codex-turn.test.ts server/src/services/native-runtime/native-safe-replacement.test.ts",
|
|
"test:slack-connector": "node server/src/services/connectors/slack/evals/run-regressions.mjs",
|
|
"test:e2e:browser-context": "playwright test --config tests/e2e/playwright-company-context.config.ts",
|
|
"test:e2e:runner:browser-support": "playwright test --config tests/runner-e2e/playwright-support.config.ts",
|
|
"test:e2e:runner:unit": "node tests/runner-e2e/run-unit.mjs",
|
|
"test:e2e:runner:stock-harness": "node tests/runner-e2e/stock-harness-checks.mjs",
|
|
"test:e2e:runner:typecheck": "node scripts/typecheck-test-projects.mjs tests/runner-e2e",
|
|
"test:e2e:runner:report": "node cli/node_modules/tsx/dist/cli.mjs tests/runner-e2e/report.ts",
|
|
"test:runner-workflow-evals": "pnpm --filter @paperclipai/paperclip-eval-kernel build && pnpm --filter @paperclipai/paperclip-runner test:runner-workflow-evals",
|
|
"test:e2e:mcp-user-stories": "node scripts/e2e-mcp-user-stories.mjs",
|
|
"test:e2e:connection-intents": "npx playwright test --config tests/e2e/playwright.config.ts tests/e2e/connection-intents.spec.ts",
|
|
"test:e2e:headed": "npx playwright test --config tests/e2e/playwright.config.ts --headed",
|
|
"test:e2e:multiuser-authenticated": "npx playwright test --config tests/e2e/playwright-multiuser-authenticated.config.ts",
|
|
"evals:smoke": "cd evals/promptfoo && npx [email protected] eval",
|
|
"test:release-smoke": "npx playwright test --config tests/release-smoke/playwright.config.ts",
|
|
"test:release-smoke:headed": "npx playwright test --config tests/release-smoke/playwright.config.ts --headed",
|
|
"test:canary-onboarding-smoke": "npx playwright test --config tests/canary-onboarding/playwright.config.ts",
|
|
"metrics:paperclip-commits": "tsx scripts/paperclip-commit-metrics.ts",
|
|
"perf:issue-chat-long-thread": "node scripts/measure-issue-chat-long-thread.mjs",
|
|
"connections:ingest-app-definitions": "node scripts/ingest-app-definitions.mjs",
|
|
"test:lifecycle-baseline": "node tests/lifecycle-baseline/run.mjs",
|
|
"test:lifecycle-baseline:support": "node --test tests/lifecycle-baseline/report.test.mjs",
|
|
"test:lifecycle-baseline:typecheck": "node scripts/typecheck-test-projects.mjs tests/lifecycle-baseline",
|
|
"verify": "node --import ./cli/node_modules/tsx/dist/loader.mjs scripts/ensure-workspace-package-links.ts && node scripts/local-verify.mjs"
|
|
},
|
|
"devDependencies": {
|
|
"@playwright/test": "^1.62.1",
|
|
"agentmail": "^0.5.31",
|
|
"cross-env": "^10.1.0",
|
|
"esbuild": "^0.28.2",
|
|
"oxc-parser": "0.127.0",
|
|
"typescript": "^7.0.2",
|
|
"vitest": "^5.0.3",
|
|
"yaml": "^2.8.1",
|
|
"turbo": "2.11.10"
|
|
},
|
|
"engines": {
|
|
"node": ">=24.11.0"
|
|
},
|
|
"packageManager": "[email protected]",
|
|
"pnpm": {
|
|
"patchedDependencies": {
|
|
"[email protected]": "patches/[email protected]",
|
|
"[email protected]": "patches/[email protected]",
|
|
"[email protected]": "patches/[email protected]",
|
|
"@agentclientprotocol/[email protected]": "patches/@[email protected]",
|
|
"@agentclientprotocol/[email protected]": "patches/@[email protected]",
|
|
"@chat-adapter/[email protected]": "patches/@[email protected]",
|
|
"@chat-adapter/[email protected]": "patches/@[email protected]",
|
|
"@chat-adapter/[email protected]": "patches/@[email protected]",
|
|
"@chat-adapter/[email protected]": "patches/@[email protected]",
|
|
"@chat-adapter/[email protected]": "patches/@[email protected]",
|
|
"@discordjs/[email protected]": "patches/@[email protected]",
|
|
"[email protected]": "patches/[email protected]"
|
|
},
|
|
"overrides": {
|
|
"@agentclientprotocol/[email protected]>@openai/codex": "0.160.0",
|
|
"@agentclientprotocol/[email protected]>@anthropic-ai/claude-agent-sdk": "0.3.286",
|
|
"rollup": ">=4.59.0",
|
|
"react": "^19.2.8",
|
|
"react-dom": "^19.2.8",
|
|
"@codemirror/state": "^6.7.2",
|
|
"@codemirror/view": "^6.43.11",
|
|
"@lezer/common": "^1.5.2"
|
|
}
|
|
}
|
|
}
|