## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Native runs save session, checkpoint, lease, and launch state, and recovery resumes that state later > - Deploys and restarts often land between the save and the resume, so one release resumes state that another release saved > - On 2026-11-06/07, staging deployed four times. Most of 182 failed runs were saved by one release and resumed by another (#19462, #19659, #19755) > - These refusals showed as the generic retryable `native_session_interrupted`. Retries repeated them, and recovery replaced a thread (now prevented by #19733). Remote-only code also shipped without a test that applied Daytona confinement (#19743) > - This pull request makes upgrade safety a required part of how we design, fix, and review run and recovery code > - The benefit is that authors, agents, and Greptile check the release cutover before a change ships, not after a deploy breaks saved runs ## Linked Issues or Issue Description Refs #19462, Refs #19516, Refs #19622, Refs #19659, Refs #19733, Refs #19743, Refs #19755 **Issue type** Missing contributor guidance for run and recovery code. **Where is the issue?** `.agents/skills/fix-failed-paperclip-run/`, `rules/`, `AGENTS.md`, and `doc/execution-semantics.md`. **What's wrong?** No rule or skill step asks how state saved by the previous release is handled. Fixes added one error code at a time. Saved-identity refusals used a generic retryable code. **Suggested fix** Add a scoped repository rule and additive skill and contract text that require a previous-release story, permanent named refusal codes, and cutover tests. ## What Changed - Add `rules/upgrade-safe-run-state.mdc`. It covers native runtime, recovery, session preparation, runner, ACPX engine, and sandbox image paths, and the `packages/db` schema for native run, checkpoint, session, and run identity state. It requires authors to state how previous-release state is handled. It requires a permanent, named code for unknown or changed saved identity, a class-level fix, an old-saved/new-resumed regression, a deploy-lands-mid-run case where relevant, and a remote-confinement test for sandbox paths. - Register the rule in `.greptile/files.json` with the same scope as its `globs`. - Link the rule from the "Scoped repository rules" section of `AGENTS.md`. - `fix-failed-paperclip-run` skill: step 1 asks if a deploy or restart separated save from resume, by comparing served revisions. Step 2 asks this early. Step 4 adds a "Persisted run identity across releases" safety row and an upgrade-safety paragraph. - `references/failure-patterns.md`: add "Saved run state meets a different release". - `doc/execution-semantics.md`: add §9.4 "Release compatibility of saved run state". ## Verification - Docs, skills, and rules only. No product code changes. - Confirm that each rule glob matches real paths, for example `server/src/services/native-runtime/`, `server/src/services/recovery/`, `server/src/services/heartbeat/session-preparation.ts`, and `packages/adapter-utils/src/acpx-engine/`. - Confirm that `.greptile/files.json` parses and that its new `scope` array equals the rule's `globs`. - `rules/README.md` names no lint check for rules. I did not edit maps, so `node scripts/check-maps.mjs` does not apply. ## Risks - Low risk. The change adds guidance only. - The rule scope includes all of `packages/paperclip-runner/**`. Greptile can apply the rule to runner docs or UI files. The rule's exception section says that changes with no persisted shape or identity need no cutover test. ## Model Used - Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, with tool use and code execution in Claude Code. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <[email protected]>
16 KiB
AGENTS.md
Guidance for human and AI contributors working in this repository.
1. Purpose
Paperclip is a control plane for AI-agent companies.
The current implementation target is V1 and is defined in doc/SPEC-implementation.md.
2. Read This First
Before making changes, read in this order:
doc/GOAL.mddoc/PRODUCT.mddoc/SPEC-implementation.mddoc/DEVELOPING.mddoc/DATABASE.md
doc/SPEC.md is long-horizon product context.
doc/SPEC-implementation.md is the concrete V1 build contract.
When adding or changing an Apps catalog connection, also follow
doc/connections/CONNECTOR-PLAYBOOK.md. It is the canonical connection
authoring runbook for provider research, supported transport/auth patterns,
credential handling, branding, implementation, testing, live proof, and PR
submission.
Scoped repository rules
Before editing or reviewing files, read the matching rules in rules/*.mdc.
Each rule has Markdown content and YAML frontmatter: globs contains
comma-separated, repository-relative file patterns, and alwaysApply: true
means the rule applies throughout the repository. Read all rules whose scope
matches the affected files. These rules supplement this file and any applicable
nested AGENTS.md instructions.
For server/src/services/chat-channels.ts and everything under
server/src/services/chat-channels/, read
rules/chat-channel-platform-boundaries.mdc.
For server/src/services/heartbeat.ts, server/src/services/heartbeat/,
heartbeat-prefixed sibling services, and heartbeat regression tests, read
rules/heartbeat-execution-boundaries.mdc.
For ACPX provider profiles, their release attestations and pins, the provider-pack
contract registry, and ACPX driver and provider code under
packages/paperclip-runner/, read
rules/acpx-profile-identity.mdc.
For native runtime, recovery, session preparation, runner, ACPX engine, and
provider-pack or sandbox image code that persists or resumes run state, and for
the native run, checkpoint, and session schema in packages/db, read
rules/upgrade-safe-run-state.mdc.
See rules/README.md for rule authoring and Greptile setup.
3. Repo Map
server/: Express REST API and orchestration servicesui/: React + Vite board UIpackages/db/: Drizzle schema, migrations, DB clientspackages/shared/: shared types, constants, validators, API path constantspackages/adapters/: agent adapter implementations (Claude, Codex, Cursor, etc.)packages/adapter-utils/: shared adapter utilitiespackages/plugins/: plugin system packagespackages/skills-catalog/: app-shipped skills catalog (@paperclipai/skills-catalog)packages/teams-catalog/: app-shipped teams catalog (@paperclipai/teams-catalog)cli/:paperclipaiCLI package (published bin, agent-facing commands)skills/: Paperclip runtime/operational skills (not part of the app catalog)doc/: operational and product docs
4. Dev Setup (Auto DB)
Use embedded PGlite in dev by leaving DATABASE_URL unset.
pnpm install
pnpm dev
This starts:
- API:
http://localhost:3100 - UI:
http://localhost:3100(served by API server in dev middleware mode)
Quick checks:
curl http://localhost:3100/api/health
curl http://localhost:3100/api/companies
Reset local dev DB:
rm -rf data/pglite
pnpm dev
5. Core Engineering Rules
- Keep changes company-scoped. Every domain entity should be scoped to a company and company boundaries must be enforced in routes/services.
Explicit exception: announcement dismissals are instance-wide user preferences, keyed by user and announcement so they persist across companies. Their audit context must still validate company membership. The announcement publication-ID registry is instance-level feed metadata; it contains no company or user data. Personal workspace presentation preferences are also instance-wide by user, with validated company membership supplying audit context. They never grant execution authority, change company capabilities or agent configuration, or rewrite canonical work records. Optional conversation guidance must retain the same action permissions.
- Keep contracts synchronized. If you change schema/API behavior, update all impacted layers:
packages/dbschema and exportspackages/sharedtypes/constants/validatorsserverroutes/servicesuiAPI clients and pages
- Preserve control-plane invariants.
- Single-assignee task model
- Atomic issue checkout semantics
- Approval gates for governed actions
- Budget hard-stop auto-pause behavior
- Activity logging for mutating actions
-
Do not replace strategic docs wholesale unless asked. Prefer additive updates. Keep
doc/SPEC.mdanddoc/SPEC-implementation.mdaligned. -
Keep repo plan docs dated and centralized. When you are creating a plan file in the repository itself, new plan documents belong in
doc/plans/and should useYYYY-MM-DD-slug.mdfilenames. This does not replace Paperclip issue planning: if a Paperclip issue asks for a plan, update the issueplandocument per thepaperclipskill instead of creating a repo markdown file. -
Attach inspectable generated artifacts. When your task produces a user-inspectable deliverable file, follow the Paperclip skill's "Generated Artifacts and Work Products" workflow before final disposition. In this repo, prefer the self-contained skill helper at
skills/paperclip/scripts/paperclip-upload-artifact.shso the file is available through the Paperclip API, create/update an artifact work product when the file is the deliverable, link the uploaded artifact in the final issue comment, and then set status. Do not rely on local filesystem paths as the only access path. If an important file intentionally remains workspace-only, create/update a work product withmetadata.resourceRef.kind: "workspace_file"and a workspace-relative path, then name that work product and path in the final comment. Treat browse/search as a fallback for recovering workspace files, not the preferred deliverable path. Seedoc/AGENT-ARTIFACTS.mdfor details and.mp4/.webmexamples. -
Name the three data paths correctly. This repo has three separate data paths. Do not confuse them. Match a change to a path by its file path, not by the word "observability" or "telemetry" alone.
- Telemetry is the Paperclip first-party event system. It is opt-out and it sends data to a Paperclip endpoint by default. Its paths are:
packages/shared/src/telemetry/- the generated contract
packages/shared/src/telemetry/generated/paperclip-telemetry.ts - each caller of
packages/shared/src/telemetry/events.tsorpackages/shared/src/telemetry/client.ts
- Observability is the OpenTelemetry trace path. An operator must set an OTLP endpoint. Until an operator sets the endpoint, the tracer is a no-operation. Its paths are:
server/src/instrumentation.tsdoc/observability.mdpackages/adapter-utils/src/duplex-observability.tsserver/src/services/duplex-observability-recorder.ts- the span attributes in
packages/adapter-utils/src/acpx-engine/startup-timing.ts
- The run log holds rows in the local
heartbeat_run_eventstable. The data stays in the instance database. Its paths are:doc/run-log-events.mdpackages/db/src/schema/heartbeat_run_events.ts- the append path
appendRunEventinserver/src/services/heartbeat.ts
Apply a review level that matches the path:
- Telemetry change (strict review). The author updates the generated contract first. The author updates
packages/shared/src/telemetry/README.mdin the same pull request. The author requests a privacy review. Reason: a Telemetry event goes to a Paperclip endpoint by default, so a mistake sends data immediately. - Observability change (lighter review). The operator endpoint gate stays in place. The no-operation behaviour stays when no endpoint is set. A privacy review is not necessary while the change stays inside the closed span-attribute allowlist.
- Run-log change (no extra review). A run-log change needs neither review level above, because the data stays in the instance database.
Exclusion. The word "observability" in a file such as server/src/services/recovery-observability.ts names a different concept. Apply this rule by path, not by word match.
6. Database Change Workflow
When changing data model:
- Edit
packages/db/src/schema/*.ts - Ensure new tables are exported from
packages/db/src/schema/index.ts - Generate migration:
pnpm db:generate
- Validate compile:
pnpm -r typecheck
Notes:
packages/db/drizzle.config.tsreads compiled schema fromdist/schema/*.jspnpm db:generatecompilespackages/dbfirst
7. Verification Before Hand-off
maps/feature-map/README.md is an optional reference for user entry points, verification recipes, and coverage gaps. The map records coverage scope, not proof that a live journey passed.
The sibling agent access map records APIs,
agent entry points and permission boundaries using the same capability IDs.
Follow the feature-map scope rule: additions require
substantial new features. Routine fixes, polish, refactors, and test changes do
not require map updates. Follow the map contract for minimal
corrections when existing entries become materially inaccurate, and run
node scripts/check-maps.mjs after map edits.
Its structural checks do not prove live behavior or authorization.
Storybook has a downstream feature catalog.
Use maintain-storybook for story/fixture drift and review-storybook for browser
evidence. Keep the feature-map skills independent of this catalog. After story or recipe changes, update the affected source annotations or
recipe and run node scripts/storybook-catalog.mjs. Indexes are generated and
ignored; never commit aggregate catalogs, refresh review fingerprints, or append
a shared review journal. Use Git diffs and keep review evidence in the PR or issue.
Structural checks are not production acceptance.
Prepare a new or updated worktree with node scripts/agent-preflight.mjs instead of running
pnpm install, the Runner build and build:rust by hand. It skips unchanged
steps and reuses other worktrees' Runner builds, debug binaries and third-party
crates from a content-keyed cache in ~/.cache/paperclip/agent-build. Do not
share CARGO_TARGET_DIR between worktrees. See "Agent worktree preflight" in
doc/DEVELOPING.md.
Default local/agent verification path:
pnpm verify
This selects affected Vitest files with the same conservative dependency policy
as CI. It includes committed, staged, unstaged, and untracked changes against the
merge base with local origin/master (or master). Use --base <ref> for another
base and --explain to inspect the selection. Passing checks with complete input
graphs can be reused within this worktree. Opaque integration checks run fresh.
Source, test, fixture, configuration, dependency, environment, and toolchain changes
invalidate matching results. Failed or interrupted attempts are never passes.
Exit 2 means the change needs broader verification: run focused checks and let
required CI supply that coverage, or explicitly use pnpm verify --full for the
full local Vitest suite. Do not automatically follow exit 2 with a full local run.
Use --no-cache to request fresh execution without reading or saving passes.
pnpm test and pnpm test:run retain their full-suite behavior.
Browser suites stay opt-in:
pnpm test:e2e
pnpm test:release-smoke
Run the browser suites only when your change touches them or when you are explicitly verifying CI/release flows.
For normal issue work, run the smallest relevant verification first. Do not default to repo-wide typecheck/build/test on every heartbeat when a narrower check is enough to prove the change.
For a PR-ready handoff, run the relevant local checks and require the applicable CI typecheck, build, and test checks to pass for the final tested revision. Record their scope, revision, environment, and result links. Fresh CI can supply broad coverage; do not duplicate a full local run merely to prepare a PR, resume a chat, or repeat unchanged passing verification. A selected local pass does not replace required CI, release qualification, browser acceptance, or a local/platform-specific regression. After edits or a rebase, rerun affected checks and invalidate evidence whose inputs or required coverage changed.
Use broad local checks when they resolve an actual remaining gap, such as a local platform fault or explicit full-suite qualification:
pnpm -r typecheck
pnpm test:run
pnpm build
If memory is limited, use node scripts/typecheck-low-memory.mjs instead of pnpm -r typecheck. See doc/DEVELOPING.md.
If anything cannot be run, explicitly report what was not run and why.
8. API and Auth Expectations
- Base path:
/api - Board access is treated as full-control operator context
- Agent access uses bearer API keys (
agent_api_keys), hashed at rest - Agent keys must not access other companies
When adding endpoints:
- apply company access checks
- enforce actor permissions (board vs agent)
- write activity log entries for mutations
- return consistent HTTP errors (
400/401/403/404/409/422/500)
9. UI Expectations
- Keep routes and nav aligned with available API surface
- Use company selection context for company-scoped pages
- Surface failures clearly; do not silently ignore API errors
- Form and wizard footers: keep Save & exit (or Cancel/Back) left and the primary action right in the same vertically aligned row. Each step owns the entire footer; never append Save & exit as a separate row. See
DESIGN.md.
10. Pull Request Requirements
When creating a pull request (via gh pr create or any other method), you must read and fill in every section of .github/PULL_REQUEST_TEMPLATE.md. Do not craft ad-hoc PR bodies — use the template as the structure for your PR description. Required sections:
- Thinking Path — trace reasoning from project context to this change (see
CONTRIBUTING.mdfor examples) - What Changed — bullet list of concrete changes
- Verification — how a reviewer can confirm it works
- Risks — what could go wrong
- Model Used — the AI model that produced or assisted with the change (provider, exact model ID, context window, capabilities). Write "None — human-authored" if no AI was used.
- Checklist — all items checked
11. Definition of Done
A change is done when all are true:
- Behavior matches
doc/SPEC-implementation.md - Relevant local verification and required CI typecheck, test, and build checks pass for the final revision, with any acceptance or platform-specific gaps resolved
- Contracts are synced across db/shared/server/ui
- Docs updated when behavior or commands change
- PR description follows the PR template with all sections filled in (including Model Used)
Design system
DESIGN.md at the repo root is the source of truth for UI design decisions. The token-only rule applies to all ui/ changes: every color, spacing, radius, type, shadow, and motion value in ui/src/components/** and ui/src/pages/** comes from the token layer in ui/src/index.css — no hex, raw px, arbitrary Tailwind bracket values, or raw font-size/fontSize declarations in components, outside the documented allowlist in ui/src/index.css. Run pnpm check:token-gates (scripts/check-token-gates.mjs) before committing UI changes — it fails on any violation not covered by that allowlist.