Files
DottaandClaude Opus 5.5 0f5ce5050e docs(plans): plan durable ownership and reconciliation for restart safety (#19831)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Agent runs must survive a control-plane restart: they must continue
on their own, keep their work, and offer a Retry that works
> - The 2026-11-06/07 restart chaos campaign found about 40 restart
failures and merged about 69 fixes (#19469 through #19807)
> - Most of these failures have one shape: a restart stops a process
between two durable writes, and each subsystem then decides on its own
whether the leftover record is safe to release
> - Each fix closed one crash window, and about 12 fixes rebuilt the
same "this owner is dead" proof
> - This pull request adds a plan for one shared ownership record,
write-ahead intents, one reconciler, and an on-demand crash-point suite
> - The plan also makes time to first token and agent boot hard
requirements, so restart safety never makes turns slow
> - The benefit is that we fix the whole class of failure once, and we
stop fixing one window at a time

## Linked Issues or Issue Description

Refs #19696, #19713, #19717, #19724, #19741, #19747, #19784, #19807

**What existing behavior does this improve?**
Recovery after a control-plane restart. Today each subsystem keeps its
own in-progress records and its own release rules.

**Subsystem affected**
Native runtime, runner, environments and sandboxes, workspaces,
accounting and budgets, and the wake queue.

**Current behavior**
A restart can leave a lease, a fence, a claim, a workspace operation, a
receipt, or a deferred wake that belongs to a dead process. If no
subsystem can prove that the owner is dead, the system fails closed. The
task then needs a Retry, the company holds new work, or a run stays
`running`. About 40 server files read a controller boot id, and about 25
files check process liveness in different ways.

**Proposed behavior**
One owner stamp and one liveness proof for every in-progress record.
Intents are written before outside effects. One reconciler brings every
record to a terminal state. Admission never waits on cleanup. Accepted
work is delivered exactly once. Time to first token and agent boot are
measured and stay within budgets.

**Reason and benefit**
Fewer restart failures, less duplicate code, and a fast first response
after a restart.

## What Changed

- Added `doc/plans/2026-11-07-durable-ownership-and-reconciliation.md`.
The plan contains:
- the five failure classes from the campaign, with example IDs and PR
counts
  - eight invariants
- a performance section with time-to-first-token, boot, and
restart-latency metrics and rules
  - the design (sections A–G)
  - an on-demand crash-point suite
  - six phases
  - risks
  - the decisions that Dotta made on 2026-11-07

## Verification

- This change adds documentation only. No code changes.
- Read the plan. Check that each failure class cites real merged PRs.
- Check the code claims:
- `git grep controller-process-identity -- server/src` shows three
users.
- `git grep task.time_to_first_token -- server/src` shows the existing
metric.

## Risks

- Low risk. The change adds one plan document and changes no code.
- The plan proposes later behavior changes. The largest is the removal
of the company-wide accounting gate. Each later phase will come as its
own pull request with tests.

## Model Used

- Claude Opus 5.5 (`claude-opus-5-5`), Anthropic, used in Claude Code
with tool use (shell, git, GitHub CLI). The model coordinated the chaos
campaign and wrote the plan from its evidence.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <[email protected]>
2026-11-07 08:36:05 -06:00
..