mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-07 06:16:06 -10:00
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Automatic skill sources check upstream every five minutes. > - The domain deadline contract can hand future work to a durable wake alarm. > - A saved deadline or a worker unit test does not prove that a stopped tenant resumes the real producer. > - This pull request checks the default cadence through the actual app and a matching service checkout. > - It checks two cold starts and three distinct persisted skill revisions with real clocks. > - The benefit is repeatable local evidence without changing production timing or authority. ## Linked Issues or Issue Description Refs: #19001, #17008, #17923. This is a test and documentation follow-up to #19001. #19001 is now merged; this PR targets master. Feature code remains in that merged foundation. #17008 covers the earlier source worker change. #17923 covers separate skill-cache retention; this PR does not change that behavior. A scoped search of public PRs and issues found no duplicate default-cadence cold-cycle acceptance test. No matching public issue was found. The enhancement fields follow. **Pre-submission checklist** This improves checks for an existing feature. It adds no product endpoint or execution authority. The public related-work search is complete. **What existing behavior does this improve?** Verification of automatic skill-source updates across a durable idle handoff and cold restart. **Subsystem affected** `server/` test fixtures and development documentation. **Current behavior** Domain tests cover claims, access and future deadlines. They do not prove the full default-300-second path from ordinary source attachment through signed tenant reports, durable alarms, stopped compute and a changed persisted skill. **Proposed behavior** An opt-in launcher checks bootstrap first, then two full real-clock cycles. Ordinary CI can run the transport controls without the host fixture. The launcher-specific worker assertion runs only when prewire is requested, and fails closed in that lane. **Reason and benefit** The test can detect an early refresh, a lost deadline, a stale proof, an ACK after physical stop, a failed cold restart or a producer that does not publish changed contents. **Breaking changes** None to product APIs, schema, cadence, authorization, claims or retry policy. Existing maintenance-helper defaults stay unchanged. **Additional context** The opt-in host flow needs an authorized matching service checkout, its normal compiled acceptance bridge, Node 22 and two owned PostgreSQL test databases. The standalone controls do not need that service or PostgreSQL. Roadmap review places this under existing Skills Manager and Cloud deployment verification, not a new core feature. ## What Changed - Add an opt-in default-cadence acceptance test and checked-in launcher. - Reuse real server startup, managed tenant authority and the ordinary project attachment route. Control only the remote Git snapshot boundary; keep actual authorization, scanning, claims and publication. - Use actual signed tenant routes and the matching service's gate, sleep controller and PostgreSQL wake-alarm store. Stop and restart real app children through a controlled local provider. - Preload the Node transport guard before coordinator, worker and child module work. Check worker-local and child PID, thread and source-hash receipts. Keep caught unexpected transport attempts fatal. - Require both persisted refresh times and actual upstream handler timestamps to be at or after each original deadline. - Join physical callbacks during bridge close, stop a late wake-created child, and preserve the original cleanup budget. Add a real IPC/HTTP-child counterfactual. - Accept valid owned HTTP options paths without allowing Unix sockets or unknown origins. Remove the unused App helper export. - Add finite transport and launcher controls. Defer only the launcher-specific parent assertion in vanilla CI. Keep all subprocess controls default-runnable. - Add optional child entry, preload and bounded pool settings to the existing test helper. Preserve its defaults and the real observer test body. - Remove added explanatory comments under the contribution policy. Preserve executable syntax, directives and literals. - Document pinned setup, fresh owned databases, bootstrap and full two-cycle commands in `doc/DEVELOPING.md`. The diff against the intended base contains nine test/helper/launcher files and this one document. It adds no production feature code or Cloud source to the App repository. ## Verification The latest two-file CI repair preserves both original failed jobs. A newer master workflow introduced two required step inputs absent from its extracted-step test environment; the faithful false/false inputs now pass all five existing target-authorization cases, after an actual missing-input counterfactual failed. A controlled socket case also reproduces legal reuse of the released runner port; the fixture now holds that reservation until the preview binds, releases it in finally, and asserts distinct ports. All original retry/result/accounting assertions and timeouts stay unchanged. The original Linux listener owner remains unknown; the control proves this fixture collision mechanism, not that original attribution. Fresh current-head full CI and complete reviews are pending. Current merge-integration candidate: `f6f686103b7bfd0a4ab83085634c622af3896ecb`, a normal merge after #19001 landed, including actual master `1c9767d323cf872a266962b9e6b4bd5cbcb57f1c`. All nine acceptance code files are byte-identical to148a82e5; both parents' documentation additions survive. Every other path equals actual master. The only conflict was the old foundation's startup block; current master was retained in full. Fresh exact-head CI and complete reviews are pending. The local measurements below belong to their explicitly named revisions; no fresh full614-second run is claimed. Current candidate: `148a82e54c502759785407357001c6b50c1439dd`, normally carrying the current base `8c912e1ba4fc66397a7341ade59373af8075068f`. All nine acceptance code blobs remain identical to qualified repair `e092746f4da181c8414c63be71474619a84b97e7`; documentation retains both parents. The matching built service for the fresh bootstrap is `389878364b42dd129835d263f91882c5774d5123`, using Node 22.23.4. The App uses Node 24.22.2 and pnpm 9.15.4. Use the exact controlled setup and commands in [doc/DEVELOPING.md](doc/DEVELOPING.md#controlled-default-cadence-sleep-acceptance). Fresh owned databases and the matching compiled service are prerequisites; the launcher does not create a cluster or build dependencies. - **Current affected semantic check passed**, using actual native TypeScript 7 and the current generated CI options/references (0.640 seconds). Normal declaration references passed in 11.726 seconds, with source-matched SDK/shared/kernel/Runner prerequisites and the frozen current lockfile. - **Current guarded controls: 44 passed**, plus the same single explicit real-PG observer deferral, in 4.790 seconds. The executing worker checked its own global PID/thread/preload source hash and matching receipt. Guard hash: `7daf9ae7d2f26e8fcd389b1d560fac1d4e218a8304a2c58e49f646af108165bc`. No unexpected attempts or PG sockets occurred. Vanilla controls on the identical repair files passed 22 cases with one explicit launcher-parent assertion deferral (2.044 seconds). - **Real cleanup counterfactual:** unchanged old helper failed the actual endpoint assertion in 1.568 seconds: its late HTTP child remained reachable after close returned. The same fixed-helper case passed in 1.485 seconds: close waited, the child exited, the port was unreachable and no IPC error occurred. The isolated driver records only the known legacy closed-IPC error long enough to inspect cleanup; current helper requires zero such errors. All recorded processes and ports were independently checked empty. - **Fresh current bootstrap passed once**, 16.159 seconds wall / 14.12 seconds Vitest / 12.237 seconds body. The ordinary enabled producer persisted useful revision 1, cleared its lease and retained exactly 300,000 ms between success and original next deadline. A signed current-generation report produced the matching generation/operation-lock SQL ACK before physical child stop (ACK 06:54:02.020 UTC; stop signal 06:54:02.025 UTC). The alarm retained the original D and D-120 eligibility. Both fixture databases ended with zero sessions; the coordinator independently audited and normally removed them. This bootstrap does not execute a due wake or the new v2/v3 timing assertions. - **Historical full two-cycle result remains 1/1 passed in 614.213 seconds wall**, at measured App `74f7c012ca17ad0d7302d76403f5033affb1430b` and service `2cb54f904031dcf69a32902cd853b43f74d6f301`. Two new processes published distinct immutable v2/v3 at original D+115 ms and D+98 ms. Exact signed SQL ACKs preceded both physical stops; real SQL D-120 claims caused both cold starts. Features, claims and original business deadlines stayed unchanged. - Both historical 10.002-second quiet windows kept app SQL, upstream and outgoing HTTP counts unchanged. Two awake PG sockets remained. Quiet counts do not mean all connections closed. - The stronger timing checks are additional assertions, not retroactively executed results. Offline inspection of the original receipts confirms persisted successes at D+115/+98 ms and earliest upstream handler entries at D+44/+30 ms; all five requests in each refresh group started after D. The exact new pre-sleep request counter and dispatch-before-handler time were not independently recorded historically. - Source review found 19 critical automatic Git producer/runtime files identical to the measured App. The current active cache helper change does not run on automatic Git publication and adds no pool. The matching service carries unchanged measured helper/sleep/controller/alarm seams through its normal master merge. The current bootstrap and focused controls qualify the changed cleanup/guard and parent integration. No fresh current-head 614-second pass or whole-tree identity is claimed. - Runtime is serialized: worker 1, per-pool limits at most 4, configured App potential 10 + shared service pool 2 + one observer against server max 40. Current bootstrap point samples observed at most 3 owned clients plus the observer; those samples are not a continuous aggregate peak. App/service guard peaks were 2 each, with zero remaining sockets at cleanup. Failures remain accounted for. The first historical bootstrap cleanup masked an unknown primary error. Later attempts exposed missing fixture-owned domain metadata and lost origin snapshots; test-only ownership/cleanup corrections preserved the original assertions and timeouts. The new causal driver first lacked a final observation, then encountered sandbox bind EPERM, then exposed the old closed-IPC error; only the final real endpoint failure above is labeled the cleanup counterfactual. An offline dependency miss and a generator-cleared ignored compiler config were retained and corrected without weakening compiler options. The former inherited policy failure (397 findings) is resolved by the genuine current-master base/scanner, with zero findings and seven scanner tests; no unrelated UI site was edited and the gate was not relaxed. Historical real TCP observer coverage passed 1/1 in 3.688 seconds. Comment-only and opt-in-selection carries remain documented in prior evidence; self-hash changes are metadata exceptions with fresh executing-worker checks. The full workspace typecheck, test, build and browser commands were not run locally for this test-only slice. The current base passes its full exact-head CI and Greptile 5/5; this leaf now has all 56 exact-head check runs terminal (54 success, one neutral, one skip), the additional legacy security status passing, complete current Greptile 5/5 and Ponytail 5/5 reviews, and all four review threads resolved across all pages. No live provider or deployed host/database suspension was exercised. ## Risks - The host test requires a matching service checkout and compiled bridge. An unrelated service revision is not a valid substitute. - Controlled localhost Git, S3 and physical process operations do not qualify deployed Railway or Neon suspension, external permissions or provider uncertainty. - The Node guard does not prove native subprocess transport isolation. Only finite local Git metadata and Node version commands were needed. - Two short SQL/HTTP quiet windows do not establish strict 90-second host acceptance. Separate plugin-host LISTEN, MCP wake and other domain gaps remain open. - Explicit test URLs are destructive fixture inputs. Use fresh, owned `_test` databases. The launcher does not start a cluster or choose a database implicitly. - Passing CI and review do not qualify deployed suspension; the current proof boundaries above still apply. > For core feature work, check `ROADMAP.md` first and discuss it in `#dev` before opening the PR. This PR adds verification to the existing feature in its base. ## Model Used OpenAI `gpt-6.1-sol` assisted with implementation, repository review and local code execution. OpenAI Codex, based on GPT-6, performed managing review and integration. The context-window sizes and exact served backend snapshots were not exposed. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change and contains no internal Paperclip ticket id - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green — fresh merge-head gates pending - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups — fresh merge-head review pending - [x] I will address all Greptile and reviewer comments before requesting merge ### Current master reconciliation (2026-11-06) Merged actual master `e28158b353fd274f0322be76d62cd223e0df599f` normally. All nine qualified acceptance code files and the held-port repair remain byte-identical. Upstream independently landed the workflow input fix in #19602; the fixture now uses that exact upstream version once, avoiding duplicate object keys in a squash merge. All other master changes are preserved. The earlier two-cycle local evidence remains source-specific; no fresh 614-second run or deployed suspension is claimed. This new head requires fresh CI and bot reviews. ### Actual combined-tree typecheck failure and repair CI job `128156887548` failed with TS1117 at workflow-target.test.ts:22: two independently landed master fixes declared the same `IMAGE_ONLY` and `TESTOPS_REQUESTED` keys. The complete original log is retained. A further normal master merge now removes the duplicate declarations, retaining exactly one false/false pair and every original assertion. All nine qualified acceptance code paths and the held-port repair remain byte-identical; unrelated master changes are preserved. This is a concrete compiler failure and source repair, with fresh exact-head CI/reviews required. --------- Co-authored-by: Paperclip <[email protected]>
59 lines
4.7 KiB
JavaScript
59 lines
4.7 KiB
JavaScript
import { createRequire } from 'node:module';
|
|
import { spawn } from 'node:child_process';
|
|
import { existsSync, readFileSync, statSync } from 'node:fs';
|
|
import path from 'node:path';
|
|
import { fileURLToPath } from 'node:url';
|
|
|
|
const root = fileURLToPath(new URL('../', import.meta.url));
|
|
const controls = process.argv.slice(2).join(' ') === '--controls';
|
|
if (!controls && process.argv.length > 2) throw new Error('Use no arguments for the witness or --controls for finite PG0 controls');
|
|
const names = ['PAPERCLIP_SKILL_DEFAULT_CADENCE_TEST_DATABASE_URL', 'PAPERCLIP_SKILL_DEFAULT_CADENCE_CLOUD_TEST_DATABASE_URL'];
|
|
if (!controls && process.env.PAPERCLIP_MAINTENANCE_PG0_CONTROLS) throw new Error('PG0 controls flag is forbidden for the witness');
|
|
if (!controls) {
|
|
const urls = names.map(name => {
|
|
if (!process.env[name]) throw new Error(`${name} requires an explicit dedicated test database`);
|
|
let url; try { url = new URL(process.env[name]); } catch { throw new Error(`${name} requires a loopback dedicated test database`); }
|
|
if (!['postgres:', 'postgresql:'].includes(url.protocol) || !['127.0.0.1', 'localhost', '::1', '[::1]'].includes(url.hostname)
|
|
|| !/^[a-zA-Z0-9_]+_test$/.test(decodeURIComponent(url.pathname.slice(1)))
|
|
|| ['options', 'dbname', 'database'].some(key => url.searchParams.has(key))) throw new Error(`${name} requires a loopback dedicated test database`);
|
|
return url;
|
|
});
|
|
if (urls[0].pathname === urls[1].pathname) throw new Error('App and Cloud require distinct test databases');
|
|
const cloud = process.env.PAPERCLIP_SKILL_DEFAULT_CADENCE_CLOUD_CHECKOUT;
|
|
if (!cloud || !path.isAbsolute(cloud) || !existsSync(path.join(cloud, 'dist/sleep/fixtures/default-cadence-bridge.js'))) throw new Error('Set the exact compiled Cloud checkout');
|
|
if (!path.isAbsolute(process.env.PAPERCLIP_SKILL_DEFAULT_CADENCE_CLOUD_NODE || '')) throw new Error('Set the actual Node22 executable');
|
|
if (!/^[a-f0-9]{40}$/.test(process.env.PAPERCLIP_SKILL_DEFAULT_CADENCE_CLOUD_SHA || '')) throw new Error('Set exact Cloud source SHA');
|
|
}
|
|
const log = process.env.PAPERCLIP_SKILL_DEFAULT_CADENCE_GUARD_LOG;
|
|
if (!log || !path.isAbsolute(log) || !existsSync(path.dirname(log))) throw new Error('Set an absolute guard log in the existing evidence directory');
|
|
if (existsSync(log) && statSync(log).size !== 0) throw new Error('Use a fresh per-attempt guard log');
|
|
if (!process.version.startsWith('v24.')) throw new Error('Launch acceptance with the actual Node24 executable');
|
|
const require = createRequire(new URL('../server/package.json', import.meta.url));
|
|
const guard = path.join(root, 'scripts/fixtures/skill-default-cadence-network-guard.cjs');
|
|
|
|
|
|
require(guard);
|
|
const vitest = path.join(path.dirname(require.resolve('vitest/package.json')), 'vitest.mjs');
|
|
const env = { PATH: process.env.PATH, HOME: process.env.HOME, LANG: process.env.LANG, TMPDIR: process.env.TMPDIR,
|
|
NODE_ENV: 'test', NODE_OPTIONS: `--require=${guard}`, PAPERCLIP_SKILL_DEFAULT_CADENCE_GUARD_LOG: log };
|
|
if (controls) env.PAPERCLIP_MAINTENANCE_PG0_CONTROLS = '1';
|
|
if (!controls) {
|
|
for (const name of [...names, 'PAPERCLIP_SKILL_DEFAULT_CADENCE_CLOUD_CHECKOUT', 'PAPERCLIP_SKILL_DEFAULT_CADENCE_CLOUD_NODE',
|
|
'PAPERCLIP_SKILL_DEFAULT_CADENCE_CLOUD_SHA', 'PAPERCLIP_SKILL_DEFAULT_CADENCE_EVIDENCE', 'PAPERCLIP_SKILL_DEFAULT_CADENCE_STAGE',
|
|
'PAPERCLIP_TEST_POSTGRES_ADMIN_URL']) if (process.env[name]) env[name] = process.env[name];
|
|
env.PAPERCLIP_SKILL_DEFAULT_CADENCE_ACCEPTANCE = '1'; env.PAPERCLIP_MAINTENANCE_KEEP_ARTIFACTS = '1';
|
|
}
|
|
const files = controls ? ['server/src/__tests__/skill-default-cadence-prewire.test.ts', 'server/src/__tests__/maintenance-harness-resources.test.ts',
|
|
'server/src/__tests__/maintenance-postgres-proxy.test.ts', 'server/src/services/skill-source-idle-schedule.test.ts']
|
|
: ['server/src/__tests__/skill-source-default-cadence-acceptance.test.ts'];
|
|
const child = spawn(process.execPath, ['--require', guard, vitest, 'run', ...files,
|
|
'--maxWorkers=1', '--no-file-parallelism', ...(controls ? ['--testNamePattern=^(?!.*maintenance observer against PostgreSQL)'] : []), `--execArgv=--require=${guard}`], { cwd: root, env, stdio: 'inherit' });
|
|
for (const signal of ['SIGINT', 'SIGTERM']) process.on(signal, () => child.kill(signal));
|
|
child.once('error', error => { console.error(error); process.exitCode = 1; });
|
|
child.once('close', (code, signal) => {
|
|
const events = readFileSync(log, 'utf8').trim().split('\n').filter(Boolean).map(line => JSON.parse(line));
|
|
const violation = events.some(event => event.event === 'blocked' || event.unexpectedAttempts?.length);
|
|
if (violation) console.error('Default cadence guard recorded an unexpected attempt; caught exceptions do not qualify');
|
|
process.exitCode = violation ? 98 : signal ? 1 : code ?? 1;
|
|
});
|