mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-07 06:16:06 -10:00
refactor(skills): keep test audit evidence in tasks and PRs (#16833)
## Thinking Path > - Paperclip manages AI agents that perform work. > - The test-audit skill guides test maintenance. > - Its Paperclip reference tells agents to commit audit receipts under doc/plans/. > - This adds repository documents to small test cleanups, as seen in #16639. > - This change keeps audit decisions and validation evidence in the task and PR description. > - Test gardening creates no repository documentation unless a human asks for it. ## Linked Issues or Issue Description **Where is the issue?** `.agents/skills/test-audit/references/paperclip.md`, with related reporting instructions in the entry point, campaign guide, and validation reference. Related example: #16639. **What's wrong?** The skill requires repository audit receipts. A six-line test deletion in #16639 added an 88-line report under doc/plans/. The task and PR already provide a place for this evidence. **Suggested fix** Keep the evidence in the task and a concise PR description. Require an explicit human request before changing repository documentation during test gardening. ## What Changed - Removed the instruction to commit dated audit receipts under doc/plans/. - Added the reporting rule to the skill entry point. - Directed campaign ledgers and plans to the task. - Kept the existing coverage, validation, and delivery requirements. ## Verification - Skill validation passed with `quick_validate.py` from the skill-creator tool. - `git diff --check` passed. - Read the entry point and all bundled references to check for conflicting reporting instructions. - This change edits skill instructions only. Repo-wide typecheck, runtime tests, and build were not run locally. - [PR CI](https://github.com/paperclipai/paperclip/actions/runs/39253050703) passed on head `d0b68db878fa4da83858da5f74fd25e6120723b5`, including typecheck, test suites, build, and E2E checks. - Greptile reviewed that head at 5/5 with no open findings. ## Risks - Evidence must remain accessible in the task and PR. The instructions still require decisions, keepers, commands, results, and limitations. - An installed copy needs a refresh to receive this source change. An operator can also apply the rule in the agent and routine instructions immediately. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI Codex, based on GPT-6. The exact runtime model ID and context-window size were not exposed. Used repository search, instruction editing, and shell validation. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes: #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass (skill validation; no runtime changes) - [x] I have added or updated tests where applicable (not applicable to this instruction edit) - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <[email protected]>
This commit is contained in:
1 parent
c6a0cf4bce
commit
f96a1e6406
4 files changed
+30
-21
No files matched your search
@@ -30,10 +30,11 @@ prior findings, model if specified, and discovery/edit/publication authority.
|
||||
Tell workers to preserve others' changes. One coordinator owns shared fixtures,
|
||||
routing, and integration. Do not edit a checkout with active test runs.
|
||||
|
||||
Keep one ledger: candidate, contract, retain/repair/consolidate/delete/defer,
|
||||
reason, keeper, baseline, validation, and branch/PR status. Record reviewed areas
|
||||
with no safe removals too. Discovery stays read-only until ownership is clear;
|
||||
qualified edits need no additional user-approval round within existing authority.
|
||||
Keep one ledger in the task: candidate, contract,
|
||||
retain/repair/consolidate/delete/defer, reason, keeper, baseline, validation, and
|
||||
branch/PR status. Record reviewed areas with no safe removals too. Discovery
|
||||
stays read-only until ownership is clear; qualified edits need no additional
|
||||
user-approval round within existing authority.
|
||||
|
||||
## 3. Plan removal of redundant layers
|
||||
|
||||
@@ -42,11 +43,11 @@ boundary. Look for suites that replay the same behavior through wrappers,
|
||||
mocked collaborators, or provider-specific copies of a shared implementation.
|
||||
Individual cases can look reasonable while the entire layer is redundant.
|
||||
|
||||
Produce a compact layer plan: retired suites, retained owner per contract,
|
||||
unique assertions to transfer, affected routing/commands, validation controls,
|
||||
and any unused fixtures or production seams. Prefer a boundary that reaches
|
||||
the responsible code; preserve distinct transport, lifecycle, and permission
|
||||
risks. If no whole layer qualifies, record that and proceed with justified
|
||||
Record a compact layer plan in the task: retired suites, retained owner per
|
||||
contract, unique assertions to transfer, affected routing/commands, validation
|
||||
controls, and any unused fixtures or production seams. Prefer a boundary that
|
||||
reaches the responsible code; preserve distinct transport, lifecycle, and
|
||||
permission risks. If no whole layer qualifies, record that and proceed with justified
|
||||
case-level changes. Large deletions are not a requirement.
|
||||
|
||||
The coordinator reconciles keeper dependencies across all batches before
|
||||
|
||||
@@ -67,10 +67,15 @@ Apply this gate to tests added or changed during gardening:
|
||||
|
||||
## Completion and reporting
|
||||
|
||||
Keep audit decisions, plans, ledgers, and validation evidence in the task and
|
||||
PR description. Do not create or modify repository documentation, including
|
||||
`doc/plans/` audit receipts or reports, unless the human explicitly requests
|
||||
documentation changes. A test cleanup does not need a repository report.
|
||||
|
||||
Report removals and keepers, retained/deferred candidates, checks and limitations,
|
||||
distinct cases versus repeated executions, and test/support versus production
|
||||
LOC. Count documentation separately. Label cost evidence as measured, estimated,
|
||||
or historical; a proven simplification does not require a measurable speedup.
|
||||
LOC. Label cost evidence as measured, estimated, or historical; a proven
|
||||
simplification does not require a measurable speedup.
|
||||
|
||||
State the actual endpoint: findings, validated local patch, published PR,
|
||||
intermediate merge, or landed in the intended target. Stop at the requested
|
||||
|
||||
@@ -53,10 +53,11 @@ Resource exhaustion must not be interpreted as a tested product failure or a
|
||||
passing control. Coordinate heavy runs and clean up only owned resources.
|
||||
|
||||
Follow existing branch/worktree conventions and protect unrelated working-tree
|
||||
edits. New repository audit plans/receipts belong in
|
||||
`doc/plans/YYYY-MM-DD-<scope>-test-audit.md`. Prefer one concise receipt per
|
||||
coherent batch or campaign. For Paperclip issue work, follow the issue's artifact
|
||||
and work-product requirements rather than relying only on local paths.
|
||||
edits. Keep audit plans, decisions, and validation evidence in the Paperclip
|
||||
execution issue and a concise PR description. Do not add repository audit
|
||||
documents or `doc/plans/` receipts. Documentation changes require an explicit
|
||||
human request. When requested artifacts are deliverables, follow the issue's
|
||||
artifact and work-product requirements rather than relying only on local paths.
|
||||
|
||||
When publishing is authorized, fill every section of
|
||||
`.github/PULL_REQUEST_TEMPLATE.md`, including exact model information that is
|
||||
|
||||
@@ -79,9 +79,11 @@ do not weaken assertions, increase budgets, skip suites, or erase a failing
|
||||
baseline to obtain a green audit. Retry a demonstrated transient problem within
|
||||
a stated bound; a recurring failure needs investigation.
|
||||
|
||||
Use PR notes for small changes. For a campaign, keep a compact dated receipt
|
||||
with revisions, decisions, keepers, commands, results, controls, measurements,
|
||||
and limitations. Link accessible CI runs/artifacts when they support claims.
|
||||
Temporary paths and hashes alone do not make evidence durably inspectable.
|
||||
Do not generate large evidence dossiers for trivial deletions or store raw
|
||||
logs containing secrets. State precisely which broad checks remain unrun.
|
||||
Use concise task and PR notes for revisions, decisions, keepers, commands,
|
||||
results, controls, measurements, and limitations. Keep campaign ledgers and
|
||||
plans in the task too; do not commit audit documentation or receipts to the
|
||||
repository. Documentation changes require an explicit human request. Link
|
||||
accessible CI runs/artifacts when they support claims. Temporary paths and
|
||||
hashes alone do not make evidence durably inspectable. Do not generate large
|
||||
evidence dossiers for trivial deletions or store raw logs containing secrets.
|
||||
State precisely which broad checks remain unrun.
|
||||
Reference in new issue
Block a user