docs(fleet): record CI-fix QA red-proofs and drift-guard PASS

Signed-off-by: John Smith <[email protected]>
This commit is contained in:
John Smith committed 2026-10-21 18:03:26 -04:00
1 parent 93d957a08c
commit d8307e8f4f
3 files changed
+124

No files matched your search

+41
View File
@@ -0,0 +1,41 @@
# QA run log — fleet-ci-fix verification (branch feat/fleet-sync-standardization @ 93d957a)
Target worktree: /home/john/Projects/.github-fleet-sync
Branch: feat/fleet-sync-standardization, tip 93d957a (parent 1cb9989, NOT pushed)
Commit under QA: 93d957a "fix(ci): satisfy zizmor self-syntax and exclude audit evidence from local-path scan"
- .github/workflows/ci.yml: uses ./.github/workflows/hygiene.yml -> $/.github/workflows/hygiene.yml + exclude block (docs/evidence, docs/reports)
- .github/workflows/ocr.yml: uses ./.github/workflows/ocr-review.yml -> $/.github/workflows/ocr-review.yml
Date (UTC): 2026-10-21. Verifier: agency-qa (independent, no commits, no subagents, no push).
## 1. Full file reads from disk
- .github/workflows/ci.yml (40 lines): hygiene job uses $/.github/workflows/hygiene.yml with audit_workflows:true and exclude docs/evidence+docs/reports; lint job = jq labels, jq manifest, validate-skills, shellcheck. CONFIRMED.
- .github/workflows/ocr.yml (42 lines): review job uses $/.github/workflows/ocr-review.yml, OCR_LLM_TOKEN forwarded. CONFIRMED.
## 2. uses-line grep (zizmor-sim for self-repository syntax)
- `grep -n "uses:" .github/workflows/ci.yml` -> 13: uses: $/.github/workflows/hygiene.yml ; 24: actions/checkout@11d5960a... (pinned). No ./ self-ref.
- `grep -n "uses:" .github/workflows/ocr.yml` -> 40: uses: $/.github/workflows/ocr-review.yml. No ./ self-ref.
- `git grep -n '\./\.github/workflows' -- .github/` -> exit 1, no matches (zero ./ self-refs in workflows). CLEAN.
- Rationale for $/ (verified against primary sources, not memory): zizmor docs (docs.zizmor.sh/audits, self-repository audit, introduced v1.30.0) state: "As of July 2026, GitHub supports a new self-repository syntax (uses: $/...) ..." Ref: GitHub Blog changelog 2026-07-30. Repo pins [email protected] in hygiene.yml, which contains this audit. $/ is therefore the auditor-recommended form. CONFIRMED.
## 3. shellcheck / bash -n / jq / validate-skills
- `shellcheck scripts/*.sh` -> exit 0, no output. GREEN.
- `bash -n` on scripts/sync-fleet.sh, sync-labels.sh, validate-skills.sh -> all exit 0. GREEN x3.
- `jq -e 'type=="array" and all(...)' labels.json` -> true, exit 0. GREEN.
- `jq -e '.schema_version==1 and ...' fleet-manifest.json` -> true, exit 0. GREEN.
- `bash scripts/validate-skills.sh` -> "Validated 4 skill(s) in <root>/skills.", exit 0. GREEN.
## 4. Hygiene local-path scan (exact PATTERN from .github/workflows/hygiene.yml)
- WITHOUT excludes (`git grep -nIE "$PATTERN" -- .`): exit 0 with matches ONLY in docs/evidence/fleet-rebase/ and docs/reports/. Exactly the two dirs ci.yml excludes. AS EXPECTED.
- WITH ci.yml excludes (... ':(exclude)docs/evidence' ':(exclude)docs/reports'): exit 1, no output. CLEAN.
## 5. zizmor (exact pinned auditor: uvx [email protected], same as hygiene.yml)
- On current ($/) state: "No findings to report. Good job! (8 suppressed)" -> exit 0. GREEN.
- (Sabotaged ./ state result recorded in red-proofs.log.)
## 6. Observation (non-blocking): local actionlint binary predates $/ syntax
- Installed actionlint reports `workflow-call` errors on ci.yml:13 and ocr.yml:40 for $/ (exit 1), and is clean (exit 0) on ./.
- Per GitHub changelog 2026-07-30 + zizmor v1.30.x docs, $/ IS valid current GitHub syntax; actionlint's rule set predates it.
- actionlint is NOT executed by any workflow step in this repo (ci.yml lint job runs jq/validate-skills/shellcheck only; hygiene.yml runs zizmor only; .github/actionlint.yaml is config synced to fleet targets). Hence observation only, not a gate failure.
## 7. Final state
- Sabotage reverted via `git checkout -- .github/workflows/ci.yml`; `git status --short` shows only untracked docs/evidence/fleet-ci-fix/ log files, zero modifications to tracked files. Nothing committed. Nothing pushed.
+26
View File
@@ -0,0 +1,26 @@
# Red-proof log — fleet-ci-fix QA (branch feat/fleet-sync-standardization @ 93d957a, NOT pushed)
Purpose: prove the $/ vs ./ check is non-vacuous (fails RED when sabotaged, GREEN when clean).
Date (UTC): 2026-10-21. Verifier: agency-qa. No commits, no push.
## Sabotage applied (temporary, production file)
- Command: sed -i 's|uses: \$/\.github/workflows/hygiene\.yml|uses: ./.github/workflows/hygiene.yml|' .github/workflows/ci.yml
- Effect: .github/workflows/ci.yml:13 became `uses: ./.github/workflows/hygiene.yml` (workspace-relative form).
## RED confirmation (sabotaged state)
1. zizmor-sim grep: `git grep -n '\./\.github/workflows' -- .github/`
-> `.github/workflows/ci.yml:13: uses: ./.github/workflows/hygiene.yml`, exit 0 (FOUND = RED). Check is non-vacuous: PROVEN.
2. Real zizmor (uvx [email protected] --format=plain .): exit 12 with 1 low finding:
`help[self-repository]: use GitHub's dedicated self-repository syntax`
`--> ./.github/workflows/ci.yml:13:11 ... uses: ./.github/workflows/hygiene.yml ... ^^^ use '$/...' instead of './...'`
`= note: audit confidence -> High / = note: this finding has an auto-fix / = help: audit documentation -> https://docs.zizmor.sh/audits/#self-repository`
`9 findings (8 suppressed, 1 unsafe fixes): 0 informational, 1 low, 0 medium, 0 high`
(The 8 suppressed are offline-mode suppressions, present in both states.)
3. actionlint on sabotaged state: exit 0 (clean) — local actionlint predates the July-2026 $/ syntax; see qa-run.log section 6. Not a repo gate.
## GREEN restoration
- Revert: `git checkout -- .github/workflows/ci.yml`
- After revert `grep -n "uses:" .github/workflows/ci.yml` -> 13: uses: $/.github/workflows/hygiene.yml (restored).
- `git status --short` -> empty for tracked files (only untracked docs/evidence/fleet-ci-fix/ logs remain). CLEAN.
- Re-ran on restored state: [email protected] -> "No findings to report. Good job! (8 suppressed)", exit 0 (GREEN); zizmor-sim grep -> exit 1 no matches (GREEN).
- Sabotage was re-applied a second time to capture the full verbatim zizmor finding above, then reverted again with identical clean proof (final `git status --short` shows no tracked modifications).
+57
View File
@@ -0,0 +1,57 @@
# Audit Report: Fleet CI Fix Verification (feat/fleet-sync-standardization @ 93d957a)
- **Auditor:** agency-drift-guard (independent, read-only; no production code changed)
- **Date (UTC):** 2026-10-21
- **Worktree:** `/home/john/Projects/.github-fleet-sync`
- **Branch:** `feat/fleet-sync-standardization` (local-only, NOT pushed; `ahead 1` of `origin/feat/fleet-sync-standardization`)
- **Tip audited:** `93d957a` "fix(ci): satisfy zizmor self-syntax and exclude audit evidence from local-path scan" (parent `1cb9989`)
- **Base:** `origin/main` (`b3afb81`, Merge PR #3 v2)
- **Inputs read in full from disk:** `.github/workflows/ci.yml` (40 lines), `.github/workflows/ocr.yml` (42 lines), `.github/workflows/hygiene.yml` (76 lines), `/home/john/Projects/.github/README.md` (S1-S4), `docs/reports/fleet-rebase-implementation.md`, `docs/reports/fleet-rebase-verification.md`, `docs/evidence/fleet-ci-fix/qa-run.log`, `docs/evidence/fleet-ci-fix/red-proofs.log`
- **Programmer claims under audit:** 2-file minimal fix (ci `$`/exclude, ocr `$/`); all gates green; QA PASS with real zizmor exit 0.
## 1. RULE 2 — Stub / shortcut hunt (assume broken until proven)
- Diff `1cb9989..93d957a` is exactly 2 files, +5/-2 (`ci.yml`, `ocr.yml`). Both files re-read in full; the only semantic changes are two `uses:` lines (`./` -> `$/`) and one added `exclude:` block. No `|| true`, `set +e`, `test.skip`, `TODO`, `FIXME`, `NotImplementedException`, or hardcoded pass verdicts in the diff.
- Repo-wide stub grep hits are **only** prose mentions inside `docs/reports/fleet-rebase-implementation.md:53` and `docs/reports/fleet-rebase-verification.md:13-14` (self-descriptions of prior grep checks) — not stubs. No `echo PASS` shims; the single `VERDICT: PASS` hit is the prior report's verdict line.
- `git grep -n 'uses: \$' -- .github/` -> exactly the two intended lines (`ci.yml:13`, `ocr.yml:40`); `./.github/workflows` grep -> rc=1 (zero `./` self-refs remain). No unreferenced modules introduced (no new modules at all).
- Programmer claim "2-file minimal fix" **confirmed byte-for-byte** against `git diff 1cb9989..93d957a`.
## 2. RULE 3 — Spec & architecture audit
### 2.1 `$/.github/...` form matches origin/main AND the pinned auditor's primary source — PASS
- `git show origin/main:.github/workflows/ci.yml` line 13 is `uses: $/.github/workflows/hygiene.yml`; `origin/main` `ocr.yml` line 40 is `uses: $/.github/workflows/ocr-review.yml`. Commit `71540de`'s message documents the `./` -> `$/` change as the zizmor self-repository fix. Tip `93d957a` therefore **restores exact parity with `origin/main`** on both lines (parent `1cb9989` had regressed to `./` via the rebase conflict resolution).
- Primary-source proof (tool output, not memory): independent sabotage `./` -> pinned `uvx [email protected]` emits `help[self-repository]: use GitHub's dedicated self-repository syntax ... use '$/...' instead of './...'` (High confidence, exit 12); restored `$/` -> `No findings to report. Good job! (8 suppressed)`, exit 0. The auditor itself mandates `$/`; the fix follows the repo's actual gate.
- Supersession note: the prior report (`fleet-rebase-verification.md` S2.7) asserted the `$` rationale "has no basis" and approved `./`. That assessment is **superseded by live zizmor v1.30.1 evidence**: with the pinned auditor, `./` is RED and `$/` is GREEN. The rebase report was correct against the linters available to it; the tool behavior observed today governs.
- Residual risk R1 (non-blocking, see S5): local `actionlint` v1.7.12 flags `$/` as malformed `workflow-call` (it predates the July-2026 syntax). `actionlint` is **not executed by any workflow step** in this repo (verified: `git grep actionlint -- .github/workflows/` hits only `fleet-sync.yml:13` path-sync config and `hygiene.yml:23` input description text; `ci.yml` lint runs only jq/validate-skills/shellcheck; `hygiene.yml` runs only zizmor). Runner-side acceptance of `$/` will be proven empirically when PR #4 CI runs.
### 2.2 `exclude: docs/evidence + docs/reports` — PASS with observation O1 (does not weaken zero-bypass for production paths)
- Mechanism legitimacy: `hygiene.yml` (identical on `origin/main`) **documents** this exact input for this exact purpose — header example `exclude: | path/to/recorded-evidence`, input description "Newline-separated paths left out of the scan". The fix uses the reusable workflow's designed feature, not a bypass hack. YAML parses (`exclude: 'docs/evidence\ndocs/reports\n'` confirmed via safe_load).
- Load-bearing and scoped: WITHOUT excludes, `git grep` with the exact hygiene PATTERN matches **only** 4 files — `docs/evidence/fleet-rebase/qa-run.log`, `docs/evidence/fleet-rebase/red-proofs.log`, `docs/reports/fleet-rebase-implementation.md`, `docs/reports/fleet-rebase-verification.md` (recorded `/home/john/Projects/...` worktree paths in committed QA logs). WITH the `ci.yml` excludes -> rc=1 (clean; rc captured directly, not through a pipe). Every match is inside the two excluded dirs; **zero matches in `scripts/`, `skills/`, `templates/`, `.github/workflows/`, configs, or manifest files**. Zero-bypass holds for all production/sync payload paths — the property the hygiene gate exists to protect (machine-local paths must not fan out to 32 downstream repos via `sync-fleet.sh`).
- O1 (non-blocking): the excluded dirs contain *tracked* files with real machine-local paths (they were untracked at the prior audit, committed under `1cb9989`). A future secret accidentally pasted into a report would now pass hygiene. Recommend keeping QA evidence untracked where possible or redacting absolute paths; inherent to the exclude feature, not a defect in this fix.
### 2.3 Security substance preserved — PASS
- `ci.yml`: `permissions: contents: read`, `audit_workflows: true` intact; lint job (jq labels, jq manifest, validate-skills, shellcheck) untouched.
- `ocr.yml`: `pull_request` trigger set, `permissions: contents: read + pull-requests: write`, explicit `OCR_LLM_TOKEN` forwarding, no `secrets: inherit` anywhere repo-wide (rc=1). `ocr-review.yml` untouched by this diff.
- All remote `uses:` remain full-40-hex SHA-pinned; `[email protected]` pin unchanged; no new remote refs.
## 3. RULE 4 — Independent verification (re-ran + own sabotage; QA logs reviewed line-by-line)
- **Gates re-run (read-only, current tip):** `jq` labels -> `true`/0; `jq` manifest -> `true`/0; `validate-skills` -> `Validated 4 skill(s)`/0; `shellcheck scripts/*.sh` -> 0; `bash -n` -> OK x3; hygiene WITHOUT excludes -> matches only the 4 excluded-dir files (as QA S4 claims); WITH excludes -> rc=1 clean. QA S1–S5 **confirmed**.
- **zizmor (pinned `uvx [email protected]`, same as `hygiene.yml`):** clean state -> exit 0, `No findings to report. Good job! (8 suppressed)`. QA S5 **confirmed**.
- **Independent sabotage (temporary, restored):** `ci.yml:13` `$/` -> `./` -> zizmor exit 12 with the verbatim `self-repository ... use '$/...' instead of './...'` finding (High confidence) — RED proven, non-vacuous. `git checkout -- .github/workflows/ci.yml` -> byte-identical restore (diffed against pre-sabotage copy), `uses: $/...` back at line 13, `git status --short` shows only `?? docs/evidence/fleet-ci-fix/`, zero tracked modifications. Reversibility **demonstrated**.
- **QA log review:** `qa-run.log` S1–S5, S7 accurate against independent re-runs (S6 actionlint-predates-`$/` observation independently reproduced with local actionlint v1.7.12 flagging both `$/` lines; correctly classified as non-gate). `red-proofs.log` RED->GREEN cycle reproduced verbatim (same finding text, same exit 12, same restore procedure). Final QA state claim (tracked-clean) matches observed `git status`.
## 4. Scope / drift / privilege / supply-chain / reversibility
- **Scope:** 2 files, CI-only, within the "PR #4 CI failing" brief. No source, script, skill, template, or manifest changes.
- **Drift:** No spec drift — restores `origin/main` parity on both `uses:` lines; exclude uses the documented input for its documented purpose. No ADR 0001 impact (no cross-repo code paths touched).
- **Privilege:** Untouched least-privilege posture (S2.3). No token/secret changes.
- **Supply-chain:** No new dependencies, pins, or remote refs; SHA pins and `[email protected]` intact.
- **Reversibility:** Fully reversible — single local-only commit (`ahead 1`, NOT pushed); `git checkout --` restore proven byte-identical.
## 5. Residual risks / observations (non-blocking)
- **R1:** `actionlint` v1.7.12 (local binary) and pinned `zizmor` v1.30.1 disagree on `$/` vs `./`. The repo gates on zizmor, so the fix is correct for CI-green, but GitHub-runner acceptance of `$/` is proven only when PR #4 CI executes. If the runner rejects it, both linters cannot be satisfied simultaneously and the team must escalate (update actionlint config or pin zizmor).
- **O1:** See S2.2 — consider keeping future QA evidence untracked or redacting absolute worktree paths to preserve the strictest hygiene posture.
VERDICT: PASS