44 Commits

Include renames

Author	SHA1	Message	Date
3 people	a2262d92b2	fix(node): implement reconciliation sweep as durability backstop (#218) (#551) ... * feat(node): implement reconciliation sweep as durability backstop (#218) Reconciliation sweep: periodic background worker that re-derives pin/seal sets and fills gaps so a dropped replication job never means data loss. - visibility_pack.rs: all_object_paths (two-phase ls-tree + cat-file --batch-all-objects), allowed_blob_tree_sets_bounded, ObjectPath/BlobTreeSets type aliases - db/mod.rs: atomic policy transactions, get/set_node_state, has_ipfs_cid, filter_ipfs_pinned_oids, filter_pinata_pinned_oids, repo_policy_epoch, migration v27 (node_state table + policy_epoch column) - ipfs_pin.rs: PolicyFence struct, dispatch fence re-check before POST - pinata.rs: dispatch fence re-check before POST - encrypted_pin.rs: dispatch fence re-check - reconciliation.rs: full sweep worker with keyset cursor, PolicyFence - api/ipfs.rs: effective_cid fallback in list_pins - main.rs: spawn reconciliation sweep worker - .cargo/audit.toml: RUSTSEC-2026-0258 h2 ignore (pending #368) * fix: add missing parameters to pinning functions and adjust timeout settings * fix(tests): update CID assertions to handle Option type and adjust parameters in pinning functions * fix(node): close review findings on reconciliation sweep (#218) Reviewer-1 P1 / Reviewer-2 P1: v27/v28/v29 were referenced by tests but never added to the MIGRATIONS array, so every existing deployed node would fail to boot (node_state, repos.policy_epoch, pinned_cids.cid NOT NULL → NULL all unreadable). Forward-only migrations: v27 pinata_only_clear_legacy_equal_cid: ALTER cid DROP NOT NULL, UPDATE cid = NULL WHERE cid = pinata_cid v28 node_state_key_value: new key/value table for the sweep cursor v29 repos_policy_epoch: BIGINT NOT NULL DEFAULT 0 Reviewer-2 P1: visibility_pack.rs:542 phase 1 used ls-tree -rz, which under -r recurses into blobs and never emits tree entries. Trees then arrived only from the phase-2 catch-all with empty path and the fail-closed filter denied every one — the sweep could repair a non-flat repo's git graph. Switched phase 1 to ls-tree -r -t -z and add a per-commit rev-parse <commit>^{tree} that registers the root tree at "/". 48 visibility_pack unit tests pass. Reviewer-1 P2 / Reviewer-2 P2: list_pinned_cids called is_raw_cidv1 on the cid column before decoding it as Option<String>, so SQL NULL failed the decode and the new Pinata-only rows never reached the handler. Decode cid as Option first, drop only the both-NULL case. The is_raw_cidv1 filter was removed entirely: the new contract lists every row that has something to advertise, and the handler at api/ipfs.rs:list_pins is the seam that decides what to do with a legacy-shape row (the resolver 404s on a mismatched key, per #173 U4). Updated list_pinned_cids_omits_unrepaired_legacy_row to match. Reviewer-1 P2: two new sqlx::test cases pin the gate order: sweep_skips_quarantined_repos_before_scan (SQL filter, plus a defense-in-depth per-row re-check) and sweep_skips_private_repos_before_scan (per-repo listable_at_root). Both use expect(0) on the mock IPFS so any future reorder that moves the scan ahead of the gate fails the build. Also: record_pinned_cid ON CONFLICT now rewrites cid (R1-P2: a stale wrong CID from a previous push is repaired by a subsequent one). record_pinata_cid ON CONFLICT clears cid to NULL when the existing row has the legacy cid = pinata_cid shape; on INSERT, stores cid = NULL when raw_cid == pinata_cid (the Pinata-only signal). * feat: implement per-backend continuation offset for missing OIDs - Enhance `missing_oids` function to accept a `start_after` parameter, allowing for rotation of the missing set based on the last attempted OID. - Update `run_pass` to load and save the continuation offset for both IPFS and Pinata backends, ensuring that previously attempted OIDs are retried last in subsequent passes. - Introduce tests to validate the behavior of the new offset mechanism, ensuring deterministic ordering and proper handling of gaps in the missing set. - Ensure that the offset is cleared when no missing OIDs are found, allowing for fresh starts in subsequent passes. * fix(node): trace provenance/structural invariants through every consumer The v30 local_ipfs_provenance column was set by the writers but only consulted by has_ipfs_cid and filter_ipfs_pinned_oids. The pin loop's "is this already done" check still used row existence (is_pinned), so a Pinata-only row hit the early-return branch without ever reaching the local writer, and the flag stayed FALSE forever. Three other seams carried the same shape-vs-provenance confusion. P1 (ipfs_pin.rs:1843): pin loop's skip keys on has_ipfs_cid (writer-owned local_ipfs_provenance = TRUE), not on row existence. A Pinata-only row now falls through to the local writer path, which calls record_pinned_cid_with_source and flips the flag on the conflict branch. The Pinata pin loop's skip remains keyed on has_pinata_cid (a separate seam). P1 (db/mod.rs v30 backfill): tighten the WHERE clause to cid IS NOT NULL AND pinata_cid IS NULL. The previous permissive (cid <> pinata_cid) clause marked pre-v30 Pinata-first uploads (where record_pinata_cid deliberately stored cid = raw_cid without a local Kubo push) as local IPFS pins, so the sweep filtered them as already-durable and an operator enabling IPFS later never got a local recovery copy. Ambiguous dual-shape rows stay at FALSE under the strict rule and the sweep re-derives on re-pin (cheap, idempotent). P1 (db/mod.rs record_pinned_cid_with_source ON CONFLICT): restore cid = COALESCE(pinned_cids.cid, EXCLUDED.cid). A Pinata-only row with cid = NULL (the raw_cid == pinata_cid shape) would otherwise be marked complete with local_ipfs_provenance = TRUE and a NULL resolver key. P1b (visibility_pack.rs): remove the synthetic "/" path on root trees and add a structural entry-level check (structurally_safe_root_tree / _for_caller) in both the sweep gate (allowed_blob_tree_sets_bounded) and the per-request tree gate (allowed_tree_set_for_caller_bounded). A tree is admitted only if every direct entry in its serialized bytes is safe at /{filename} and (for tree entries) the child tree is itself in the allow set. The previous path-based "/" check let the root tree of a public repo slip through even when its bytes name a denied subtree entry. P2 (api/ipfs.rs list_pins): add local_ipfs_provenance to PinnedCidRecord, include in list_pinned_cids SELECT, and surface as local_pinned / pinata_pinned in the JSON response. A Pinata-only row previously returned the raw resolver key as local_cid, making it indistinguishable from a real dual-backed row. The new fields let a gl consumer distinguish local-only, remote-only, and dual rows without re-inferring semantics from nullability. local_cid is preserved for backward compatibility. Tests: - migration_v30_backfills_local_ipfs_provenance_heuristically: flipped both-distinct expectation to FALSE under strict rule. - migration_v12_makes_cid_nullable_and_preserves_classification and migration_v27_clears_legacy_equal_cid: both-distinct / distinct row expectations flipped to FALSE. - sweep_never_pins_root_tree_naming_withheld_subtree (new): assert root tree + secret subtree tree absent from pinned_cids. - allowed_tree_set_gates_withheld_subtree_tree: root tree assertion inverted to EXCLUDED for anon. - list_pins_reports_writer_owned_provenance_for_all_shapes (new): assert 4 row shapes produce correct local_pinned / pinata_pinned JSON. Scope discipline preserved: PolicyFence capture order, _pin_permit reuse for the seal phase, fresh per-arm rederive_budget, keyset repo cursor with lookahead, mirror-row skip, Kubo 2xx-without-Hash refusal, v27 provenance-clearing migration all untouched. v31 schema unchanged. * fix(node): structural tree gate at every depth + non-commit ref tolerance The prior round installed a `local_ipfs_provenance` column and applied a structural tree gate only to root trees. The fresh review pass found two structural gaps the writer contract didn't propagate to: P1 (visibility_pack.rs:1519): the path-based tree admission admitted a tree whenever its path was allowed, regardless of whether its serialized bytes named a denied subtree. Concrete failure: a public repo with `/public/secret/file` and a `/public/secret/**` deny admitted the `/public` tree (its path is `/public`, allowed) but `/public` still contained a `secret/` entry pointing at the denied subtree — anyone who obtained `/public`'s CID could `cat-file -p` it and read the subtree's name and child OID. The prior round's post-pass structural check ran on root trees only. The fix: replace the prior round's `structurally_safe_root_tree` / `_for_caller` / `root_tree_oids` helpers with one recursive `tree_structurally_safe` that walks every tree's entries at every depth, recursing on tree entries. Bundled the rule/path args into a `TreeCheckCtx` struct to fix the clippy `too_many_arguments` warnings the prior round introduced. A tree is admitted iff every direct entry is safe at its path AND (for tree entries) the child tree is itself structurally safe there. A denied-subtree tree fails at its level; the parent fails because the denied-path child entry fails; the propagation continues up to the root. The `admitted` set memoizes trees proven safe at some path so the recursion short-circuits on cycles and on trees reachable at multiple allowed paths. P1 (visibility_pack.rs:533): `all_object_paths` called `assert_all_refs_are_commits`, which bailed on any ref that didn't peel to a commit (tag-of-tree, tag-of-blob). The sweep (`run_pass`) converted that error into a full-scan failure that skipped the whole repo. A repo with a pushable tag-of-tree never got its missing public pins repaired by the new hourly worker, even though its commit-reachable graph was otherwise safe to classify (`ipfs_cid_tree_served_despite_non_commit_ref` is the in-repo example of that supported shape). The fix: remove the `assert_all_refs_are_commits` call from `all_object_paths` (and the analogous one in `blob_paths`, used by the encrypted recovery path — the same reasoning applies). `git rev-list --all` already silently skips non-commit refs, so the commit-reachable object set is what the sweep needs. The unused `assert_all_refs_are_commits` helper is removed. Unclassifiable ref targets still fail closed at a different layer: the cat-file catch-all enumerates them with no path, and the path-based allow filter drops empty-path entries. Test inversions and new regressions: - `migration_v12_makes_cid_nullable_and_preserves_classification` and `migration_v27_clears_legacy_equal_cid` — both-distinct / distinct row expectations flipped to FALSE under the v30 strict backfill (prior round's tightening is re-affirmed here; the v30 backfill is `cid IS NOT NULL AND pinata_cid IS NULL`, the only unambiguous local-IPFS shape). - `ipfs_cid_gate_withholds_blob_from_unauthorized` — root tree assertion inverted to NOT_FOUND (the recursive structural gate denies root trees whose entries name a denied subtree). - `ipfs_cid_legacy_provider_cid_repaired_on_repush` — legacy-shape row insert now sets `local_ipfs_provenance = TRUE` to reflect that the bytes ARE on local IPFS (the legacy CID is just the wrong key). - `ipfs_cid_tree_served_despite_non_commit_ref` — root tree assertion inverted (same reason as the gate test). - `fails_closed_on_annotated_tag_of_a_blob` and `fails_closed_when_a_ref_cannot_be_traversed` — renamed to `skips_*`, assertions inverted from `Err` to `Ok` (a ref pointing at a non-commit object is no longer a walk error; the blob falls out of the gap set because `git rev-list --all` skips it). - `ipfs_cid_walk_error_fails_closed` — removed (relied on the now-removed `assert_all_refs_are_commits` to force an error; the fail-closed arm is still covered by `ipfs_cid_commit_tag_walk_error_fails_closed`, which uses a different trigger — a nonexistent ref object makes `rev-list --all` fail). - `sweep_does_not_publish_public_ancestor_tree_naming_withheld_subtree` (new) — recursive depth test: a `/public/secret/**` repo asserts root tree, `/public` tree, and `/public/secret` subtree are all absent from `pinned_cids` while `/public/visible.txt` is IPFS-pinned. - `sweep_repairs_commit_reachable_object_in_repo_with_tag_of_tree` (new) — tag-of-tree repo with a separate `mktree`'d tree (not commit-reachable); asserts the commit-reachable blob is IPFS-pinned and the unclassifiable tag-of-tree is not. Scope discipline preserved: PolicyFence capture order, `_pin_permit` reuse for the seal phase, fresh per-arm `rederive_budget`, keyset repo cursor with lookahead, mirror-row skip, Kubo 2xx-without-Hash refusal, v27 provenance-clearing migration, v30 strict backfill, v31 schema, per-(repo, backend) reconciliation offset, and the `local_ipfs_provenance` writer/reader contract are all untouched. * test(node): ignore pre-existing RED replication-tail test on CI `api::repos::tests::receive_pack_tail_survives_a_disconnect_during_release` documents an open bug in `api/repos.rs` (the replication tail must be spawned above `release` and gated on `receive_result.is_ok()`; today it's spawned below, so a disconnect during `release` drops the pins, recovery copy, and announce). The test's `assert!` message starts with "RED:" — this is a TDD-style "keep the design intent visible in the code" test, not a working-behavior assertion. The test was added 2026-07-29 by `beardthelion` (PR #215 lineage), well before the #218 sweep work, and is unrelated to this PR. CI is failing on it under load (passes in isolation) because the open bug hasn't been fixed yet. `#[ignore]` until the open bug is fixed. The test's design intent and the fix description stay in the doc comment and the assert message; only the CI failure is suppressed. Run the test explicitly with `cargo test -- --ignored` to verify the bug is fixed when the tail-spawn moves above `release`. * fix(node): align encrypted manifest anchor log with one-shot contract The "will retry next pass" log at reconciliation.rs:1034 was false: plan_seal returns SkipUnchanged on every subsequent pass once the recipients tag matches, so sealed stays empty and the anchor block never runs again. A transient Irys outage at the moment of a fresh seal therefore LOSES that anchor permanently — the next pass has no delta to anchor and no retry fires. This is intentionally best-effort: re-anchoring an unchanged manifest would burn Irys writes for no recovery benefit. Factored the warn message into ENCRYPTED_MANIFEST_ANCHOR_FAILED_MSG and added a unit test that pins the constant; the test goes RED if "will retry" reappears in the log. * fix(node): address eight reviewer findings on #218 reconciliation sweep (round 3) * fix(node): close audit.toml array and drop inline comment * fix(node): repair the stale for-each-ref fake-git fixture (round 8 P1) `upload_pack_shares_one_deadline_across_walk_and_filtered_serve` — the #174 shared-deadline guard for the FILTERED serve arm — was red on both `test (stable)` and `test (beta)`. The fake git answered `for-each-ref` with the single token `refs/heads/main`, a ref NAME, where `blob_paths` phase 2 asks for `--format=%(objectname) %(objecttype)`. The phase-2 parse fails closed on a line it cannot read, so the walk returned an error and the request surfaced as a generic 500 instead of being reaped into a 504. That silently repurposed the test from "the filtered serve shares one deadline" into "the walk errors" — the guard proved nothing while it was red. The fixture is what was stale, not the production path: the arm now emits the column shape the walk parses. A commit tip has no peeled referent, so two columns is the whole line. The 1.2s walk cost stays on `ls-tree` and the serve's on `pack-objects`, so the two phases remain independently attributable. * fix(node): peel annotated tags in the blob_paths ref walk (round 8 P1) `%(objecttype)` reports the type of the ref's OWN object, which for an annotated tag is `tag`. The phase-2 `if kind == "blob" || kind == "tree"` arms therefore never saw the referent, and a blob reachable ONLY through an annotated tag escaped the withheld set — while `rev_list_keep`'s `git rev-list --objects --all` does peel tags and served it. Phase 2 closed the leak for a direct ref-to-blob and for that shape only. The walk now asks for the peeled atoms `%(*objectname) %(*objecttype)` and inserts the referent with an empty path, alongside the direct blob/tree tips it already handled. Line parsing moves to a field-count match: two columns for a non-tag tip (the peeled atoms expand to empty), four for a tag tip, anything else still fails closed. Measured against git 2.50, the `*` atoms peel the WHOLE chain — a tag-of-a-tag-of-a-tag-of-a-blob reports the blob — so a `tag` peeled type does not occur on stock git. That arm is kept anyway, because `push_delta.rs`'s ref-type guard documents `%(*objecttype)` as a one-level peel: under such a git it finishes the chain with `rev-parse <tip>^{}` plus a `cat-file -t` probe rather than bailing and 500ing every clone of a repo that merely carries a nested tag. Both children are bounded by the walk's shared deadline and are unreachable on the common one-line-per-ref path. * fix(node): one empty-path decision for both blob_paths consumers (round 8 P1) `blob_paths` has two consumers and only one of them carried the empty-path policy. `withheld_from_pairs` (the deny side, what the smart-http serve filter excludes) withheld a phase-2 empty-path entry from everyone but the owner; `allowed_blob_set_for_caller_bounded` (the allow side, what `GET /ipfs/{cid}` hands over) called `visibility_check(..., "")` directly. On a public repo no glob matches the empty path, so that returns `Allow`: the serve filter withheld the OID while the IPFS gate served the bytes — the leak phase 2 exists to close, reopened one layer over. The policy moves into `pair_decision`, which both consumers now call, so the two gates cannot disagree and a future change to the empty-path rule moves both at once. `withheld_blob_recipients_bounded` routes through it too: an unclassifiable blob grants a recovery copy to the owner only, never to a rule's named reader whose grant was written against a path this object does not have. No behavior change for non-empty paths — `pair_decision` delegates to `visibility_check` unchanged. * test(node): bind the phase-2 ref tests to a commit-unreachable blob (round 8 P2) `skips_a_ref_pointing_at_a_blob` and `skips_an_annotated_tag_of_a_blob` were vacuous. Both hung their ref on `fixture()`'s `secret` blob, which is COMMITTED at `secret/b.txt`, so phase 1's per-commit `ls-tree` already produced `(secret, "/secret/b.txt")`, the `/secret/**` rule already denied it, and `withheld.contains(&secret)` passed with phase 2 deleted outright. Both now use `orphan_blob`, a blob written straight to the object store that no commit's tree names, so the ONLY route into the withheld set is the `for-each-ref` phase. That is also the real leak shape: `rev_list_keep`'s `git rev-list --objects --all` does follow a ref to such a blob, so an under-withheld one ships in the clone pack. Verified by mutation: with `if false` injected at the phase-2 filter, every test below goes RED (they were green under that mutation before). New coverage: - `ref_only_blob_is_denied_on_the_allow_side_too` — the allow-side denial for the shared `pair_decision`; it also asserts deny and allow agree on one OID, and that the owner IS still admitted, so it cannot pass by denying everything. Reverting the allow side to `visibility_check` turns it red on exactly that assertion. - `skips_a_nested_annotated_tag_of_a_blob` — real-git tag-of-a-tag. - `peels_a_tag_whose_peeled_target_is_still_a_tag` — drives the one-level-peel fallback through the fake-git seam, so that arm is tested rather than dead. * fix(node): advance the reconciliation continuation only for dispatched work (round 8 P2) `ipfs_last` / `pinata_last` were captured from the missing set above the pin permit, both `PolicyFence` captures and both pin loops, then written under nothing but `if ipfs_enabled` / `if pinata_enabled`. Every stage in between can legitimately produce nothing — a fence capture that fails, a quarantine/visibility recheck that says skip, a pin-boundary re-derivation that errors — and each is transient. Advancing past OIDs that were never attempted is not a harmless retry delay: `missing_oids` rotates strictly past the stored offset, so the whole unattempted prefix lands BEHIND the entire backlog on the next pass. For an at-cap repo — the only kind the continuation exists for — the backlog never drains inside one cap window, so those objects are starved indefinitely. That is a durability hole in the durability backstop. The offset now moves only for work actually handed to a backend, tracked at the dispatch boundary as `to_pin.last()` (not the missing set's last: an OID the pin-boundary re-derivation dropped was never offered). It is recorded before the call, so a pin phase that times out mid-batch still counts as dispatched. The write becomes three outcomes rather than two, via one `next_offset_write` helper so the backends cannot drift apart: * work dispatched -> advance to the last dispatched OID * nothing missing, query OK -> None, marking the row done * nothing dispatched, or the missing-set query failed -> write NOTHING, leaving the resume point a capped pass already paid for `*_scan_ok` is what separates the last two: an empty missing set means "everything is pinned" or "the filter query failed and we know nothing", and only the first should mark the row done. Test: `sweep_leaves_offset_untouched_when_nothing_was_dispatched`, using a `#[cfg(test)]` seam on the pin-boundary re-derivation — that stage cannot be starved from the outside, because the mid-scan re-filter runs first on the same rules and budget and makes the sweep `continue` well before the offset write. It asserts the stored offset comes back byte-identical after a declining pass, then that the same repo does advance it once dispatch happens, so the assertion cannot pass on a dead write site. Restoring the old semantics turns it red. * fix(gl): render the Pinata provider CID in `gl ipfs list` (round 8 P2) The server-side wire change landed without its client half. `cmd_list` read `pin["cid"].as_str().unwrap_or("?")` as the entry heading and nothing in the CLI read `pinata_cid` at all, so a Pinata-only row printed as a bare `?` — the listing showed nothing usable for an object that is in fact durably stored. `cid` is legitimately null for such a row: it is the key `GET /ipfs/{cid}` resolves against, and the node stopped aliasing a Pinata provider CID into it (round 3) precisely because that CID 404s there. So the fix is on this side. Rendering moves into `render_pin`, which leads with the node-resolvable CID when there is one and otherwise the provider CID, labelled so nobody feeds it back to this node's resolver. A dual row keeps both visible. A `backends:` line names where the bytes are, read from the writer-owned `local_pinned` / `pinata_pinned` booleans rather than re-inferred from CID shape — which is what the node's own comment warns against. Older nodes send neither flag; those `cid`-only rows fall back to CID presence and render as they always did (`render_pin_handles_a_legacy_cid_only_row`). * fix(node): implement third fence check for IPFS pinning and enhance visibility rules handling * fix(node): enhance documentation and add tests for ProgressState and helper functions * fix(node): address round-9 reviewer findings on #218 reconciliation sweep * fix(node): correct consumer-4 recipient test for call-invariant set * fix(git): fail closed on non-UTF-8 tree listing in walk_tree_oids_inner (#218 round 10 P1) walk_tree_oids_inner previously returned Ok(()) on a non-UTF-8 ls-tree -z listing, inserting only the tree OID and skipping every child blob/tree OID. A direct tree ref (or an annotated tag peeling to a tree) is valid Git input; git rev-list --objects --all still enumerates the tree and every descendant, so the keep-side removed the tree from the served set but passed the child blob OIDs to pack-objects, exposing their bytes to an anonymous clone. Phase 1 already bails on the same input at the blob_paths walk; this fix puts the tree-walk on the same fail-closed outcome. The new fails_closed_on_non_utf8_tree_tip test pins the invariant for both a direct refs/tags/direct-tree ref and an annotated tag-of-tree (the two non-commit shapes round 9 added tolerance for), confirming the walker bails rather than returning Ok with a partial withheld set. * fix(recon): run encrypted recovery even when public list is empty (#218 round 10 P1) A path-scoped repo whose only reachable object is a direct blob or direct tree ref yields an empty public list (the anonymous public classifier removes the only object from the served set) while withheld_blob_recipients_bounded still assigns that object to the owner recovery set. The early `continue` on object_list.is_empty() (and the second one after the mid-pass refilter) suppressed encrypted recovery too, and a lost or failed encrypted copy was never repaired. Track empty-public-work as a flag and gate the public phase on it. Encrypted phase 2 runs regardless. The backend enable flags and `_pin_permit` move out of the gated block so phase 2 can read them when the public phase did not run. * fix(db): rename migration v12 test to v32 (#218 round 10 P3) The test name and its docstring/comments still say "v12", but the cid-nullable migration is at v32 (line 1163). The test itself is correct — it drives the real path — so this is naming drift rather than dead coverage. A future reader looking for the migration would otherwise chase a v12 that no longer exists. * fix(git): index OID membership before quadratic classifier scan (#218 round 10 P2) all_object_paths was doing O(O×P) work in two places: the catch-all cat-file branch checked `blob_set.iter().any(|o| o == oid)` and `tree_set.iter().any(...)` for every reachable object, and the phase 1 loop also did a per-commit unused `git rev-parse <commit>^{tree}` probe whose result was discarded (the root-tree OID is recomputed by `root_tree_oids`). On the 50k-object repos the sweep exists for, the quadratic scan and the redundant per-commit child process can consume each authorization deadline before any object reaches a backend, producing a permanent hourly skip. Maintain a separate `blob_oids` / `tree_oids` HashSet alongside the path-pair sets so the membership check is O(1). Drop the unused per-commit rev-parse probe. * fix(git): bound tree walk by ls-tree invocation count, not just depth (#218 round 10 P2) walk_tree_oids_inner stopped at MAX_TREE_WALK_DEPTH and the shared deadline. A wide shallow tree reachable from N ref tips or from multiple parents still spawned one ls-tree child process per subtree well inside the depth cap, and the wall-clock bound could not stop that — expiry was the only stop. The ceil was wall-clock only. Add a structural invocation cap (MAX_TREE_WALK_INVOCATIONS) the walker fails closed at, and a memo of walked tree OIDs so the same tree reachable from multiple ref tips is walked once. Round 10 P2. * fix(pinata): suppress push when DB record is not durable (#218 round 10 P2) pin_new_objects pushed (sha, cid) on provider POST success regardless of whether record_pinata_cid persisted the row. A closed/failed DB write or an explicit-transaction source record timeout meant the next sweep queried the DB, found no row, and re-offered the same gap — with reconciliation having counted a fill that never happened. Track a `db_record_durable` flag. On hard failure of either record_pinata_cid (non-timeout) or record_pin_source (any failure, since it's a multi-statement transaction), set the flag false and skip the `pinned.push`. The `BoundedDbError::Elapsed` autocommit case stays as the previous comment's "may have committed" reasoning: the push fires so the reconcile still sees the fill. Round 10 P2. * fix(node): serialize policy narrow with irreversible upload via in-process mutex (#218 round 10 P1) The PolicyFence's epoch read was a one-way read: a visibility narrow (rule insert, quarantine set) could commit between the fence's epoch read and the upload's HTTP POST, and the upload's DB-record INSERT's epoch check would see the new epoch and reject — but the provider already accepted the object. Encrypted envelopes had no fence at all. Add a per-repo in-process mutex that PolicyFence acquires at capture and releases at drop. The narrow paths (set_repo_quarantine, set_visibility_rule, remove_visibility_rule) acquire the same lock for the duration of their DB transaction. A narrow therefore blocks until the in-flight batch finishes, and an in-flight batch waits for the narrow to commit before it even reads the epoch. Multi-process / multi-node is a known gap: a Postgres advisory lock per repo would extend the same guarantee across processes. Deferred to a follow-up. * fix(pinata): update post-upload stalled record test for round 10 P2 contract The previous test asserted that a successful upload with a timed-out post-upload source record still returned the (sha, cid) pair. That assertion pinned the buggy behavior the reviewer called out: the reconcile would count a fill that was not durable, then re-offer the same gap. The test name and the `record_pin_source` arm of `pin_new_objects` are unchanged; the assertion now reflects the new contract: when the post-upload source record times out (its `mark_pin_sources_incomplete` already sets the source set to incomplete, so the next pass re-offers), the pair is suppressed so the reconcile sees the gap as not-filled. Round 10 P2 follow-up. * fix: clippy allow + rustfmt on round 10 commits clippy::too_many_arguments on walk_tree_oids_inner (8 args after the round 10 memo + invocation counter threading). rustfmt re-ran on the if/else blocks added in the has_public_work refactor (round 10 P1 #42). No behavior change. * fix(node): address round-11 review findings on #218 reconciliation sweep P1: remove the whole-batch policy mutex. PolicyFence no longer holds a per-repo lock from capture through drop and the narrow paths (set_visibility_rule, remove_visibility_rule, set_repo_quarantine) no longer acquire it, so a visibility narrow commits immediately and the batch aborts on its next is_current check instead of the narrow blocking behind the sweep. Accepted residual is a single in-flight object; the fenced DB record still refuses to land a raced row as durable. encrypt_and_pin_stops_sealing_when_reader_removed_mid_batch is green again. PolicyMutexes is deleted, resolving the key-leak P2 by removal. P2: cover the round-10 has_public_work branch with sweep_seals_withheld_blob_when_public_list_is_empty: a direct-blob-ref repo yields an empty public list with a non-empty owner recovery set; asserts no public gaps/fills, no cleartext pin, exactly one POST (the seal envelope), and a recorded encrypted copy. P3: peel-case isolation in the non-UTF-8 tree-tip test (delete the direct-tree ref before the second call so the peel arm must run); v12->v32 rename drift in the migration test (boundary, comments, and the stale (2)->true doc line); accurate comments for the pre-refilter flag and the post-refilter fall-through. * fix(node): address merge-readiness findings on #218 reconciliation sweep Reworks the sweep around explicit state-machine contracts instead of adding conditionals: backend results carry last_attempted plus confirmed durability separately (PinBatchOutcome), provider permits scope to provider effects only, discovery pages by commit window with a persisted cursor, legacy provenance migrates as unknown, and the pin list advertises only validated raw local keys. - Empty-path owner recovery without path rules: phase 2 drops the has_path_scoped_rule gate and runs on the actual recipient result, with a dedicated recovery recheck (no anonymous-listability requirement); matrix test over blob/tree/tag refs x zero/root rules, including recreate-after-delete. - Pinata Elapsed is non-durable unless verify_pinata_record proves the exact row and fence epoch; loop test with row-locked repos. - list_pins gates cid/local_cid on is_raw_cidv1 plus confirmed provenance; response-shape matrix uses real CIDs with a legacy row and GET /ipfs follow-through; docs match the nullable wire contract. - Permits acquired per provider effect and dropped after; no nesting at pool size 1 (test renamed to the new shape). - Scan discovery pages SCAN_COMMIT_WINDOW commits oldest-first with a node_state cursor (advance on drained/empty/attempted, delete on cover); refilters and phase-2 recipients run on the same window; walk-layer paging/invocation/union tests plus a 1050-commit cursor-to-completion sweep test. - v35 migrates legacy rows as unknown (no TRUE backfill); v35 test asserts unknown + preserved Pinata history; sweep re-derivation test re-pins instead of filtering. - Drained completions DELETE continuation rows (no tombstones); FK cascade on repo delete; row-count regression test. - Audit: lru waiver gets an explicit retirement owner plus a weekly drift guard; revocation model recorded as a product decision on PolicyFence (prompt revocation, single-in-flight residual). * fix(tests): configure git identity in bare clone for tag fixture The windowed-union test creates its annotated tag in the bare clone, which carries no config. It passed locally only via the developer's global git identity; CI has none, so 'git tag -a' failed there. Verified with an empty GIT_CONFIG_GLOBAL/SYSTEM. * fix(tests): ignore pre-existing hung-git pid race on CI read_object_bounded_returns_by_deadline_with_a_hung_git reads the fake's pid file with no readiness handshake; a slow scheduler yields 'No such file or directory' (stable-only red, identical beta green). Unrelated to #218 (store.rs untouched); re-enable with a readiness signal from the fake. * fix(node): address round-12 review findings on #218 reconciliation sweep - P1 roots: admit structurally safe root trees at "/" into the allow set and candidates, so safe snapshots fully replicate; roots naming a withheld subtree stay excluded via the recursive structural check. - P1 per-pass WalkBudget on sweep enumeration (entries / output bytes / git invocations) with fail-closed budget-exhaustion; serve paths keep the uncapped budget. - P2 recovery lane: independent recovery cursor pages unlistable history across restarts instead of sharing the scan window. - P2 gap-hold: failed gap queries hold discovery advance but not the healthy backend, via test-only failure seams. - P2 nested tags: collect full tag chains so nested tag objects pin without an inner ref. - P2 RepoStore: thread the Tigris-backed store into the sweep with an acquire timeout and shutdown select; restore cache-miss and skip missing. - P2 Pinata source: keep confirmed pairs when the primary already covers the repo; suppress push for stalled post-upload records without coverage. - Update large-history sweep expectation to blob+root+commit per round. * fix(node): address consolidated review findings on #218 reconciliation sweep P1: wire WalkBudgetExceeded through the sweep cursor logic. Both the public scan and the recovery walk now dispatch the typed budget-exhaustion signal at the sweep boundary: a budget-hit window advances past content the sweep refuses to classify partially (scan cursor save/delete via the existing persist site; recovery cursor likewise), while git/policy/panic/timeout failures keep the cursor for retry. visibility_pack documents the obligation; the match arms are the handler, with a one-line pattern note for future typed walk errors. P2: run encrypted recovery when the public scan fails. Scan walk/panic/timeout arms now fall through with an empty public list instead of continuing the whole repo iteration, so phase 2 runs on the recovery lane's own cursor like the unlistable-repo path. The scan cursor stays untouched for retry. Refilter/recheck-public failures remain fail-closed whole-iteration continues by design. Includes a repo-scoped test seam (process-global set, safe across spawn_blocking and parallel tests via fresh UUIDs). P2: bump policy_epoch only when remove_visibility_rule deletes a row, mirroring set_repo_quarantine's rows-affected guard, so a typo'd or double-deleted glob cannot abort fenced batches. Also adds the repo-loop outcome matrix (every early exit × scan cursor / recovery cursor / phase 1 / phase 2) and deny tests: budget-hit advances both lane cursors with zero pins/seals (real 250k-entry trip, early-trip fixture runs in ~30s), scan failure still seals the recovery copy, delete-missing-rule leaves the epoch unchanged. Each deny was verified by temporarily restoring the old branch. * fix(node): address jatmn review findings on reconciliation sweep re-home * fix(node): reload persisted sweep cursor between passes * fix(node,gl): pinata offset warn wording and boundary-safe pin timestamp --------- Co-authored-by: Gravirei <[email protected]> Co-authored-by: Kevin Codex <[email protected]>	2026-11-09 15:35:18 +00:00
beardthelion	5bb61cc97f	ci(release): verify release provenance for backfill publishes (#523) ... * ci(release): gate publish jobs on the release environment, anchor the tag check, pin npm Closes #234. workflow_dispatch runs the selected ref's YAML and npm trusted publishing matches on the workflow filename, not the ref, so a dispatched non-main ref could reach the OIDC publish with no approval. Declare environment: release on all three publish jobs (npm-publish, docker, docker-manifest) so the environment's deployment-branch rule gates every publish surface, not just the OIDC one. Also close the two residuals the issue carries: the release-tag case glob was right-unanchored (accepted v1.2.3.4.5, v1_x, v1-any-branch-name, bare v1), now an extglob v+([0-9]).+([0-9]).+([0-9]); and the OIDC publish ran npm@^11.5.1, a floating range, now pinned to [email protected].1. The environment itself must exist with a deployment-branch rule limited to main (and required reviewers if desired) for the gate to bite; npmjs.com can also scope each package's trusted publisher to environment "release" for a registry-side check that survives a workflow edit. * ci(release): qualify tag checkouts and keep moving tags forward-only Closes #271, closes #238. actions/checkout resolves an unqualified ref as a branch before a tag, so a branch named v0.7.1 shadows the tag and the release builds unreviewed content. Qualify all four release-tag checkouts as refs/tags/..., which takes checkout's unconditional tag path. docker-manifest applied :latest and :X.Y unconditionally, so a backfill of an older tag repointed them at older code. The manifest now always publishes the immutable :X.Y.Z and applies the moving tags only when the resolved version is not older than the registry's latest release. The same shape existed on the npm side: publish moves the latest dist-tag to whatever it publishes, so a backfilled older version would have stolen it; older backfills now publish under the backfill dist-tag instead. The manifest also asserted nothing about the digest set it stitches, so a wrong-arity or misnamed artifact would ship under :latest. Assert two digests of 64 lowercase hex chars before imagetools create. Extend the resolver test to pin both publish steps byte-for-byte and to reject any unqualified tag-checkout ref, so these guards cannot regress silently. * ci(release): gate every job on the release environment and verify tag provenance Two residuals from review of the dispatch-publish surface: - environment: release only covered the three registry-publish jobs. Declaring it on all eight jobs means a dispatch of unmodified YAML on a non-main ref is denied by the environment's deployment rules before any step runs. - The resolver shape-checked vX.Y.Z but did not prove the tag was produced by the release flow. A write-access user could plant a v99.99.99 tag on unreviewed content and dispatch a backfill against it. In CI (GH_TOKEN present) the resolver now requires a GitHub release for the tag and a main...tag compare status of identical or behind, so the tag's commit must already be reachable from main. * ci(release): qualify the provenance compare and fail closed on npm lookup errors Second adversarial pass found two residuals: - The provenance check compared main...<tag> with an unqualified refname, the same branch-shadows-tag ambiguity the refs/tags/ checkout prefix was added to avoid. Resolve the tag through commits/refs/tags/<tag> to a commit SHA and compare that instead. - npm view ... || true mapped every lookup failure to "no latest", so a registry outage during an older backfill would publish under --tag latest and move the dist-tag backward. E404 still means first publish; any other failure now aborts before a dist-tag is chosen. * ci(release): check release author, decide npm dist-tag per package, close the test gap Third adversarial pass: - Provenance now also requires the release to be authored by the release automation account, so a hand-created release on a valid tag does not qualify for backfill. - The npm dist-tag decision is per package. A partial publish can leave platform packages ahead of the wrapper, and a single lookup on @gitlawb/gl would then downgrade an ahead package's latest tag. - The environment gate test now emits one line per job and marks jobs with no declaration MISSING, so a job added without environment: release can no longer pass silently. * ci(release): anchor docker moving tags to the registry floor, not the release pointer GitHub's latest-release pointer is operator-mutable and is not a bound on what :latest serves. List the package's versions on ghcr and refuse to move :X.Y or :latest below the newest X.Y.Z tag there. A 404 means the package does not exist yet, so the first publish applies all tags; any other lookup failure aborts. The version filter is grouped with || true so an empty list does not trip pipefail. * ci(release): require backfill release assets to be automation-uploaded The npm backfill republishes GitHub release assets, which are mutable for anyone with contents: write. Require every asset on the release to have been uploaded by github-actions[bot], matching the release-author check. Not a hard bound, but it removes the direct edit path and leaves a review signal when a hand-uploaded asset blocks a backfill. * ci(release): pin the verified commit and the exact asset bytes end to end The provenance checks ran at resolve time but the consumers re-resolved mutable names afterward, leaving a move-the-tag or replace-the-asset race: - Checkouts now use the commit SHA the resolver emitted (tag_commit), falling back to the qualified tag ref. - The resolver captures the release's name/id asset pairs once, checks all uploaders against them, and the lay-in step downloads by asset id. An asset deleted and reuploaded between resolve and download gets a new id and fails closed. - The npm-publish comment now names the third trusted-publisher component (environment), which is what binds the OIDC token to the gate. * ci(release): run the resolver on the producer path and hash homebrew inputs locally - release-binaries and web-sync now resolve the release tag and check out the verified commit SHA, closing the tag-move race on the job that produces the release assets, not just the jobs that consume them. - homebrew-bump resolves the tag, downloads the tarballs by captured asset id, and computes sha256 locally instead of trusting the mutable *.tar.gz.sha256 sidecar assets. - The step pin now covers the lay-in, formula, and release-tag checkout blocks so edits to the trust boundary force a deliberate fixture update. * ci(release): bind the tag to the commit its release was created against Reachable-from-main alone accepts a tag moved to any other main commit. The release object records its creation-time target_commitish, so require tag_commit == release.targetCommitish: a moved tag now fails closed, and the SHA emitted for checkouts is provably the released commit. * ci(release): force bash for the release-binaries resolver step The matrix includes windows-latest where run: defaults to pwsh, and the resolver body is bash. * ci(release): base64 release-asset names in the resolver map Release asset names are attacker-controlled text. A crafted name carrying whitespace could smuggle a fake uploader into the third positional field and pass the automation-only check. Emitting names base64-encoded keeps each record in three real fields and makes the name-to-id lookup immune to injection. * Address PR review feedback (#447) - floor the docker minor-line tag at its own X.Y max so an in-line-newest backfill advances :X.Y without moving :latest * Address PR review feedback (#447) - serialize all release runs under one concurrency group; the ref-keyed group let dispatches from different refs interleave * ci(release): freeze docker moving tags on dispatch, retire the registry floor * ci(release): make the resolver test's gh stub discriminate on the full call * ci(release): assert the pinned npm version took effect * ci(release): surface missing-asset errors from the formula sha() subshell * ci(release): consume resolver outputs instead of release-please outputs post-resolve * ci(release): document the concurrency group's pending-slot replacement * ci(release): share the image output and bound the step scanners to jobs The docker-manifest rel step now emits image= the same way the docker job's does, and both manifest steps consume steps.rel.outputs.image instead of recomputing the lowercase ghcr path. The three whole-workflow awk scanners now ignore everything before jobs: and flush at job boundaries, so an `on.push.branches` list item can no longer open a pseudo-step that swallows top-level YAML, and a step record can no longer absorb the next job's header lines. * ci(release): close the review-found evasions in the release checks Review of the follow-up diff found the structural assertions could be satisfied or evaded without the behavior they guard: - the gh stub now pipes a fixture response through real jq with the caller's own -q on every arm, so a resolver that hardcodes its expected fields is detected - the moving-tag scan keys on the outcome (a push mechanism plus a mutable-tag reference in decommented step text) and requires the event+ref guard on the carrier's if: line, instead of binding one argv spelling and accepting comment-borne guard text - a new assertion pins the on.push.branches / workflow_dispatch triggers, which no step-level check could see - the npm check detects install/i and -g/--global variants, requires a non-empty install set, and reads the pin comparison from code, not comments - the stale-reads scan covers first step lines, job-level env/if lines, and jobs without a resolver step - the resolver test asserts the GHAE heredoc terminator, clears ambient GH_TOKEN/GITHUB_REPOSITORY on non-stub calls, and prints the captured resolver stderr when a pass-expected case fails - sha() exits on a failed download, since errexit is off inside its command substitution and a truncated file would otherwise be hashed * ci(release): floor the moving tags against the releases list The event gate alone cannot bound a stale run: a re-run of an older push run replays its stored needs outputs under the original event and ref, and would move :latest and :X.Y backward to that run's version. The gated step now derives the newest published release from the live releases list (contents:read, drafts and prereleases excluded) and refuses when this run's version is not the maximum. The check reduces to VERSION == max, because the run's own release already exists by the time the step runs. The all-steps assertion now requires the carrier to carry the floor as well as the event+ref guard, and the red-check spec gains a gated-but-floorless carrier mutation to prove the clause binds. * ci(release): pin the rebranded Homebrew formula text in the resolver-steps test * ci(release): keep npm stderr out of the dist-tag lookup The Publish step captured `npm view ... dist-tags.latest` with 2>&1, so a warning on stderr from a successful lookup became part of registry_latest and could make a new release compare as older than the registry latest, publishing it under the backfill dist-tag instead of latest. Send stderr to a temp file and read it only when the lookup fails. E404 still means the package was never published, and any other failure still fails closed. The pinned copy of the block in the resolver test is updated to match, and a stubbed-npm test covers a warning on success, a clean lookup, a genuine backfill, E404 and a non-E404 failure. * ci(release): bound the moving tags against every release page The Move floating tags step floored a stale re-run against the semver maximum of the first page of the releases list only. That list is ordered by creation, not semver, so once history passes one page the true newest version could sit on a later page and a stale re-run would move :latest and :X.Y backward. Read every page with gh api --paginate and take the semver maximum in bash. The gh call stays its own command substitution so a failure on any page still fails closed, and an empty list still yields no floor. The pinned copy in the resolver test is updated to match, and a stub-gh test serving two pages covers a stale run whose newer release is only on page 2, the run that is the newest, a gh failure, and an empty list. * ci(release): fail closed when a release asset has no uploader login The resolver ran its automation-only asset check only when the collected uploader list was non-empty. A null uploader login made jq emit an empty third field, the list collapsed to nothing, and provenance passed although no asset was shown to be automation-uploaded. Substitute a sentinel for a missing login so every asset row carries a value and a null, missing, or login-less uploader fails the existing comparison. The base64 name and id map is unchanged, and a release with no assets behaves as before. The resolver test covers an all-null uploader, a null login, a missing uploader key, a bot asset mixed with a null one, and the all-bot release.	2026-11-05 21:51:35 +00:00
beardthelionandKevin Codex	cd27756134	ci(triage): match test as a name segment, not just a whole component (#277) ... * ci(triage): match test as a name segment, not just a whole component #202 landed the namespaced-attribute fix, so main already detects #[tokio::test] and #[sqlx::test]. What it still misses is a third-party harness whose final path component carries `test` as an underscore segment. #[test_case(1)] and #[wasm_bindgen_test] both fall through and wrongly earn the PR a needs-tests label. Widen the final component to (?:[A-Za-z0-9]+_)*test(?:_[A-Za-z0-9]+)*, keeping the leading ::, the r# form, and the whitespace tolerance #202 added. Differential over 972 real file-patches from 400 commits: no verdict changes either way, so this is a forward-looking net for harnesses the tree does not use yet, not a fix for a current mislabel. Segment rather than substring, on purpose. addsInlineTest suppresses the label when it matches, so a false positive is the quiet failure: #[contest] or #[latest] would clear needs-tests on a PR that added no tests and nobody would notice. The cost is the harness names with no underscore, such as #[rstest], which stay unmatched and produce the loud, author-correctable failure instead. The segment classes are [A-Za-z0-9]+ rather than \w+ so that `_` is only ever the literal delimiter. \w+ contains `_`, which makes the repetition ambiguous and backtracks exponentially. That matters here specifically because the workflow triggers on pull_request_target, so f.patch is fork-controlled text on a privileged runner: with \w+, `#[test` followed by a long `_a` run took 71ms at 22 characters and roughly quadrupled every two more, which stalls the triage job for any contributor who can open a PR. The character-class form measures linear across every adversarial input tried. * fix(ci): allow Rust token separators before test attribute delimiters The needs-tests inline detector required ] or ( immediately after horizontal whitespace, so valid spellings like #[test /* rationale */] or a path split across added patch lines still triggered needs-tests. Match block comments, line breaks, and split-line closers instead. * ci(triage): scan attribute closers token-aware, bound to adjoined added lines The two-regex fallback matched a path-only attribute line and a ]/( line anywhere else in the patch, so unrelated records could complete each other and silently clear needs-tests; the mandatory separator regex also dropped legal //-to-newline and nested block comments, relabeling real tests. Replace both with one forward token scan that starts on the attribute's added line, continues only through immediately adjoined added lines under a hard bound, and answers 'no inline test' on anything uncertain. Fence the detector for extraction and add scripts/test-pr-triage-detect.mjs, run by a new pr-checks job, since pull_request_target executes the target branch's workflow and can never exercise the proposed body. * fix(ci): skip test attributes inside raw strings, block comments, and strings patchAddsInlineTest had no Rust lexical state, so a #[test_case] inside a raw string (r#"..."#), block comment, or regular string literal, followed by ] or ( on an adjoining added line, falsely cleared needs-tests. That is the silent failure direction the detector is supposed to fail loud against, and it is newly reachable because the underscore-segment regex introduced by this branch matches forms the old test\b did not. A pre-pass now walks every patch line (context and added, not removals) tracking raw strings (r#*", br#*", cr#*"), block comments (/* */ with nesting), regular strings (which can span newlines in Rust), and line comments. Added lines that start inside any of these are skipped as candidates. The pre-pass resets at hunk headers, is O(n) in patch length, and only sees what the patch shows: a non-code region opened before the first context line of a hunk is invisible, and the candidate is still scanned (conservative, fails toward the loud label). Also corrects the failure-direction comment in test-pr-triage-detect.mjs: false positives suppress needs-tests (they set touchedTests, which clears the label), while false negatives apply it (the loud, corrigible direction). The previous text had them backwards. Adds 7 regression cases: raw-string, block-comment, no-hash raw-string, and multiline regular-string false positives (reject), plus raw-string, block-comment, and regular-string closing before a real test attribute on the next added line (accept). Revert-check confirmed the pre-pass is load-bearing for both the nonCodeStart skip and the inString carry: disabling either makes the corresponding reject cases go RED, restoring makes all 31 pass. * fix(ci): recognize char literals so their contents cannot open string state The lexical pre-pass in patchAddsInlineTest missed real tests when an earlier added line contained a double-quote char literal like '"' or b'"'. The scanner saw the " inside the char token and set inString, which persisted to the next line and suppressed a following #[test]. Add bounded char-literal recognition: on seeing ', look ahead 2-4 chars (or to } for \u escapes) for a closing '. If found, skip the whole literal. If not, it is a lifetime or label, and the apostrophe is skipped as a regular char. Not a skip-to-next-' rule; lifetimes and labels fall through unchanged. * fix(ci): bound the \u{ escape scan to prevent quadratic stall The char-literal pre-pass scanned to the next } with no bound when it saw \u{, so a fork-controlled patch repeating '\u{ thousands of times made the triage job quadratic (21s for 30000 reps). Bound the scan to 20 chars past the { (a valid \u{...} holds at most 6 hex digits). Add a runtime probe for the malformed-escape shape. * chore: refresh PR head --------- Co-authored-by: Kevin Codex <[email protected]>	2026-11-05 16:33:22 +00:00
JATMNandClaude Opus 5.5	e2ae21795a	chore: rebrand user-facing gitlawb references to Twigpine (#520) ... * chore: rebrand user-facing gitlawb references to Twigpine Rename the product name in docs, README, CLI help/output, MCP tool descriptions, macOS app strings, installers, npm metadata and license headers from gitlawb/Gitlawb to Twigpine. Point GitHub links at the renamed Twigpine org (and fix the release workflow's repository guard, which still matched Gitlawb/node), and switch the public node hosts and website/install URLs to twigpine.com. Protocol and operator identifiers are intentionally unchanged (did:gitlawb, gitlawb://, signing domains, GITLAWB_* env vars, crate and binary names, X-Gitlawb-* headers, icaptcha and peer-network URLs); they need a follow-up migration. Co-Authored-By: Claude Opus 5.5 <[email protected]> * docs: point image pulls at ghcr.io/twigpine/node and fix macOS DMG name The release workflow publishes the container image to ghcr.io/${GITHUB_REPOSITORY,,}, which is ghcr.io/twigpine/node since the org rename; the old ghcr.io/gitlawb/node path no longer resolves for anonymous pulls. Point the RUN-A-NODE docker run example, the OSS readiness audit's claim about it, and the macOS app's generated compose file at the published path. The README listed the macOS DMG as "Twigpine Node.dmg", but build-macos-app.sh writes GitlawbNode-macOS.dmg (kept on purpose), so document the real file name. Co-Authored-By: Claude Opus 5.5 <[email protected]> * fix(infra): keep the AWS security group description to avoid replacement `description` on aws_security_group is ForceNew, so renaming it forced a replace of the in-use group on the next apply of an existing deployment. Restore the original text and note why it must stay until a separate infra migration. Co-Authored-By: Claude Opus 5.5 <[email protected]> * fix(macos): regenerate the app's compose file instead of treating it as a repo's Config.repoPath fell back to the data directory when no repo clone was found, and DockerCompose then picked up the compose file it had generated there as if it belonged to a repo. The generator therefore only ran on first launch, so upgraded installs kept their original compose, including the old ghcr.io/gitlawb/node image, which no longer serves anonymous pulls. repoPath now returns nil when there is no clone, and the app regenerates its compose file whenever no repo compose is in use. Project directory and named volumes are unchanged. Co-Authored-By: Claude Opus 5.5 <[email protected]> * fix(infra): default the AWS node image to ghcr.io/twigpine/node release.yml publishes to ghcr.io/${GITHUB_REPOSITORY,,}, which is now ghcr.io/twigpine/node; ghcr.io/gitlawb/node denies anonymous pulls, so the default made new deployments fail to start and the upgrade command fail on existing ones. On an existing deployment `terraform apply` only updates the upgrade SSM document in place (user_data drift is ignored); the new image is pulled on the next upgrade_command. Co-Authored-By: Claude Opus 5.5 <[email protected]> * fix(macos): keep operator edits to the generated compose, migrate only the image e04efe4 regenerated the app's compose file on every access, overwriting edits operators make to pass settings the generated environment block omits (e.g. GITLAWB_OPERATOR_STRICT_MODE). Generate the file only when it is missing; when it exists, keep it and rewrite only references to the retired ghcr.io/gitlawb/node image, keeping tags and digests. Co-Authored-By: Claude Opus 5.5 <[email protected]> * fix(macos): only migrate complete ghcr.io/gitlawb/node references Without a left boundary the migration also rewrote the retired path when it appeared inside a longer reference, such as a registry mirror (registry.example.com/ghcr.io/gitlawb/node), pointing it at a path the mirror may not have. Co-Authored-By: Claude Opus 5.5 <[email protected]> --------- Co-authored-by: Claude Opus 5.5 <[email protected]>	2026-11-04 21:58:29 +00:00
Kevin Codex	6668525f41	Merge pull request #286 from Gitlawb/fix/icaptcha-release-versioning ... fix(release): bump icaptcha-client with the rest of the workspace	2026-08-10 12:11:19 +08:00
Kevin Codex	7b365b0dc3	Merge pull request #249 from MikeTomlin19/agent/harden-release-tag-239 ... fix(ci): reject unsafe release dispatch tags	2026-08-10 10:22:10 +08:00
mike	1b65f70403	fix(ci): add tested release tag resolver	2026-08-05 01:22:10 -04:00
mike	0b990d687f	fix(ci): reject unsafe release dispatch tags	2026-08-05 01:22:10 -04:00
euxaristia	13f6948b3e	fix(ci): use anchored regex for pr-triage needs-tests detection ... Drop the head-file fetch and Rust lexer. Match test attributes on added lines only, with horizontal-whitespace anchoring, covering bare #[test], #[cfg(test)], namespaced forms (#[tokio::test(...)]), optional leading ::, and raw-identifier spellings. needs-tests is advisory, so the simpler matcher is enough.	2026-08-04 12:00:07 -05:00
euxaristia	8319c20660	fix(ci): make bounded scan inconclusive, bound blob size, handle no-newline marker, recognize raw-ident test ... Addresses jatmn's review: an exhausted 15-file scan or an oversized/ unreadable blob no longer asserts "no test found" (needs-tests was being applied on unverified evidence), blobs over 5MB are skipped before decoding, the "\ No newline at end of file" patch marker no longer miscounts as an added context line, and #[r#test] / #[tokio::r#test] raw-identifier attributes are now recognized.	2026-08-04 12:00:07 -05:00
euxaristia	fa51d428cb	fix(ci): stop bounded scan from aborting triage, use blob API, tighten matching ... Addresses jatmn's second round of review on pr-triage's needs-tests check: hitting the per-PR fetch cap now logs and stops scanning instead of throwing (which was skipping label reconciliation entirely), file content comes from the Git blob API instead of the Contents API to avoid the encoding: none gap for 1-100MB files, the added-line check now requires a non-whitespace attribute character (not just added whitespace) on the added line, stripped literals leave a placeholder so they can't glue adjacent tokens into a false match, and the namespaced-test regex now accepts a leading :: for absolute-path attributes.	2026-08-04 12:00:07 -05:00
euxaristia	24952b16be	fix(ci): handle nesting, multiline attributes, precise terminal matching, and bound Contents API scans	2026-08-04 12:00:07 -05:00
euxaristia	51f72ee3be	fix(ci): lex the full head file so state survives across diff hunks ... A diff hunk carries no lexical context from lines outside it, so scanning patch text alone could misclassify an added line that sits inside a comment or raw string opened earlier in the file (or outside the hunk's context window). Fetch the head-version file content via the contents API and lex it in file order, checking only the lines the diff actually added.	2026-08-04 12:00:07 -05:00
euxaristia	e458cc2d84	fix(ci): robustly strip comments/strings and match parameterized tests	2026-08-04 12:00:07 -05:00
euxaristia	474b42cfc5	fix(ci): update pr-triage needs-tests regex to anchor attributes and cover parametrized tokio::test	2026-08-04 12:00:07 -05:00
euxaristia	6452e996f8	fix(ci): recognize namespaced test attributes in pr-triage's needs-tests check ... The needs-tests detection regex only matched a literal #[test] or #[cfg(test) immediately after #[, so it missed namespaced test attributes like #[tokio::test] or #[async_std::test]. Any PR whose only new tests were async (the common pattern in this codebase, e.g. crates/gl/src/agent.rs) got falsely labeled needs-tests even when tests were added in the same commit. Reproduced against PR #198: it added two #[tokio::test]-only tests in the same commit as the source change, and the triage bot still commented needs-tests. Widens the regex to also match any attribute ending in ::test], verified against representative patches including PR #198's actual diff. Closes #201	2026-08-04 12:00:07 -05:00
beardthelion	4f424693b8	fix(release): bump icaptcha-client with the rest of the workspace ... icaptcha-client became a workspace member in #138, after #130 last wrote release-please-config.json, and nothing updated extra-files. release-please rewrites a version line only when the line carries the annotation AND the manifest is listed, so the crate was skipped every release and sat at 0.4.0 while the other five walked to 0.7.1. Nothing broke: every intra-workspace path dep is bare, so no version requirement could go unsatisfied; no crate is published to a registry; and the crate reads no CARGO_PKG_VERSION of its own. The cost was the workspace Cargo.toml comment asserting a lockstep rule the config did not keep, which is what the next person adding a crate would read and copy. Add the missing entry, and gate the rule so it cannot silently lapse again. The gate derives its required set from `cargo metadata`, so a new crate reds CI whether or not its author knows the gate exists, and it covers all three release-silent shapes: a member absent from extra-files, a member whose version line lost the annotation, and an extra-files path that is no longer a member. Next release moves icaptcha-client 0.4.0 to the current workspace version in one step. Nothing depends on that version, so the jump is cosmetic.	2026-07-30 08:54:33 -05:00
t	ebfe89d1c9	ci: declare the lock-sync toolchain and extend --locked to the shipping builds ... sync-release-lock ran cargo update --workspace on whatever Rust the ubuntu-latest image shipped, so the lock it wrote for the release PR came off an undeclared input. Install the same stable toolchain the other cargo jobs use. This does not make resolution identical (stable moves, and the MSRV gate checks the same lock on 1.91), it just removes the runner image from the equation. The release-binaries build, both Dockerfiles and the darwin path of build-bins.sh also resolved without --locked, so a lockfile disagreeing with the manifests was caught on the PR gates but not where artifacts are actually produced. Add --locked to all five. Two of those five cannot reach resolution today and are hardened for when they can: the Dockerfile dep-cache layer at line 24 copies four of the workspace's six member manifests, so cargo dies at workspace load and || true swallows it, and Dockerfile.bins pins rust:1.85 against a 1.91 MSRV. Both are pre-existing and left alone here. The live coverage comes from release.yml, Dockerfile:37 and build-bins.sh. The gitlawb-core dependency-purity gate still resolves without --locked on purpose, but its comment justified that by saying the build and test jobs do the same, which stopped being true when those jobs gained --locked. Restate the real reason: a stale lock would red that gate for something unrelated to the dependency closure it checks.	2026-07-28 16:28:27 -05:00
t	5091f2dba0	ci: fail the release-lock job on a failed dispatch, and scope release-PR discovery ... Two review findings on the sync-release-lock job. Fail on every dispatch failure. The step tolerated the 422 a release branch cut before pr-checks gained its workflow_dispatch trigger returns, on the theory that release-please would rewrite the branch and heal it. A rewrite only happens when a later releasable commit lands, so an open release PR nobody touches is never rewritten, and the tolerated path left the lock-sync commit this job had just pushed on the release head with no --locked validation. That is the exact state the job exists to prevent. No release PR is open today, and after this lands every release branch is cut from a main that already carries the trigger, so the tolerance protects nothing. Scope the fallback discovery. `gh pr list` includes fork PRs and a fork's headRefName is bare, so matching on the `release-please--` prefix alone let any fork nominate the branch this job checks out, commits to, and dispatches on: actions/checkout then cannot resolve that fork-only ref here, failing the job and leaving the real release PR unsynced and unchecked. Discovery now requires the head repo to be this repository and the author to be the release-please app, and selects the highest PR number instead of relying on gh's undocumented default ordering. Verified the new filter against the hostile inputs, using the jq program as extracted from the parsed YAML: a fork lookalike, a fork PR claiming the app author, and a same-repo human branch named `release-please--*` are all rejected; the real release PR is still selected when listed after a fork and when a second component release PR is present; an empty listing yields empty. The previous expression selected the fork's branch on all three hostile cases.	2026-07-27 18:34:28 -05:00
t	b605b2269a	ci: dispatch pr-checks on the release branch so GITHUB_TOKEN pushes still get validated	2026-07-27 18:27:13 -05:00
t	52a8f010b7	ci: lock-sync the release PR and build --locked so lock drift fails loudly (#242)	2026-07-27 18:27:13 -05:00
beardthelion	e9d6aadd58	ci(core): add dependency-purity allowlist gate for gitlawb-core ... gitlawb-core is embedded by gl, git-remote-gitlawb, and the node daemon, so it must stay lean. Add a CI gate that fails if its normal (non-dev, non-build) transitive dependency tree gains any crate not on an explicit allowlist. The allowlist is exhaustive-by-construction: a new heavy dependency reds CI whether or not anyone thought to ban it, which a denylist cannot do. The checker resolves like the build/test jobs (no --locked, since the committed lockfile can lag the manifests) and snapshots/restores Cargo.lock so a local run leaves no working-tree side effects. - ci/gitlawb-core-allowed-deps.txt: the 83 current normal deps, plus the regeneration command and rationale in the header - scripts/check-gitlawb-core-deps.sh: recompute and diff; hard-fail on a non-allowlisted crate, note-only on a stale (removed) allowlist entry - pr-checks.yml: new hard-fail core-deps-purity job, matching the pinned action SHAs and style of the sibling jobs	2026-07-22 12:04:34 -05:00
Kevin Codex	b484a24955	Merge pull request #237 from Gitlawb/fix/issue-228-windows-ci ... ci: add a non-blocking Windows test lane for the shipped CLIs	2026-07-22 23:28:53 +08:00
Kevin Codex	c681af70db	fix(gl): anchor cert --verify to a trusted issuer; pin canonical payload form ... Settles the review's design question: a valid signature alone proves internal consistency — the payload was signed by whatever key the certificate itself names — so a hostile source could self-sign and pass. --verify now also requires the issuing node_did to match a trusted anchor: --expect-node when given, else the DID of the node being queried; unanchorable (node info unreachable, no --expect-node) fails closed. The displayed verdict wording says what it proves. Also from review: - A frozen-literal test pins gl's payload serialization to the node's canonical byte form, so serialization drift (new field, a preserve_order feature unifying across the workspace) fails a test instead of silently rendering every real certificate INVALID. - The printed unalias remediation uses the idempotent-safe 'unalias gl 2>/dev/null' form everywhere (doctor, install.sh, brew caveats), matching what doctor already suggested. - Acknowledged, deliberately left: bash-side alias detection and 127/8-range loopback matching (noted as out of scope for a warning-level heuristic).	2026-07-22 21:43:44 +08:00
t	c71bba5cd1	ci: add a non-blocking Windows test lane for the shipped CLIs (#228) ... We ship `gl` and `git-remote-gitlawb` to Windows users via release.yml but run no Windows tests: pr-checks.yml is ubuntu-only, and the single windows-latest in the repo is the release build step, which never runs `cargo test`. So changes to the shipped Windows CLIs (e.g. the multi-round fetch fix in #192) reach Windows users with zero test runs on Windows. Add a `test-windows` job that runs `cargo test -p gl -p git-remote-gitlawb` on windows-latest with `continue-on-error: true`, mirroring the informational beta lane: a Windows-only failure is visible on the PR but does not block merge. The two crates are the client CLIs and depend on no database, so the job needs no Postgres service (service containers do not run on windows-latest regardless); git is preinstalled on the runner for the real_git_fetch harness. This makes the #[cfg(windows)] regressions from #192 load-bearing instead of never-run. Un-gating the currently #[cfg(unix)] harness tests is the deferred follow-on in #228.	2026-07-22 08:01:06 -05:00
Kevin Codex	e3df51d76c	fix: warn about the oh-my-zsh gl alias at install time ... install.sh detects the common shadowing setup (omz git plugin, no unalias in ~/.zshrc) and prints the fix right after installing; the generated Homebrew formula gains a caveats block saying the same, so brew install/upgrade surfaces it too. Catching this at install time beats letting the first gl invocation print git's error.	2026-07-22 20:51:23 +08:00
Kevin Codex	4465beb0c2	ci: make the red-main alert a state tracker, not an event echo ... Address review: the alerter reflected CI events rather than main's current state, which could both cry wolf and stay silent. - Freshness gate: a run only opens or closes anything when its commit is still main's tip — re-running an old red run after a fix has landed no longer opens a false stop-ship issue (and a stale green can't close a current one). - Trigger covers timed_out and startup_failure — both are red mains that previously alerted nothing. cancelled stays excluded as a human abort. - Recovery: a green run on the tip comments and closes open ci-main-red issues, turning the alert into a current-state signal. - queue: max — the single-slot pending queue let a later completion displace a pending red alert under burst merges. - Issue lookup filters out pull requests carrying the label. - github-script converged on v8.0.0 (SHA verified against the tag) in pr-triage too.	2026-07-22 16:20:15 +08:00
Kevin Codex	82efd60229	ci: serialize main-red-alert runs to prevent duplicate issues ... Two near-simultaneous red-main events could both pass the open-issue lookup and each create a ci-main-red issue (or 422 racing createLabel). A workflow concurrency group with cancel-in-progress: false serializes them; the default single-slot pending queue is enough since a superseded alert's state is re-read by the newest run anyway.	2026-07-22 16:18:38 +08:00
Kevin Codex	e8f193a5df	ci: guard against semantic merge races breaking main ... Two individually-green PRs can break main in combination when the second merges on checks that ran against a stale base (#113 + #145 compiled fine apart, but #145 added a field to ReceivedRefUpdate that #113's new test initializer never set — E0063 on merged main, fixed by #229). - pr-checks: add merge_group trigger so the suite can gate a GitHub merge queue, which re-validates every PR against the true merged result. - main-red-alert: new workflow that opens/updates a labeled issue whenever a push-to-main PR Checks run fails, so a red main is loud instead of silent (release-please PRs inherit main breakage, as #182 showed).	2026-07-22 16:18:38 +08:00
Kevin Codex	05e434d906	ci: lowercase the ghcr image reference for the digest push ... buildx's --output name= does no lowercasing (unlike metadata-action), so ghcr.io/Gitlawb/node is an invalid reference and the per-arch digest push would fail outright. Compute the lowercase image once in the resolve step and reuse it, matching the manifest job.	2026-07-22 15:23:42 +08:00
Kevin Codex	7593540f62	ci: build release Docker images natively per-arch, merge with a manifest ... The v0.6.0 release run's multi-arch Docker build wedged for 3+ hours emulating the arm64 Rust release build under QEMU, and — because the workflow concurrency group is cancel-in-progress:false — every queued release run sat blocked behind it. - docker is now a matrix: amd64 on ubuntu-latest, arm64 on ubuntu-24.04-arm — both native, mirroring release-binaries. Each leg smoke-tests the image (--version must report the released version) and pushes by digest only. - docker-manifest stitches the digests into the :X.Y.Z, :X.Y and :latest tags with buildx imagetools, so a half-finished matrix can never publish a partially-tagged image. - workflow_dispatch grows docker_backfill_tag (and npm_backfill_tag becomes optional): each channel backfills independently for an already-cut release — needed to publish the missing v0.6.0 image.	2026-07-22 13:13:37 +08:00
Kevin Codex	1fddc81b7c	ci: skip release-please on manual npm backfill dispatch ... A backfill run must be a narrow npm-only operation. Without an event guard, workflow_dispatch also ran release-please with contents+PR write permissions and could create or advance a release PR as a side effect. npm-publish is unaffected: its !cancelled() condition tolerates the skipped need, and all other jobs already gate on release_created.	2026-07-22 12:10:25 +08:00
Kevin Codex	62d73c4165	ci: switch npm publishing to trusted publishing (OIDC) + manual backfill ... The npm-publish job silently skipped every release since 0.4.x because NPM_TOKEN was never configured — the guard made the job green while @gitlawb/gl sat at 0.3.7 on the registry. Replace token auth entirely: - npm Trusted Publishing: the job's GitHub OIDC identity (id-token: write) is exchanged with the registry; no long-lived secret to configure, leak, or rotate. Provenance is attested automatically. All five @gitlawb packages exist on the registry now, so OIDC no longer has the first-publish bootstrap problem that forced token auth in #127. - Fail loud: the secret guard is gone. If a package's trusted publisher isn't configured on npmjs.com, the publish errors visibly instead of green-skipping. - setup-node 24 + npm >=11.5.1 (OIDC exchange support; older npm falls back to absent token auth with a confusing error). - workflow_dispatch backfill: publish an already-cut GitHub release (e.g. v0.6.0) to npm by tag — binaries come from the existing release assets, so only the npm-publish job runs.	2026-07-22 11:50:44 +08:00
Kevin CodexandClaude Fable 5	6cff5286b4	fix(release): build aarch64-musl natively on arm64 runners, replace retired macos-13 (#155) ... The aarch64-unknown-linux-musl leg cross-compiled a musl target with the glibc toolchain (CC=aarch64-linux-gnu-gcc), which aws-lc-sys cannot build with — the Build step failed on every release and no aarch64-linux asset has ever shipped. Build it natively on ubuntu-24.04-arm instead (free for public repos), using the same stock musl-tools recipe as the x86_64 leg, and drop the smoke-test skip since the binaries now run on their build runner. The x86_64-apple-darwin leg targeted macos-13, which GitHub has retired; the job queues forever and holds the release concurrency group open. Move it to macos-15-intel. Both failures also skipped the downstream npm-publish, homebrew-bump, and web-sync jobs, so v0.4.0 and v0.5.0 were never published there. Co-authored-by: Claude Fable 5 <[email protected]>	2026-07-06 10:39:12 +08:00
beardthelionandt	e3a20317d5	ci: pin cargo-audit on the blocking gate, float the weekly canary (#151) ... The PR-blocking audit job ran `cargo install --locked cargo-audit` with no version, so the scanner floated to the latest release on every run; a regressed release could red the PR gate repo-wide with no code change. Pin it to 0.22.2, the version the unpinned install currently resolves to and the one the audit-gate learnings were validated against. `cargo install --version X` is an exact pin (not a caret), so this freezes the binary while the advisory DB is still fetched fresh at scan time, which is what the gate actually keys on. Keep a forcing function so the pin cannot silently rot: leave the weekly Scheduled Audit deliberately unpinned as a drift canary. That job is cron-only (never pull_request/push-triggered), so it never gates a merge; floating there surfaces a regressed latest as a red weekly run or a changed report, which is the drift signal, without ever blocking a PR. Also assert the installed version after the pinned install so an accidental future unpin fails loud instead of silently running a different scanner, include the version in the rust-cache key so cache hits stay deterministic, and document the yank/bump path in the step comment. Co-authored-by: t <t@t>	2026-07-05 17:10:45 +08:00
Kevin Codex	3f08d9936b	Release automation (#127) ... * ci(release): add Windows target and cross-platform packaging Add x86_64-pc-windows-msvc to the release matrix. Windows ships a .zip (7z) while unix keeps .tar.gz; a portable sha256 helper handles sha256sum/shasum/certutil. Smoke test and packaging now resolve the .exe suffix on Windows. Release attach uses globs to cover both formats. * feat(release): vendor npm wrapper and auto-publish to npm Vendor the @gitlawb/gl npm wrapper (optionalDependencies platform-package pattern) into npm/, replacing the standalone gl-npm repo. A new npm-publish job downloads the four unix release tarballs from the GitHub release, lays in the binaries, stamps every package.json with the release version, and runs npm publish --provenance. Guarded on NPM_TOKEN so it no-ops until the secret is configured. * feat(release): auto-bump Homebrew tap on release Add a homebrew-bump job that regenerates Formula/gl.rb in Gitlawb/homebrew-tap from the release tarballs' sha256 checksums (macOS+Linux, arm64+x64) and pushes the bump. Guarded on HOMEBREW_TAP_PAT so it no-ops until the secret and tap repo exist. Enables 'brew install gitlawb/tap/gl'. * feat(release): Windows installer and web auto-sync on release Add install.ps1 (Windows PowerShell installer, source of truth in node). Add a web-sync job that opens a PR to Gitlawb/web on each release syncing install.sh + install.ps1 + public/version.json, plus idempotent drift fixes (remove orphaned public/bin, fix stale skill.md releases link). Guarded on WEB_SYNC_PAT. Vercel deploys on merge. * docs(readme): document npm, Homebrew, and Windows install methods * ci(release): build only CLI binaries on Windows The gitlawb-node daemon targets Linux/macOS; building it on Windows risks a compile failure that would block the whole release (npm/brew/web jobs all depend on release-binaries). Windows now builds only gl + git-remote-gitlawb; the smoke-test and package loops follow the per-target binary list. * fix(release): reconcile release-please manifest with v0.3.9 and bump action to v5 The manifest still said 0.3.8 while the only tag is a manual v0.3.9 and Cargo.toml is 0.3.9, so release-please had no valid baseline (no v0.3.8 tag) and never cut a release. Set the manifest to 0.3.9 so release-please diffs v0.3.9..main and opens a proper release PR for the next version. Also pin release-please-action to v5.0.0 to match the openclaude pipeline. * ci(release): set explicit minimal permissions on homebrew-bump and web-sync CodeQL flagged both jobs for lacking a permissions block, leaving GITHUB_TOKEN at broad default scopes. Both only need contents: read (cross-repo writes use dedicated PATs). * fix(release): harden installers and workflow per review - install.ps1: fail closed if the .sha256 cannot be fetched; pure ASCII (no BOM) - npm install.js: fail (exit 1 / throw) when the platform payload or a binary is missing instead of reporting a successful but broken install - workflow: top-level permissions {} + persist-credentials: false on read-only checkouts - docs: PowerShell-only Windows note; text language tag on npm/README fenced block * fix(installer): require core CLI binaries on Windows install.ps1 only failed when zero binaries were present, so a partial archive missing gl.exe could still report success. Require gl.exe + git-remote-gitlawb.exe (gitlawb-node.exe stays optional, matching the Windows build) and fail early if a required binary is absent. * ci(release): make Windows non-blocking and stop persisting cross-repo PATs - release-binaries: continue-on-error for the Windows leg so a failure there no longer fails the job or blocks npm/Homebrew (matches the best-effort intent) - homebrew-bump/web-sync: persist-credentials: false on the PAT-backed checkouts; supply the token only on the git push via an x-access-token URL * fix(release): gate web-sync on binaries and detect native arch in installer - web-sync now needs release-binaries so the site never advertises a release whose artifacts failed to build - install.ps1 reads PROCESSOR_ARCHITEW6432 before PROCESSOR_ARCHITECTURE so a 32-bit PowerShell host on x64 Windows is not wrongly rejected * ci(release): make npm publish idempotent across reruns Skip any @gitlawb package version already on the registry so rerunning the job after a partial publish no longer errors on the first existing package and leaves the wrapper/platform packages out of sync. * fix(release): harden web-sync and tap commit per review - web-sync: verify the Windows ZIP + .sha256 exist on the release before syncing install.ps1/version.json (Windows is best-effort, so a failed leg must not make the site advertise a missing installer) - stage generated files before the no-op check so brand-new untracked files (formula, version.json) are committed instead of skipped - gh pr create: only swallow the 'PR already exists' case; surface auth/API errors - npm README: scope npm/yarn/pnpm/bun install to macOS/Linux/WSL, point native Windows at the PowerShell installer * ci(release): guard npm-publish before doing any work Check NPM_TOKEN in an up-front guard step (like homebrew-bump/web-sync) and skip checkout, release downloads, extraction, version rewrites, and publish when it is absent, instead of doing all that work and bailing in Publish. * fix(release): robust web PR error handling and Windows installer hardening - web-sync: capture gh pr list output so a real API/auth failure surfaces instead of being misread as 'PR already exists' - install.ps1: enforce TLS 1.2 and use -UseBasicParsing so downloads work on stock Windows PowerShell 5.1; remove binaries no longer shipped on upgrade so an in-place update can't leave a mixed-version install; handle an empty user PATH without a leading separator	2026-06-30 16:56:58 +08:00
beardthelion	1960f6b28f	ci(triage): label fork PRs and stop auto-closing PRs that only lack an issue link (#107) ... * ci(triage): run PR triage on pull_request_target so fork PRs can be labeled Fork PRs run the triage job with a read-only GITHUB_TOKEN under the pull_request event regardless of the workflow's declared write permissions, so addLabels/createComment fail with 403 'Resource not accessible by integration' for every external contributor (e.g. #103). Switch the trigger to pull_request_target, which runs in the base-repo context with the declared write token. The job never checks out or runs PR head code (it only reads context.payload and the pulls/files API and manages labels), so it carries none of the untrusted-code-execution risk that makes pull_request_target dangerous. * ci(stale): drop needs-issue from auto-close trigger, keep it advisory needs-issue flags PRs that link no issue, but linking one is a SHOULD in CONTRIBUTING (required only for protocol-level changes), and many legitimate small PRs (docs, obvious fixes) have none. Coupling it to the 14/21-day auto-close was overreach: a substantive, well-described PR could be closed purely for missing an issue reference. Keep needs-issue as an advisory label and guidance nudge; leave only needs-description (an empty/thin body, a far stronger low-effort signal) as the auto-close trigger. * ci(stale): align stale message with description-only close condition The auto-close trigger narrowed to needs-description, but the stale message still told contributors a PR could close for a missing linked issue. Drop that clause so the warning matches what actually closes a PR. * style(ci): replace em dashes in triage/stale contributor messages * docs(contributing): match stale close policy to description-only trigger Auto-close is driven by needs-description only; a missing issue link stays advisory. Align the contributor docs with the workflow so the OR wording no longer implies a missing issue link can close a PR.	2026-06-27 23:14:41 +08:00
beardthelion	2202b0097f	feat(node): enforce per-route authorization across the REST and GraphQL surface (#87) ... * test(node): add HTTP-API integration harness with CI Postgres Adds a #[cfg(test)] test-support module: a migrated AppState over a real #[sqlx::test] pool (test_state), a DB-free variant (test_state_lazy), the assembled router (app), and signed_request_as for injecting an authenticated DID without RFC-9421 signing. A test-callable run_migrations reuses the production migrate() path, and sqlx gains the macros/migrate features so #[sqlx::test] is available. CI's test job gets a postgres service behind a pg_isready health gate so the DB-backed tests can run. The proving test exercises the owner gate on PUT /visibility end to end: non-owner is rejected with 400, owner succeeds. signed_request_as sets a JSON content-type so the Json extractor does not 415 before the handler runs. * feat(node): enforce per-route authorization on REST mutations Closes the missing-authorization cluster on the signed REST surface, where 'authenticated' meant 'any participant' because identities are permissionless. Adds two shared helpers in api/mod.rs: require_repo_owner (403) and did_matches, a method-safe DID comparison that collapses the full did:key vs bare short form without matching across methods (did:web/did:gitlawb share the base58 space). Per-route gates: - merge_pr is owner-only (subsumes branch protection); close_pr and close_issue allow the repo owner or the PR/issue author (the issue author is read from the git-JSON blob, with a None author falling back to owner-only). - create_review, create_comment, create_issue_comment, and create_bounty are read-gated (participants need read access, not ownership; this also closes the private-repo bounty leak). - the task handlers bind the acting DID to the authenticated signer. - dispute_bounty, previously ungated, now requires the creator or claimant; the other bounty and replica identity checks switch from raw equality to did_matches. A source-level guard test asserts every in-scope mutation handler still carries its expected gate marker, so a removed gate or an unclassified new route fails CI. DB-backed tests (via the integration harness) cover merge owner-only and the task signer-binding; did_matches has unit coverage for the cross-method case. * feat(node): authenticate GraphQL mutations GraphQL mutations were a fully unauthenticated parallel write path to the task system (N2): the /graphql route had no auth layer and the resolvers took the acting DID as plain arguments. Apply optional_signature to the /graphql POST so a verified DID is attached when a signature is present (queries stay open), and thread it into request-scoped data. Each mutation now reads the verified signer via require_signer and binds the acting DID to it (rejecting an unsigned request or one whose claimed delegator/assignee/by_did differs from the signer), reusing did_matches for the comparison. Read-only subscriptions (/graphql/ws) are left open. A resolver-level test covers the unsigned, mismatched, and matching cases. * fix(node): path-scope the get_tree visibility gate get_tree gated on the repo root ("/") regardless of the requested subtree, so a caller denied a withheld subtree could still enumerate its filenames and blob SHAs (N3). Gate on the requested path instead, mirroring get_blob, and reject traversal segments. A cross-caller test shows the rejection is path-scoped: a non-reader is denied the withheld subtree but passes the gate on a non-withheld path. * fix(node): authorize task completion and read-gate PR/issue creation Two authorization gaps surfaced by an adversarial review of the auth stack, both necessary-but-insufficient siblings of fixes already in it. complete_task/fail_task bound the acting DID to the signer but never checked the caller against the task's assignee, and finish_task updated by id alone, so any authenticated did:key could finish or fail any task in any state. Load the task and require the caller to be its assignee (REST and GraphQL), and add an 'AND status=claimed' predicate so only a claimed task transitions. The now-unused by_did request fields are removed. create_pr/create_issue resolved the repo with get_repo but skipped authorize_repo_read, so a non-reader could open a PR (firing the owner's webhooks) or file an issue against a private repo they cannot read. Gate both on read access like their create_review/create_comment/create_bounty siblings. Adds DB-backed tests covering non-assignee rejection (including the empty-body bypass), the claimed-state predicate, and private-repo PR/issue denial. * test(node): harden the authz drift-guard against false passes The guard asserted a marker STRING appeared in each handler body, which a bare identifier in a comment or log line could satisfy even after the real gate was deleted, and its body slice over-ran on any handler not declared exactly 'pub async fn'. - Markers are now gate-shaped: a call (require_repo_owner(, did_matches(, authorize_repo_read() or a binding/comparison expression (caller != &record.owner_did, let owner_did = auth.0), never a field name. - Full-line comments are stripped before matching, so a gate that survives only as a comment no longer counts as enforced. - The body slice bounds at the next top-level fn item across pub async, pub(crate) async, async, pub, and bare fn forms. - Adds rows for the now-read-gated create_pr/create_issue and for fork_repo, create_repo, set_profile, claim_bounty; register is excluded with a note (it trusts the body did, tracked as P3 D3-1). - Adds a self-test proving a comment-only marker does not satisfy a row. * fix(node): P3 consistency batch from the adversarial review Five low-severity items from the stack review. The sixth (a did_matches false-negative on non-canonical multibase encodings of the same key) is left as-is: it is a self-inflicted owner lockout unreachable with the conforming client, never an impersonation vector, so decoding keys in the auth path is not warranted. - register bound the registered DID to the request body, not the signer, so a signed caller could create or refresh a trust row under a victim DID. Bind it to auth.0 (403 on mismatch) and add a drift-guard row now that it is gated. - Bounty submit/approve/cancel returned 400 for an identity-mismatch denial where dispute returned 403. Return Forbidden, matching the rest of the surface. - The visibility/protect owner gates (require_owner and the protect.rs inline checks) returned 400 for a non-owner; return 403 like require_repo_owner, and assert the exact code in the harness. - get_tree rejected ./.. segments but not empty interior segments, so a path like secret//x could reach git while the gate saw a different string. Reject empty interior segments too (the empty path remains the root listing). - Tighten the get_tree path-scope test to assert an exact 200 on the non-withheld branch so a future upstream 4xx/5xx cannot masquerade as gate-pass. Adds a register-binding regression test. * fix(node): unify protect/visibility owner checks on did_matches CodeRabbit flagged that protect_branch/unprotect_branch and visibility's require_owner used a trailing-segment owner compare (split(':').next_back()) distinct from the hardened did_matches, so the two owner-match idioms could drift. It is not exploitable (the authenticated caller is always the full did: form while the short form is a bare suffix, so no wrong caller passes), but it is a real inconsistency, and the trailing-segment form is a false-negative for a bare-stored owner. Replace both with did_matches (collapses did:key full vs bare on both sides, never across methods), and extend the drift guard to assert require_owner itself uses did_matches, not just that it is called.	2026-06-23 13:16:35 +08:00
beardthelion	7533b74f79	ci: harden PR pipeline (audit, MSRV, beta matrix, release + docker smoke) (#57) ... * ci: harden PR pipeline (audit, MSRV, release build, beta matrix, docker smoke) Restructure the single stable-only fmt+clippy+test job into separate jobs: fmt+clippy (stable), a test matrix on stable+beta (beta non-blocking), build --release, cargo audit, an MSRV check pinned to Rust 1.91, and a Docker build smoke test mirroring the release workflow. Add concurrency cancellation for superseded PR runs and per-job timeouts. Declare rust-version = "1.91" in [workspace.package] and inherit it in every member crate so the stated MSRV is a real, CI-enforced contract. * ci: disable credential persistence on checkout steps Set persist-credentials: false on all six actions/checkout steps. These jobs run untrusted PR code via cargo and docker build, and checkout's default leaves the GITHUB_TOKEN in the local git config where that code could read it. None of the jobs push or otherwise need git credentials. * fix(deps): bump rustls-webpki to 0.103.13 Clears three advisories flagged by the new cargo audit job: a reachable panic in CRL parsing and two name-constraint bypasses (wildcard and URI). Semver-compatible patch, lockfile only, no manifest or code change. cargo also corrects a stale lockfile reference while rewriting Cargo.lock: data-encoding-macro-internal 0.1.17 (a build-time macro crate) requires syn 1.x, and its dependency entry is re-pinned from syn 2.0.117 to the 1.0.109 it actually resolves to. Unrelated to the bump, unavoidable when cargo touches the lock, and the more-correct state. * ci(audit): suppress no-fix advisories with justification and a re-check The cargo audit job stays hard-fail. The remaining three advisories have no available upstream fix, so they are suppressed in .cargo/audit.toml (the location cargo-audit actually reads; a repo-root audit.toml is not read), each with an inline rationale and removal condition: - RUSTSEC-2026-0118/0119 (hickory-proto 0.25.2, via libp2p-dns 0.44): reachable, since the node wraps QUIC in the system DNS resolver. No fix until libp2p-dns ships hickory 0.26. Accepted, reachable risk. - RUSTSEC-2023-0071 (rsa Marvin): in the lockfile via sqlx's unconditional sqlx-mysql, but not linked (sqlx is postgres-only), so no RSA oracle exists; no upstream fix regardless. Two controls keep the suppression honest. The PR audit job echoes the active ignore list to the job summary, so a green check is not read as advisory-clean. A weekly scheduled job re-runs cargo audit without the ignores for visibility and hard-fails if hickory-proto moves off 0.25.x while the ignores remain, so the suppression cannot silently outlive the upstream fix. * ci(audit): reference tracking issue #76 in the hickory ignores * ci(audit): treat hickory-proto removal as drift too The scheduled drift guard only failed when hickory-proto was present and off 0.25.x, so removing it from Cargo.lock entirely left the RUSTSEC-2026-0118 /0119 ignores in place with the guard still passing. Treat an absent version as drift as well, so dead ignore entries are caught whether the dependency is bumped past 0.25.x or removed.	2026-06-22 19:02:38 +08:00
beardthelion	197c4e1cf1	Add contributor templates, issue forms, and PR triage automation (#58) ... * docs: add pull request template * docs: add structured issue forms and template chooser config * chore: add CODEOWNERS for CI, release, and security paths * ci: add quality-signal PR triage workflow * ci: add label-gated auto-stale workflow for ignored PRs * docs: document PR quality bar and triage/stale policy in CONTRIBUTING * ci: recognize inline #[cfg(test)] tests and fix triage comment grammar	2026-06-19 22:52:32 +08:00
3 people	a5d2264cc0	ci(release): smoke-test packaged binaries' --version before attaching (#59) ... The release-binaries job packaged gl, git-remote-gitlawb, and gitlawb-node into the release tarball but never ran them, so a broken artifact could ship. This is the follow-up promised in #30 and the gap noted in OSS-READINESS-AUDIT ("release binaries are not smoke-tested after packaging"), now unblocked by git-remote-gitlawb gaining --version. Run each binary's --version after build and assert the released version string is present. Skipped for aarch64-unknown-linux-musl, which is cross-compiled and can't execute on the x86_64 runner. Co-authored-by: FrozenRaspberry <[email protected]> Co-authored-by: Claude Opus 4.8 <[email protected]>	2026-06-19 22:50:35 +08:00
Kevin CodexandClaude Opus 4.7	14a1cdd656	chore: bump v0.3.9, fix all clippy warnings, enforce strict CI lint (#12) ... * chore: bump v0.3.9, fix all clippy warnings, enforce strict CI lint - Bump workspace version 0.3.8 → 0.3.9 - Fix 22 clippy warnings across 17 files: - Replace .last() with .next_back() on DoubleEndedIterators (8 instances) - Remove useless .into() conversions on anyhow::Error (3) - Remove needless borrows in gl init (3) - Replace map_or with is_some_and/is_none_or (3) - Collapse consecutive str::replace into char array pattern (2) - Change &PathBuf to &Path in function signatures (2) - Remove dead code: unused init_bare_repo function - Collapse if-let into outer match arm - Replace vec![...] with array literal - Introduce RecordAnchorInput struct (10 params → 1 struct) - Fix always-true boolean condition in gossip task guard - Promote clippy from advisory to strict (-D warnings) in CI Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]> * style: cargo fmt Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]> --------- Co-authored-by: Claude Opus 4.7 (1M context) <[email protected]>	2026-05-27 10:28:14 +08:00
Kevin CodexandOpenClaude	ec4ef81770	feat: embed bootstrap peers seed list for automatic network discovery (#1) ... * feat: embed bootstrap peers seed list for automatic network discovery A fresh \`docker compose up\` now joins the Gitlawb network with zero manual peer configuration. The node parses an embedded \`bootstrap-peers.json\` on startup and merges the entries into both the HTTP gossip task and the libp2p Kademlia bootstrap. - Add \`bootstrap-peers.json\` at repo root (versioned schema, PR-friendly) - New \`bootstrap\` module in the node crate (parse + merge_seeds) - Wire into \`main\` after \`Config::parse\` - Operators can opt out via \`GITLAWB_BOOTSTRAP_DISABLE_SEEDS\` for isolated dev networks Also in this commit: - \`cargo fmt --all\` over the entire workspace (no logic changes) - Downgrade CI clippy step to advisory (\`continue-on-error: true\`) until the existing lint backlog is cleared. fmt + tests stay strict. Co-Authored-By: OpenClaude <[email protected]> * test: cover bootstrap merge logic with unit tests Refactor bootstrap.rs into pure functions (parse_seed_list, merge_into_vecs) so the parse + merge logic can be tested without constructing a Config or mutating process-global env vars. Adds 11 tests covering: - valid v1 list parses - unknown version is rejected - malformed JSON is rejected - empty / missing peers array - merge appends new http + p2p entries - merge dedupes against existing entries - invalid p2p_multiaddr is skipped (http still added) - empty strings are skipped - null optional fields are tolerated - the canonical bootstrap-peers.json shipped in the repo always parses (regression guard against future schema changes) Co-Authored-By: OpenClaude <[email protected]> * fix(security): reject path traversal in repo_store local_path Closes CodeQL alert #4 (Uncontrolled data used in path expression). Both `owner_did` and `repo_name` come from URL parameters and were used unsanitized to build a filesystem path, so a request like `/did:key:foo/../../../etc/passwd.git/info/refs` could escape the repos directory. Fix: - New `validate_owner_did` and `validate_repo_name` enforce a strict allowlist before path construction (alphanumeric + `: . _ -` for DIDs; alphanumeric + `. _ -` for repo names; rejects empty, `..`, leading `.` or `-`, slashes, backslashes, null bytes, overlong inputs). - `RepoStore::local_path` now returns `Result<…>` and the 5 callers propagate the error. - Defence in depth: even after the allowlist passes, the joined path is checked to still be rooted at `repos_dir`. - 17 unit tests cover normal DIDs/repo names plus every malicious shape (`..`, `/`, `\`, leading dot/dash, null byte, overlong, did:web with dots). Co-Authored-By: OpenClaude <[email protected]> * fix(security): add Path::components barrier for CodeQL rust/path-injection CodeQL's rust/path-injection query did not recognise the existing allowlist validator as a sanitiser, so it kept flagging local_path even though every malicious input was already rejected. Adds an explicit Path::components() walk after path construction that rejects any ParentDir or CurDir segment. This is the barrier pattern CodeQL recognises — and is also genuine defence-in-depth in case a future change ever weakens the allowlist. Allowlist + starts_with + components walk = three independent layers between attacker-controlled HTTP input and filesystem access. All 17 existing tests still pass. Co-Authored-By: OpenClaude <[email protected]> --------- Co-authored-by: OpenClaude <[email protected]>	2026-05-13 19:11:54 +08:00
Kevin CodexandOpenClaude	6b9c5278d4	chore: initial public release ... Open-source release of the gitlawb node, CLI, and git remote helper. - gitlawb-node: Axum HTTP server with git smart-HTTP and DID auth - gl: command-line client (identity, repos, PRs, MCP server) - git-remote-gitlawb: git remote helper for gitlawb:// URLs - gitlawb-core: shared crypto primitives (DID, CID, HTTP sigs, UCAN) Dual licensed under MIT or Apache-2.0. Co-Authored-By: OpenClaude <[email protected]>	2026-04-29 16:34:27 +08:00