Compare commits

...
Author SHA1 Message Date
beardthelion 20f725e25e Merge remote-tracking branch 'origin/main' into feat/commit-status-surface
# Conflicts:
#	crates/gitlawb-node/src/db/mod.rs
2026-11-12 19:18:24 -06:00
beardthelion e8952f517a fix(node): close_pr fallback head fills an empty head only (#304)
A head stored by a push between the handler's read and the UPDATE is no
longer replaced by the fallback.
2026-11-12 18:54:07 -06:00
beardthelionandanandh8x 5a7576b682 feat: add push, fetch, and sync latency metrics (#580)
* feat: add push fetch and sync latency metrics

* Merge remote-tracking branch 'origin/main' into pr-362

* fix(node): sample sync lag at mirror visibility, clamp untrusted origin ages

- the lag observation moves to the upsert point, before the
  encrypted-blob tail and mark_sync_done, so the metric reads
  push-to-visible rather than push-to-fully-replicated
- a quarantined mirror is admitted but withheld from readers and now
  reports no visibility lag
- MAX_SYNC_LAG_OBSERVATION (24h) discards absurd sender-supplied origin
  ages that would park fabricated values in the tail buckets; the
  timestamp is unauthenticated peer input
- the migration lands at v47 after main's slot moves; the test
  initializer gains origin_timestamp

* fix(node): split sync-lag origin from gossip publish freshness

The single origin_timestamp captured after the durability wait was reused
as the signed RefUpdateEvent timestamp, so a tail that published more
than 600s later minted an already-stale event that verified ingest drops
in check_freshness, silently disabling gossip auto-sync on mirrors.

- Capture the lag origin at push landing, before the durability wait, and
  carry it on a new optional signed field RefUpdateEvent::lag_origin
  (serde default, skipped when absent, so events from older peers still
  parse and the null-optional signing shape is byte-identical).
- Sign the publish instant in event.timestamp again, matching merge-base,
  so check_freshness sees the time the event actually went on the wire.
- Ingest now feeds enqueue_sync lag_origin with a timestamp fallback, so
  sync-lag still measures push-to-visible instead of notify-to-visible.
- Re-pin the all-Some GOLDEN_SIGNING_BYTES for the new field; the
  all-None production golden is unchanged. Round-trip test asserts the
  field survives serde.

Also restore the v18 pinned-cid upgrade oracle: the test's rollback
delete targeted schema_migrations version 47 instead of 18, so v18 never
re-applied and the index_exists assertion failed on stable and beta.

* fix(node): version-gate lag_origin, read persisted quarantine, keep push-land instant

- lag_origin ships under v=1: CURRENT_REF_UPDATE_VERSION bumped, ingest
  rejects a v0 event carrying the v1 field, goldens re-pinned for the
  v1 publish shape, and the legacy-parse test now pins the fleet rule
  (v0 still parses, v1 refuses loudly on the v key)
- the sync-lag observation gate reads is_repo_quarantined instead of
  the attempt-local flag, so a re-sync of a quarantined mirror no
  longer samples visibility readers still deny; fail-closed on lookup
- post_receive_replication_tail takes push_landed_at so a deferred
  drain reports the request row's landing instant rather than its own
  drain time; live callers pass the acceptance instant, tests pass None

* fix(node): tighten the two new lag-observation edges

- deferred tails source push_landed_at from the accepted transitions'
  applied_at (the instant refs actually moved), falling back to the
  request row's created_at rather than drain time
- the visibility gate confirms the mirror's ref tip matches the queued
  outcome before sampling, so a row superseded by a newer origin
  update (or a deletion the mirror absorbed) never records lag

* fix(node): split notify publish/land clocks and unify live push-land instant

- HTTP sync/notify now stamps timestamp with send time (publish
  semantics, matching what gossip ingest stores on the feed row) and
  carries the land instant in a new optional lag_origin field; the
  receiver plumbs lag_origin to sync_queue.origin_timestamp with a
  timestamp fallback for older senders
- both live replication-tail spawns pass the request row's acceptance
  instant (the same clock the deferred drain's created_at fallback
  uses) instead of minting a fresh clock at spawn time

* fix(node): use the deferred land-instant rule on live replication tails

Both live post_receive_replication_tail spawns now resolve
push_landed_at through db.request_push_landed_at: the latest
applied_at across this request's applied transitions, falling back to
the request row's created_at, the same rule the deferred drain uses.
The pre-receive-pack acceptance stamp no longer flows into gossip
lag_origin, the HTTP notify land field, or sync_queue.origin_timestamp
on the live path, so live and deferred pushes report one lag origin.
Regression: request_push_landed_at_returns_max_applied_or_none covers
the max-applied arm and both fallback cases.

* fix(node): align notify land surfaces and resolve land instant in-task

- the multi-ref notify regression now matches lag_origin for the
  supplied land instant instead of timestamp, which is a live send
  instant after the publish/land split; the tail comment now names the
  notify lag_origin field rather than timestamp
- both live replication-tail spawns resolve push_landed_at inside the
  spawned task so the handler no longer awaits the lookup while
  holding the reclaimed write lease or leaving the tail un-owned on
  disconnect

* test(node): realign the inv22 tail-spawn gate and metrics help text (#580)

- inv22_replication_tail_spawns_at_the_durability_boundary matched the
  literal tokio::spawn(post_receive_replication_tail( shape, which went
  blind when production wrapped the tail in tokio::spawn(async move {
  ... }). The gate now matches the tail call itself (filtering the fn
  definition) while keeping the disposition.spawn_tail gate, brace
  depth, no-second-spawn, landed-set, and ordering assertions.
- metrics help for gitlawb_push_duration_seconds described receive-pack
  subprocess time but observes the whole successful handler interval;
  gitlawb_sync_lag_seconds described the origin ref-update timestamp but
  samples lag_origin (the land instant). Both strings and the module
  bullet now name what is measured.

* fix(node): tri-state mirror tip probe, full-interval push duration (#580)

- mirror_ref_tip returns a TipProbe (Tip/Absent/Unknown): a failed
  rev-parse is no longer a confirmed-absent ref, so a delete row only
  samples sync lag on proven absence (regression test included)
- observe_push_duration now fires at the response paths (claim-loser and
  post-effects), so the sample covers the whole handler interval the
  client waits on rather than stopping before inline effects

* fix(node): tighten inv22 landed-ref gate, align push-duration help (#580)

- inv22_gates U5: match the third call argument against landed/landed_refs
  instead of contains("landed"), which push_landed_at also satisfied and
  made the proven-landed check vacuous.
- metrics: gitlawb_push_duration_seconds help now describes handler wall
  time through the response including claim-loser returns.
- sync: log mirror-repo registration failures instead of silently
  discarding them.

---------

Co-authored-by: anandh8x <[email protected]>
2026-11-12 23:17:40 +00:00
beardthelion ddc4a2c931 fix(node): refuse re-closing a closed PR, drop 429 Retry-After, pin test git identity
- close_pr refuses a PR that is not open and the UPDATE is conditional on
  status='open', so a repeated close cannot store the branch's current tip
  on a historical pull request.
- Remove Retry-After from 429s. The status-claim caps are per-commit and
  append-only, so a 1s hint invited retries that cannot succeed.
- bare_repo_with_ref sets git author and committer env so the
  heads_still_at_tip tests pass on hosts with no global git identity (CI).
2026-11-12 17:14:01 -06:00
f426fe8399 docs(node): document the two-tier post-push pinning model (#576)
* docs: document the two-tier storage and pinning model

* docs: clarify pin de-duplication is best-effort, not atomic

* docs: address review — fix table/no-op/peer/budget claims, link from README

* docs: name the hot tier as the unbounded side of the parked-list memory

Also record the two wording residuals from the same round: dedup isolation
is one-directional (warm row presence suppresses a later hot pin), and an
unset Pinata JWT disables uploads but not the tail's permit + walk cost.

* docs(node): narrow the storage doc to its pinning charter, fix the Durability bullet

- opening now says what the page is: optional post-push pinning tiers in
  ipfs_pin.rs and pinata.rs, with one pointer to bare-repo storage so the
  page does not read as the answer to "where is my data?"
- the "Running both" Durability bullet now carries the warm-first caveat
  the dedup paragraph already documents: an object warm-pinned before
  Kubo was configured keeps only the warm copy
- README links the doc from the pin configuration row, where operators
  configuring pins actually look; the original staking-list link stays

* docs(pinning): keep the config table contiguous, document local_ipfs_provenance

- README: the duplicate GITLAWB_MAX_CONCURRENT_PIN_TASKS row and the
  free-standing paragraph it introduced broke the table; the single row
  now carries the STORAGE-AND-PINNING cross-link inline
- dedup: the hot tier skips on has_ipfs_cid (local_ipfs_provenance =
  TRUE), not row presence; a Pinata-only row is local_ipfs_provenance =
  FALSE, so it enters the hot path on later pushes once Kubo is
  configured instead of being permanently suppressed
- the "Running both" durability bullet now agrees: enabling Kubo
  backfills local copies for warm-only objects

* docs(pinning): qualify pin admission, per-copy durability, and loop sizing

- the opening now names the admission gates: publish-durability
  confirmation plus root replication; private and non-replicating repos
  and withheld objects are out of "every push pins" wording
- warm-table and durability copy drop the Filecoin backing claim the v3
  API does not document
- the JWT-unset paragraph now says the permit and re-derivation walk
  only run when the push was admitted, and an empty IPFS_API can still
  briefly touch the pin pool on a non-empty list
- the semaphore section and README row state the two-loops-per-push
  fan-out so the default of eight reads as roughly four dual-tier
  pushes
- the backfill and running-both wording is conditional on object
  selection and per-sink success, naming the reconciliation sweep

* docs(pinning): drop unsupported Filecoin claim, normalize punctuation (#576)

* docs(pinning): qualify warm admission and loop count for coalesced pushes (#576)

A push that coalesces onto an in-flight encrypt/pin task never runs its
own withheld walk, yet still spawns the warm Pinata worker, which
re-derives the withheld set inside its pin permit. Scope "up to two
loops per push" to pushes that start their own hot task; a coalesced
push adds at most one warm loop.

---------

Co-authored-by: andreolf <[email protected]>
Co-authored-by: Kevin Codex <[email protected]>
2026-11-12 22:20:39 +00:00
beardthelion 9519078316 test(node): drive the status and push-events routes in the deny sweep
The mounted-route completeness gates require every repo-scoped route to be
a driven registry row: the status read surfaces and the push-events poll
are read-gated (existence-hiding 404 for anon and signed non-readers,
owner reach twin proves the gate) and the claim write is owner-gated on a
public repo. {sha} fills as a fixed 40-hex placeholder.
2026-11-12 12:32:29 -06:00
beardthelion 862293c558 fix(node): add head_commit to test-harness PR fixture 2026-11-12 12:23:02 -06:00
beardthelion 600c358a55 fix(node): write status surfaces from proven landings in the effect executor
The stored-head and catch-up poll writes ran pre-release inside the repo
write lock, so a refused publish left rows claiming a landing the node had
deleted, and a landing reconcile proved only later emitted nothing at all.
Both are durable accounting and belong to the effect executor: the live
path reaches it only after the publish verdict, and the drain replays it
identically once reconcile proves the landing.

The ordering the in-lock placement guaranteed is kept differently: a head
write is conditioned on the branch still sitting at the landing's tip, so
a superseded landing never pins the rollup to a commit the branch moved
past. Event ids derive from (request_id, ordinal) so a recovery replay
collapses on ON CONFLICT instead of double-delivering.

Also: drop the node-global seq from the create response (id already names
the row), pin the claim body's signed format (deny_unknown_fields + v=1),
and carry Retry-After on 429.
2026-11-12 11:48:07 -06:00
beardthelion 48e88972f0 fix(node): harden status surface writers and close-path head freeze
Review-driven fixes on the merged surface:

- close_pr freezes the source head it decided against, resolving the
  recorded push for the branch (then disk) when head_commit is unset;
  a PR closed before its first status read no longer reports
  head_resolved=false forever.
- insert_status_claim_capped bounds its advisory-lock wait with
  SET LOCAL lock_timeout and retries on transient classes, matching
  the push-events writer's contract.
- record_push_events retries only transient failures (the inner bound
  already retries lock waits; a permanent error's second run is a
  guaranteed re-failure) and update_open_pr_heads retries once on
  transient and escalates to error naming the lost branch@sha rows.
- parse_ref_updates lowercases old/new SHAs so pushed values match
  the claim side's lowercase normalization.
- combined_state folds unrecognized stored states to failure, never
  silently to success.
2026-11-12 10:54:37 -06:00
beardthelion 41bbe70853 refactor(node): simplify commit-status surface after main merge
Land the merge on one mechanism per concern instead of two:

- landed_refs now intersects the declared set with the ok_set already
  derived for the outcome commit, instead of re-parsing the report in
  accepted_ref_updates; the helper's unreachable keep-declared arm (a
  trap its own doc contradicted) is gone with the second parse.
- bound_declared_refs is deleted: main's parse_ref_updates already
  refuses past MAX_REF_UPDATES inside the command loop, so the
  post-parse check was dead code whose tests pinned an abandoned
  design; the ordering test now pins the parse itself before git.
- merge_pr resolves the source head with read_ref (exact-name
  show-ref) instead of listing every ref in the repo.
- normalize_sha reuses is_hex_sha; the refs/heads/ mapping reuses
  store::local_branch_ref; test-only db methods and their SQL are
  cfg(test)-gated; duplicated body_json/status_and_bytes test
  helpers are hoisted to test_support.

Merge-artifact repair found by running the tests: scrape guards
track the renamed landed_refs binding and the reclaimed.release /
receive_pack_raw_with_reflog call names; the status migration
upgrade test rolls back to and asserts version 47 (not 24, which
main now owns); a doubled cfg(test), an orphaned doc block, and a
stale dead_code allow on StatusClaim.seq are corrected.
2026-11-12 07:19:08 -06:00
beardthelion e9e5e3d302 Merge remote-tracking branch 'origin/main' into feat/commit-status-surface 2026-11-12 00:40:22 -06:00
beardthelion 55ca89bfff test(node): real-node deny harness for trust-boundary regressions (owner-gated mutations, path-scoped reads, client-surfaced denials) (#194)
* refactor(node): split gitlawb-node into lib+bin so integration tests can spawn a real node

Move the module tree and boot logic from main.rs into a new lib.rs crate
root exposing the boot surface (build_router, AppState, Config, migrations)
as pub; main.rs becomes a thin #[tokio::main] shim over run(). No behavior
change: both targets build and the full node suite (488 tests) stays green.

Prerequisite for the real-node deny harness (U1).

* test(node): real-node deny harness rig + INV-8 unsigned-push case (U2-U4, U5a)

Add a feature-gated (test-harness) spawn surface (src/test_harness.rs) that
boots a real node on 127.0.0.1:0 over an ephemeral #[sqlx::test] pool through
the production axum::serve stack with connect-info, and an integration crate
(tests/deny_harness.rs) that drives deny paths with a real reqwest client:

- U2 signing client: wraps gitlawb_core::http_sig::sign_request for reqwest;
  self-checks that a valid signature clears require_signature and a tampered
  body is rejected (400 content_digest_mismatch).
- U3 spawn_node: real socket, p2p disabled, per-test DB, shutdown-on-drop.
- U4 assert_denied: 4xx AND body-no-leak AND not-empty-200 (INV-8); pure core
  unit-tested for clean-403 / empty-200 / leaking-403 / wrong-status.
- U5a INV-8: unsigned git-receive-pack is denied 401 with no leak.

Widens the three cfg(test) test builders (Db/RepoStore::for_testing,
run_migrations) to also compile under the feature. No production behavior
change: prod build (no feature) excludes test_harness; node suite stays green
(488) and the 7 integration tests pass.

* test(node): INV-1 owner-gate deny case over the real stack (U6)

A validly signed non-owner PUT /visibility is rejected 403 by require_owner
(no x-ucan, so require_ucan_chain passes through to the gate); the owner's
signed PUT reaches the handler (reachability proof, guards against a 404/415
masquerading as a pass). Adds seed_repo/withhold_path seeding helpers to the
test harness. Mutation-verified load-bearing: with require_owner forced Ok the
non-owner PUT returns 201 and the INV-8 assertion flips the test RED.

* test(node): INV-2 read + content-addressed withhold deny cases (U5b, U7)

Adds seed_bare_repo (shells git to build a real bare repo at the served path,
sha1 or sha256 object format) and two INV-2 deny cases over the real stack:

- U7: a public repo with a /secret/** withhold rule denies an anonymous blob
  read of the withheld path (404) with no content/OID leak, while the sibling
  public path is served (path-scoped, not blanket).
- U5b: the same withhold denies an anonymous /ipfs/{cid} read of the withheld
  blob's content-addressed id (404, no leak), while the public blob's CID is
  served. Completes U5 (INV-8) alongside U5a.

Both mutation-verified load-bearing: forcing visibility_check to allow leaks
the secret at 200 and the INV-8 assertion flips each test RED.

* test(node): INV-2 replication-path deny case over git-upload-pack (U8)

Drives the git-upload-pack POST directly (v0 stateless-RPC: want HEAD, flush,
done) via a bounded reqwest client rather than a vanilla `git clone` (which
negotiates protocol v2 and deadlocks against the node's v0 server, and would
otherwise wedge the suite). The served pack is indexed with git index-pack and
its objects listed with verify-pack -v: a packfile-aware assertion, since a raw
byte scan cannot see an OID inside the zlib-compressed stream.

A public repo with a /secret/** withhold rule must serve a pack that omits the
withheld blob's object while keeping the sibling public blob. Mutation-verified
load-bearing: forcing visibility_check to allow puts the withheld blob back in
the pack and flips the test RED.

Completes the harness (8 units, 11 integration tests). Prod build (no feature)
and the 488-test node suite stay green.

* ci(node): run the real-node deny harness on every PR

cargo test --workspace skips the harness because it lives behind the
test-harness feature (kept off the production binary). Add an explicit step
that runs it with the feature and the same Postgres service, so the INV-1/
INV-2/INV-8 trust-boundary regression cases execute on every PR instead of
only when run by hand.

* test(node): cover the deny-assertion guard branches (U4)

Add unit tests for the two remaining check_denied branches: a non-4xx expected
status is rejected as a test bug, and an empty withheld token is skipped rather
than matching every body. Closes the last unexecuted branches in the deny
assertion.

* test(node): full-stack owner-gate deny cases for 5 high-value mutations

Fan-out of U6 to the security-sensitive owner-gated mutations that had only the
source-level authz-table guard and no runtime deny test: protect_branch,
unprotect_branch, create_webhook, delete_webhook, remove_visibility. Each
rejects a validly-signed non-owner with 403 and lets the owner reach the
handler (not 403). Mutation-verified load-bearing on their shared root gate:
did_matches forced true opens all five (non-owner protect_branch returns 201)
and the test flips RED.

Replica register/unregister were intentionally excluded: they are signer-self
(you register your own node), not owner-gated, so there is no owner-deny to
assert.

* style(node): cargo fmt the deny-harness sources

* Address PR review feedback (#194)

- Create the node identity key atomically with create_new + mode 0600, closing
  the umask-derived 0644 disclosure window and the exists()->write overwrite
  race; on AlreadyExists load the winner's key (bounded retry so a loser can't
  read a half-written PEM) and tighten looser perms on load.
- Enforce the configured shutdown grace: bound the axum drain by grace measured
  from the signal (extracted as drive_serve_with_grace), abandoning in-flight
  requests once it expires instead of waiting indefinitely. Removes the
  discarded grace value.
- Route the deny-harness reqwest clients through a shared bounded_client (30s
  timeout) so a wedged node path fails fast instead of hanging to the CI limit.

Tests: 0600-on-create, load-tighten, concurrent-start convergence (create race),
and grace-race abandon / normal-drain / signal-gated-clock. All RED-then-GREEN
by execution.

* fix(node): remove a failed first-write key file; verify key mode after tightening (#194)

F1 (P1): create_new(true) closed the permission window, but a write_all failure
after the file was created left an empty/partial PEM behind. Every later start
then took the key_path.exists() branch, re-parsed that corrupt file in load_racing,
and exited 'invalid PEM key' instead of regenerating — a transient ENOSPC/EIO on
first boot permanently wedged the node. Extract write_key_or_cleanup, which removes
the just-created file on write failure, and wire it into both the unix and non-unix
create branches.

F2 (P2): the load path tightened a loose existing key to 0600 but discarded the
set_permissions result. A chmod that failed (read-only mount, ownership/ACL
mismatch) left a world/group-readable private key in use while logging a normal
'loaded existing identity'. Surface the tighten failure (propagate it) and add
ensure_key_mode_0600, which fails closed if the key is not 0600 after the attempt.

RED->GREEN: failed_write_removes_the_partial_key_file (a failed write leaves no
file; RED without the remove_file). loose_key_mode_is_rejected_not_used (a 0644 key
is rejected; RED without the mode check). Existing created_key_is_mode_0600,
existing_key_is_loaded_and_tightened, and concurrent_starts_converge_on_one_identity
stay green. Full node lib+bin suite 497 passed, fmt + clippy clean.

* fix(node): publish the identity key atomically and wait a real deadline

The create path did create_new(final)+write_all, so the final path appeared as
an empty inode before the PEM was flushed, and a losing/fast-path start only
retried ~100ms (50x2ms). On a slow or stalled filesystem the winner's write can
exceed that window, so every other start failed boot with 'invalid PEM key'
(#194). Publish atomically instead: write the full PEM to a sibling temp, then
hard_link it into place. hard_link is atomic and fails if the target exists, so
the final path only ever appears COMPLETE (no partial-read window), a lost race
never clobbers the winner, and a crashed writer leaves only a temp rather than a
partial final that would wedge later starts. load_racing_key now polls on a
wall-clock KEY_RACE_DEADLINE (5s) instead of a fixed 100ms count. Hoisted
load_existing_key/load_racing_key to module level for testability.

Adversarial RED->GREEN: a 250ms-slow winner is waited out (RED at the old 100ms);
a reader watching the final never sees a partial file (RED with create_new+write);
a losing publish does not clobber the winner; 500 tests pass.

* fix(node): recover key publish from stale temps, make it durable; join TestNode teardown (#194)

- pick the temp key name with a bounded per-call retry instead of a
  process-global counter: a crashed start's leftover temp no longer wedges
  a PID-stable restart, and cross-container publishers stop colliding
- fsync the temp PEM before it can be linked and the key directory after
  publication, so a power crash cannot leave a durable-but-truncated
  identity.pem that permanently blocks startup
- TestNode::shutdown() joins the serve task and closes the pool before
  sqlx's DROP DATABASE cleanup runs; every deny-harness test now ends
  with it instead of relying on drop-only teardown

* test(node): let the partial-final reader wait out the fsynced publish (#194)

The added temp-file and directory fsyncs can push the hard_link past the
reader's old 400ms window under parallel-suite disk load, flaking
publish_never_exposes_a_partial_final (observed ~1 in 5 local runs). The
reader now polls a 5s deadline and stops at the first complete
observation; the never-see-partial assertion still runs before the
break, so the exposure mutation still fails the test.

Also moves the allow-unbounded-git marker in seed_bare_repo directly
above the spawn line, where the pre-push gate looks for it.

* fix(node): fall back to a direct create_new publish when hard_link fails (#194)

On a non-AlreadyExists hard_link error, publish the identity key by
creating the final path directly with create_new(0600): no-clobber is
preserved via the exclusive create (AlreadyExists routes to the lost-race
load), write failures remove the partial final, and fsync failures keep
the complete final on disk since a concurrent loser may already have
loaded it (the Won-arm policy). The fallback chains the original link
error and the rustdoc now scopes the complete-only-visibility guarantee
to the hard-link path. Link step is injectable via compile-time fault
hooks; six new tests drive both fallback arms, the lost race, the
concurrent convergence, and both failure phases.

* test(node): abort the server task in TestNode::drop (#194)

A test that returns without calling shutdown() previously relied on the
watch signal alone to unwind the server before sqlx's DROP DATABASE.
Drop now also aborts any remaining serve handle, so teardown holds even
when the graceful chain is broken; signal and abort are independently
sufficient (verified by a four-quadrant neuter matrix in the new
drop_without_shutdown_unblocks_database test, which observes leaked
sessions via a standalone connection with sqlx pool reaping disabled).

* chore: sync Cargo.lock with the 0.5.1 workspace manifests

* test(node): surface axum::serve's result through TestNode::shutdown (#194)

The spawned serve task previously discarded its io::Result; shutdown()
now joins and checks it. On axum 0.8.8 the Err arm is unreachable
(graceful shutdown hard-codes Ok and accept errors retry forever), so
this is forward-compat insurance documented as such inline; the Ok path
is exercised by every harness test.

* fix(node): verify key file mode before writing PEM bytes at both creation sites (#194)

Both creation sites (the temp on the hard-link path, the final on the
fallback path) now run the loader's tighten-then-verify pair right after
create_new and before any private bytes are written: set_permissions
repairs a umask-narrowed mode (0277 landed 0400 and previously shipped
that way), and on mode-ignoring mounts where the tighten silently no-ops
the verify fails closed with the empty file removed so retries are not
wedged. A verified temp implies a verified hard-linked final (shared
inode). Both call sites are mutation-vetted; umask cases run in child
processes because umask is process-global.

* docs(node): state the real shutdown-grace contract and lock it with a real-socket test (#194)

GITLAWB_SHUTDOWN_GRACE_SECS promised a 503 to in-flight clients at
expiry; drive_serve_with_grace (correctly) abandons them instead, and a
retroactive 503 on an in-flight response is not constructible under
hyper. The config prose now states the abandon contract, and a new test
drives a real listener with a slow handler through the driver at a
250ms grace, asserting the client never sees a 503 status line and the
driver returns bounded with grace_expired set.

This commit changes only a doc comment and a test, so it is typed docs()
rather than fix() to avoid a spurious release-please patch bump.

* test(node): close review gaps on the key-publish fallback and Drop abort (#194)

Round-five review follow-ups, no production behavior change beyond a
pure fsync_parent_dir extraction:

- Recondition the KEY_RACE_DEADLINE and publish_key_atomically rustdocs:
  a present key is complete-on-first-read only on the hard-link path; on
  the fallback the final name is visible mid-write (the deadline covers
  it) and the fallback also weakens durability (d), failing loud at next
  start, never a silently used key.
- Cover the fallback create_new open-error arm via a new fault hook,
  asserting the returned error chains the original link error.
- Make the Drop abort load-bearing (INV-21): a test-only flag suppresses
  the graceful signal so handle.abort() carries teardown alone; removing
  the abort turns the new test RED while the existing one stays green.
- Observe the crash-residue contract: empty/garbage finals fail loudly.
- Extract fsync_parent_dir, shared by the Won and fallback arms.
- Split the permissive-umask test so a 0000 regression can't hide behind
  0077.

* fix(node): recover identity-key fallback fsync failure and abort wedged serve task

F1: the hard-link fallback wrote the PEM directly to the final path then
fsync'd it separately; an fsync failure left a durable name over non-durable
content, so a later crash wedged startup (load_or_create_keypair parses the
existing truncated file instead of regenerating). Fold the file fsync into
write_key_or_cleanup so the final is removed on fsync failure, matching the
hard-link temp policy; the next start regenerates. Parent-dir fsync still
keeps-on-failure.

F2: TestNode::shutdown moved the serve JoinHandle into tokio::time::timeout and,
on elapse, the panicking .expect dropped it, detaching the still-running task
and leaking the test DB across DROP DATABASE. Retain the handle (&mut), abort
and reap it on timeout before panicking. Also: honor a task that finishes in the
timeout race as graceful (not a false wedge), propagate a real serve-task panic,
and close the pool on every path (including the wedged arm) before returning or
panicking so the leak is closed on the path that needs it most.

* test(node): invariant deny-prober (driven registry + completeness cross-check) (#195)

* test(node): U1 deny-bearing route registry (owner-gate + signature tranches)

Foundation for the invariant deny-prober (plan 002): a declarative registry of
ONLY the routes that carry a runtime deny (owner-gate 403 / read-gate 404 /
signature 401), each classified by reading its handler and recording the fn
name. Owner-gate (12 rows: merge/close_pr, close_issue, webhooks, labels,
protect, visibility set/remove/list incl the GET-is-403 correction) and
signature-required (git-receive-pack) tranches are verified; read-gate tranche
stubbed with a TODO (per-handler 404-vs-filter verification pending). A
consistency self-test asserts no dup method+path, every row names its handler,
and every read-gate row carries a positive-twin Reach. Decoupled from the #194
primitives (U1 uses none), so classification is not hostage to harness churn.

* test(node): U2 fixture state matrix + per-gate-class probe generators

Two-repo fixture (public owner-gate substrate + private read-gate substrate)
plus probes_for(): each deny-bearing row expands to a hostile probe asserting
the exact deny status and a positive twin (owner-reachability for owner-gate,
authorized read for read-gate) so a deny for the wrong reason cannot false-pass.
Three self-checks cover the owner-gate, read-gate, and signature classes.

* test(node): U1 read-gate tranche (14 repo-scoped 404-deny reads)

Each row's handler was read to confirm it gates on "/" via
authorize_repo_read / visibility_check and returns RepoNotFound (404) to a
non-reader of a private repo, with the owner re-read as the ReaderReads
positive twin. Sub-entity gets (get_issue/get_pr/get_cert/get_bounty and the
comment/review lists) are deferred to the fixture expansion that seeds them;
global list-filter surfaces and the KNOWN_UNGATED reads are excluded with
reasons. Runtime behavior is asserted by U3.

* test(node): U3 drive the deny-bearing registry over a real node

Boots a real node, seeds the two-repo fixture (plus a PR and an issue so the
author-or-owner close gates are reached, not 404'd on an absent entity), then
walks every deny-bearing route: the hostile probe must return the exact deny
status and leak nothing, and the positive twin must reach the handler (owner:
not 403; read: 2xx). A terminal invariant asserts one hostile probe per row
was driven and the count equals the registry, so a row that produced no probe
fails loud rather than passing vacuously. Adds seed_pr/seed_issue to TestNode.

Mutation-verified load-bearing: breaking did_matches drove the merge_pr
owner-gate hostile to 500 (RED, named); neutering authorize_repo_read's deny
drove the get_repo read-gate hostile to 200 (RED, named); both reverted green.

* test(node): U4 completeness cross-check keeps the registry from drifting

A pure source scrape of server.rs (multiline .route parser, floor 90) that the
runtime sweep cannot give: no deny-bearing row points at a route that no longer
mounts (anti-stale), and no handler carrying an unambiguous owner-gate marker
(require_repo_owner / require_owner) escapes the registry (orphan). The api dir
is read at test time so a new module is covered. Complements authz_guard (which
proves handlers are gated) by proving the deny-bearing ones are actually driven,
and reaches the non-API git mounts authz_guard never sees.

Mutation-verified: a bogus row path tripped anti-stale (named the row); removing
the merge_pr row tripped the orphan guard (named merge_pr); both reverted green.

* test(node): close vacuous-green gaps in the deny prober's own assertions

Vetting the harness against itself surfaced branches that ran green but were
never exercised adversarially:
- Ok2xx twin dropped its unused token Option; the arm now asserts the authorized
  read returns a NON-EMPTY 2xx, so an empty-200 denial-rendered-as-success is
  caught (executed by every read twin).
- U4 orphan guard gains a floor on the owner-marker scan (>=6), so a marker-scan
  regression to zero fails loud instead of passing by checking nothing.
- Added probe unit tests that execute the previously-uncovered probes_for arms:
  the SiblingPublic path-scoped twin, and the Reach::None fail-loud panic.

All 22 harness tests green. Every send_probe/probes_for branch is now executed;
the Deny(401), Not403, and Ok2xx-status arms were additionally mutation-verified
RED (signature-expect flip; did_matches->false; authorize_repo_read->deny) and
both U4 floors RED via threshold flips, all reverted.

* test(node): close 3 completeness gaps in the deny prober's own guards (#195)

jatmn's review flagged three ways the prober's self-guards stay green when
the protection they check is removed:

- F1: the U4 completeness check enforced owner-gate orphans but not
  read-gate ones, so dropping authorize_repo_read on get_issue/get_cert/
  get_bounty/etc. left the sweep green. Add a symmetric read-gate orphan
  guard that scans src/api for authorize_repo_read/visibility_check handlers
  and requires each to be a driven ReadGate row or an enumerated
  READ_GATE_NOT_DRIVEN entry with a reason, replacing the free-text prose.
  A staleness check keeps the allowlist from carrying dead exemptions.

- F2: the U3 registry sweep called assert_denied with an empty withheld
  list, proving status only. Seed the private fixture with a distinctive
  secret and its blob OID, carry them on the read-gate hostile probes, and
  pass them through so a denial body echoing withheld data fails.

- F3: the owner-gate orphan scan matched only require_owner/require_repo_owner
  and missed the inline did_matches(caller, &record.owner_did) idiom that
  protect_branch/unprotect_branch use. Fold in a has_owner_did_matches
  discriminator that catches the owner form without false-matching the
  signer-self (register_replica) or author (close_pr) did_matches forms,
  pinned by unit tests on the exact bodies.

* test(node): make the deny-prober guards load-bearing (#195, F1-F4)

Resolves jatmn's four INV-21 completeness findings — guards that were green but
not load-bearing:

F1: every ReadGate hostile probe was anonymous, so the signed-non-reader branch of
each private read was untested — a regression treating any valid signature as
authorized would leak while the sweep stayed green. Add a Signer::Stranger 404/
no-leak probe per read row, with a dedicated readgate_stranger counter so a dropped
probe fails loudly. RED: authorize any signed caller -> the stranger probe leaks
(200) while the anon probe stays green.

F2: get_star_status, get_icaptcha_proof, and replicate_encrypted_blobs were
source-only exemptions (a marker scan, never driven). Drive them as real ReadGate
rows and remove the allowlist entries. RED: a runtime bypass that keeps the
authorize_repo_read marker leaks -> the driven probe RED, the source scan green.

F3: git_receive_pack's protected-branch owner 403 was never driven (its registry
row is the unsigned-401 path). Add signed_stranger_protected_branch_push_is_forbidden
(a signed non-owner push to a protected branch -> 403, owner control -> not-403),
plus a seed_protected_branch harness helper. RED: invert the gate -> the probe RED.

F4: the read-gate no-leak assertion omitted the private repo id (seed discarded it).
Preserve it on the Fixture and withhold it; a synthetic 404 body carrying the UUID
must fail check_denied. RED: a UUID-bearing body passed before the token was added.

Each guard proven load-bearing by reverting the exact production auth-line (reverts
not shipped). deny_harness suite 30 passed, fmt + clippy clean.

* test(node): drive every arm of the multi-principal close/dispute gates

close_pr, close_issue (owner-or-author) and dispute_bounty (creator-or-claimant)
were each driven with a single owner/creator twin, so reverting the untested arm
stayed green. Model gates by their authorizing-principal set (MultiPrincipalGate
+ Principal), seed distinct author/creator/claimant identities, and emit one
Not403 twin per arm plus the stranger 403. registry_internal_consistency now
invokes probes_for and asserts a twin per declared arm, so a future multi-arm
handler cannot register with an untested arm.

Each arm proven load-bearing: reverting is_owner/is_author on close_pr/close_issue
and is_creator/is_claimant on dispute_bounty turns that arm's twin RED.

* test(node): drive the deferred private reads with per-read leak assertions

The deferred reads (get_issue, get_pr, get_pr_diff, list_issue_comments,
list_reviews, list_comments, get_cert, get_bounty, path-scoped get_tree) sat in
READ_GATE_NOT_DRIVEN, checked only for an authorize_repo_read marker, so an
ignored result or early return stayed green. Seed each sub-entity in the private
repo (issues/PRs via the real create handlers, a cert via a signed owner push,
a private-repo bounty), thread id-keyed paths through IdSource, and drive each as
a ReadGate row: anon/signed-stranger 404, owner 2xx.

Each seeded entity carries a distinctive marker added to the read-gate withheld
set, so a 404 that leaks the entity's own content fails (not just a status
check). get_pr_diff exercises both its repo-root gate and its per-path
visibility_check loop. get_encrypted_blob stays deferred (owner twin needs live
IPFS; reclassified structurally next).

Load-bearing: an ignored-result bypass on each read turns its hostile probe RED;
a 404 leaking a seeded marker fails the body assertion.

* test(node): close the prose-excuse hatch with a mounted-route completeness gate

READ_GATE_NOT_DRIVEN allowed a free-text reason to park a read-gated handler out
of the runtime sweep. Replace the reasons with a closed NotDrivenReason enum
(ReadGatingMutation, GitSmartHttpRead, ContentAddressedRead, GlobalListFilter,
ExternalDependencyUnavailable); get_encrypted_blob moves to the last. A drivable
404-deny GET no longer type-checks a reason and is caught by a driven-not-
relabeled assertion.

Add a completeness gate that derives the required read-gate set from the mounted
route table (scrape_mounts, INV-21b), not only the source-marker scan: every
mounted repo-scoped GET must be a driven ReadGate row, a deny-bearing GET,
structurally excused, or a declared public GET — else it fails. Catches a read
that gates via a helper the marker scan does not recognize.

Load-bearing: re-parking a driven read trips the relabel assert; removing a
driven ReadGate row trips the mounts cross-check.

* test(review): harden multi-principal twin check against signer collision

Add a distinct-signer assertion to multi_principal_rows_emit_one_twin_per_arm:
the per-arm any() check is satisfied by a single twin when two Principals map to
the same Signer, so a signer_for_principal collision would leave one arm silently
untested. Assert the emitted twins carry principals.len() distinct signers
(proven RED by colliding Author->Owner). Also correct the get_cert / priv_markers
comments, which claimed a withheld cert-content marker that is not seeded — no
cert marker is needed because get_cert 404s at the repo-root gate before the cert
is fetched, so cert fields cannot reach the deny body.

* style(test): apply rustfmt to the deny-prober additions

* test(node): drive the bounty mutation gates and pin fixture object format (#195)

- submit/approve/cancel_bounty become MultiPrincipalGate registry rows
  with status-appropriate bounty fixtures (claimed/submitted/open): their
  status check fires before the auth gate, so each row seeds its own
  bounty through the real API and the arm twins consume the status
- pin --object-format=sha1 in both fixture worktrees so a sha256 default
  object format cannot break the OID equality with the served sha1 repos

* test(node): drive the caller-self task/register gates in the deny-prober (#195)

Resolve the three review findings on head 07abcc6.

Bound the two new real-node probes (protected-branch push, PR-diff) on a
timeout-bearing client, so a wedged git subprocess fails the suite rather than
hanging it until CI kills the job.

Withhold the pushed secret blob's OID (full and [..12] short form) in the PR-diff
denial. The old branch was dead code: the OID was never in the fixture seed map,
so build_branch_push_body now returns the pushed blob OIDs.

Drive every caller-self runtime deny the registry claimed but only source-marker
checked. create_task, claim_task, and register are a new SignerSelfGate class (a
body DID field bound to the authenticated signer); complete_task and fail_task
are a MultiPrincipalGate assignee arm (the caller must equal the task's stored
assignee). Each runs hostile-then-authorized against a real node. A completeness
guard, derived from a scan of src/api, now requires every did_matches(caller, X)
handler to be a driven registry row, and the module doc is reconciled to the REST
surface (the parallel GraphQL mutation gates are fenced separately, #219).

Every gate is mutation-verified load-bearing: reverting a production did_matches
line drives its hostile probe RED, and the completeness guard goes RED when a
driven row is dropped or the classifier is starved. No production code changed.

* fix(node): bracket the fallback identity-key publish with a durable marker

The hard-link fallback exposes the final key name before the PEM is
complete, so a crash in that window used to leave a partial final that
wedged every later start. Create a bounded-name .publishing marker and
fsync its name durable before create_new(final), dispose of it on every
exit with the final's removal ordered first, and remove it only once the
final is complete and durable. The marker dir fsync is warn-and-continue
so dir-fsync-hostile mounts degrade to reduced power-loss protection
instead of never provisioning.

* fix(node): recover a crash-interrupted fallback key publish at boot

A marker plus a content-failed final (empty or unparseable PEM) is now
provably an interrupted first write: boot claims one marker by atomic
rename to a .recovering file, quarantines the final preserving its
bytes, sweeps the remaining markers, and regenerates, at most once per
start. Every other failure keeps today's loud fail bit-for-bit: a bad
final without a marker, any non-content load failure (transient EIO,
mode, stat), and a second-iteration failure never touch the key. The
error class is carried as a typed root error so recovery cannot key on
message text.

* fix(node): make the fallback marker removal a commit check for stalled winners

A fallback winner stalled past the race deadline can have its round
claimed by a recovering peer: the peer renames the marker, quarantines
the in-progress final, and republishes. The winner's own-marker removal
now decides the round: removal succeeding linearizes it ahead of any
recovery, while any failure demotes it to Lost, after waiting out live
or stale .recovering claims so the follow-up load cannot observe the
pre-quarantine key. Demotion is always safe: absent interference the
Lost path re-loads the winner's own key.

* fix(node): sweep stale key-publish markers on every healthy boot

A marker orphaned between its durability fsync and create_new(final),
or a claim left by a crashed recovery, must not linger to misclassify a
much-later corruption of a good key as an interrupted first write. Both
healthy-boot arms (successful load, and a Won publish on the generate
path) now sweep stale .publishing and .recovering entries, but only
after the sweeper itself has made the final durable: a loadable final
is not necessarily synced, and removing a live winner's marker before
its sync_all would recreate the marker-less partial-final wedge on
power loss. Any durability failure skips the sweep and leaves the
markers in place.

* fix(node): make recovery claims complete, releasable, and themselves recoverable

Three review findings against the marker protocol. Recovery now claims
every publishing marker and stale recovering claim (per-item bounded
claim names), so a live publisher's own-marker commit check can no
longer falsely succeed while its round is stolen through a stale
marker; a post-claim re-parse aborts to reload when a publisher
completed in the window. Claims count toward the crash signature and
are claimable by rename, so a recoverer crash between claim and
quarantine no longer recreates the permanent wedge. Claims are released
on every exit including publish failures, closing the leak that made
demoted publishers ride the full race deadline on a dead claim. The
crash signature also needs two consecutive observations, giving a live
mid-write publisher one poll of grace before a reader can classify its
round as a crash.

* test(node): cover the quarantine-exhaustion arm and pin recovery latency

Add the missing coverage on recover_crashed_publish's degraded arms:
quarantine-name exhaustion must fall back to removing the bad final and
still regenerate, and a blocked removal must surface the chained error
loudly, with the leftover claim proven claimable on the next boot. Route
all marker-class name construction through publishing_prefix and
recovering_prefix as the single source of truth, state the dir-fsync
degradation caveat in publish_key_atomically's guarantee prose, and pin
the crash-signature short-circuit with a timing assertion so recovery
never silently regresses to riding the full race deadline.

* fix(node): protect live recoveries from the sweep and restore hijacked keys

Second-model review findings. The healthy-boot sweep now age-gates
recovering claims (2x the race deadline; a live claim is always
younger), because sweeping a live recovery's claim cancelled a demoted
publisher's clearance wait and reopened two-sided divergence; publishing
markers keep the unconditional sweep since stealing one only forces a
safe demotion. The post-claim re-parse quarantines only on content-class
failures (a transient error there releases the claims and stays loud),
and a key completed inside the re-parse-to-quarantine window is detected
by parsing the quarantined bytes and restored via no-clobber link
instead of being discarded. The crash signature must now persist for
250ms of consecutive observations, above realistic mid-write latency on
shared volumes, so concurrent starts stop hijacking live publishes.

* fix(node): keep no-clobber on the quarantine restore and scope cleanup to owned inodes

The restore now republishes the quarantined bytes through the atomic
publisher, so every tier refuses an existing final and a concurrent
winner survives; a lost restore keeps the quarantine as forensics and
loads the winner. The fallback's post-write error arms remove the final
only after proving the name still refers to the inode this publish
created, so a failing publisher whose round was claimed away can no
longer unlink the recoverer's republished key. Pre-write arms keep
by-name removal: they complete inside the crash-signature persistence
window, before any recovery can have claimed the round.

* fix(node): hand post-write publish failures to boot recovery, stamp claim freshness

A failed write or fsync on the fallback path now removes nothing: the
partial final and its marker stay in place as exactly the crash state
boot recovery claims, quarantines, and regenerates from. That deletes
the stat-then-unlink cleanup whose race could destroy a recoverer's
republished key, and can no longer strand a marker-less partial final
when an unlink fails. Pre-write failures keep removal and disposal;
they complete inside the crash-signature persistence window. Claim
renames also stamp the claim's mtime to now, because rename preserves
the source mtime and an aged stale marker otherwise births a live claim
the healthy-boot sweep would treat as orphaned mid-round.

* fix(node): key recovery claims by process nonce and harden the freshness stamp

Two PID-1 containers on a shared identity volume derive identical claim
names, and rename replaces an existing destination, so one recoverer's
release could unlink the other's live claim. Claim names now carry a
per-process nonce between pid and counter, so the release set can never
alias another process's claims. The claim-freshness stamp gains a write
fallback (a one-byte append bumps mtime where set_modified is
unsupported), and its failure warning names the real hazard. Also lock
the vanished-final re-parse arm with a dedicated regression test.

* fix(node): adopt stranded quarantined identities and age-gate claim stealing

A completed key stranded in quarantine by a failed or crash-interrupted
restore is no longer abandoned: the generate arm first scans quarantines
newest-first and republishes the first parseable one through the atomic
publisher, preserving identity continuity instead of silently minting a
fresh DID; unparseable quarantines stay as forensics. Recovering claims
become claimable only past the same age floor the sweep uses, so a
second starter can never steal a live recoverer's round mid-flight; a
claim-crash state recovers after the bounded age floor passes. The
crash-signature persistence doc now states the slow-mount trade-off
plainly: a pre-write window exceeding the floor yields churn that
converges, never divergence.

* fix(node): bound quarantine adoption, expire orphan claims, align the waiter

Adoption of a stranded quarantined identity is now bounded by the claim
age floor, so a current round's completed key is preserved while a
historical forensic quarantine can never resurrect an old DID or defeat
the delete-identity-for-fresh-DID operator procedure. A healthy boot
also schedules one delayed re-sweep after the age floor passes, so
claims spared as possibly-live cannot linger as aged residue that would
let later real corruption of a good final masquerade as a crash state
and silently regenerate. The demotion waiter now shares the sweep's
liveness rule: it waits out claims younger than the age floor rather
than a flat race deadline, and skips aged orphans quickly.

* fix(node): sweep every healthy return, supersede outcompeted quarantines, fail closed on unsettled rounds

Every arm that returns a loaded, adopted, or reloaded key now funnels
through one sweep-then-return closure, so stale markers cannot outlive
any healthy boot path. Quarantines that provably lost their round (a
Lost restore, or siblings of an adopted quarantine) move to an inert
superseded class: bytes preserved as forensics, never adoptable, so the
delete-identity-for-fresh-DID operator procedure can no longer
resurrect an outcompeted key. The demotion waiter fails closed when its
bound expires while a live young claim persists: an unsettled round now
surfaces as a loud failed publish instead of resolving a key that disk
may be about to contradict.

* fix(node): supersede on every Lost arm, heartbeat live claims, retire old forensics

Complete the supersede rule across its mirror arms: a lost adoption and
a restore-Won round now both demote every outcompeted adoptable
quarantine, so no arm leaves bytes the fresh-DID procedure could
resurrect. Live recovery rounds now heartbeat their claim mtimes every
quarter of the age floor under an RAII stop-and-join guard, making
age-as-orphan sound: an aged claim provably lost its recoverer, and a
stalled-but-live round on a slow volume can no longer be mistaken for
residue by the waiter, the sweep, or a competing recoverer. Superseded
forensic files are retention-swept after 24 hours, bounding growth on
long-lived key directories.

* fix(node): order supersede before consume, retire quarantine forensics, key the waiter on absence

Both Won arms now supersede sibling quarantines before consuming the
chosen one, so a crash in the gap cannot leave an outcompeted key
adoptable. Plain quarantine forensics get the same 24h retention sweep
as superseded files, bounding growth on long-lived key directories. The
demotion waiter drops age logic entirely: it returns Ok only when every
claim file is gone and fails the publish closed while any claim
survives the bound, young or aged, so a heartbeat failure on a hostile
volume can no longer convert a live round into orphan residue mid-wait.
Doc prose reconciled with the shipped wait contract.

* test(node): drive the #113 read/owner gates in the deny-harness fence

After merging main, the completeness fence flagged three repo-scoped list
GETs that #113 gated but the harness still treated as ungated public reads:
list_webhooks (authorize_repo_read + require_repo_owner), list_replicas and
list_protected_branches (authorize_repo_read). The runtime marker scan caught
its own hand-maintained PUBLIC_REPO_GETS list drifting stale.

Drive each real deny instead of exempting it:

- list_replicas and list_protected_branches take Option auth, so they slot in
  as standard ReadGate rows (anon and signed-non-reader both 404, owner 2xx,
  no-leak body).
- list_webhooks is doubly gated and 401s a headerless caller before any
  lookup, so a vanilla ReadGate row's hardcoded anon->404 leg does not fit.
  Add a per-Row anon_read signal (Deny404 default, Deny401 for auth-required
  reads) so its anon leg expects 401 while the signed-non-reader still drives
  the existence-hiding 404. Its owner layer is a sibling OwnerGate row on the
  public repo (signed stranger -> 403). The two share the /hooks path, so the
  consistency dedup key moves from (method, path) to (method, path, gate),
  which still catches an accidental same-gate double-registration.
- Empty PUBLIC_REPO_GETS: the three are now driven rows, classified via the
  driven-path set rather than declared public. The mechanism stays for the
  next genuinely-public repo-GET.

Each new row proven load-bearing: reverting the production gate turns the
matching hostile probe red (403->200 owner, 404->403/200 read), and flipping
list_webhooks' anon_read to Deny404 reddens its anon probe (got 401). Full
deny_harness green (39/39). No production changes.

* test(node): harden deny-harness so its guarantees can't pass vacuously

Resolve three review findings on the real-node deny harness, all test-only.

- assert_denied no longer folds a body-read error into an empty body: an
  unreadable denial (a mid-body reset/truncation) now fails loud instead of
  being certified leak-free by a withheld-token scan over "".

- The three sqlx::test cases that returned via TestNode's async Drop
  (signed_stranger_protected_branch_push_is_forbidden,
  get_pr_diff_withheld_path_is_denied,
  deny_bearing_registry_denies_hostile_and_admits_authorized) now end with
  node.shutdown().await, releasing the serve task's pool clones before sqlx's
  synchronous DROP DATABASE cleanup instead of racing it.

- A source-scrape guard, every_spawn_node_test_shuts_the_node_down, makes that
  contract load-bearing for every spawn_node test, with a closed allowlist for
  the two Drop-regression tests that deliberately skip shutdown. Removing any
  one shutdown() turns it RED (verified).

- The eight fixture seeders in support/probe.rs use the bounded client, so a
  hung setup route fails fast rather than running to the CI job timeout.

Full deny_harness suite green (40 tests).

* test(node): close two vacuity holes in the shutdown-completeness guard

An adversarial review flagged that the new every_spawn_node_test_shuts_the_
node_down guard could itself pass vacuously - the exact class it exists to
prevent (INV-21 on the guard).

- A spawn_node test whose only `node.shutdown(` occurrence was a comment
  (`// node.shutdown().await`) satisfied the raw substring check while still
  returning via Drop.
- An unbalanced `{` inside a string literal in a test body could over-extend
  the brace-matched body slice and swallow a later test's shutdown, clearing
  the earlier one (latent: no current test triggers it).

Both are closed by sanitizing the source through a new `code_only` pass
(blanks comment and string/raw-string content, length-preserving) before the
scan sees it, and the offender check is extracted to a pure
spawn_node_tests_missing_shutdown so it can be driven with synthetic sources.
Three adversarial unit tests pin both holes shut; stubbing code_only to a
no-op turns the comment and brace tests RED (verified), so the sanitizer is
load-bearing.

deny_harness suite green (43 tests); clippy clean.

* fix(ci): stub the gitlawb-node lib target in the Docker dependency cache

This branch gives gitlawb-node a lib target alongside its binary, and the
dependency-cache layer only wrote a main.rs stub for it. Cargo resolves every
declared target before it fetches anything, so the layer aborted on the missing
src/lib.rs and took the whole image build with it.

Adds the matching lib.rs stub. The layer deliberately has no `|| true`, so this
was a hard failure rather than a silently cold cache, which is the behavior that
block's comment asks for.

* fix(node): open the identity key without following symlinks, and close the temp before removing it

Two findings from the last review.

Loading an existing key ran metadata, set_permissions, a mode verify, and the
read as four separate path lookups, all of which follow symlinks. A key path
inside a writable directory could therefore be pointed at any service-readable
file, and a privileged start would chmod that target to 0600 before it ever
looked at the contents. On Unix the load now opens the path once with O_NOFOLLOW
and does the mode check, the tightening, and the read through that one
descriptor, which refuses a symlink outright and closes the window between the
check and the use. A non-regular file is refused for the same reason. A regular
key with loose bits is still narrowed rather than rejected, since rejecting it
would break existing deployments.

The failed-write path called the cleanup helper while the temp file handle was
still open. Removing an open file fails on Windows, so a write or fsync failure
left the temp behind, and the temp names are deterministic per process id with
64 of them, so a service restarting under a reused pid could exhaust the set and
stay stuck after storage recovered. The handle is now closed first, with fsync
still ahead of it since it needs the live descriptor. A removal that fails is
logged, because it is the precursor to that exhaustion, while the write error
stays the one surfaced to the caller.

* fix(node): do not block on a FIFO planted at the identity key path

The non-regular-file refusal added in the previous commit sits after the open,
and opening a FIFO read-only waits for a writer to connect. A named pipe at the
key path therefore hung startup indefinitely instead of being refused, reachable
by the same actor the symlink case assumes. The directory case did not catch it,
since opening a directory fails immediately and does reach the refusal.

Opening with O_NONBLOCK as well makes the open return so the file-type check can
run. It has no effect on a regular file, and a non-regular descriptor never gets
past the check, so nothing needs to clear it afterwards.

* fix(node): apply review fixes for identity key dir and deny harness

Tighten identity key parent directories to 0700 before load/publish,
create fallback publish markers with 0600, and extend U5b /ipfs withhold
coverage with signed non-reader denial and allowlisted-reader grant legs.
Add OwnerGate/ReadGate registry probe-shape tests.

* fix(node): close deny harness gaps after main integration

Tighten identity key parent dirs only on publish, extend INV-8 header
scanning, add inline claimant orphan guard, and align gossip ping tests
with the /ready-first readiness probe.

* ci(codeql): exclude test paths and drop secret tokens from panic text

CodeQL flagged intentional deny-harness fixtures (HTTP secret_cid probe and
pack listing assertion). Ignore tests and test_support for analysis, remove
an unused import, and keep the allow-unbounded-git marker on the harness
git spawn line.

* test(node): clear CodeQL false positives on deny harness fixtures

Reuse the signed anon probe path for the loopback GET so the cleartext-
transmission query does not flag secret_cid in the URL builder, and mark
the assert_denied panic as an intentional leak witness for cleartext-logging.

* fix(node): restore lib/bin boot regressions dropped in the deny-harness extraction

Re-wire Config::validate(), the advisory-lock pool, and gossip peer-reachability
hysteresis on the library boot path, and narrow CodeQL suppressions on fixture-only
harness assertions.

* fix(node): close remaining #194 gaps (CI lib, key parent, gossip shutdown)

Run join_or_abort lib tests in the deny-harness CI step, repair
group/world-writable key parents on existing-key load, abort gossip on
HTTP shutdown with per-peer shutdown polling, and log list_peers errors.

* test(node): restructure deny-harness fixtures to clear CodeQL alerts

Route the anonymous withheld-CID GET through an anon_get helper (matching
the unflagged signed_request pattern) so CodeQL no longer traces the
secret into a single inline HTTP expression. Redact withheld tokens from
the assert_denied panic witness so a leak failure does not itself leak
the secret to stderr. Both are test-only changes; no production code moved.

* fix(node): tighten identity-key parent-dir reload mask and add owner-push socket coverage

The reload fast-path checked only write bits (0o022) while the publish
helper checked all group/world bits (0o077), so a 0755 parent survived
reload but was tightened on publish. Align the reload mask to 0o077 so
the owner-only 0700 invariant holds across restarts, not just first
publish. Adds a regression test proving a 0755 parent is tightened on
reload (RED with the old mask, GREEN after).

Adds real socket coverage for enforce_owner_push on an unprotected
branch, isolating the owner-push gate from branch protection. The
existing protected-branch test could not distinguish which gate fired
the 403.

Updates sweep_failure_tolerated to reflect the tightened reload path: a
0555 parent is now tightened to 0700 before the sweep, so the marker is
removed rather than surviving behind a read-only directory.

* test(node): clear CodeQL alert and restore gossip readiness tests

Two findings from the 2026-09-08 review on head 988e8ea:

1. CodeQL rust/cleartext-logging on assert.rs:118. The panic sink
   interpolated redact_withheld(&reason, withheld), but CodeQL's taint
   analysis traces withheld tokens through redact_withheld without
   recognizing it as a sanitizer. Replace the interpolated panic with a
   fixed string so no tainted data reaches the sink. redact_withheld is
   still called (keeping it live) and its unit tests still pass; only the
   panic message changes. This is a test-only fixture false positive: no
   production path logs secrets.

2. Four gossip readiness tests dropped in the lib+bin split. Merge-base
   main.rs had 7 ping_peer_* tests; head had 3 renamed ping_peer_health_*
   tests. Restore the 4 missing: the /ready 200 direct happy path, the
   /ready 503 + /health 200 must-not-be-ready case, the legacy fallback
   with explicit expect-count, and the shared-deadline probe. No
   production code change; the function signatures are unchanged.

Verified: cargo fmt --check, cargo clippy -- -D warnings, 9 gossip
tests pass, 53 deny_harness tests pass (including assert unit tests).

* test(node): refresh deny harness for post-July main drift

Main's quarantine read pre-checks and task visibility layering landed
after the registry froze, so the merged table needed three classes of
refresh:

- scan hygiene: mask string/comment contents and strip cfg(test) mods
  before the marker scans, so fixture text (the resolver_body helper)
  cannot fake a decl boundary or inherit markers
- registry refresh: allowlist the read-gating mutations that gained
  authorize_repo_read (merge_pr, visibility/protect/webhook/label/
  bounty/close/replica writes, git_receive_pack), drop the task_claimable
  helper and create_task's dedicated-test-driven conditional owner gate,
  and class list_visibility (owner-driven GET) and list_anchors
  (query-scoped read) explicitly
- sweep substrate: scope the visibility twins to /swept-scope so
  set_visibility cannot 404 later strangers on the shared repo, and seed
  completable/failable tasks on the public repo so strangers reach the
  assignee 403 through get_visible_task

* test(node): restore main's lock-pool-direct for_testing semantics

The merge reconstruction derived an 8-connection lock pool inside
for_testing, so callers passing a pool observed nothing checked out.
Main's contract is that the argument IS the lock pool, which the
exhaustion, deadline, backend-reuse, and waiter-isolation tests observe
directly. Route through Self::new like for_testing_with_tigris so the
cfg(test) fields stay gated.

* fix(node): close review leaks in the deny harness and the grace knob doc

- config.rs: GITLAWB_SHUTDOWN_GRACE_SECS documented an enforced drain
  bound that production never applies; the value is only logged and axum
  drains in-flight requests unbounded. Say so.
- test_harness.rs: join_or_abort unwound a serve-task panic before
  pool.close() ran, leaking the test database on exactly the path where
  teardown matters. Carry the payload as ServeTeardown::Panicked and
  resume unwinding only after the pool and repos dir are released.
- probe.rs: the cert-push workdir was keyed on pid+server_main_tip, which
  two parallel fixtures seeding within one second share; one fixture's
  cleanup could delete another's in-progress tree. Use a tempfile
  TempDir, which is unique and also cleans up on the panic path.

* test(node): point the inv26 purge gate at lib.rs after the lib/bin split

The queue-lifecycle purge boot wiring moved from main.rs to lib.rs in the
crate split, but the source gate still scanned main.rs, so
inv26_step4_queue_lifecycle_purge_is_wired panicked in CI while the
purge itself stayed correctly wired. Scan lib.rs and update the doc and
assertion strings; the purge contract is unchanged.
2026-11-11 20:55:39 +00:00
7ae4707c48 docs: language-agnostic PROTOCOL.md wire spec (#578)
* docs: add initial language-agnostic PROTOCOL.md wire specification

* docs: add language-agnostic PROTOCOL.md wire spec

* docs: scope PROTOCOL.md to shipped alpha behavior

Address all five review findings: did:key-only resolution (did:web /
did:gitlawb marked planned), SHA-1 object format with IPNS demoted to
planned and the REST refs CID mapping described instead, the receive-pack
route and the 404-fetch vs 401-push auth split documented, iCaptcha wire
shape corrected (client sends only x-icaptcha-proof; url/level are 403
response headers; ICAPTCHA_MODE defaults off over a small endpoint set),
and the core gitlawb/ref-update/v1 schema spelled out field-for-field
with node push receipts explicitly split off as a different document.

* docs(protocol): close the outstanding review findings on the wire spec

- @path binds path plus query, not RFC 9421's path-only value; a second
  implementation signing info/refs without ?service= fails verification
- content-digest is a mandatory covered component; a body-less request
  signs the SHA-256 of the empty body, also accepted when absent
- TLS transport stated in the signature section with a pointer from
  smart-HTTP
- the ref-update certificate section is marked planned: the schema is
  defined in code but no shipped endpoint serves it, and its 64-hex
  hash fields reserve the SHA-256 width while repos are sha1 today
- smart-HTTP documents the mandatory ?service= values, pkt-line
  framing, and the upload-pack/receive-pack result content types

* docs(protocol): correct digest, transport, path, and icaptcha claims

- Content-Digest is required on every signed request: the node refuses
  a signed request without it (400 missing_content_digest); a body-less
  request still sends the SHA-256 of the empty body, never omits it
- transport: deployments should run TLS in front, but the node verifies
  signatures on whatever transport reaches it, no scheme gate
- path normalization documented: one trailing .git stripped, owner may
  be full did:key or bare multibase, and @path is the literal
  path_and_query bytes; the signing example is now signable as written
- icaptcha enforce with no loaded verifier key stays inert, so the 403
  reads as enforce-plus-loaded-key, not enforce alone

* docs(protocol): stop requiring TLS on signed smart-HTTP in §5

§5 still told readers signed git-receive-pack traffic follows §2
"including TLS transport" after §2 was corrected to state the verifier
has no scheme check. A second implementation could infer TLS is part of
the signed push contract and reject cleartext requests the node accepts.
Point the path at §2's RFC 9421 header rules and defer transport policy
to §2's non-normative deployment note.

---------

Co-authored-by: tatulatu700-lang <[email protected]>
Co-authored-by: Kevin Codex <[email protected]>
2026-11-11 17:40:47 +00:00
72c31ddeca gl: add --icaptcha-proof flag to register command (#579)
* gl: add --icaptcha-proof flag to register command

This allows users to provide a pre-obtained iCaptcha proof token when
registering with a node, sent as the x-icaptcha-proof header on the
initial POST /api/register request.

Usage:
  gl register --icaptcha-proof "<token>"
  GITLAWB_ICAPTCHA_PROOF="<token>" gl register

The existing automatic iCaptcha retry flow (403 -> solve -> retry) remains
unchanged and is still available if no initial proof is provided.

* gl: disable redirects, add proof test, improve error context

- Disable HTTP redirects in NodeClient to avoid leaking proof/signature headers
- Add send_signed_does_not_follow_redirect test
- Use specific error context for proof-bearing registration requests
- Add register_forwards_icaptcha_proof_header test

* gl: disable redirects for signed/proof requests, add end-to-end test

* fix(gl): land the outstanding review asks on the icaptcha-proof flag

- help text on --icaptcha-proof so Clap renders what the flag accepts
- register_forwards_icaptcha_proof_header pins the supplied proof onto
  the register request as x-icaptcha-proof
- revert the 186 file-mode flips the contributor commits carried
  (644->755 noise unrelated to the feature)
- the redirect-forwarding concern is already covered on main: the
  client runs a same-origin-only redirect policy, so the proof header
  can never ride a redirect to a different origin; the cross-origin
  and proof-header tests already exist in http.rs

---------

Co-authored-by: root <root@Rocinante>
Co-authored-by: jilt <[email protected]>
Co-authored-by: jilt <[email protected]>
2026-11-11 16:00:56 +00:00
beardthelionandAyush7614 4fc041a399 fix(node): encrypt webhook HMAC secrets at rest and redact from Debug (#575)
* fix(node): encrypt webhook HMAC secrets at rest and redact from Debug

Webhook secrets are signing keys and were stored in cleartext, exposed by the
Webhook struct's derived Debug, and reachable through JSON serialization.

- Introduce a WebhookSecret newtype whose Debug and Serialize redact to "***",
  so the secret never reaches a log line or an API body.
- Seal the secret at rest with a node-held key (HKDF-derived from the identity,
  domain-separated as gitlawb/webhook-secret/v1) via XChaCha20-Poly1305; the
  column holds enc:v1:<nonce||ciphertext>.
- Open the secret only at delivery-signing time; legacy plaintext rows still
  sign and are re-encrypted once at startup by Db::reencrypt_legacy_webhook_secrets.

Closes #506

* test(node): derive webhook test keys from the OS CSPRNG

CodeQL flagged the hard-coded [0x42u8; 32] / [0x43u8; 32] helper keys
as a hard-coded cryptographic value. Derive them with OsRng.fill_bytes
instead, matching gitlawb-core's scan_token::new_key, so no hard-coded
key material sits in the tree. A wrong key still fails
XChaCha20-Poly1305 authentication with overwhelming probability.

* test(node): draw webhook test keys via OsRng.gen to clear CodeQL flag

CodeQL's 'Hard-coded cryptographic value' query treats the [0u8; 32]
zero-init flowed through OsRng.fill_bytes as a hard-coded key, so the
CSPRNG fix still tripped the gate. Generate the key array directly with
OsRng.gen() so no constant byte array reaches the key sink.

* fix(node): withhold unsigned webhook deliveries, classify sealed rows by open()

- delivery path now decides via SigningOutcome: a hook whose stored
  secret cannot be opened (foreign or rotated key, truncated
  ciphertext) is withheld with a warn and the secret_unopenable metric
  label, instead of posting unsigned and marking the event sent; a
  hook with no secret still delivers unsigned as before
- reencrypt_legacy_webhook_secrets treats "already sealed" as "opens
  under the node key", not "has the enc:v1: prefix": a plaintext secret
  that literally begins with the prefix, or a value sealed elsewhere,
  is re-sealed raw instead of being skipped and wedged. The UPDATE is
  compare-and-set on the value read so a concurrent secret change is
  not clobbered, and prefixed-but-unopenable rows log a warning
- WEBHOOK_SECRET_KEY_INFO doc now says a version bump renders every
  sealed row unopenable until a re-seal path exists, rather than
  advertising rotation the code cannot do
- shared hkdf_sha256 helper in state.rs: derive_scan_token_key and
  derive_webhook_secret_key were byte-identical HKDF bodies
- new pins: signing_decision across all four outcomes, reveal on a
  sealed value, the prefixed-plaintext re-encryption wedge

* fix(node): leave foreign ciphertext alone in webhook re-encrypt, count CAS outcome

- has_seal_shape splits "prefixed plaintext or corrupt blob" from
  "well-formed ciphertext": a sealed value that fails open() under the
  node key is left unchanged so delivery keeps withholding, instead of
  re-wrapping the outer blob and signing with it as a fake plaintext
- rewritten counts rows_affected, not iterations; a lost CAS race is
  logged and not counted
- derive_webhook_secret_key rustdoc now agrees with
  WEBHOOK_SECRET_KEY_INFO: a version bump orphans sealed secrets, it is
  not a rotation
- the signing_decision test key comes from OsRng.gen() like the other
  webhook test keys, clearing the hard-coded-key CodeQL alert
- pins: foreign ciphertext survives the pass untouched and reveal still
  returns None for it

* chore: re-trigger CodeQL check evaluation

---------

Co-authored-by: Ayush7614 <[email protected]>
2026-11-11 03:22:39 +00:00
beardthelionandVasanthdev2004 3a0d91a801 feat(node)!: anchor the UCAN proof chain and honour delegated git/push (#567)
* feat(core)!: verify_chain returns the proof chain's root issuer

verify_chain returned Result<()>, so a caller learned that a chain was
internally consistent and nothing else. `did:key` is self-certifying: anyone
can mint a keypair, self-issue `*` on `*`, and produce a token that verifies.
That is correct UCAN behaviour rather than a defect — a token with no proofs is
its own root — and it still verifies after this change. But it does mean "the
chain verifies" is not an authorization answer on its own, and without the root
identity no caller could ask the question that actually decides a push: does
this chain rest on someone this repository trusts?

Returning the root means a caller cannot accept a chain without being handed
the identity it rests on. Callers that legitimately do not care — the node
middleware checking that a bootstrap token is well-formed, for one — discard it
explicitly. The doc comment now states outright that this establishes internal
consistency and not trust, and names what a caller has to compare the root
against.

Multi-proof chains are refused: more than one proof means more than one root,
and nothing says which root authorized a given capability, so returning any
single one would be unsound. The old loop followed every proof but returned
nothing, so a two-proof chain verified silently; the new test pinned that by
observing the pre-fix code accept one. Ucan::delegate only ever writes one
proof, so no token this codebase produces is affected.

BREAKING CHANGE: Ucan::verify_chain returns Result<Did> instead of Result<()>.

* fix(node): gate two Unix-only tests so the crate compiles on Windows

Two tests build a fake git as a `/bin/sh` script, mark it executable through
`std::os::unix::fs::PermissionsExt`, and reap the hung `rev-list` with
`libc::kill(SIGKILL)`. None of that exists on Windows, and neither test was
cfg-gated, so `cargo test -p gitlawb-node` failed to compile there with six
errors before running anything. A Windows checkout could not run a single test
in the crate, including the ones that are platform-independent.

Gate both with `#[cfg(unix)]`. The attribute is a no-op on Linux, so CI keeps
running them exactly as before; only the Windows build changes, from "does not
compile" to "runs the platform-independent tests".

Refs #228

* feat(node): carry the verified UCAN and its root into the request

require_ucan_chain validated a presented token and discarded it, so no handler
could read the result and Ucan::can had no call site anywhere in the node. A
UCAN could only ever fail a request, never authorize one.

The middleware's own behaviour is unchanged — an absent header passes through,
an invalid chain is still 401. It now also parks the verified token and the root
issuer its chain rests on in request extensions, so an authorization decision
downstream can use them. The root is stored rather than recomputed so the chain
is walked once per request.

Holding a VerifiedUcan is deliberately not an authorization decision. did:key is
self-certifying, so a chain that verifies proves only internal consistency; the
caller has to compare the root against an identity it trusts for the resource
being touched. The next commit adds that comparison for git/push.

* feat(node): add ucan_grants_push, anchored at the repo owner

The predicate that decides whether a delegated capability authorizes a push.
The chain must root at the repo owner — data the node holds independently of
the token, which is what a self-minted chain cannot forge — and the leaf must
carry git/push for this repo.

Only the leaf is examined for the capability. verify_chain has already
established that each leaf capability is attenuated by its proof, transitively
to the root, so a surviving leaf capability is no broader than what the root
granted; re-walking the chain would duplicate that guarantee.

Resource matching is structural rather than a string compare because owner_did
is stored full on canonical rows and bare on mirror rows; a literal match would
deny valid delegations for every mirror, which is a defect that would have
looked like a permissions bug rather than a parsing one.

A capability carrying nb authorizes nothing while constraints stay
uninterpreted, so an owner who writes a ref restriction never accidentally
grants repo-wide push.

Not yet wired into the push path — the next commit does that — so a non-test
build still reports VerifiedUcan::ucan as unread.

* feat(node): honour a delegated git/push capability on the push path

caller_authorized_to_push becomes owner || delegated, exactly the Phase 2 its
own doc comment described. The owner check is unconditional and runs first, so
the UCAN path can only ever turn a 403 into a 200, never the reverse. A push
carrying no X-Ucan reaches the same owner-only decision it did before.

git_receive_pack takes the verified token as Option<Extension<VerifiedUcan>>;
axum extracts an absent extension as None rather than rejecting, so the optional
header stays optional.

The denial message is deliberately identical whether a delegation was absent,
expired, or named a different repository. A behavioural test asserts the two
bodies are byte-identical, because a difference would turn the refusal into an
oracle for which capabilities exist.

The behavioural test drives both auth layers with a real RFC 9421 signature and
a real invocation, and discriminates on status: 500 means the request passed
require_signature, passed require_ucan_chain, cleared the owner gate, and
reached git on a repo with no disk backing. A bare `!= 403` would let a 401
regression through. It needs no fake-git shim, so unlike the rest of the push
path it is not cfg(unix) and runs on every platform.

Both new tests were verified to fail: with the delegation branch removed the
behavioural test reports 403 where it wants 500, and the unit test's assertion
fires. A test written after its implementation proves nothing until it has been
watched failing.

* feat(gl): gl ucan import stores a delegation for the push helper

The git remote helper needs a delegation on disk in a location it can derive
from a gitlawb:// URL alone. Files key on the bare base58 owner key because
did:key contains a colon, which Windows will not accept in a filename, and the
same identity appears in both full and bare form across this codebase — storing
under one and looking up by the other would silently miss.

Import decodes the token so a malformed delegation fails here, where the error
is actionable, rather than surfacing as an unexplained 403 in the middle of a
git push. A delegation naming no repository is rejected with the resources it
did carry, since that is almost always a wrong --cap argument.

* feat(git-remote): send a delegated push capability as X-Ucan

Wraps a stored delegation into an invocation (iss=agent, aud=node,
prf=[delegation]) and attaches it to the receive-pack POST, which is what lets a
CI or delegated key push once owner-only push is enforced. Only on the push: a
fetch is gated by read visibility, not by git/push.

Capabilities are copied from the delegation unchanged, so an invocation can
never be broader than what was delegated. No expiry is set on the invocation
itself — the delegation`s own exp still bounds the chain because verify_chain
checks each proof`s expiry as it recurses. A test proves that rather than
asserting it: an already-expired delegation still fails the chain after
wrapping, so an expired grant cannot be laundered into an open-ended one.

The whole path is best-effort. A missing delegation, an unreachable node, or an
unreadable stored token all send the request without the header. The node
decides whether one was required, and a node denial has to reach the user
instead of being pre-empted by a local guess.

delegation_path duplicates gl`s six-line version because gl is not a dependency
of this crate; both carry a pointer to the other.

* test(git-remote): cover pack-POST URL splitting

split_pack_post_url decides which repo a delegation is looked up for, so a
parsing slip silently means no delegation was found, surfacing as a confusing
403 rather than a visible error. Covers the optional .git suffix, a repo
genuinely named x.git, and the malformed shapes that must not parse.

* build: sync Cargo.lock for the git-remote-gitlawb dependencies

Adding serde_json and chrono to crates/git-remote-gitlawb/Cargo.toml updated
Cargo.lock locally, but the lockfile lives at the repo root and was never
staged. Every CI job builds with --locked, so all seven compiling jobs failed
immediately with "cannot update the lock file ... because --locked was passed".

* fix(core,gl,git-remote): close two review findings on delegated push

Both are mine, both were found in review, and both are load-bearing.

1. Stripping `nb` mid-chain escalated a constrained delegation.

`Capability::is_attenuated_by` compared only `with` and `can`, so the holder of
a capability constrained by `nb` could re-delegate the same resource and action
with the constraints removed and `verify_chain` still accepted the chain. The
node's push gate refuses constrained capabilities at the LEAF, so it then saw an
unconstrained one and granted repo-wide push — guarding the leaf guarded the
wrong end of the chain.

Constraints now participate in attenuation: an unconstrained parent permits
anything, an identical child is attenuated, and both a differing child and a
child that drops the constraints are refused. `nb` has no interpreted semantics
yet, so "different" cannot be shown to be narrower and is refused with the
stripping case. Tests cover the forged strip, the legal unchanged case, and
adding constraints under an unconstrained parent.

2. `gl ucan import` wrote to a path taken from an untrusted token.

`repo_from_resource` split the resource with `rsplit_once('/')`, so the owner
half could carry separators, `..`, or an absolute prefix. `Path::join` with an
absolute component discards the base entirely, so a resource of
`gitlawb://repos/C:/Windows/System32/x` did not merely climb out of the
delegations directory — it replaced it. The value then reached `std::fs::write`.

The resource must now be exactly `gitlawb://repos/<owner>/<repo>`, with each
half an allow-listed component: alphanumerics plus `.`, `-`, `_` and `:` (a DID
carries colons), never `.` or `..` or anything containing `..`. An allow-list
rather than a deny-list of separators, which would miss whichever ones the next
platform introduces.

The same shape existed in the helper's `split_pack_post_url`. There the value
comes from the remote URL and the path is only read, but a traversing owner
would read an arbitrary file and send its contents to the node as `X-Ucan`, so
it carries the same guard.

Also: the node-DID probe now uses a 5s timeout instead of inheriting the shared
client's 300s. It is best-effort metadata, and a stalled node should not delay
every delegated push by five minutes before falling back to sending no header.

* fix(core,node,gl,git-remote): close the second review round on delegated push

Each finding was reproduced against the code before being changed; two were
confirmed with the mutation the reviewer described.

A delegation could be perpetual
--------------------------------
`exp` is optional, `is_expired` reports false when it is absent, `gl ucan
delegate` defaulted to no expiry, and there is no revocation path. So the
default flow minted a permanent push grant: once the token leaked, the owner's
only remedy was rotating the DID the repo is keyed on. The PR body claimed the
damage window was the token's `exp`, which was simply untrue when `exp` was
`None`.

`Ucan::chain_lifetime_is_bounded` walks every link, and `ucan_grants_push`
requires it. It is deliberately NOT enforced inside `verify_chain`: an
unbounded token is well-formed and may suit a read-only capability; whether an
unbounded grant is acceptable is the consumer's policy, not the format's.
`gl ucan delegate` now defaults to 720 hours with an explicit `--no-expiry`
opt-out, and the helper carries the delegation's expiry onto the invocation so
the leaf is bounded too.

The recursion to the root was untested
---------------------------------------
Every chain in the suite was depth two, where the immediate proof IS the root,
so nothing distinguished walking to the true root from returning the proof's
issuer. Confirmed by mutation: keeping full recursive validation but returning
`proof.payload.iss` left gitlawb-core at 92 passed, the node's UCAN tests at 17,
and the e2e green. A three-link owner -> lead -> agent test now pins it, with
`assert_ne!` against the middle issuer as well as `assert_eq!` against the root
— without the former, returning the middle would still pass.

A path-prefixed node base broke delegated push entirely
--------------------------------------------------------
`GITLAWB_NODE` may carry a path (`https://host/gitlawb` behind a proxy), and
`repo_base` passes it through verbatim. `split_pack_post_url` read the FIRST two
segments as owner/repo, so the prefix became the owner: the delegation lookup
missed, the DID probe hit the wrong URL, no `X-Ucan` was sent, and a valid
delegate got a 403 — silently, because every failure on that path is
best-effort. It now strips the known trailing `<owner>/<repo>/<service>` instead,
which is correct at any prefix depth, and prefix segments carry the same
allow-list so a `..` cannot redirect the probe.

A wildcard delegation grew without limit
-----------------------------------------
`with: "*"` covered every repo the owner created AFTER signing, a scope nobody
chose. `build_invocation` now narrows to the concrete repo, which
`is_attenuated_by` accepts under a `*` parent, so a captured invocation is worth
one repo rather than all of them. Constraints are copied from the covering
capability rather than dropped, since dropping them is a widening.

Branch protection and the owner gate now disagree, on purpose
--------------------------------------------------------------
A delegate clears the owner gate and is still refused on a protected branch. That
is the intended policy: a protected branch is the owner's explicit marker that
even routine writes should stop, and if a delegation overrode it, issuing any
capability would weaken every protection already set. The comment claiming
non-owners never reach that loop is corrected, and
`delegated_push_is_still_refused_on_a_protected_branch` pins it — asserting the
body names the branch, so the refusal is provably branch protection rather than
the owner gate.

Smaller items
-------------
The denial body no longer says "only the repo owner may push", which stopped
being true once a delegation could authorize one. It stays a single unconditional
message: varying it by whether a delegation was presented, expired, or named
another repo would turn the refusal into an oracle for which capabilities exist.

`gl ucan import` writes the delegation 0600, matching the sibling identity key.
The token alone cannot push — the node requires `iss` to equal the request
signer — but it discloses the delegation graph.

`docs/RUN-A-NODE.md` told operators not to enable owner-push until every pusher
was the owner, which is the workflow this change adds. It now documents the
delegation flow, the four requirements the node enforces and why, that a
delegation does not override branch protection, and that withdrawal is by expiry
only. `README.md` no longer describes UCAN as a future workflow.

* fix(gl): restrict the delegations directory, and state the contract

The imported delegation was written 0600 on Unix and left to inherit the
directory ACL elsewhere. Tightening only that one file would have been theatre:
identity.pem sits in the same directory under exactly the same assumption, and
its disclosure is strictly worse.

The directory itself is now 0700 on Unix, and the comment states the contract
that already governed the private key — gitlawb_dir accepts any path, std::fs
has no portable ACL API, so on other targets the caller must supply a
user-private directory, which a per-user profile provides by default.

* fix(gl,git-remote): make a delegation issued the documented way actually work

Two silent failures, both of which end as a 403 telling the delegate to obtain
the delegation they are already holding.

The narrowing fix broke the documented issuing form
----------------------------------------------------
`build_invocation` built `gitlawb://repos/{owner}/{repo}` from the push URL and
compared it to the delegation's `with` with `==`. The URL always carries the BARE
owner, since `parse_gitlawb_url` takes the last colon-delimited segment, while
`docs/RUN-A-NODE.md` tells the owner to issue
`--cap gitlawb://repos/<owner-did>/<repo>` — the full DID. Those strings never
match, `find` returns None, and because every failure in `delegation_header` is
best-effort the push goes out with no `X-Ucan`.

This was introduced by the previous round: copying `att` through unchanged worked
because the node normalizes both forms in `did_matches`. Narrowing to a
URL-derived string did not.

The owner segment is now compared on the bare key, and the parent's `with` is
kept VERBATIM whenever it already names this repo — `is_attenuated_by` compares
`with` by exact equality, so re-emitting the bare form under a full-DID parent
would fail attenuation at the node and trade one silent refusal for another. Only
a `*` parent uses the URL-derived resource, which is the case narrowing exists
for. Both combinations are now tested; neither side exercised them before.

The two halves disagreed about where delegations live
------------------------------------------------------
`git-remote-gitlawb` resolves its store from `resolve_key_path().parent()`, which
honors `GITLAWB_KEY`. `gl ucan import` wrote under `gitlawb_dir(None)`, which was
always `~/.gitlawb`. With `GITLAWB_KEY=/data/keys/identity.pem` — the shape
`.env.example` documents — the import stored the token in one directory and the
helper read another, empty one.

`gitlawb_dir` now falls back to the parent of `GITLAWB_KEY` before `~/.gitlawb`,
so both halves derive the store from the same setting.

Also: `.env.example` still said a push from a non-owner DID is rejected, which is
the behaviour this branch removes.

* fix(gl): refuse a relative or non-UTF-8 GITLAWB_KEY when deriving the store

Two defects in the previous round's fix, both of which put the delegation
somewhere the helper does not look and end as an unexplained 403.

A relative GITLAWB_KEY resolved differently in each half. `gl ucan import` and
`git-remote-gitlawb` do not share a working directory, so `keys/identity.pem`
sends the import to one `delegations` directory and the lookup to another. A
one-component value is worse: `parent()` yields "", so the store becomes
`./delegations` relative to whatever directory happened to be current. It is now
refused with a message that says why, rather than silently resolved.

`std::env::var` returns Err for a non-UTF-8 value, which the code treated as
unset — indistinguishable from having no GITLAWB_KEY at all, and silently
selecting ~/.gitlawb instead of the operator's real key directory. `var_os`
keeps the OsString, so a valid non-UTF-8 path now works and an empty one is
still treated as unset.

* fix(identity): one resolver for GITLAWB_KEY across gl and the helper

`gl` and `git-remote-gitlawb` each carried their own answer to "where is my
identity?", and so did five call sites inside `gl` itself. The round-three fix
hardened one of them — `gitlawb_dir` — and left the rest, which is why the same
misconfiguration kept surfacing in a new place each round.

The rules now live in `gitlawb-core::identity_path`, the crate both binaries
already depend on:

  identity_key_path()  $GITLAWB_KEY, else ~/.gitlawb/identity.pem
  identity_dir()       its parent — the delegation store

read through `var_os` (so a non-UTF-8 path is refused rather than folded into
"unset" by `var`), with `~/` expanded on the first path component, a relative
value refused, and a bare `~`, `~/`, or `/` refused rather than resolved to a
directory whose parent is not where anything lives.

Both take the home directory as an argument rather than looking it up: core is
held to an explicit dependency allowlist and both callers already carry `dirs`,
so the rules can be shared without widening core's tree. It also lets every
case be tested against a fixed home instead of the machine's.

Call sites moved onto it:

  gl   identity::gitlawb_dir            delegates to identity_dir
  gl   identity::load_keypair_from_dir  was ~/.gitlawb even after `gl identity
                                        new` wrote elsewhere, so `gl ucan
                                        delegate` signed with a stale DID — or
                                        found nothing — for exactly the
                                        operators who moved their key. Every
                                        `load_keypair_from_dir(None)` caller
                                        (register, repo, pr, clone, mcp) is
                                        fixed with it.
  gl   doctor::run                      the one command whose job is to explain
                                        a broken setup was reporting on a
                                        directory the setup does not use
  gl   init (ucan.json, generate_identity)
  gl   mcp ucan_show
  gl   ucan_cmd::cmd_show
  helper resolve_key_path / the delegation store behind delegation_header

The helper's version was the loosest of the set: `env::var`, a literal `"~/"`
prefix, `HOME` falling back to `"."` (never set on Windows), and no absolute
check at all.

Also in this commit:

- `gl ucan import` creates the store and the token at their final mode.
  `create_dir_all` then chmod leaves 0755 under the usual umask, and
  `fs::write` then chmod leaves 0644, both readable by any local user until the
  second call lands. `DirBuilder::mode` and `OpenOptions::mode` close the
  window; the trailing `set_permissions` now only matters for a store an older
  `gl` left behind.

- `gl ucan import` refuses a delegation the push path cannot use. It filtered on
  resource shape only, so a `pr/open` token printed "Stored delegation for
  owner/repo" and was then dropped by `build_invocation` behind a
  `tracing::warn`, surfacing as a 403 with nothing connecting it to the earlier
  success. Import now applies the same push-class filter the helper does.

- README's write-authorization limitation is narrowed to what is actually
  missing (revocation, `nb` interpretation, non-push capabilities) rather than
  claiming delegated push is not implemented.

Every new guard was verified by disabling it and watching the matching test go
red: the absolute-path check, the bare-`~` refusal, the `load_keypair_from_dir`
routing, and the import action filter each own a failing test.

* test(git-remote): cover delegation_header end to end

`delegation_header` was the one piece of the delegated push path with no test.
Its parts each had one — `split_pack_post_url`, `build_invocation`,
`delegation_path` — but nothing checked they compose, and this is the function
where a regression is silent by construction: every failure inside it returns
`None` and the push goes out without `X-Ucan`, so a break surfaces as a 403 at
the node with nothing locally to connect it to.

Five cases, driven against a mockito node with the store seeded where
`gl ucan import` would have left it:

- the delegated push: a stored token becomes an invocation issued by the agent,
  addressed to the node's DID, rooted at the repo owner, with every link bounded
- the owner's own push: no store read, no node round-trip. The comparison has to
  survive the form mismatch — the keypair holds `did:key:z…` while the URL
  carries the bare key — so the mock asserts zero hits
- no usable delegation: an empty store, a token that does not decode, and a
  `git/fetch` capability that carries nothing to wrap. All three yield no header
  rather than an error
- an unreadable node DID: a 500, a JSON body with no `did`, a proxy's HTML error
  page, and a `did` that does not parse. The push loses its header, never aborts
- a path-prefixed node base: the DID probe must go to `/gitlawb`, not `/`.
  `split_pack_post_url` is unit-tested for the prefix, but nothing checked the
  probe followed it; the mock on `/` asserts zero hits

Each case was watched failing before it was kept, against five separate
mutations: the owner short-circuit removed, the prefix dropped from the node
base, `build_invocation`'s push-class filter widened, the node-DID lookup given
a fallback, and the invocation addressed to the agent instead of the node.

* fix(gl): finish the identity-path sweep on the write side

The previous commit unified where `gl` READS the identity from and stopped
there. Four resolvers were left holding their own `~/.gitlawb`, and one of them
is a write:

  register.rs   ucan_path        the bootstrap UCAN from POST /api/register
  quickstart.rs the wizard dir   identity generation AND the UCAN it stores
  name.rs       identity_dir     reads identity.pem
  node_stake.rs load_did         reads identity.pem

`register.rs` is the one that actually breaks a working setup. With
`GITLAWB_KEY=/data/keys/identity.pem` — the shape `.env.example` documents —
`gl register` loaded the key from `/data/keys/identity.pem` and then wrote
`ucan.json` to `~/.gitlawb/`. Registration reported success; `gl doctor`,
`gl ucan show`, `gl init`, and `gl mcp ucan_show` all read `ucan.json` from the
key's directory, found nothing, and reported an unregistered identity. Split
storage, no error anywhere.

`quickstart` had the same shape end to end: it generates the identity and stores
the bootstrap token, so an operator who ran the wizard under `GITLAWB_KEY` got
both in a directory nothing else consults.

`name.rs` and `node_stake.rs` only read, but both fell back to `PathBuf::from(".")`
when the home directory could not be determined, which makes the identity path
depend on the working directory — the same defect the helper's `resolve_key_path`
carried before it moved to the shared resolver.

All four now go through `crate::identity::gitlawb_dir`. Afterwards the only
`dirs::home_dir()` calls left in `gl` are the one inside `gitlawb_dir` itself,
one test expectation, and `doctor`'s shell-rc scan, which is unrelated to the
identity directory.

`gl register` also no longer prints a path it did not use: the success line was
hardcoded to `~/.gitlawb/ucan.json`, so an operator with `GITLAWB_KEY` set was
told to look somewhere the file was not. It now prints the real destination, and
says so plainly when the node returned no token at all.

The `GITLAWB_KEY` test guard moves to `identity::test_env` so the resolver's own
cases and the new register case share one lock. Two suites each holding their
own mutex over the same process-global variable would not serialise against each
other.

Verified by reverting `ucan_path` to the hardcoded form and watching
`register_saves_the_bootstrap_ucan_beside_the_key` fail on exactly that
assertion.

* fix(gl,core): close round five — lazy home, envelope reads, honest register

Five findings from the round-five review, four of them introduced by round four's
own centralisation.

resolve_key_path and resolve_identity_dir opened with `home_dir()?`, but
identity_key_path only needs a home when the key is unset, empty, or `~/`-prefixed.
On a host where `dirs::home_dir()` returns None — no HOME and no passwd entry, an
ordinary container shape — a perfectly valid absolute GITLAWB_KEY was discarded and
the push went out unsigned, blaming the home directory for a setting the operator
had got right. The pre-round code resolved an absolute key without consulting home
at all, so this was a regression, not an inherited gap. The core resolvers now take
`Option<&Path>` and demand a home only where the value being resolved needs one.

`gl ucan show` called Ucan::decode on the whole of ucan.json, but register, init,
and quickstart all write an envelope — {"ucan", "node", "did", "saved_at"} — and
doctor and quickstart already read it as one. Ucan is {payload, s}, so show failed
with "missing field `payload`" immediately after a successful `gl register`. It now
reads the envelope like every other reader, and still accepts a bare token so files
written by an older gl stay readable. The predating mismatch belongs to this round
because this round rerouted cmd_show and claimed to unify the identity flow.

`gl register` printed "You are now a verified agent" even when the node returned no
bootstrap UCAN — the exact shape the previous commit message said it existed to
remove, one line below the fix. The closing line is now inside the branch that
actually stored a token.

The MCP `ucan_show` tool called gitlawb_dir(None) while every sibling tool in
call_tool honours the server's --dir, so one session read two identity directories.
The parameter was already in scope.

Six `--dir` help texts still promised "default: ~/.gitlawb" after the resolver
stopped having that default.

Both behavioural fixes were verified by mutation: demanding a home unconditionally
turns an_absolute_key_resolves_without_a_home_directory red, and dropping the
envelope branch turns the_register_envelope_decodes red.

* fix(node,gl,git-remote): enforce repo narrowing at the node, honour the key basename

Round seven. Two P1s, and the first changes an authorization outcome.

The node accepted a leaf resource of `*` for delegated push. Narrowing a wildcard
delegation to the concrete repository happened only in `git-remote-gitlawb`, which
is one client's behaviour and not a protocol boundary: a delegate can sign an
otherwise valid `agent -> node` invocation that keeps `with: "*"`, it passes
attenuation because its proof is `"*"` too, and the node then authorised a push to
any repository owned by the root DID — including repositories created after the
delegation was issued. `docs/RUN-A-NODE.md` promised exactly the opposite, and
named the growth it was preventing, while resting that promise on the helper.
`repo_capability_matches` now requires a concrete `gitlawb://repos/<owner>/<repo>`.
Wildcard delegations still work through the normal path, because the helper
narrows before signing; only a hand-built wildcard leaf is refused.

`honours_the_resource_wildcard_and_repo_admin` asserted the old behaviour, so it
is now `refuses_a_resource_wildcard_and_honours_repo_admin`, and it separates the
two axes: a `*` ACTION on a concrete repository is still fine, since attenuation
bounds it. `a_wildcard_delegation_cannot_reach_a_second_repository` covers the
attack directly.

`gl` discarded the basename of `GITLAWB_KEY`. The shared resolver returns the
configured PEM path, but `gl` kept only its parent and re-appended `identity.pem`,
while the helper opens the configured path. With
`GITLAWB_KEY=/data/keys/ci-agent.pem` the CLI wrote and read
`/data/keys/identity.pem` while every push loaded `/data/keys/ci-agent.pem`, so
the two either disagreed on identity or the helper found no key — and both owner
enforcement and the delegation proof key off that identity. This was mine, from
last round's centralisation: a directory-level contract substituted where callers
needed a key-file-level one. `key_path_for` now serves both, and the seven
production sites that still hardcoded the basename go through it: `identity`
(load/new/backup/restore), `doctor`, `init`, `name` (x2), `node_stake`,
`quickstart` (x2). An explicit `--dir` keeps `<dir>/identity.pem`.

Also:

- `gl ucan import` binds the delegation's audience to the local identity and
  verifies its chain, expiry, and bounded lifetime before storing. It previously
  accepted a token addressed to a different agent, reported success, and left the
  push to fail on proof linkage with nothing pointing back.

- `build_invocation` selects only a capability the node can authorise. It took the
  first push-class capability, including a constrained one, while the node refuses
  constrained leaves outright — so a delegation holding both a constrained and an
  unconstrained grant succeeded or failed purely on their order in `att`.

- Delegation files are replaced atomically: written to a sibling temp at 0600,
  fsynced, then renamed. `truncate(true)` on the live path emptied a working
  delegation before the replacement was written, so an interruption left the
  helper with an unreadable token and pushes that silently drop `X-Ucan`.

- `gl quickstart` no longer reports "UCAN saved to ..." when the node returned no
  token. Both reviewers flagged this independently; it is the same false success
  fixed in `register` last round, one file over.

Every fix was verified by mutation: restoring the node's wildcard acceptance,
restoring `<parent>/identity.pem`, disabling the audience check, and dropping the
helper's constraint filter each turn the matching test red.

* fix(gl): align the MCP ucan_show tool with gl ucan show

The two surfaces answer the same question and answered it in different shapes:
`gl ucan show` decodes the envelope and prints fields, while the MCP tool returned
the file verbatim. That was the round-seven non-blocking note.

Aligning them removes a second problem the note did not name. The file contains
the bootstrap token, so the MCP tool handed the caller the credential itself —
and an MCP response travels further than a terminal does, into a model's context,
transcripts, and logs. Nothing needs the token to answer "am I registered, with
what, until when", so the tool now returns issuer, audience, version,
capabilities, expiry, expired, signature validity, and the path it read.

The tool description said "Show the saved bootstrap UCAN token"; it now says what
is returned and that the token is not.

Verified by mutation: restoring the raw-file return turns
`ucan_show_returns_decoded_fields_and_withholds_the_token` red on the assertion
that the token is absent.

* fix(gl): drop a needless borrow in quickstart's unix key write

`generate_identity` now takes `path: &Path` rather than deriving it from an owned
directory, so `set_permissions(&path, ..)` became a double borrow. Clippy refuses
it under `-D warnings`.

It sits inside `#[cfg(unix)]`, which is why the local gate passed: this machine is
Windows, so that block is never compiled or linted here. Only CI sees it. The
sibling `fs::write` on the line above was already corrected; this one was missed
because the same pass could not fail locally.

The other `&path` / `&dest` / `&tmp` borrows in the unix blocks this PR touches
are on owned `PathBuf`s, where the borrow is required — `quickstart` is the sole
place a parameter changed from owned to reference, and so the sole place the lint
applies.

* fix(node,gl,git-remote): fix a delegation's scope at issuance

Round eight. The round-seven wildcard fix closed the wrong half, and I confirmed
that by execution before changing anything:

    PROBE ucan_grants_push = true

An owner issues `with: "*"`, the delegate narrows it to a concrete leaf for a
repository created afterwards, `verify_chain` passes, and the node authorised the
push. Refusing a wildcard LEAF did nothing, because `is_attenuated_by` accepts a
concrete child under a `*` parent — and that narrowing is exactly what
`build_invocation` performed, so the first-party helper was the working mint path.
The round-seven tests never saw it: they injected a self-issued `*` leaf with no
`prf`, so they never touched a proof chain at all.

The contract is now that a delegation's scope is fixed when it is issued, which is
what both operator documents already promised:

- The node requires EVERY link to name the repository, not just the leaf
  (`proofs_name_repo` walks `prf` after `verify_chain` has established the chain).
- `git-remote-gitlawb` no longer selects or narrows a `*` capability, so a stored
  wildcard fails closed locally with a clear message instead of minting an
  invocation the node would refuse.
- `gl ucan delegate` refuses to issue a push-class wildcard at all.

Also, all from round eight:

- `gl quickstart` loads the exact key it selected. It resolved `pem_path` through
  `key_path_for` but then loaded through `Some(&dir)`, which re-derived the
  basename as `identity.pem` — so with `GITLAWB_KEY=/data/keys/ci-agent.pem` the
  existence check and the load disagreed, and the Err arm regenerated ONTO
  `ci-agent.pem`, destroying a working key and changing the DID that repository
  ownership, registrations, and delegations all key off.

- `gl ucan import` binds every capability's owner segment to the VERIFIED ROOT.
  `verify_chain` proves a chain is internally valid; it says nothing about which
  repository it applies to. Without this, any key holder could issue a valid
  bounded token naming `gitlawb://repos/<victim>/repo` and displace the working
  delegation for a repository they have no authority over. The whole set is
  validated before anything is written, so a later bad capability cannot leave a
  multi-repository import half applied.

- Import applies the same push-class rule the helper and node use, including
  `constraints.is_none()`. A constrained-only grant used to import cleanly and
  then authorise nothing.

- Delegation replacement stages to a per-call path (`create_new`, pid + counter)
  rather than a shared `.<name>.tmp`, so two concurrent refreshes cannot write the
  same inode and publish each other's bytes. The non-Unix path stages and renames
  too; it previously truncated the live file, so a failed refresh destroyed a
  working token on that platform.

Three tests that were weaker than they read, all mine:

- `a_refused_reimport_leaves_the_stored_delegation_intact` never reached
  `write_private_file` — it stopped at the audience check. Renamed to say what it
  covers, and `a_failed_write_leaves_the_stored_delegation_intact` now forces a
  real write failure against a read-only store.
- The MCP token-withholding assertion used `!out.contains(&token)`, which passes
  whether or not the token leaked: the token is itself JSON, so embedding it
  escapes every quote. It now asserts on the signature, a bare base64 string that
  survives escaping.
- `build_invocation_narrows_a_wildcard_to_the_pushed_repo` asserted the behaviour
  this commit removes; it is now `build_invocation_refuses_a_wildcard_delegation`.

* fix(gl): drop an unused import in the unix-only refresh tests

`refresh_atomicity_tests` moved from `token_for_agent` to `owned_token` when
import began requiring the resource owner to equal the verified root, and the
import was left behind. The module is `#[cfg(all(test, unix))]`, so it does not
compile on this Windows host and no local clippy run could see it.

Cross-linting is not available either: `cargo clippy --target
x86_64-unknown-linux-gnu` fails in ring's build script for want of a C
cross-compiler. CI is the only linter for cfg(unix) code from here, which is worth
recording rather than rediscovering.

The sibling `import_binding_tests` still uses both helpers; only the unix module's
import changed.

* fix(node): pass the UCAN extension at the last two receive-pack call sites

Two `git_receive_pack` calls in `#[cfg(unix)]` test blocks still supplied six
arguments. Like the rest of that class they do not compile on this Windows host,
so neither the local build nor clippy could see them; CI found them one at a time.

Rather than wait for a third round, every call site was audited by grep instead of
by compiler: 25 calls, 2 missing, both fixed, none others. The other signatures
this branch changed — `caller_authorized_to_push`, `identity_key_path`,
`identity_dir`, `key_path_for` — were audited the same way and are consistent.

Both sites push as a synthetic non-owner DID under an owner-push-disabled config,
so `None` is correct: no delegation is involved in what they test (write-lock
release on disconnect, and on push success).

* fix(core,node,gl,git-remote): one definition of a usable push capability

Round nine. Both reviewers named the same pattern independently: the rule for
"can this token push to this repository?" was reconstructed at every boundary —
issuance, import, invocation construction, node authorization — and each round
found a token one boundary accepted and the next refused. This round's own
instance: the node walked the whole proof chain, import checked only the leaf, so
a concrete leaf on a wildcard proof imported cleanly and 403'd on every push.

The rule now lives once, in `gitlawb_core::ucan::push`, the crate all four
already depend on:

  did_key_eq            the node's exact did:key collapse rule — bare and full
                        forms are one identity; a bare id never matches across
                        methods (`did:gitlawb:X` is not `X`)
  is_push_action        git/push, "*", repo/admin
  parse_repo_resource   gitlawb://repos/<owner>/<repo>, exactly two segments
  is_push_wildcard      a push-class action on a "*" resource
  Capability::grants_push_to(owner, repo)
                        push-class, unconstrained, names exactly this repo
  Ucan::chain_grants_push_to(owner, repo)
                        the leaf AND every proof behind it, depth-bounded,
                        failing closed on undecodable or multi proofs

and every boundary calls it:

  node       ucan_grants_push = root anchored && bounded && chain_grants_push_to.
             `repo_capability_matches` is a thin wrapper; `proofs_name_repo` and
             `push_class_names_repo` are gone. `did_matches` delegates to
             did_key_eq so there is one collapse rule, not two.
  import     the same walk, before any write. A leaf naming the repo on a "*"
             proof is refused with a message that names the proof. Everything is
             validated before anything is written.
  helper     `build_invocation` refuses a stored token whose chain does not grant
             push to this repo, with a message that says why, before selecting a
             leaf — instead of minting an invocation the node is certain to refuse.
  issuance   `gl ucan delegate` and the MCP `ucan_delegate` tool share
             is_push_wildcard. The MCP arm had no guard at all.

Two tests that failed CI, both invisible on this Windows host because they are
cfg(unix):

- `import_creates_the_store_and_token_owner_only` still issued against a
  synthetic `z6MkAbc` owner, so import's owner-root binding refused it before the
  mode assertions. Now built from `owned_token`, like its siblings.

- `a_failed_write_leaves_the_stored_delegation_intact` chmod'd the store to 0500
  and expected the staging create to fail. It did not: `create_private_dir`
  repairs the store to 0700 before every write, so the second import succeeded
  and the test failed on `is_err()` — and a privileged runner would have made the
  arrangement pass anyway. Failure is now injected through a test-only seam
  (`fault::FailStagingWrites`) that fires after the bytes are written and before
  anything is published, on every platform, and the test also asserts no staging
  file is left behind. The module is no longer unix-only.

Also: the node's rustdocs for `repo_capability_matches` and `ucan_grants_push`
described the pre-round-8 behaviour (helper narrowing; leaf-only coverage) and now
match the code. An orphaned docblock above `staging_path` — the old
`write_private_file` doc, left behind in round 8 and claiming "not create_new"
when the writer uses exactly that — is removed and the writer has a correct one.

Verified by mutation, each restored byte-for-byte afterwards:

  core walks no proofs        → core 3-link, node 2-link, node 3-link, import,
                                and helper wildcard tests all red
  core walks ONE proof only   → ONLY the two 3-link tests red; every 2-link test
                                stays green — the recursion is load-bearing
  MCP guard removed           → MCP deny test red, MCP named-resource test green
  staging seam disconnected   → atomicity test red
  import chain check removed  → import deny test red, delegate deny test green

fmt, clippy -D warnings, check --locked --workspace --all-targets clean.
gitlawb-core 122, gl 389, git-remote-gitlawb 65, node auth:: 23, and the DB-backed
delegated-push tests pass alongside main's owner-push test.

* fix(node): drop repo_capability_matches, orphaned by the shared push rule

Round nine routed the node's authorization through
Ucan::chain_grants_push_to, which parses the resource and compares
owners with did_key_eq itself. The thin wrapper left behind had no
callers, and CI's -D warnings build rejects dead code. Nothing else in
the crate referenced it.

* fix(core,gl): bound every proof-chain walk and make verify report the chain

Round ten of #331.

`verify_chain` and `chain_lifetime_is_bounded` recursed with no depth
guard while `chain_grants_push_to` carried one. The node verifies the
chain on the `X-Ucan` header of any signed request before the scope walk
runs, so the bound protected nothing: a deep chain met the unbounded
walk first. Both now stop at `MAX_CHAIN_DEPTH`, which moves to the
module root since all three walks share it, and a chain at the bound
still validates end to end.

The MCP `ucan_delegate` tool issued an unbounded token when
`expiry_hours` was omitted, which import and the node both refuse for
push. It now defaults to the CLI's 720 hours and refuses a non-positive
value.

`gl ucan verify` and the MCP `ucan_verify` tool answered from the leaf
alone, so a good signature on a broken proof chain reported valid and
then failed at import and at the node. Both report through one
`VerifyReport`, which walks the chain; `valid` requires it.

The non-Unix `write_private_file` removed the live delegation before
`rename`, on the belief that Windows refuses to replace an existing
destination. std maps `rename` to `MoveFileExW` with
`MOVEFILE_REPLACE_EXISTING`, so the remove only opened a window with no
delegation at all. Dropped; a refresh over an existing token is now
tested on every platform.

Also: `cmd_import` and `cmd_verify` share one token-argument reader that
reports an unreadable path as a file problem and caps the file size,
instead of falling through to a decode error on the path string; and
the MCP `ucan_delegate` description says import is CLI-only.

* docs: state the two gates a delegated push crosses, and finish the sweep

Round eleven of #331, documentation half.

A `git push` crosses two independent gates: read visibility on the
`info/refs` advertisement, and push authorization on the receive-pack
POST. This PR implements the second and left the first as it was, and a
UCAN grants no read access — so on a private repository a delegate must
also be a reader. Nothing said so. RUN-A-NODE now states the contract
once, in the owner-push section, and the delegation section carries a
"Private repositories" note with the `gl visibility set` invocation and
a "One repository per token" note matching what import enforces.

The rest is the drift the earlier doc edits left behind: the README
env-var table still said a `git/push` capability was "verified but not
yet honored", the known-limitations bullet still said enforcement
defaults to `false`, and the RUN-A-NODE section opened with a blanket
403 for every non-owner two paragraphs above the text that says a
delegate can push. All three now describe the same node.

* fix(gl): store one repository per imported delegation; pin the two-gate composition

Round eleven of #331, code half.

`gl ucan import` validated every capability up front and then wrote one
file per repository, and `write_private_file` is atomic per file, not
across files: a token naming two repositories could publish the first
delegation, fail on the second, and report failure for both. `gl ucan
delegate` and the MCP tool issue one capability per token, so such a
token is hand-built; import now refuses it with the supported shape in
the message. Several capabilities on one repository are still one file.

`delegated_push_on_a_private_repo_needs_read_visibility_too` pins the
composition the docs now describe, through the middleware production
gives each route: on a private repository a valid delegation is refused
at the `info/refs` advertisement (404) until the owner names the agent
as a reader, clears the receive-pack POST on its own, and a reader with
no delegation is still refused on the push. Making the visibility check
admit everyone reddens the advertisement assertion.

* fix(gl,core): sanitize token-derived output, prove the publish step, add MCP import

Round eleven of #331, second half.

Every string a token carries is chosen by whoever built it: `Did`
deserializes any string (only `FromStr` validates), and a capability
field is free text. `gl ucan verify`, `gl ucan show`, and the import
refusal messages printed them raw, so an ANSI or OSC sequence in `with`,
`can`, `iss` or `aud` — or in the error text `verify_chain` quotes them
into — reached the operator's terminal. The strip is now one definition
in core, `sanitize::strip_terminal_controls` (both halves of INV-6), and
`gl`'s existing `sanitize_node_msg` is that plus its length cap. In
`ucan_cmd`, `shown()` wraps every token-derived value on its way out.

The refresh-replacement test could not tell the current non-Unix writer
from the one that removed the live file before renaming, because a
rename that succeeds ends in the same state either way. A second seam,
`fault::FailPublish`, fails the rename after the staged bytes are
complete; the new test requires the old token to survive that, and
reinstating the remove reddens it on the platform that had it.

The MCP server could issue and verify a delegation but not store it
where `git-remote-gitlawb` looks. `cmd_import` is now a printer over
`import_delegation`, and the `ucan_import` tool is the same function
with a JSON answer: one set of checks, one store.

* test(gl): count the MCP tool list at 41 with ucan_import

`e3314c6` added the tool and left the count test at 40, so `test
(windows)` went red first. The test also carried "42" in its name while
asserting 40; it is now named for what it checks, and the names test
covers `ucan_import` alongside the other two UCAN tools.

* fix(node,gl): reader gate on receive-pack POST, GITLAWB_KEY in optional load, sanitize + bound UCAN surfaces (#331)

* test(gl): pin GITLAWB_KEY honouring in load_optional_keypair (#331)

* fix(gl): drop needless borrow caught by clippy

* fix(gl): doctor reports refused resolvers, sanitize MCP UCAN JSON, atomic key write (#567)

* fix(ucan): match push capabilities against the canonical repo name

A capability written gitlawb://repos/<owner>/<repo>.git passed import
and filed under the suffixed name, but the helper looks the store up by
the canonical name and the node compared the resource verbatim against
record.name, so the delegation missed at every hop and the push died at
the owner gate with a bare 403.

Introduce ucan::push::canonical_repo_name - the same at-most-one-.git
strip smart_http_repo_name applies - and run it everywhere a push
capability is issued (gl ucan delegate, MCP ucan_delegate), imported
(repo_from_resource now returns the canonical segment, so the file
lands where the helper looks), loaded (split_pack_post_url), and
matched (grants_push_to, hence chain_grants_push_to and the node gate).
At-most-once preserves the literal foo.git repo, reachable as
foo.git.git. Document the canonical form in RUN-A-NODE.md.

* fix(ucan): refuse key clobber on unreadable identity, fix doc gate sites and message escapes

- quickstart: bail on an existing-but-unreadable identity instead of
  regenerating over it, and write the fresh key with create_new so a
  file that appears mid-setup is not clobbered (same rule gl init uses)
- RUN-A-NODE: gate 1 read visibility applies on both GET info/refs and
  the git-receive-pack POST (same 404 for a client that skips the
  advertisement); visibility set example carries an explicit --mode
- ucan delegate: replace literal \n escapes in the wildcard refusal
  messages (CLI and MCP) with line continuations so the error prints as
  one sentence; MCP expiry bound uses the shared MAX_EXPIRY_HOURS
- MCP ucan_show: run the token version field through shown()
- helper tests: restore GITLAWB_KEY via a drop guard so a panicking case
  cannot leak a tempdir path, and run the advertisement signature test
  under the shared lock with a deterministic empty store

* docs(node): gate 1 allow-list semantics, fix comment-strip order in fn_body

- RUN-A-NODE: a `/` rule is an allow-list that replaces the public flag,
  so "public repo = open to everyone / reader step unnecessary" only
  holds when no `/` rule exists; gate-1 bullet, the both-gates summary,
  and the private-repositories section now tell that one story
- authz_guard fn_body strips // line comments before /* */ blocks so a
  line comment containing /* cannot open a range that swallows a gate

* fix(node,gl): fail closed on protection-lookup errors, surface push readiness in verify

- git-receive-pack: a non-owner whose is_branch_protected lookup errors
  is now refused instead of treated as unprotected; the owner path is
  unchanged (skip the check outright)
- VerifyReport gains bounded_lifetime and push_ready so `gl ucan verify`
  and MCP verify show what import and the node would do with an
  unbounded push chain instead of a bare valid:true
- RUN-A-NODE.md: the owner-push bullet names the delegation exception,
  and the delegation walkthrough now states the GITLAWB_KEY absolute-path
  prerequisite (import --dir must land in the same store the helper reads)
- pins: delegate push refused when the protection lookup errors; an
  unbounded push-shaped delegation verifies but is not push_ready

* fix(gl): judge push readiness with the shared push predicates in ucan verify

- push_class now uses the same is_push_action rule import and the node
  apply, so repo/admin and * actions classify as push-shaped instead of
  omitting the Push-ready line
- push_ready is a stored verdict, not just bounded+valid: every concrete
  gitlawb://repos/... push capability on the leaf must pass
  chain_grants_push_to, so a wildcard proof behind a concrete leaf
  verifies cryptographically yet reports not push-ready, matching what
  import and git-receive-pack would do
- pins: repo/admin and * actions are push-ready; a wildcard-proof chain
  verifies but is not push-ready

* fix(git-remote): gate delegation_header on import's push predicates and the file cap

- a stored delegation that is expired or has an unbounded link is now
  ignored instead of attached: the node would refuse it at middleware
  with a distinguishable invalid_ucan response, and omitting the header
  keeps the denial on the unified owner-push path
- the store file is size-checked against the same 1 MiB cap
  MAX_TOKEN_FILE_BYTES applies at import; an oversize entry is ignored
  like an unreadable one rather than read into memory on every push
- pins: an expired on-disk delegation produces no X-Ucan (and no node
  probe); an oversize store entry produces none either

---------

Co-authored-by: Vasanthdev2004 <[email protected]>
2026-11-11 02:49:14 +00:00
beardthelionandAyush7614 d15f194f7f fix(node): pin no-redirect and bound the body read on the iCaptcha public-key fetch (#574)
* fix(node): pin no-redirect and bound the body read on the iCaptcha public-key fetch

The node fetches the iCaptcha public key that anchors its proof gate, but built
its client with reqwest's default redirect policy and trusted `.json()` to
buffer the response. A redirect from the configured ICAPTCHA_URL could move the
trust anchor, and an oversized body was buffered without bound.

- Build the key client with `redirect::Policy::none()`, matching every other
  outbound client the node runs.
- Read the response body through a capped chunk loop (8 KiB) and parse from the
  received bytes instead of buffering unboundedly.
- Add mockito tests proving a valid JWKS still parses, a 302 to a host serving
  a real key is NOT followed, and an oversized body is refused.

Closes #507

* test(node): make iCaptcha oversized-body test use a valid padded JWKS

coderabbit flagged that the size-limit test fed fetch_key a body of
repeated x bytes, which is invalid JSON: even without a byte cap the
parse would return None, so the test passed regardless of the limit.
Pad jwks_body() with JSON whitespace past KEY_MAX_BODY_BYTES so an
unbounded read would still parse successfully, making the assertion
actually exercise the cap.

* chore(ci): re-trigger PR checks on the flaky test (beta) run

* fix(node): close the anchor paths the redirect pin left open, pin the boundary

- fetch_key refuses any non-2xx status before parsing the body: under
  Policy::none() a 3xx surfaces rather than being followed, but its own
  body could still carry a parseable JWKS and move the anchor anyway
- read_key_body accepts a body of exactly the cap (the loop now probes
  for end-of-body instead of exiting at the limit); over-cap is still
  refused, matching the documented "exceeds" contract
- the no-redirect test now points Location at the mocked /v1/pubkey so a
  redirect-following client would return a real key, making the pin
  load-bearing, and asserts the target is never contacted
- new pins: at-cap JWKS parses, a non-2xx with a valid JWKS body is
  refused, an unreachable endpoint is fail-safe None

---------

Co-authored-by: Ayush7614 <[email protected]>
2026-11-10 20:38:29 +00:00
beardthelionandcarlitos973 f67688eae2 fix(gl): resolve webhook owner from caller identity, not node DID (#568)
* fix(gl): resolve webhook owner from caller identity, not node DID

resolve_owner() in webhook.rs unconditionally fetched the node's own
root DID via GET / and used it as the repo owner, ignoring both an
explicit owner/repo argument and the caller's own loaded identity.
This made gl webhook create/list/delete effectively unusable for any
repo not owned by the node operator itself -- every call resolved to
the wrong owner and 404d.

cert.rs's resolve_repo already has the correct pattern: check for an
explicit slash first, then prefer the caller's own keypair, falling
back to the node DID only if no local identity is available. This
applies that same pattern to webhook create/list/delete.

Verified against a live self-hosted node: webhook create/list/delete
all resolve the correct owner now. Existing webhook test suite passes
unchanged (8/8).

* fix(gl): print fully-qualified owner/repo in webhook create delete hint

Per CodeRabbit review on #159: the resolve_owner destructuring shadowed
repo with the bare name, so the printed delete hint lost the owner
qualification -- only correct by coincidence if the caller runs it
under the same identity used at create time. Print owner/repo
explicitly so the hint always works regardless of default identity.

* fix(gl): thread identity resolution and status checking through webhook list

Apply the same fix already applied to create/delete to the list
subcommand, per review feedback on this PR:

- gl webhook list now accepts --dir and resolves the caller identity
  the same way create/delete do, instead of always falling back to
  the node root DID for bare repo names
- cmd_list checks the HTTP status before parsing, so a non-2xx error
  (e.g. repo not found, not authorized) surfaces as an error instead
  of silently rendering as an empty webhook list
- list output now prints the fully-qualified owner/repo, matching the
  fix already applied to the create/delete hints

Also tightens resolve_owners doc comment: it falls back to the node
DID on any keypair-load failure, not only when no keypair exists.

* fix(gl): thread caller-identity repo resolution through MCP webhook tools (#159)

* test(gl): pin unsigned webhook_list node-denial fallback on MCP (#568)

---------

Co-authored-by: carlitos973 <[email protected]>
2026-11-10 18:53:03 +00:00
beardthelionandJaime Alfaro Salazar af6a759330 test(node): pin task list limit ceiling and default on the cursor handler (#572)
* Fix #399: GET /api/v1/tasks: clamp limit to match siblings; negative limit returns 500 with raw DB error

* test(api): test exact limit boundaries for negative and i64::MAX limit requests

* test(node): pin task list limit ceiling and default on the cursor handler

Port the #405 regression coverage onto the current cursor-paginated
handler: i64::MAX clamps to the 200 ceiling and an omitted limit falls
back to the 50-row default. Negative-limit behavior is already pinned
by negative_limit_returns_empty.

---------

Co-authored-by: Jaime Alfaro Salazar <[email protected]>
2026-11-10 17:12:41 +00:00
beardthelionandAyush7614 66fe646320 fix(node): close quarantine bypass on encrypted blobs, visibility routes, and sibling mutations (#566)
* fix(node): close quarantine bypass on encrypted blobs and visibility routes

Rebuilt per review: this branch now carries ONLY the quarantine fix
(dropped the unrelated task-claim and Windows fetch commits, which live
in #275/#312, and the author-local .cursor symlink). The ipfs.rs and
db-layer halves of the original commit were superseded by main's paged
scan, which already carries per-row quarantine flags; what remains is
routing encrypted-blob and visibility handlers through
authorize_repo_read so a quarantined repo is an opaque 404 there too,
plus the three regression tests for exactly that surface.

* fix(node): gate sibling mutation endpoints on quarantine, reconcile 404 contract in gl clone (#276)

* fix(node): release quarantined mirror fixtures, bound 404 body read in gl clone (#566)

* test(node): pin read-before-owner order on every dual-gated handler (#566)

* test(node): pin list_webhooks in the order loop, close-owner second assert, tighter release legs (#566)

---------

Co-authored-by: Ayush7614 <[email protected]>
2026-11-10 15:29:23 +00:00
JATMNandClaude Opus 5.5 309fc8360d ci: run PR checks and CodeQL only for the surfaces a PR touches (#519)
* ci: run PR checks and CodeQL only for the surfaces a PR touches

Every PR ran every PR Checks job on the whole workspace, and CodeQL default
setup analyzed every language, including a ~13 min macOS Swift autobuild
(55 min when it hung on main) for an 8-file macos-app/ that 4 of the last
~280 commits touched.

scripts/ci-surfaces.sh maps a PR's changed paths to the jobs that validate
them, and scopes cargo test / build --release to the touched crates plus
their transitive reverse dependencies (derived from crates/*/Cargo.toml).
Root Cargo.toml/Cargo.lock/.cargo, the CI wiring itself, and unknown crate
dirs widen to the full suite. Push to main, merge_group, and the release
flow's workflow_dispatch always run everything on the whole workspace.

codeql.yml replaces default setup with the same scoping per language and a
full analysis on push to main and weekly. It needs default setup disabled.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* docs: say which changes widen PR CI to the whole workspace

ci-surfaces.sh has two widening tiers. A root Cargo.toml/Cargo.lock/.cargo
change (or toolchain config, or an unknown crate dir) runs the Rust jobs on
--workspace, while the other jobs still follow their own inputs. Only the CI
wiring itself (pr-checks.yml, codeql.yml, ci-surfaces.sh) and non-PR events
run every surface. AGENTS.md and the script header described both tiers as
"everything" / "the full suite" and left "CI wiring" undefined, so an edit
to, say, release.yml read as if it ran the full PR Checks suite; it runs
none of those jobs and only CodeQL's Actions analysis.

Comment and docs only; the mapper's output is unchanged.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* ci: test gitlawb-node when bootstrap-peers.json changes

ci-surfaces.sh attributed Rust crates only by crates/<dir>/ paths, but
gitlawb-node also compiles in the repo-root bootstrap-peers.json through
include_str!. A PR that changed only that file therefore ran the Docker
smoke (docker build + --version, which embeds the JSON without parsing
it) and skipped cargo test, so embedded_seed_list_parses_successfully
("embedded bootstrap-peers.json must always parse") no longer guarded it.

Add a path -> crate table for files outside crates/ that a crate
compiles in, routed through the same check as crates/<dir>/ paths (an
unknown crate still widens to --workspace). bootstrap-peers.json now
yields rust=true with -p gitlawb-node and keeps its Docker trigger; every
other mapper output is unchanged.

The header also said anything the script cannot attribute precisely
widens the run. That holds only inside the Rust surface; a path no
surface lists (docs, README, infra/, other workflows) runs no PR Checks
job, as AGENTS.md says. Reword it to the two tiers the script implements.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* ci: feed ci-surfaces NUL-delimited paths and pin its mapping in a test

The mapper read newline-delimited `git diff --name-only` output. In that
mode git C-quotes any path with non-ASCII, tab, double-quote or newline
characters, so crates/gl/src/é.rs arrived as "crates/gl/src/\303\251.rs",
matched no surface, and a PR whose only Rust change was such a file ran no
Rust job (Café.swift likewise skipped Swift CodeQL). Newline-delimited
input also cannot carry a path that contains a newline.

Both workflows now pass `git diff -z` output and the mapper reads it with
`read -r -d ''`; input that is not NUL-terminated is rejected (exit 2)
instead of being misread. Mapping output for every ASCII path is
unchanged (checked against the last 43 commit ranges).

scripts/test-ci-surfaces.sh pins one must-not-skip case per surface row,
the crate fan-out and widening rules, --all, the wiring rows, quoted
names, and rejection of newline-delimited input. PR Checks runs it in the
`changed surfaces` job before mapping, on every event, so a broken mapper
turns the run red rather than silently narrowing it.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

---------

Co-authored-by: Claude Opus 5.5 <[email protected]>
2026-11-10 00:41:03 +00:00
f68d0dd3ff fix(node): durable post-receive outbox for receive-pack (#26 split 1/4) (#558)
* fix(node): durable post-receive outbox at the DB layer (#26 split 1/4)

Reviewer 2 closed PR #224 on 2026-08-28 with a directive: split the
work into four narrow PRs. This is Split PR 1 (durable post-receive
lifecycle) at the DB layer; the handler refactor in
crates/gitlawb-node/src/api/repos.rs:2007 (git_receive_pack) lands in
the next slice so the test can drive the failure injection end-to-end.

The pre-outbox crash window the reviewer flagged: receive_pack can
apply a ref to disk and return Ok, and a process exit, a dropped
future, or a DB failure before the bookkeeping at
crates/gitlawb-node/src/api/repos.rs:2361 (push event + cert + webhook)
loses the recovery record. Startup drain enumerates only sources
written from that bookkeeping, so it cannot reconstruct the missing
work. The partial fallback that re-derives from a row present in the
bookkeeping substitutes did:key:recovered and an empty attestation,
which is not equivalent to the original authenticated push.

This commit adds the durable boundary the handler will lean on.

NEW TABLE pending_ref_transitions (migration v27):
  - Written by the handler BEFORE smart_http::receive_pack, in state
    'prepared', carrying the verified pusher DID, the raw RFC 9421
    signature header, signature-input, and content-digest that
    authorized the push, the request id, and the parsed ref update.
  - The handler transitions the row to 'applied' on receive_pack Ok
    or 'cancelled' on Err. The drain reads only 'applied'.
  - A failed or cancelled receive-pack therefore leaves the row in
    'prepared' or 'cancelled', which the drain never promotes. This
    is what closes the reviewer's second proof ("a failed or cancelled
    receive-pack does not turn a prepared intent into completed
    accounting or anchoring").

NEW TABLE anchor_jobs (migration v27, owned by PR 1, consumed by PR 2):
  - One row per (repo_id, ref_name, old_sha, new_sha) transition.
    PR 1 inserts it on 'applied'; PR 2 reads it and updates claimed_at.
  - ON CONFLICT (id) DO NOTHING makes the insert idempotent on the
    deterministic id, so a recovery re-pass cannot create a second
    upload request. This is the handoff boundary; the bundler call
    itself is PR 2.

NEW DB METHODS on Db:
  - insert_pending_ref_transitions: writes one 'prepared' row per
    ref update, returns the persisted rows.
  - mark_pending_ref_transitions_applied / _cancelled: state flip,
    gated on the FROM state, idempotent.
  - list_pending_ref_transitions_applied: drain query, oldest first.
  - delete_pending_ref_transition: called by recovery after the
    artifacts land; a third pass is a no-op.
  - record_push_with_id: ON CONFLICT (id) DO NOTHING on the
    deterministic id.
  - insert_ref_certificate_idempotent: ON CONFLICT (repo_id, ref_name)
    DO NOTHING, returns None if a live-path cert already exists.
  - insert_anchor_job_idempotent: ON CONFLICT (id) DO NOTHING on the
    deterministic per-transition id.

NEW HELPERS in db/mod.rs:
  - deterministic_id: SHA-256 hex with an ASCII Unit Separator
    between fields so two distinct tuples never collide on prefix
    overlap.
  - push_event_id_for, ref_cert_id_for, anchor_job_id_for: the
    derived ids above, one helper per artifact so a caller cannot
    derive a wrong id by mistake.

NEW STRUCTS:
  - PendingRefTransition: the row shape.
  - AnchorJob: the handoff row shape.
  - pending_state: const strings ('prepared' / 'applied' /
    'cancelled') shared by tests, the producer, and the drain so a
    typo on one side cannot silently mismatch the other.

NEW TESTS in db::pending_ref_transition_tests (8 tests, all green):
  - insert_then_mark_applied_flips_state_for_every_ref: producer
    contract.
  - mark_applied_is_idempotent_on_repeat: re-fire is a no-op.
  - cancelled_rows_are_not_returned_by_the_drain: reviewer's second
    proof at the DB layer.
  - prepared_rows_are_not_returned_by_the_drain: same proof for the
    pre-flip state (handler crashed before reaching post-Ok).
  - mark_cancelled_is_idempotent_on_repeat: counterpart.
  - drain_then_re_derive_is_idempotent: reviewer's first proof at
    the DB layer. Inserts a row in 'applied' state directly via
    insert_pending_ref_transition_for_test, drains it, derives the
    artifact ids twice, exercises record_push_with_id and
    insert_anchor_job_idempotent directly, asserts exactly one push
    event row and exactly one anchor job row regardless of how many
    times the drain runs.
  - deterministic_id_avoids_prefix_overlap_collisions: the
    separator regression test.
  - push_event_id_for_is_stable: derived ids match across calls and
    differ on each varied input.

OTHER:
  - Make RefUpdate and its fields pub(crate) so the DB methods can
    iterate the parsed ref updates. No public API change.

NOT IN THIS SLICE (the handler refactor, next commit):
  - The receive-pack handler does not yet call insert_pending_ref_
    transitions before the receive_pack call, nor mark_applied /
    mark_cancelled after. The DB layer is in place for it; the
    handler will call these methods and the startup drain will be
    wired in main.rs.
  - The startup drain in main.rs is not yet called; it will iterate
    list_pending_ref_transitions_applied, re-derive the artifacts,
    and delete the row.
  - The cert/push event issuance in cert.rs and the bookkeeping in
    api/repos.rs:2361 are not yet changed to use the deterministic
    ids. The helper functions exist and are tested; the callers
    follow.

Compiles clean, clippy clean under -D warnings, fmt clean.

* fix(node): wire durable outbox into the receive-pack handler (#26 split 1/4)

This is the handler-level half of Split PR 1. The previous commit
added the migration and the DB methods; this one threads them
through crates/gitlawb-node/src/api/repos.rs:2007 (git_receive_pack),
the cert issuer, and the startup drain.

CHANGES IN THE HANDLER
======================

In git_receive_pack, AT THE LAST POSSIBLE MOMENT before the
smart_http::receive_pack call, the handler now:

  1. Generates a per-handler request_id (UUID).
  2. Captures the raw Signature, Signature-Input, and Content-Digest
     headers from the request.
  3. Calls db.insert_pending_ref_transitions(request_id, ...) which
     writes one row per ref update in state 'prepared'.

The receive_pack call runs as before. After it returns:

  4. On Ok: db.mark_pending_ref_transitions_applied(request_id) —
     the row is the ONLY thing that promotes a 'prepared' row to
     'applied', and the drain reads only 'applied' rows. A process
     crash before this call leaves the row in 'prepared', which the
     drain never promotes.
  5. On Err: db.mark_pending_ref_transitions_cancelled(request_id) —
     a failed receive_pack leaves the row in 'cancelled', which the
     drain never promotes.

This is what closes the reviewer's two proofs:

  Proof 1 (crash window): if the process dies after
  mark_pending_ref_transitions_applied but before the bookkeeping
  writes, the row is in 'applied' and the next startup drain
  re-derives the push event, the per-ref certificate (carrying the
  ORIGINAL pusher DID, not a placeholder), and the anchor handoff.
  The drain uses the persisted authentic pusher DID and signature
  header, not a recovered placeholder.

  Proof 2 (failed receive-pack): the row is only ever flipped to
  'applied' in the explicit Ok branch above. A 'prepared' or
  'cancelled' row is invisible to the drain, so a failed or dropped
  receive_pack cannot turn a prepared intent into completed
  accounting or anchoring.

BOOKKEEPING IS NOW DETERMINISTIC-ID
===================================

The post-Ok bookkeeping at api/repos.rs:2448 now uses:

  - record_push_with_id with push_event_id_for(request_id, first_ref)
    — ON CONFLICT (id) DO NOTHING, so a recovery re-pass is a no-op.
  - issue_ref_certificate_idempotent with
    ref_cert_id_for(request_id, ref_name) — ON CONFLICT (repo_id,
    ref_name) DO NOTHING, returns None if a live-path cert already
    exists.
  - insert_anchor_job_idempotent with
    anchor_job_id_for(repo_id, ref_name, old_sha, new_sha) — the
    per-transition tuple key, so two pushes to the same ref produce
    one anchor upload per landed state.

The legacy entry points (record_push, issue_ref_certificate,
insert_ref_certificate) remain for callers that prefer a fresh UUID
per cert; they are #[allow(dead_code)] for the PR 3 cert/CLI compat
pass to decide whether to keep or remove.

STARTUP DRAIN
=============

crates/gitlawb-node/src/main.rs calls
durable_outbox::drain_pending_ref_transitions(state, 1000) ONCE
before serving, after migrations and after the existing peer /
quarantine prunes. Non-fatal: a transient drain failure logs and
leaves the rows for the next startup.

durable_outbox::drain_pending_ref_transitions reads every 'applied'
row, calls derive_one (which re-derives the three artifacts using the
persisted authentic pusher DID and signature header), then deletes
the row. A second drain pass is a no-op for both the artifacts
(idempotent inserts) and the row (gone after the first pass).

NEW END-TO-END TESTS
====================

crates/gitlawb-node/src/durable_outbox.rs adds three end-to-end
tests in drain_tests, complementing the eight DB-layer tests in
db::pending_ref_transition_tests:

  - drain_re_derives_all_three_artifacts_for_an_applied_row: the
    reviewer's first proof. Inserts a row in 'applied' state (the
    crash window), drains, asserts exactly one push event row,
    exactly one cert row carrying the original pusher DID (not a
    placeholder), and exactly one anchor job row. Asserts the
    deterministic cert id matches. Asserts a second drain pass is a
    no-op.
  - cancelled_row_produces_no_artifacts: the reviewer's second proof
    for the cancelled state. A row in 'cancelled' (receive_pack
    returned Err) is invisible to the drain.
  - prepared_row_produces_no_artifacts: the reviewer's second proof
    for the prepared state. A row in 'prepared' (handler crashed
    between insert_prepared and the post-Ok branch) is invisible to
    the drain.

Each test names the invariant it pins and the production line it
covers. Reverting that line turns the named assertion red.

Compiles clean, 1099 tests pass with 0 regressions, clippy clean
under -D warnings, fmt clean.

Cross-PR overlap (declared in the PR description):

  - #134 (anchors auth): composes. The /arweave/anchors route
    already requires auth; this PR does not change the route.
  - #285 (advisory-lock session affinity): composes. The durable
    intent is written inside the same handler that holds the lock
    from #285; no changes to the lock layer.
  - #306 (Content-Digest on signed requests): composes. PR 1
    persists the Content-Digest header that #306 makes mandatory.
  - #314 (small-order Ed25519): independent. PR 1's tests use strong
    keys.
  - #324 (libp2p keypair persistence): independent. PR 1 does not
    touch p2p identity.
  - #325 (gossip ref-update auth): independent. PR 1's signed
    envelope is the HTTP-side equivalent, not the gossip-side.
  - #382 (replication withheld-subtree trees): independent. PR 1
    does not touch replication or pin selection.

* fix(node): address four reviewer findings on #26 split 1/4

- P1-A: add startup reconcile step that promotes `prepared` rows to
  `applied` when the on-disk ref matches the row's `new_sha`. The
  recovery drain (which only reads `applied` rows) can now pick up
  a ref that landed when the live handler's
  `mark_pending_ref_transitions_applied` call errored or was
  interrupted. Strict SHA equality is the load-bearing check — a
  `prepared` row whose target did NOT actually land stays
  `prepared`.
- P1-B: route the live handler's cert issuance through
  `cert::issue_ref_certificate` (the upsert) instead of
  `issue_ref_certificate_idempotent` (DO NOTHING). A re-push to the
  same ref now updates the cert's `old_sha` / `new_sha` /
  `pusher_did` / `issued_at` / `signature` to the new transition
  while preserving the deterministic `cert_id`. The recovery drain
  keeps the idempotent variant; both paths collapse to one row.
- P2-A: refactor the drain into a `drain_pending_ref_transitions_with`
  testable seam that does per-row log-and-continue, and add
  `drain_pending_ref_transitions_all` that loops
  `DRAIN_PER_PASS_LIMIT=1000` rows for `DRAIN_MAX_PASSES=10` passes.
  A failing row no longer stalls the batch; a backlog above 1000
  rows is fully processed across passes.
- P2-B: add a `first_ref_name` column to `pending_ref_transitions`
  via migration v28. The live handler hoists a `first_ref_name`
  local and persists it on every row of the same `request_id`. The
  drain's `derive_one` keys the push event id on
  `row.first_ref_name` instead of `row.ref_name`, so live and
  recovery produce the same id and `ON CONFLICT (id) DO NOTHING`
  collapses a multi-ref push to one push event row (and one trust-
  score bump). Cert and anchor ids stay per-ref / per-transition.

* fix(node): address four reviewer findings on #26 split 1/4 (round 2)

* fix(node): address reviewer round-3 findings on #26 split 1/4

- P1: Delete outbox rows after live durable effects complete so they
  don't replay on every restart
- P1: Parse git report-status for per-ref ok/ng results; mark only
  proven rejections as cancelled, uncertain outcomes as recoverable
- P1: Introduce 'uncertain' state for receive-pack errors where some
  refs may have landed; reconcile checks these against disk at startup
- P2: Promote deletion transitions during reconcile (new_sha == ZERO_SHA
  with absent ref = successful deletion)
- P2: Loop reconcile across multiple passes so backlogs beyond the first
  page are processed in the same startup

Closes review round 3 findings from reviewer-1 and reviewer-2.

* fix(node): fix CI failures from round-3 changes

- Add COMMENT ON TABLE to v29 migration so migration_bodies_are_non_empty passes
- Return error on non-zero receive-pack exit (preserving backward compat
  with tests that expect Err(AppError::Git(_))) while still parsing
  report-status for outbox row handling

* fix(node): update inv22 gate tests for receive_pack_raw refactor

- F3 gate: scan for smart_http::receive_pack_raw( instead of receive_pack(
- U5 gate: scan for all_refs_ok instead of receive_result.is_ok()

* fix(node): require reflog proof before the reconcile promotes a row (#26 split 1/4)

Closes the second half of the reviewer's round-3 [P1]: the startup
reconcile still promoted on `disk_refs.get(ref) == row.new_sha`, which
also matches a coincidental current tip (old=B, new=A while the ref is
already A). That is not an exotic case — it is the ordinary shape of a
REJECTED push, since git refuses an update whose expected old value does
not match the ref. Both the SHA check and the MAX_RECONCILE_AGE window
pass, the push never happened, and the drain then writes a push event, a
signed certificate and an anchor for it.

The live path proves landings from git's report-status body; that body is
gone by the time the reconcile runs, so the proof is re-derived from the
repository. `git::store::ref_reflog_entries` reads `logs/<ref>` and a row
is promoted only when an entry carries its exact `old -> new` pair,
stamped at or after the row was written (REFLOG_CLOCK_SKEW absorbs git's
whole-second truncation against the row's sub-second `created_at`). The
timestamp floor is what separates a landing from a later push that
re-introduced the same pair: proof must postdate the intent it proves.

Deletions stay exempt from the reflog half and keep the absence-plus-age
rule already in place — git removes a ref's reflog along with the ref, so
no other evidence can exist for one.

`init_bare` now sets `core.logAllRefUpdates`, which bare repos default
off; a repo with no reflog yields no proof and its rows stay put for
human-attended recovery rather than becoming accounting the node cannot
substantiate.

Tests: reconcile_refuses_a_coincidental_tip_with_no_reflog_proof,
reconcile_promotes_a_row_the_reflog_proves,
reconcile_leaves_a_row_prepared_when_the_repo_keeps_no_reflog,
reconcile_refuses_a_reflog_entry_older_than_the_row,
reconcile_still_promotes_a_landed_deletion_which_can_have_no_reflog,
init_bare_keeps_reflogs_so_a_landing_can_be_proven,
ref_reflog_entries_is_none_when_the_repo_kept_no_log.

No migrations added or renumbered.

* fix(node): walk the reconcile backlog on a keyset cursor (#26 split 1/4)

The multi-pass reconcile looped but never advanced. Each pass re-issued
the same cursor-less `LIMIT n` query, and unlike the drain — which
deletes every row it finishes, so its next page is always new work — the
reconcile leaves every row it cannot promote exactly where it was. One
unprovable row at the head of the ordering therefore pinned page one on
every pass, and the backlog behind it was never examined at all, which
is the condition the multi-pass loop was added to remove. Rows keep
ageing toward MAX_RECONCILE_AGE while they wait, so "picked up next
restart" can mean "never recovered".

Pages now resume strictly after the `(created_at, id)` of the last row
EXAMINED, via `list_pending_ref_transitions_prepared_or_uncertain_after`.
A keyset cursor rather than an OFFSET because rows leave the set
underneath the walk when they are promoted: an absent row simply does
not appear on a later page, and it can never shift an unvisited row into
a page that was already read. Pass budget, per-pass limit, short-page
exit and the residual-backlog warning are unchanged.

This matters more with the reflog gate in place: a landing in a repo
that keeps no reflog is PERMANENTLY unprovable, so before this it would
jam page one on every startup rather than once.

Tests: reconcile_all_walks_the_backlog_past_the_first_page,
reconcile_all_advances_past_an_unprovable_row (whose blocker is a
no-reflog row, the class the gate introduces).

* fix(node): bound the reflog read to a recent tail (#26 split 1/4)

`ref_reflog_entries` slurped the whole reflog. A reflog grows one line
(~150 bytes) per ref update and how often a ref is updated is
PUSHER-controlled, so that was an attacker-sized allocation, taken once
per stranded row, at startup — the moment the node can least absorb it.
Every other attacker-influenced read in this crate is bounded
(read_body_capped, MAX_SIGNATURE_ENTRIES, the pin/walk budgets); this one
should be too.

The bound costs nothing in proving power, which is why it is a ceiling
rather than a tradeoff: the landing gate only ever accepts an entry
stamped at or after `created_at - REFLOG_CLOCK_SKEW`, and reflogs are
append-ordered oldest-first, so every entry that could possibly qualify
is at the tail. The read now seeks to the last REFLOG_TAIL_BYTES (256KB,
~2000 recent updates to one ref) and `take`s that many bytes, so the
allocation stays bounded even if the file grows between the stat and the
read.

A window into the middle of a file starts mid-record, so the bytes before
its first newline are discarded: without that, a boundary landing inside
a record's old SHA leaves a suffix that still tokenizes and yields a
truncated SHA as a bogus `old -> new` pair — and landing proof is an
exact pair match, so a bogus pair is a fabricated proof. An entry older
than the window reads as absent, which is the safe direction: the caller
treats "no matching entry" as unproven and leaves the row where it is.
`Ok(None)` for a missing file, and the absence-of-evidence contract, are
unchanged.

Tests: ref_reflog_entries_reads_whole_records_from_the_recent_tail (lays
the file out so the boundary lands 20 bytes into a record's old SHA — the
alignment that would otherwise parse), an_entry_older_than_the_reflog_tail_reads_as_unproven.

* fix(node): read the report through both side-band frames

The report-status is framed twice whenever the client negotiates
side-band-64k, which `git push` over smart HTTP does: the outer
side-band envelope carries a stream that is itself pkt-line encoded.
One strip pass left the first line as `000eunpack ok`, so the unpack
check failed and parse_report_status returned None — read by the caller
as "no report", which keeps every declared ref. The per-ref landing
gate was therefore inert for exactly the pushes it exists to filter.

The regression test is a byte-for-byte capture from git 2.50.1
rejecting one ref of a two-ref push, because a hand-written
single-framed fixture parses under the old code and hides the case.

* fix(node): address round-4 reviewer findings on #26 split 1/4

* fix(node): rustfmt round-4 reviewer fixes

* fix(node): remove empty line after doc comment

* fix(node): drop redundant reflog gate; implicit-ok on exit-zero no-report

* fix(node): address round-5 reviewer findings on #26 split 1/4

* fix(db): v30 migration — add receive_pack_requests table and ordinal column

The reviewer's round-5 finding: the push event identity was encoded
into a mutable per-ref column (first_ref_name) whose correct value is
knowable only after git, and the correction was not committed
atomically with the per-ref outcomes. A crash between git updating
a later ref and the rewrite left the durable rows naming the
rejected ref, and derive_one's row.ref_name == row.first_ref_name
guard then meant no push event ever landed for the accepted child.

Per the state-transition model at
.gravirei/plans/state-model-durable-post-receive.md, this migration
introduces the request-level record that owns the push event and the
trust score, and extends the per-ref child with an ordinal column
the drain and the effect executor walk together. first_ref_name is
dropped; the push event id is keyed on
(request_id, accepted_ordinal) and lives on the request row.

The next commit (handler rewrite) wires the live path and the
recovery drain against the new tables.

BREAKING CHANGE: pending_ref_transitions.first_ref_name is dropped.
Callers that read or write that column must move to
receive_pack_requests.accepted_ordinal.

* fix(node): rewrite receive-pack handler against the request-level model (#26 split 1/4 step 2)

The v30 migration added receive_pack_requests and dropped first_ref_name,
but no Rust code used either. The handler still ran a four-branch per-ref
state flip plus a first_ref_name rewrite after git returned, which is the
mixed-outcome bug a push with the first ref rejected exposes: the live
path and the drain computed different push-event ids.

- Insert a receive_pack_requests row in state `received` BEFORE git runs
  (insert_receive_pack_request), carrying the request bytes and the
  SHA-256 of the body. Crash between intent and git return is now
  recoverable via the reconcile step.
- Drop the first_ref_name rewrite at api/repos.rs:2454. The push event
  identity is now (request_id, accepted_ordinal); accepted_ordinal is
  computed once at the per-ref state flip.
- Re-key push_event_id_for and ref_cert_id_for on (request_id, ordinal).
  Anchor job id stays per-transition.
- Transition the request row to outcomes_committed (with parsed_report
  and accepted_ordinal) or rejected_at_git (no report + non-zero exit)
  in the same handler tail as the per-ref state flip. The drain (step 3)
  picks up outcomes_committed rows; today the live path also runs the
  per-ref effects inline.
- Stub the step-3 surface (mark_request_effects_pending, complete,
  list_receive_pack_requests_due, update_request_attempt) with
  #[allow(dead_code)] + contract-pin comments so the rewrite is
  bisectable: reverting this commit restores the pre-rewrite live path.

The U5 gate (replication tail spawn inside push_succeeded, before
guard.release) is preserved; inv22_replication_tail_spawns_at_the_
durability_boundary stays green.

* fix(node): use raw bytes for request_bytes_hash (#26 split 1/4 step 2)

The v30 migration defined request_bytes_hash as BYTEA but the handler
was binding a hex-encoded String. Postgres rejected the insert with
"column request_bytes_hash is of type bytea but expression is of type
text" and the handler shed the push with a 503.

Switch the handler to bind the raw 32-byte digest and update the
ReceivePackRequest struct field type to Vec<u8> to match. The
drain-side test fixture already used Vec<u8> and needs no change.

Pinned by the bin test suite going from 24 failures (all the
receive-pack-cap tests that send a 4-byte body) to 0 regressions
on the new model.

* fix(node): factor out apply_request_effects; drain walks receive_pack_requests (#26 split 1/4 step 3)

The step-2 commit moved the request row to be the unit of work but
left the per-ref effects fan-out inline in the handler and the
drain walking pending_ref_transitions per-ref. Step 3 factors the
effects fan-out into apply_request_effects(state, request_id) and
rewrites the drain to walk receive_pack_requests.

- apply_request_effects lives in durable_outbox.rs; the live handler
  in api/repos.rs and the drain both call it. Idempotent on
  (request_id, accepted_ordinal). Returns EffectsOutcome::{Done,
  Nothing, Retry}.
- Drain switches from list_pending_ref_transitions_applied +
  derive_one to list_receive_pack_requests_due +
  apply_request_effects. derive_one and the per-ref drain entry
  points are deleted.
- The step-3 stubs in db/mod.rs (mark_request_effects_pending,
  mark_request_complete, list_receive_pack_requests_due) lose
  their #[allow(dead_code)] annotations; every stub has a caller
  in this PR.
- Reconcile (reconcile_prepared_from_disk_all) is unchanged.
  The crash window between intent durable and outcomes commit
  still routes through per-ref reflog proof.
- Drain tests rewritten to stage receive_pack_requests rows; the
  per-ref fixtures (make_row, lookup_accepted_ordinal) are gone.
- New inv26_step3_live_and_drain_share_apply_request_effects
  assertion pins the live/drain sharing and the per-request
  drain walk.

* fix(node): bounded retirement purge for terminal request rows (#26 split 1/4 step 4)

The v30 partial index idx_receive_pack_requests_completed_at
exists but no code reads it. Step 4 wires a periodic purge task
that deletes terminal `complete` and `rejected_at_git` rows
older than the retention window, along with their per-ref
children. `quarantined` (a step-5 state) is never purged by the
timer.

- `purge_completed_receive_pack_requests(older_than, limit)` —
  deletes parent requests using the v30 partial index.
- `purge_completed_pending_ref_transitions(older_than, limit)` —
  deletes children of purged parents, gated on the child's own
  applied_at / cancelled_at.
- `purge_request_queue(db, retention_days, limit)` — orchestrator
  in durable_outbox.rs that calls both and returns the totals.
  The deletion order (parents first, then children) is the
  contract; a crash mid-purge leaves orphaned children that the
  next pass will pick up.
- `spawn_queue_lifecycle_sweep` in main.rs — periodic task on
  the same detached pattern as `spawn_legacy_cid_sweep`, 24-hour
  interval, shutdown-aware. The interval matches the spec's "one
  per cluster per day" target.
- Config knobs `queue_retention_days` (default 7, range 1..=365)
  and `queue_purge_batch` (default 1000, matches
  DRAIN_PER_PASS_LIMIT).
- 3 new `durable_outbox::drain_tests::purge_*` tests pin the
  contract: only old terminal rows are deleted, the second pass
  is a no-op, and a row inside the retention window survives.
- New `inv26_step4_queue_lifecycle_purge_is_wired` gate asserts
  the wiring (main.rs calls the purge, the DB helpers exist, the
  Config knobs exist).

* fix(node): reference-transaction marker + quarantined state + failure matrix (#26 split 1/4 step 5)

Steps 2-4 gave the request row the unit of work, the shared
executor, and the bounded retirement policy. Step 5 closes the
evidence gap: the reconcile now requires a per-request marker
ref (refs/gitlawb/requests/<id>) whose value matches
request_bytes_hash. A missing or mismatched marker quarantines
the request; an operator reclassifies it.

- v31 migration: adds `quarantined` to the state vocabulary and a
  partial index for operator queries.
- Handler writes the marker ref before git-receive-pack; the
  marker is causally bound by being in the same async task as
  the receive-pack call. The marker's value is content-addressed
  (git hash-object of the request bytes), so the gate compares
  consistent SHAs on both sides.
- git::store::read_ref reads a single ref's value, returning
  Ok(None) for absent refs. Used by the marker gate.
- git::store::marker_value_for computes the content-addressed
  marker value; both the live handler and the reconcile use it
  so the write and the read agree.
- Reconcile gains a marker gate between the age check and the
  reflog proof. Mismatch or absent ⇒ mark_request_quarantined +
  mark_children_rejected_for_quarantined_parent.
- effects_max_attempts bound (config knob, default 8) flips
  retry-stuck requests to `quarantined` after N attempts,
  closing the infinite-retry DoS window.
- New failure_matrix_tests submodule covers the spec's outcome ×
  ref-kind × exit-point × recovery-scenario matrix (6 cells).
- New inv26_step5_marker_quarantine_and_bound_are_wired gate
  asserts the marker gate, the bound check, the handler's
  pre-receive-pack ordering, and every load-bearing helper.
- Existing 7 reconcile tests updated to stage a marker ref via
  the new `stage_marker` test helper.

* fix(node): converge durable post-receive to singular request-level lifecycle

Unify producer, evidence, aggregate outcome, executor, retry, and
retirement on the request row: atomic intent and outcome commits,
synthetic normalized reports for implicit-ok/reconciled paths,
reconcile parent promotion with fail-closed deletions and
competing-claimant guard, retry progression from both executable
states with backoff/quarantine, terminal-parent-gated purge with
marker cleanup and hidden refs, minimal digest-only intent plus
request-level RFC9421 proof (v32).

* fix(node): harden request occurrence lifecycle, proof, retry, and retirement

Reject incomplete report framing and omitted refs as indeterminate,
preserve uncertain children on completion, terminalize all-rejected,
bind reconcile with landing history, durable proof with ack gate,
occurrence-keyed anchors, background due loop, deletion quarantine
with operator resolve, children-before-parent purge with marker
tombstones, recovery prereq upgrade check, and bounded marker runner.

* fix(node): keep recovery prereq upgrade best-effort on push path

Refusing pushes when git config upgrade fails broke fake-git and
non-repo disk-path tests with 503. Reconcile already fails closed
on missing reflogs, so downgrade to warn-and-proceed.

* fix(node): close reviewer round on outcome authority and lifecycle ownership

Gate effects on unpack_ok with cleared ordinal and terminal no-effects,
intersect executor with applied children, treat absent report-status as
indeterminate regardless of exit, require landing history before child
delete, claim due requests with recoverable leases, dedupe webhooks by
occurrence ledger, ack-gate proof retention with terminal disposition,
fair marker tombstones with dead-letter, kill_on_drop marker bounds,
and loud migration version collision guard.

* fix(node): load-bearing coverage for intent, guards, and delivery ledger

Handler intent test plus inv22 gate so disabling the durable insert
turns red; unpack-false, mixed ok/ng, divergent cancelled, partial
sibling, and proof-ack tests pin each executor guard; webhook
deliveries record sent_at with stale-pending reclaim; stale
parsed_report comment corrected.

* fix(node): centralize live retry, bound outcome commit, close retention orphans

Route live Retry/Err through schedule_request_retry_or_quarantine so
attempt progression matches the drain; retry post-git outcome commits
3x before restart repair; block terminal purge while non-terminal
children exist; add v27 upgrade-path test.

* fix(node): document attended-restart commit policy and pin executor retry arms

Track outcome-commit success, skip metrics/touch on failure, correct
the lease-expiry comment to startup reconcile; add restart-repair test
for stuck received parents plus RED tests for landing-history and
proof-ack failure paths.

* fix(node): correct marker prereq comment to warn-and-proceed

* fix(node): single partial-completion model with disposition gate

Split apply_request_effects into phase-1 bundle (push, cert, anchor,
history, webhook, proof ack before child deletion) and phase-2 gate
that retries while non-cancelled siblings remain; Nothing only when
nothing owed. Gate tail, Tigris, and effects on PostGitDisposition
with documented replication carve-out; add partial-sibling webhook
and disposition tests; fix stale comments.

* fix(node): close review round on outbox ordering, marker visibility, and repair reachability

Address the P1/P2 review findings: await the webhook delivery claim
before accepted-child deletion; exempt the marker namespace from the
fail-closed ref peeling guard so visibility walks tolerate blob-pointing
request markers; keep request marker deletion bounded in the purge path;
promote cross-page siblings via idempotent report superset; run
stuck-parent repair when the prepared queue is empty; bound the durable
intent write while admission permits are held; remove dead proof
verification scaffolding.

* fix(node): close review round on pre-git refusal lifecycle, namespace gate, and promotion merge

* fix(node): close review round on namespace gate, intent lifecycle, and promotion atomicity

Namespace gate hardening:
- Refuse non-UTF-8 ref-update lines (git accepts bytes ≥0x80; gate never sees them)
- Catch bare 'refs/gitlawb' name (starts_with missed it)
- Add deny test for both holes plus refusal wiring verification

Intent lifecycle guards:
- Add IntentDropGuard: terminalize stranded intents on dropped futures
- Bound refuse_receive_pack_before_git to 10s (no statement_timeout on pool)
- Guard mirror fetch: hideRefs blocks non-exempt refs/gitlawb/* from +refs/*:refs/*

Promotion merge atomicity:
- Take row under SELECT FOR UPDATE so merge is atomic with state read
- Prevents racing due worker from stranding newly applied children outside report

* fix(test): correct inv22 namespace gate assertions to match actual implementation

The test was looking for the wrong patterns:
- UTF-8 check: error message string, not pre-loop condition
- Namespace check: `*r == "refs/gitlawb"`, not `ref_name ==`
- Visibility exemption: `&&` logic (both false to exclude), not `||`

* chore: cargo fmt

* fix(node): phase-aware intent guard, mirror import prune, terminal-state merge

* fix(node): guard cleanup retry with fates, fail-closed mirror prune, refusal coverage

* fix(node): bound guard attempts, sibling-wait outcome, fail-closed prune enumeration

* fix(node): sibling-wait outcome, no-report path, bounded prune and guard retry

* fix(node): no-report durability, cert landing order, bounded reconcile git, flush checks

* fix(node): gate trust-score bump on a new push event row

* chore: update rustls 0.23.37 → 0.23.45 to resolve RUSTSEC-2026-0285

* fix(node): surface git's report-status when a stdin EPIPE follows child exit

* fix(node): cap receive-pack ref-update count before per-ref work and intent

* fix(node): cap ref-update parsing in-loop and dedupe identical commands

* docs(node): correct round-4 comment claims on client outcome and gate cost

* fix(node): single replication tail, proven marker taxonomy, claimed webhook delivery (#558)

- One post_receive_replication_tail per push: the carve-out spawn and the
  durability-bound spawn are mutually exclusive positions on the same
  proven-landed ref set; inv22 now fails on any unfiltered or ungated spawn.
- Marker reconcile: bounded-read errors (git Err/deadline) stay prepared
  and retry instead of quarantining as 'missing marker ref'; only a
  successfully read marker proves missing or mismatch. Err-injection test
  pins no-quarantine on unproven reads.
- Claimed webhook delivery: the synchronous claim phase returns the
  claimed Webhook rows so the send task never re-lists; a list failure
  after claim can no longer drop claimed deliveries. Source gate pins it.

* docs(node): name claimed-hooks variant in fire_event_occurrence doc

* fix(node): order pending_ref_transitions create before outbox alters, fill v29 (#558)

* fix(node): proven-missing marker taxonomy, v38 upgrade test, report-ok tail fixtures (#558)

* fix(node): read marker refs via rev-parse so present refs return their oid (#558)

---------

Co-authored-by: Gravirei <[email protected]>
Co-authored-by: Kevin Codex <[email protected]>
2026-11-09 23:40:38 +00:00
beardthelionandcarlitos973 3c1ee27b60 fix(node): make creation-route rate limit configurable; fix(gl): surface real errors on pr create failure (#560)
- gitlawb-node: rate limiter was hardcoded to 10 req/hour per DID across all
  creation routes (repo/issue/PR create, register, fork). Adds
  GITLAWB_RATE_LIMIT_MAX and GITLAWB_RATE_LIMIT_WINDOW_SECS config fields
  (clap env-var pattern matching the rest of Config), defaults unchanged.
  Lets node operators tune this for real agent/bot usage without recompiling.

- gl pr create: previously called resp.json() before checking response
  status, so any non-2xx response with an empty or non-JSON body crashed
  with a generic "invalid JSON" error instead of surfacing the real cause.
  Now reads the raw response body first and includes it in the error message
  on failure.

Co-authored-by: carlitos973 <[email protected]>
2026-11-09 23:02:05 +00:00
beardthelionandKRATOS 209618a2fe feat(node): add scheduled inbound mirror worker (#555)
* feat(node): persist upstream mirror transition state

Origin-Session: local-d6a143 | Codex | 12 prompts

* fix(node): reject null mirror transition state

Origin-Session: local-d6a143 | Codex | 14 prompts

* fix(node): address mirror state review findings

Origin-Session: local-d6a143 | Codex | 16 prompts

* fix(node): close mirror review blockers

Origin-Session: local-d6a143 | Codex | 20 prompts

* fix(node): align peer URL validation message

Origin-Session: local-d6a143 | Codex | 20 prompts

* fix(node): close remaining mirror review findings

Origin-Session: local-d6a143 | Codex | 21 prompts

* feat(node): add scheduled inbound mirror worker

Origin-Session: local-d6a143 | Codex | 18 prompts

* fix(node): harden scheduled mirror fetches

Origin-Session: local-d6a143 | Codex | 24 prompts

* fix(node): close inbound mirror review blockers

Origin-Session: local-d6a143 | Codex | 24 prompts

* fix(node): reject special-use upstream literals at configure time (#555)

validate_mirror_upstream_url used only the peer public-host classifier, so
an IP literal the fetch worker's stricter permit denies (192.0.2/24,
198.18/15, 2001:db8::/32, ...) could be stored and then hard-fail on every
scheduled scan. The IANA special-use permit moves to peers::
mirror_upstream_ip_permitted, shared by configure validation and the
resolve-path check; the negative test set now covers the denied literals.

* test(node): scrub inherited git env in mirror fixture spawns (#555)

The repository_local_* fixture tests ran plain Command::new("git") and
inherited any GIT_DIR/GIT_CONFIG_* state from an outer checkout, so a
fixture spawn could act on the wrong repository. A shared fixture_git
helper now sets the cwd and removes git environment vars; all five spawn
sites use it.

* fix(node): bound mirror preflight probes separately, align operator docs (#555)

* chore(node): retrigger checks after code-scanning alert triage (#555)

* chore(node): retrigger checks after code-scanning alert triage (#555)

---------

Co-authored-by: KRATOS <[email protected]>
2026-11-09 22:48:47 +00:00
beardthelion a00d7d5c48 fix(gl): pin secret-file modes at creation instead of chmod after (#354) (#557)
* fix(gl): pin secret-file modes at creation instead of chmod after (#354)

identity.pem and ucan.json were written with fs::write and then chmod'd,
leaving the private key world-readable between the two syscalls, and
ucan.json was never chmod'd at all. A shared helper now opens with
mode(0o600) so no permissive window exists, adds O_NOFOLLOW so a
pre-planted symlink is refused rather than written through, and re-pins
the mode after write so a pre-existing loose file is tightened. The
containing directories are created 0700 via DirBuilder and re-pinned the
same way. gitlawb-node's own key write gets the same treatment.

* fix(gl): tighten secret handling before, not after, replacing contents (#459)

Address PR review feedback:

- secret_file::write and the node key write now open without truncate,
  pin mode 0600 on the descriptor, then set_len(0) and write, so a
  pre-existing permissive file never holds new secret contents while
  still group/world-readable.
- Directory re-pins skip symlinked paths instead of chmodding through
  them to the link target.
- The node's existing-key read opens with O_NOFOLLOW and reads from the
  descriptor instead of read_to_string on the path.
- An existing permissive key parent dir is re-pinned to 0700.

* fix(gl): adopt writable --dir leaves the user cannot chmod (#557)

create_dir re-pinned every existing final directory to 0700. On a leaf the
caller can write but does not own (group 0775, ACL grant, a system dir like
/tmp), fchmod fails EPERM and the whole command aborted, a regression over
create_dir_all's no-op. The re-pin now applies only when the euid owns the
opened directory; residual EPERM on an owned leaf adopts rather than aborts.
Regression test proves /tmp is adopted with its mode unchanged.
2026-11-09 20:13:51 +00:00
4cd0a6a306 fix(node)!: gate agent-task reads behind visibility rules (#553)
* fix(node): gate agent-task reads behind the same visibility rules as repo data

list_tasks and get_task had no authorization at all: any anonymous caller
could enumerate every task on the node, including another party's
repo-less task, its ucan_token, and its payload (#268). Add task_visible,
mirroring the repo read-visibility gate already used by the ref-updates
feed: the delegator and assignee can always read their own task, a
repo-scoped task follows that repo's normal visibility rules, and a task
naming no repo (or a repo this node doesn't host) is visible only to its
delegator/assignee. Both REST and GraphQL now route through the same
collect_visible_tasks/get_visible_task collectors so the two surfaces
cannot drift, and neither read path echoes ucan_token back, since the
holder already received it via the create/claim response.

Fixes #268

* fix(node): query the task limit-clamp test as the delegator

tasks_limit_ceiling_clamped_to_200 seeded 201 repo-less tasks and read
them back anonymously, expecting all 200. That read is exactly the
enumeration #268 closes, so the new visibility gate correctly returns
none of them and the test went red. The clamp ceiling is what this test
pins, not the gate, so query as the tasks' delegator, who can legitimately
see all 201 rows.

Refs #268

* fix(node): scope task visibility lookups to the page's repos

collect_visible_tasks loaded every repo on the node and every visibility
rule in order to gate at most 200 tasks, so an anonymous request paid for
the whole node's repo and rule set. Narrow both lookups to the repo ids the
fetched page actually names, and skip them when no task names a repo.

The deduped repo snapshot stays the source of truth for resolving a
repo_id: it collapses mirror and canonical pairs and omits quarantined
repos, and an id missing from it has to keep failing closed. Resolving ids
straight from the repos table would surface exactly those withheld rows.

Add GraphQL denial tests as well. Nothing pinned that the task resolvers
delegate to the shared collectors, so a resolver that queried the database
directly would not have gone red.

Refs #268

* Preserve authorized task reads across shipped clients.

Refs #268

* Bound denied task history scans.

Refs #268

* Signal incomplete task scans with recoverable cursors and map read errors to AppError.

Refs #268

* Stop disclosing denied task rows in the scan-wall cursor and add GraphQL pagination state.

Refs #268

* Normalize task cursors and gate mutation endpoints behind visibility.

Canonicalize RFC 3339 timestamps in parse_after_cursor to handle URL-decoded spaces, reject mixed cursor alias families, gate complete_task and fail_task behind get_visible_task so unreadable tasks 404 instead of leaking existence with 403, and only flag incomplete when hitting candidate ceilings on full SQL batches.

Refs #268

* Update complete_task test in test_support to exercise both 404 on unreadable and 403 on non-assignee tasks.

Refs #268

* Bind task to repo id in complete_task test in test_support.

Refs #268

* Preserve timestamp fractional width and anchor continuation tests on legitimate cursors.

Refs #268

* Match on Value::Object to extract task ID in GraphQL continuation test.

Refs #268

* Remove unused mut binding on visible_newer in GraphQL query test.

Refs #268

* Hide unauthorized claim and keep assigned tasks from being stolen

Gate REST and GraphQL claim behind the same visibility check as complete and fail, refuse claim when another assignee already holds the task, and only broadcast publicly visible task events. Treat a full list page as incomplete when more candidates remain. Surface HTTP errors from CLI and MCP claim and complete helpers.

Refs #327

Co-Authored-By: cairn-code <[email protected]>

* Keep task-list incomplete for the scan ceiling only.

A full visible page was flagged incomplete whenever the SQL batch was full, so the first page of any list with more than 200 candidates looked stalled. Align the GraphQL claim test with the visibility gate's not-found message.

Refs #268

* fix(node): pin claim guards and map mutation errors through AppError

Review required tests that go red if the pre-assigned claim predicate or
the anonymous announce gate is deleted, and incomplete must not stay
true when the candidate stream is exhausted at the scan ceiling. Route
claim, complete, and fail through AppError so closed-pool outages stay
503 and 404s match the read envelope.

Refs #268

* fix(node): match bare and did:key assignee forms in claim and list

create_task stores the supplied assignee unchanged, so a raw SQL
equality check drops a designated assignee who presents the other
did:key form. Compare the normalized key so claim and filtered list
agree with did_matches.

Refs #268

* fix(node): check task creation status and add assignee expression index

- Add error_for_status() to cmd_create and task_create MCP tool
- Update test_create_task_server_error to assert failure on 500
- Add migration v18 creating expression index idx_agent_tasks_assignee_key matching ASSIGNEE_DID_CASE_SQL
- Add did:web:z6Mkfoo single-residual shape to parity boundary matrix

Refs #327

# Conflicts:
#	crates/gitlawb-node/src/db/mod.rs

* fix(node)!: Replace task cursors with opaque continuation tokens

The task read path treated visibility, pagination, and error vocabulary as
separate edits, so each one broke where they met. Rework them as one contract.

A raw (created_at, id) cursor forced a choice between two broken options: it
could name the last visible row, and then a denied window longer than the
1,000-candidate scan budget was unpageable forever; or it could name the last
examined row, and then a denied read leaked the id and timestamp of a task
GET /tasks/{id} otherwise 404s. Continuation tokens remove the choice. They
carry the last examined candidate, so paging always advances a full scan budget
per request, and they are encrypted and authenticated under a node-derived key,
so the caller learns nothing from one and cannot forge one naming a row of
their choosing. Encryption is a synthetic-IV construction over the hmac/sha2
pair already used for webhook signatures, so it adds no dependency and needs no
randomness source.

Making the token the only accepted cursor also gives the ordering key one
domain. agent_tasks.created_at is TEXT and compared as TEXT, so a caller-typed
'...Z' and '...+00:00' denote one instant but sort differently, and a client
could silently skip or repeat same-time rows. The token carries the stored
string verbatim, so the value compared is always one the server wrote. The raw
after_*/cursor_* pairs are removed rather than kept alongside it, since a second
domain is the bug.

Separate the two facts the old single incomplete flag conflated: has_more says
candidates remain, incomplete says this page is short only because the
authorization scan hit its ceiling. Both REST and GraphQL now return has_more,
incomplete, and next_cursor from the shared collector, and REST echoes the limit
it actually applied so a clamped request is visible as clamped.

Have gl task list and MCP task_list follow next_cursor instead of issuing one
request: --limit 500 returned a successful but silently truncated 200 rows.
Following is bounded by a page cap and a no-progress guard, and a run stopped by
either reports an explicit incomplete result with a resume cursor.

Route claimTask, completeTask, and failTask through the same task_write_conflict
classifier the REST handlers use, via curated helpers in the graphql module so
the map_err source guard still holds. A claim race or stale finish reached
GraphQL clients as a generic database error while REST clients got an actionable
conflict; genuine sqlx faults stay opaque on both.

Refs #327

* fix(node): End the candidate stream only after the batch is examined

A short SQL batch means no rows exist past it, not that every row in it was
examined. When the page filled mid-batch the collector treated the two as the
same, marked the stream ended, and suppressed the continuation, so every row
after the one that filled the page was unreachable. The equal-timestamp paging
tests caught it: three rows with a limit of one returned only the first.

Track how much of each batch was consumed and end the stream only when the
whole of a short batch has been examined. Otherwise leave `has_more` to the
probe row, which resumes from the last examined candidate.

Refs #327

* fix(gl): Reject a non-positive task-list limit and fault the write under test

A `--limit 0` reached the node, which clamped it to zero and answered with
an empty page marked complete, so an invalid request read as proof that no
tasks exist. Reject a non-positive limit in `fetch_tasks()`, the helper the
CLI and MCP share, so the guard cannot drift between the two surfaces.

`task_write_sql_faults_stay_opaque` did not exercise what it named. Dropping
`updated_at` also broke the SELECT in `get_task()`, so the fault surfaced
from the `get_visible_task()` pre-check through `graphql_app_err` and never
reached `graphql_claim_conflict`. A `BEFORE UPDATE` trigger keeps every read
valid and faults only inside `Db::claim_task`, and the test now also asserts
that a write-time fault is not reclassified as a claim race.

Refs #268

* fix(node): Bind task cursors to the caller and brake the task read routes

A continuation token names the last candidate a scan examined, not the last
row it returned, so it encodes how far that scan got under one caller's
visibility. The MAC bound the page filter but not the presenting identity,
so resuming a token as a different caller started the scan past rows that
caller was entitled to read and dropped them from the answer with nothing
to signal the loss. Bind the caller's normalized DID into the MAC, with
anonymous flagged absent rather than encoded as empty. Normalization goes
through normalize_owner_key so the two spellings of one did:key identity
bind identically, matching did_matches on the read path: a caller who
presents the other form of their own DID keeps their own page. A mismatched
token renders the existing single rejection message, so this adds no oracle.

GET /api/v1/tasks and GET /api/v1/tasks/{id} are anonymously reachable, and
the visibility gate costs a task lookup plus deduped-repo and
visibility-rule queries before it can return the opaque 404. An
unauthenticated prober therefore pays nothing while the node pays per
request, whether or not the id exists. Attach the per-IP limiter already
used on /ipfs/{cid}, configurable through GITLAWB_TASK_READ_RATE_LIMIT and
swept by the periodic task like every other per-key limiter.

Refs #268

* fix(node): Brake anonymous GraphQL task reads on the same per-IP bucket

The per-IP brake added for the task read routes covered only
/api/v1/tasks*, so an anonymous caller reached the same
collect_visible_tasks and get_visible_task gate over /graphql with no
bucket at all. The fence had an open lane beside it.

Carry the brake as GraphQL request data and debit it in the tasks and
task resolvers rather than layering rate_limit_by_ip onto the GraphQL
router: /graphql is one endpoint for every operation, so a router layer
would charge unrelated queries and every mutation against the task-read
bucket. Debiting per resolved field also prices an aliased query
honestly, since ten aliased tasks fields run the gate ten times.

Extract RATE_LIMIT_MESSAGE so the GraphQL surface, which cannot return a
429 status inside a 200 envelope, refuses with the same text the REST
routes use.

/graphql/ws serves the query root as well and stays unbraked; closing it
needs a WebSocketUpgrade handler and is left for a follow-up.

Refs #268
Refs #327

* fix(gl,node): parse typed task pages, track cursor cycles, and normalize assignee filter MAC

* fix(node): Cap aliased GraphQL task reads per request and derive has_more from visible rows

- Cap aliased GraphQL task read fields per request using an atomic counter
  on TaskReadBrake (MAX_GRAPHQL_TASK_READS_PER_REQUEST = 5).
- Derive has_more in collect_visible_tasks by scanning for bounded_limit + 1
  visible rows, eliminating the un-gated keyset probe that could leak the
  presence of trailing denied tasks.
- Add regression tests covering aliased GraphQL capping and trailing denied
  task has_more privacy.

Refs #327

* fix(node): Check database continuation when scan budget ends on exact batch boundary

When candidate scanning reaches MAX_TASK_SCAN_CANDIDATES without finding
a target_visible row and the final batch was full, probe the database for
rows beyond the scan position so an exhausted candidate stream is not
erroneously marked incomplete.

Refs #327

* fix(node): Document and pin the bound on the task scan-ceiling probe

The scan-ceiling branch of collect_visible_tasks settles has_more with an
un-gated LIMIT 1 probe, so a caller can learn whether any row - readable
or not - trails the position the scan stopped at. Withholding the probe
does not remove that bit: enumeration past a denied window longer than
one scan budget requires handing back a continuation, and following that
continuation returns the same terminal page one round trip later.

State what the probe discloses (one bit, only at server-chosen positions
a full scan budget apart, reachable only through a MAC'd cursor, never a
denied row's id, payload or ucan_token) and pin it end to end. Also
correct the comment above the branch, which claimed has_more never comes
from an un-gated probe while the code below it did exactly that.

Refs #327

* Index task keyset pages so the scan ceiling bounds database work.

Optional IS NULL predicates kept the planner from using a created_at/id order, so every list_tasks_keyset batch could sort a growing match set before LIMIT. Dedicated per-domain SQL plus v28 indexes make the candidate ceiling a database bound.

Refs #327

* Reject oversized task pages so list never returns more than the requested limit.

fetch_tasks asked for the remaining total, then appended every row on a valid-shaped page. A remote that sent more tasks than want could make gl and MCP expose more than --limit. Treat that page as protocol-invalid before any extra row is kept.

Refs #327

* Enforce task page limits on legacy responses and drop redundant single-column indexes.

Refs #327

* Address review feedback on task-read authorization gating and client safety.

- Wire TaskReadBrake into /graphql/ws subscriptions and verify per-request field caps and per-IP rate limiting over WebSocket connections.
- Define open-claim eligibility separately from read visibility so unassigned tasks on readable/unscoped domains can be claimed without making task bodies enumerable.
- Propagate identity errors on explicit key directories in CLI and MCP instead of silently falling back to anonymous mode.
- Enforce response byte limits before deserializing task pages, validate row schema and page-local uniqueness before row commit, and sanitize continuation cursors in terminal diagnostics.
- Add GraphQL denial assertions for repo-less tasks and token isolation.

Refs #268
Refs #327

* Address review feedback on task-read authorization gating, WebSocket budgets, and test framing.

Refs #395

* Address review feedback on quarantined task claims, WebSocket auth, and MCP limit parsing.

Refs #395

* Reject forged signatures during WebSocket authentication upgrade.

Refs #395

* Accumulate forged WebSocket handshake response until headers terminate.

Refs #395

* Withhold quarantined-repo tasks on all read surfaces before party checks.

Refs #395

* docs: Align task-read documentation with enforced visibility and rate limits.

Refs #395

* docs: Distinguish task list and get behavior in rate-limit guidance.

Refs #395

* fix(node): Batch task quarantine checks to bound read costs.

Refs #395

* docs(node): clarify the shared task-read budget

Document that GraphQL task queries over HTTP and WebSocket debit the same per-IP hourly budget as REST task reads.

Refs #395

* fix(node,gl): bound scoped repo lookup and validate MCP cursor

Bound repository access in list_repos_deduped_by_ids to requested groups via idx_repos_owner_key_name, leaving dedup_cte unscoped. Strictly validate MCP task_list cursor argument before network dispatch.

Refs #396

* test(db): verify scoped lookup work in prepared plans

Measure discarded rows and verify actual custom and generic prepared-plan counters, including the previous full-index-scan regression.

Fixup for a0ea4c78e4.
Refs #396

* docs(db): remove stray dedup comment escapes

Restore readable root-path notation in the deduplicated listing documentation.

Fixup for a0ea4c78e4.
Refs #396

* fix(node): address #464 review findings

- Hold migration 27/28 numbering: reservation comment now names the
  competing Gitlawb/node#384 range (27-35) and states the numbering is
  on hold pending #384. No renumber.
- Restore crates/gl/src/identity.rs to LF endings so the diff shows
  only the added load_optional_keypair function.
- Drop the dead single-column idx_agent_tasks_assignee_key build from
  v27 (v28 builds the keyset-ordered variants directly); adjust the
  v27 test to assert the raw index is dropped and the expression
  index is never built.
- Align task_claimable with claim_task's conditional write: drop the
  delegator short-circuit and correct the docstring.
- Pin error_for_status on the gl task list path with a 403 mock that
  serves a well-formed empty page, so the test goes red if the call
  is removed.

* test(node): pin the delegator deny on a pre-assigned claim

- Add claim_task_delegator_cannot_claim_preassigned_task: pins opaque 404
  for delegator claiming a pre-assigned task (regression guard for P2)
- Fix test_list_tasks_server_error_surfaces regex: /api/v1/tasks\? ->
  /api/v1/tasks\? (escaped ? for literal question mark)
- Assert actual reqwest StatusCode::FORBIDDEN instead of matching "403"
  in error string (avoids spurious pass from random mock port)

* test(node): exercise streamed byte-budget guard with chunked body

Use with_chunked_body instead of with_body so mockito sends no
Content-Length, pinning the no-length streamed accumulation guard.
Assert the "exceeded" message to distinguish from the declared-length
check.

* fix(node): pin declared-length byte-budget arm in oversized content-length test

Addresses beardthelion review finding (2026-09-26): test_read_task_page_json_oversized_content_length now asserts on the "(declared" arm the way the chunked twin asserts "(exceeded", so the Content-Length pre-check at crates/gl/src/task.rs:211 is genuinely pinned. Verified the pin by neutering the pre-check (test goes red), then restored. fmt clean.

* fix(node): fail closed on claim for unresolvable repo

task_claimable returned true for an unassigned task whose repo_id is
slash-form (mirror row) or names a repo this node does not host, while
task_visible fails closed on the same rows. A signed stranger could claim
such a task even though reads 404. Return false for a Some(repo_id) the
node cannot resolve to a locally-hosted repo, matching task_visible's
fail-closed check; keep the None arm open (repo-less tasks claimable by
design). Adds a regression test asserting a stranger's claim 404s and
leaves the assignee unchanged for both row classes.

* fix(node): pad sealed task cursor to fixed width

The scan-ceiling token seals the last examined row's (created_at, id) as
variable-length JSON under a length-preserving keystream, so token length
leaked the withheld row's field widths: a 32-char to_rfc3339() rendering
mints a 148-char token where a 35-char one mints 161 chars. Pad the
serialized payload with trailing JSON whitespace to a fixed 128-byte
bucket before the SIV tag is computed; decode tolerates the padding
(serde_json ignores trailing whitespace) and tokens minted before this
change still verify. REST and GraphQL share task_cursor::encode, so one
fix covers both.

Adds token_length_does_not_vary_with_position_width: asserts identical
token length across narrow/wide timestamp positions and that the padded
token still decodes to the exact stored position.

* fix(node): address beardthelion round-six P3 findings on task cursor

- Pin the encode docstring compat claim: add unpadded_legacy_token_still_verifies
  test minting a token via the pre-padding recipe (siv + apply_keystream with
  no padding) and asserting decode returns the position.
- Pin the PAYLOAD_BUCKET round-up claim: add oversized_payload_rounds_up_to_next_bucket
  test with a 140-char id crossing the 128-byte bucket, asserting round-trip
  at two buckets.
- Correct timestamp-width figures: to_rfc3339() renders 25 chars with no
  fractional seconds (not 32); 32 is the microsecond width.

* fix(node): address beardthelion round-seven P3 findings on task cursor

- Correct wire-form docstring to v1.<iv>.<body>
- Fix payload width figures: 101 bytes total, to_rfc3339() precision ladder
- Tighten oversized-payload test to assert exactly 2*PAYLOAD_BUCKET
- Pin legacy back-compat test with a frozen v1 token fixture

* test(node): address beardthelion round-eight P3 findings on task cursor

- Pin the one-bucket width: round_trips_position_verbatim now asserts the
  decoded body of a normal token is exactly PAYLOAD_BUCKET, catching a pad
  that skips next-multiple or a drifted bucket constant. Every other width
  assertion here is bucket-relative, so only an absolute check goes red.

- Assert the frozen fixture is the unpadded shape:
  unpadded_legacy_token_still_verifies decodes the body segment and asserts
  raw.len() < PAYLOAD_BUCKET, so a literal re-minted under the padded
  recipe would retire the back-compat pin loudly instead of passing
  silently.

- Narrow the recipe-pin claim: the docstring now names what the frozen
  literal actually binds (domains, MAC input, keystream, segment layout,
  field names) instead of claiming padding and serialization changes break
  the test. The padding-width claim stays with
  oversized_payload_rounds_up_to_next_bucket.

* fix(node): trim stored assignee on every task gate; align SECURITY.md task-read wording

* test(node): pin mirror-row list-vs-get asymmetry; fix plan-test bind (#553)

- EXPLAIN helper binds normalize_owner_key(assignee) to match the
  production list_tasks_keyset bind.
- Pin the deliberate asymmetry jatmn asked about: a delegator sees a
  slash-form (mirror-hosted) task in a list, while get fails closed for
  every caller because the row has no local repo backing. Notes in
  get_visible_task and a regression test keep it from drifting.
- Repair test-mod drift from the main merge: build_schema gains its
  TaskCursorKey arg at three sites and list_tasks calls move to
  list_tasks_keyset.

* test(node): reconcile tasks complexity contract with TaskPageType (#553)

The four GraphQL task/brake tests failed on this branch: contract tests
still queried the flat tasks { id } shape that TaskPageType replaced, and
the alias-brake tests ran default-size lists whose six aliases exceed
GRAPHQL_MAX_COMPLEXITY before the brake can debit.

- Contract tests now select through items { ... } with refreshed cost
  arithmetic.
- Brake tests use tasks(limit: 1), keeping six aliases under the budget
  so the 6th field reaches TaskReadBrake and fails RATE_LIMIT_MESSAGE.
  Complexity stays the outer gate (#552); the brake stays the per-request
  field-count gate.

* docs: note mirror-hosted task list-vs-get asymmetry (#553)

Option 3 for slash-form mirror repo_id rows was pinned in code and test at
6c0926de; this adds the operator-facing half jatmn asked for. README and
SECURITY task-read bullets now state that mirror rows list for party readers
while get-by-id and GraphQL task fail closed for all callers.

* ci: retrigger codeql scan after alert 86 dismissal

---------

Co-authored-by: euxarist <[email protected]>
Co-authored-by: cairn-code <[email protected]>
Co-authored-by: cairn-intern <[email protected]>
Co-authored-by: Marlowe <[email protected]>
2026-11-09 19:02:54 +00:00
0919cf7c62 fix(core): ignore invalid signatures in satisfies_threshold (#349) (#554)
* fix(core): ignore invalid signatures in satisfies_threshold (#349)

RefUpdateCert::satisfies_threshold called verify_all, which fails closed
on the first malformed or invalid signature entry. Signature entries live
outside the signed body, so any third party can append a junk entry with
no key material — appending one denied an otherwise valid certificate, a
DoS on the threshold check.

Add a lenient valid_signers() helper that skips entries failing to parse,
decode, or verify, and have satisfies_threshold count distinct valid
signers from it. verify_all stays strict (fail-closed) so its API and the
tampered_signature_fails_verify_all test are unchanged. A forged entry
naming a real maintainer DID with a bad signature is skipped, so it cannot
inflate the count either.

Adds tests: appended junk entries do not deny a valid 2-of-2 cert, and a
forged maintainer signature is not counted toward the threshold.

Closes #349

* fix(core): bound the tolerant signature scan before cryptographic work

satisfies_threshold verified every entry, so a peer could append
arbitrarily many syntactically valid entries with bogus signatures and
buy attacker-controlled CPU and allocation on every threshold check.
Bound each stage ahead of the loop: reject a list over
MAX_SIGNATURE_ENTRIES loudly (a prefix scan would let junk starve real
signatures out of the window), refuse to base64-decode any encoded
signature that is not exactly the 86 chars of a 64-byte Ed25519
signature, skip non-maintainer and already-counted signers on a string
compare before any curve work, and return as soon as the threshold is
met.

* docs(cert): correct ENCODED_SIG_LEN length-gate comment (#554)

The comment claimed anything longer cannot decode to a valid signature.
The same 64 bytes appear in padded or non-url-safe encodings; the gate is a
canonical-form DoS bound for the tolerant scan, not a crypto impossibility.

---------

Co-authored-by: andreolf <[email protected]>
Co-authored-by: Kevin Codex <[email protected]>
2026-11-09 17:42:13 +00:00
beardthelionandRapyBus S.A.C. 05afd68e41 bootstrap-peers: add moonwake-ledger with a dialable p2p multiaddr (#559)
Co-authored-by: RapyBus S.A.C. <[email protected]>
2026-11-09 17:21:54 +00:00
beardthelion 6c971e7ea0 ci(audit): ignore RUSTSEC-2026-0459, unlinked libsqlite3-sys advisory (#564)
The advisory covers bundled SQLite in libsqlite3-sys 0.30.1, which sits in
Cargo.lock only because sqlx resolves sqlx-sqlite unconditionally. sqlx builds
postgres-only, so the bundled C source is never compiled or linked. The fix
needs libsqlite3-sys >= 0.39 but sqlx-sqlite 0.8.6 pins ^0.30.1; the waiver is
documented to retire if the sqlite feature ever enters the build.
2026-11-09 16:54:05 +00:00
beardthelion 10d5b98b2b fix(gl): report quickstart registration failure when the node returns no UCAN (#355) (#556)
A 2xx with a non-JSON or ucan-less body still printed "Registered
successfully / UCAN saved to ..." while nothing was written, and left a
stale ucan.json from another node in place. Print the success lines only
when a UCAN was actually persisted.
2026-11-09 16:11:08 +00:00
a2262d92b2 fix(node): implement reconciliation sweep as durability backstop (#218) (#551)
* feat(node): implement reconciliation sweep as durability backstop (#218)

Reconciliation sweep: periodic background worker that re-derives pin/seal
sets and fills gaps so a dropped replication job never means data loss.

- visibility_pack.rs: all_object_paths (two-phase ls-tree + cat-file
  --batch-all-objects), allowed_blob_tree_sets_bounded, ObjectPath/BlobTreeSets
  type aliases
- db/mod.rs: atomic policy transactions, get/set_node_state, has_ipfs_cid,
  filter_ipfs_pinned_oids, filter_pinata_pinned_oids, repo_policy_epoch,
  migration v27 (node_state table + policy_epoch column)
- ipfs_pin.rs: PolicyFence struct, dispatch fence re-check before POST
- pinata.rs: dispatch fence re-check before POST
- encrypted_pin.rs: dispatch fence re-check
- reconciliation.rs: full sweep worker with keyset cursor, PolicyFence
- api/ipfs.rs: effective_cid fallback in list_pins
- main.rs: spawn reconciliation sweep worker
- .cargo/audit.toml: RUSTSEC-2026-0258 h2 ignore (pending #368)

* fix: add missing parameters to pinning functions and adjust timeout settings

* fix(tests): update CID assertions to handle Option type and adjust parameters in pinning functions

* fix(node): close review findings on reconciliation sweep (#218)

Reviewer-1 P1 / Reviewer-2 P1: v27/v28/v29 were referenced by tests but
never added to the MIGRATIONS array, so every existing deployed node
would fail to boot (node_state, repos.policy_epoch, pinned_cids.cid
NOT NULL → NULL all unreadable). Forward-only migrations:
  v27 pinata_only_clear_legacy_equal_cid: ALTER cid DROP NOT NULL,
       UPDATE cid = NULL WHERE cid = pinata_cid
  v28 node_state_key_value: new key/value table for the sweep cursor
  v29 repos_policy_epoch: BIGINT NOT NULL DEFAULT 0

Reviewer-2 P1: visibility_pack.rs:542 phase 1 used ls-tree -rz, which
under -r recurses into blobs and never emits tree entries. Trees then
arrived only from the phase-2 catch-all with empty path and the
fail-closed filter denied every one — the sweep could repair a
non-flat repo's git graph. Switched phase 1 to ls-tree -r -t -z and
add a per-commit rev-parse <commit>^{tree} that registers the root
tree at "/". 48 visibility_pack unit tests pass.

Reviewer-1 P2 / Reviewer-2 P2: list_pinned_cids called is_raw_cidv1 on
the cid column before decoding it as Option<String>, so SQL NULL
failed the decode and the new Pinata-only rows never reached the
handler. Decode cid as Option first, drop only the both-NULL case.
The is_raw_cidv1 filter was removed entirely: the new contract lists
every row that has something to advertise, and the handler at
api/ipfs.rs:list_pins is the seam that decides what to do with a
legacy-shape row (the resolver 404s on a mismatched key, per #173 U4).
Updated list_pinned_cids_omits_unrepaired_legacy_row to match.

Reviewer-1 P2: two new sqlx::test cases pin the gate order:
sweep_skips_quarantined_repos_before_scan (SQL filter, plus a
defense-in-depth per-row re-check) and
sweep_skips_private_repos_before_scan (per-repo listable_at_root).
Both use expect(0) on the mock IPFS so any future reorder that
moves the scan ahead of the gate fails the build.

Also: record_pinned_cid ON CONFLICT now rewrites cid (R1-P2: a
stale wrong CID from a previous push is repaired by a subsequent
one). record_pinata_cid ON CONFLICT clears cid to NULL when the
existing row has the legacy cid = pinata_cid shape; on INSERT, stores
cid = NULL when raw_cid == pinata_cid (the Pinata-only signal).

* feat: implement per-backend continuation offset for missing OIDs

- Enhance `missing_oids` function to accept a `start_after` parameter, allowing for rotation of the missing set based on the last attempted OID.
- Update `run_pass` to load and save the continuation offset for both IPFS and Pinata backends, ensuring that previously attempted OIDs are retried last in subsequent passes.
- Introduce tests to validate the behavior of the new offset mechanism, ensuring deterministic ordering and proper handling of gaps in the missing set.
- Ensure that the offset is cleared when no missing OIDs are found, allowing for fresh starts in subsequent passes.

* fix(node): trace provenance/structural invariants through every consumer

The v30 local_ipfs_provenance column was set by the writers but
only consulted by has_ipfs_cid and filter_ipfs_pinned_oids. The pin
loop's "is this already done" check still used row existence
(is_pinned), so a Pinata-only row hit the early-return branch
without ever reaching the local writer, and the flag stayed FALSE
forever. Three other seams carried the same shape-vs-provenance
confusion.

P1 (ipfs_pin.rs:1843): pin loop's skip keys on has_ipfs_cid
(writer-owned local_ipfs_provenance = TRUE), not on row existence.
A Pinata-only row now falls through to the local writer path,
which calls record_pinned_cid_with_source and flips the flag on
the conflict branch. The Pinata pin loop's skip remains keyed on
has_pinata_cid (a separate seam).

P1 (db/mod.rs v30 backfill): tighten the WHERE clause to
cid IS NOT NULL AND pinata_cid IS NULL. The previous permissive
(cid <> pinata_cid) clause marked pre-v30 Pinata-first uploads
(where record_pinata_cid deliberately stored cid = raw_cid
without a local Kubo push) as local IPFS pins, so the sweep
filtered them as already-durable and an operator enabling IPFS
later never got a local recovery copy. Ambiguous dual-shape rows
stay at FALSE under the strict rule and the sweep re-derives on
re-pin (cheap, idempotent).

P1 (db/mod.rs record_pinned_cid_with_source ON CONFLICT): restore
cid = COALESCE(pinned_cids.cid, EXCLUDED.cid). A Pinata-only row
with cid = NULL (the raw_cid == pinata_cid shape) would otherwise
be marked complete with local_ipfs_provenance = TRUE and a NULL
resolver key.

P1b (visibility_pack.rs): remove the synthetic "/" path on root
trees and add a structural entry-level check
(structurally_safe_root_tree / _for_caller) in both the sweep
gate (allowed_blob_tree_sets_bounded) and the per-request tree
gate (allowed_tree_set_for_caller_bounded). A tree is admitted
only if every direct entry in its serialized bytes is safe at
/{filename} and (for tree entries) the child tree is itself in
the allow set. The previous path-based "/" check let the root
tree of a public repo slip through even when its bytes name a
denied subtree entry.

P2 (api/ipfs.rs list_pins): add local_ipfs_provenance to
PinnedCidRecord, include in list_pinned_cids SELECT, and surface
as local_pinned / pinata_pinned in the JSON response. A
Pinata-only row previously returned the raw resolver key as
local_cid, making it indistinguishable from a real dual-backed
row. The new fields let a gl consumer distinguish local-only,
remote-only, and dual rows without re-inferring semantics from
nullability. local_cid is preserved for backward compatibility.

Tests:
- migration_v30_backfills_local_ipfs_provenance_heuristically:
  flipped both-distinct expectation to FALSE under strict rule.
- migration_v12_makes_cid_nullable_and_preserves_classification
  and migration_v27_clears_legacy_equal_cid: both-distinct /
  distinct row expectations flipped to FALSE.
- sweep_never_pins_root_tree_naming_withheld_subtree (new):
  assert root tree + secret subtree tree absent from pinned_cids.
- allowed_tree_set_gates_withheld_subtree_tree: root tree
  assertion inverted to EXCLUDED for anon.
- list_pins_reports_writer_owned_provenance_for_all_shapes (new):
  assert 4 row shapes produce correct local_pinned /
  pinata_pinned JSON.

Scope discipline preserved: PolicyFence capture order,
_pin_permit reuse for the seal phase, fresh per-arm
rederive_budget, keyset repo cursor with lookahead, mirror-row
skip, Kubo 2xx-without-Hash refusal, v27 provenance-clearing
migration all untouched. v31 schema unchanged.

* fix(node): structural tree gate at every depth + non-commit ref tolerance

The prior round installed a `local_ipfs_provenance` column and
applied a structural tree gate only to root trees. The fresh
review pass found two structural gaps the writer contract
didn't propagate to:

P1 (visibility_pack.rs:1519): the path-based tree admission
admitted a tree whenever its path was allowed, regardless of
whether its serialized bytes named a denied subtree. Concrete
failure: a public repo with `/public/secret/file` and a
`/public/secret/**` deny admitted the `/public` tree (its path
is `/public`, allowed) but `/public` still contained a
`secret/` entry pointing at the denied subtree — anyone who
obtained `/public`'s CID could `cat-file -p` it and read the
subtree's name and child OID. The prior round's post-pass
structural check ran on root trees only.

The fix: replace the prior round's
`structurally_safe_root_tree` / `_for_caller` / `root_tree_oids`
helpers with one recursive `tree_structurally_safe` that walks
every tree's entries at every depth, recursing on tree entries.
Bundled the rule/path args into a `TreeCheckCtx` struct to fix
the clippy `too_many_arguments` warnings the prior round
introduced. A tree is admitted iff every direct entry is safe
at its path AND (for tree entries) the child tree is itself
structurally safe there. A denied-subtree tree fails at its
level; the parent fails because the denied-path child entry
fails; the propagation continues up to the root. The
`admitted` set memoizes trees proven safe at some path so the
recursion short-circuits on cycles and on trees reachable at
multiple allowed paths.

P1 (visibility_pack.rs:533): `all_object_paths` called
`assert_all_refs_are_commits`, which bailed on any ref that
didn't peel to a commit (tag-of-tree, tag-of-blob). The
sweep (`run_pass`) converted that error into a full-scan
failure that skipped the whole repo. A repo with a pushable
tag-of-tree never got its missing public pins repaired by the
new hourly worker, even though its commit-reachable graph
was otherwise safe to classify (`ipfs_cid_tree_served_despite_non_commit_ref`
is the in-repo example of that supported shape).

The fix: remove the `assert_all_refs_are_commits` call from
`all_object_paths` (and the analogous one in `blob_paths`,
used by the encrypted recovery path — the same reasoning
applies). `git rev-list --all` already silently skips
non-commit refs, so the commit-reachable object set is what
the sweep needs. The unused `assert_all_refs_are_commits`
helper is removed. Unclassifiable ref targets still fail
closed at a different layer: the cat-file catch-all
enumerates them with no path, and the path-based allow
filter drops empty-path entries.

Test inversions and new regressions:

- `migration_v12_makes_cid_nullable_and_preserves_classification`
  and `migration_v27_clears_legacy_equal_cid` — both-distinct /
  distinct row expectations flipped to FALSE under the v30
  strict backfill (prior round's tightening is re-affirmed
  here; the v30 backfill is `cid IS NOT NULL AND
  pinata_cid IS NULL`, the only unambiguous local-IPFS
  shape).
- `ipfs_cid_gate_withholds_blob_from_unauthorized` — root
  tree assertion inverted to NOT_FOUND (the recursive
  structural gate denies root trees whose entries name a
  denied subtree).
- `ipfs_cid_legacy_provider_cid_repaired_on_repush` —
  legacy-shape row insert now sets `local_ipfs_provenance
  = TRUE` to reflect that the bytes ARE on local IPFS (the
  legacy CID is just the wrong key).
- `ipfs_cid_tree_served_despite_non_commit_ref` — root
  tree assertion inverted (same reason as the gate test).
- `fails_closed_on_annotated_tag_of_a_blob` and
  `fails_closed_when_a_ref_cannot_be_traversed` — renamed
  to `skips_*`, assertions inverted from `Err` to `Ok` (a
  ref pointing at a non-commit object is no longer a walk
  error; the blob falls out of the gap set because
  `git rev-list --all` skips it).
- `ipfs_cid_walk_error_fails_closed` — removed (relied on
  the now-removed `assert_all_refs_are_commits` to force an
  error; the fail-closed arm is still covered by
  `ipfs_cid_commit_tag_walk_error_fails_closed`, which uses
  a different trigger — a nonexistent ref object makes
  `rev-list --all` fail).
- `sweep_does_not_publish_public_ancestor_tree_naming_withheld_subtree`
  (new) — recursive depth test: a `/public/secret/**` repo
  asserts root tree, `/public` tree, and `/public/secret`
  subtree are all absent from `pinned_cids` while
  `/public/visible.txt` is IPFS-pinned.
- `sweep_repairs_commit_reachable_object_in_repo_with_tag_of_tree`
  (new) — tag-of-tree repo with a separate `mktree`'d tree
  (not commit-reachable); asserts the commit-reachable blob
  is IPFS-pinned and the unclassifiable tag-of-tree is not.

Scope discipline preserved: PolicyFence capture order,
`_pin_permit` reuse for the seal phase, fresh per-arm
`rederive_budget`, keyset repo cursor with lookahead,
mirror-row skip, Kubo 2xx-without-Hash refusal, v27
provenance-clearing migration, v30 strict backfill, v31
schema, per-(repo, backend) reconciliation offset, and the
`local_ipfs_provenance` writer/reader contract are all
untouched.

* test(node): ignore pre-existing RED replication-tail test on CI

`api::repos::tests::receive_pack_tail_survives_a_disconnect_during_release`
documents an open bug in `api/repos.rs` (the replication tail must
be spawned above `release` and gated on
`receive_result.is_ok()`; today it's spawned below, so a
disconnect during `release` drops the pins, recovery copy, and
announce). The test's `assert!` message starts with "RED:" —
this is a TDD-style "keep the design intent visible in the code"
test, not a working-behavior assertion.

The test was added 2026-07-29 by `beardthelion` (PR #215
lineage), well before the #218 sweep work, and is unrelated to
this PR. CI is failing on it under load (passes in
isolation) because the open bug hasn't been fixed yet.

`#[ignore]` until the open bug is fixed. The test's design
intent and the fix description stay in the doc comment and the
assert message; only the CI failure is suppressed. Run the
test explicitly with `cargo test -- --ignored` to verify
the bug is fixed when the tail-spawn moves above `release`.

* fix(node): align encrypted manifest anchor log with one-shot contract

The "will retry next pass" log at reconciliation.rs:1034 was false:
plan_seal returns SkipUnchanged on every subsequent pass once the
recipients tag matches, so sealed stays empty and the anchor block
never runs again. A transient Irys outage at the moment of a fresh
seal therefore LOSES that anchor permanently — the next pass has no
delta to anchor and no retry fires. This is intentionally best-effort:
re-anchoring an unchanged manifest would burn Irys writes for no
recovery benefit.

Factored the warn message into ENCRYPTED_MANIFEST_ANCHOR_FAILED_MSG
and added a unit test that pins the constant; the test goes RED if
"will retry" reappears in the log.

* fix(node): address eight reviewer findings on #218 reconciliation sweep (round 3)

* fix(node): close audit.toml array and drop inline comment

* fix(node): repair the stale for-each-ref fake-git fixture (round 8 P1)

`upload_pack_shares_one_deadline_across_walk_and_filtered_serve` — the
#174 shared-deadline guard for the FILTERED serve arm — was red on both
`test (stable)` and `test (beta)`.

The fake git answered `for-each-ref` with the single token
`refs/heads/main`, a ref NAME, where `blob_paths` phase 2 asks for
`--format=%(objectname) %(objecttype)`. The phase-2 parse fails closed on
a line it cannot read, so the walk returned an error and the request
surfaced as a generic 500 instead of being reaped into a 504. That
silently repurposed the test from "the filtered serve shares one
deadline" into "the walk errors" — the guard proved nothing while it was
red.

The fixture is what was stale, not the production path: the arm now
emits the column shape the walk parses. A commit tip has no peeled
referent, so two columns is the whole line. The 1.2s walk cost stays on
`ls-tree` and the serve's on `pack-objects`, so the two phases remain
independently attributable.

* fix(node): peel annotated tags in the blob_paths ref walk (round 8 P1)

`%(objecttype)` reports the type of the ref's OWN object, which for an
annotated tag is `tag`. The phase-2 `if kind == "blob" || kind == "tree"`
arms therefore never saw the referent, and a blob reachable ONLY through
an annotated tag escaped the withheld set — while `rev_list_keep`'s
`git rev-list --objects --all` does peel tags and served it. Phase 2
closed the leak for a direct ref-to-blob and for that shape only.

The walk now asks for the peeled atoms `%(*objectname) %(*objecttype)`
and inserts the referent with an empty path, alongside the direct
blob/tree tips it already handled. Line parsing moves to a field-count
match: two columns for a non-tag tip (the peeled atoms expand to empty),
four for a tag tip, anything else still fails closed.

Measured against git 2.50, the `*` atoms peel the WHOLE chain — a
tag-of-a-tag-of-a-tag-of-a-blob reports the blob — so a `tag` peeled
type does not occur on stock git. That arm is kept anyway, because
`push_delta.rs`'s ref-type guard documents `%(*objecttype)` as a
one-level peel: under such a git it finishes the chain with
`rev-parse <tip>^{}` plus a `cat-file -t` probe rather than bailing and
500ing every clone of a repo that merely carries a nested tag. Both
children are bounded by the walk's shared deadline and are unreachable
on the common one-line-per-ref path.

* fix(node): one empty-path decision for both blob_paths consumers (round 8 P1)

`blob_paths` has two consumers and only one of them carried the
empty-path policy. `withheld_from_pairs` (the deny side, what the
smart-http serve filter excludes) withheld a phase-2 empty-path entry
from everyone but the owner; `allowed_blob_set_for_caller_bounded` (the
allow side, what `GET /ipfs/{cid}` hands over) called
`visibility_check(..., "")` directly. On a public repo no glob matches
the empty path, so that returns `Allow`: the serve filter withheld the
OID while the IPFS gate served the bytes — the leak phase 2 exists to
close, reopened one layer over.

The policy moves into `pair_decision`, which both consumers now call, so
the two gates cannot disagree and a future change to the empty-path rule
moves both at once. `withheld_blob_recipients_bounded` routes through it
too: an unclassifiable blob grants a recovery copy to the owner only,
never to a rule's named reader whose grant was written against a path
this object does not have.

No behavior change for non-empty paths — `pair_decision` delegates to
`visibility_check` unchanged.

* test(node): bind the phase-2 ref tests to a commit-unreachable blob (round 8 P2)

`skips_a_ref_pointing_at_a_blob` and `skips_an_annotated_tag_of_a_blob`
were vacuous. Both hung their ref on `fixture()`'s `secret` blob, which
is COMMITTED at `secret/b.txt`, so phase 1's per-commit `ls-tree`
already produced `(secret, "/secret/b.txt")`, the `/secret/**` rule
already denied it, and `withheld.contains(&secret)` passed with phase 2
deleted outright.

Both now use `orphan_blob`, a blob written straight to the object store
that no commit's tree names, so the ONLY route into the withheld set is
the `for-each-ref` phase. That is also the real leak shape:
`rev_list_keep`'s `git rev-list --objects --all` does follow a ref to
such a blob, so an under-withheld one ships in the clone pack.

Verified by mutation: with `if false` injected at the phase-2 filter,
every test below goes RED (they were green under that mutation before).

New coverage:
- `ref_only_blob_is_denied_on_the_allow_side_too` — the allow-side
  denial for the shared `pair_decision`; it also asserts deny and allow
  agree on one OID, and that the owner IS still admitted, so it cannot
  pass by denying everything. Reverting the allow side to
  `visibility_check` turns it red on exactly that assertion.
- `skips_a_nested_annotated_tag_of_a_blob` — real-git tag-of-a-tag.
- `peels_a_tag_whose_peeled_target_is_still_a_tag` — drives the
  one-level-peel fallback through the fake-git seam, so that arm is
  tested rather than dead.

* fix(node): advance the reconciliation continuation only for dispatched work (round 8 P2)

`ipfs_last` / `pinata_last` were captured from the missing set above the
pin permit, both `PolicyFence` captures and both pin loops, then written
under nothing but `if ipfs_enabled` / `if pinata_enabled`. Every stage in
between can legitimately produce nothing — a fence capture that fails, a
quarantine/visibility recheck that says skip, a pin-boundary
re-derivation that errors — and each is transient.

Advancing past OIDs that were never attempted is not a harmless retry
delay: `missing_oids` rotates strictly past the stored offset, so the
whole unattempted prefix lands BEHIND the entire backlog on the next
pass. For an at-cap repo — the only kind the continuation exists for —
the backlog never drains inside one cap window, so those objects are
starved indefinitely. That is a durability hole in the durability
backstop.

The offset now moves only for work actually handed to a backend, tracked
at the dispatch boundary as `to_pin.last()` (not the missing set's last:
an OID the pin-boundary re-derivation dropped was never offered). It is
recorded before the call, so a pin phase that times out mid-batch still
counts as dispatched.

The write becomes three outcomes rather than two, via one
`next_offset_write` helper so the backends cannot drift apart:
  * work dispatched            -> advance to the last dispatched OID
  * nothing missing, query OK  -> None, marking the row done
  * nothing dispatched, or the missing-set query failed
                               -> write NOTHING, leaving the resume point
                                  a capped pass already paid for

`*_scan_ok` is what separates the last two: an empty missing set means
"everything is pinned" or "the filter query failed and we know nothing",
and only the first should mark the row done.

Test: `sweep_leaves_offset_untouched_when_nothing_was_dispatched`, using
a `#[cfg(test)]` seam on the pin-boundary re-derivation — that stage
cannot be starved from the outside, because the mid-scan re-filter runs
first on the same rules and budget and makes the sweep `continue` well
before the offset write. It asserts the stored offset comes back
byte-identical after a declining pass, then that the same repo does
advance it once dispatch happens, so the assertion cannot pass on a dead
write site. Restoring the old semantics turns it red.

* fix(gl): render the Pinata provider CID in `gl ipfs list` (round 8 P2)

The server-side wire change landed without its client half. `cmd_list`
read `pin["cid"].as_str().unwrap_or("?")` as the entry heading and
nothing in the CLI read `pinata_cid` at all, so a Pinata-only row
printed as a bare `?` — the listing showed nothing usable for an object
that is in fact durably stored.

`cid` is legitimately null for such a row: it is the key
`GET /ipfs/{cid}` resolves against, and the node stopped aliasing a
Pinata provider CID into it (round 3) precisely because that CID 404s
there. So the fix is on this side.

Rendering moves into `render_pin`, which leads with the node-resolvable
CID when there is one and otherwise the provider CID, labelled so nobody
feeds it back to this node's resolver. A dual row keeps both visible. A
`backends:` line names where the bytes are, read from the writer-owned
`local_pinned` / `pinata_pinned` booleans rather than re-inferred from
CID shape — which is what the node's own comment warns against.

Older nodes send neither flag; those `cid`-only rows fall back to CID
presence and render as they always did
(`render_pin_handles_a_legacy_cid_only_row`).

* fix(node): implement third fence check for IPFS pinning and enhance visibility rules handling

* fix(node): enhance documentation and add tests for ProgressState and helper functions

* fix(node): address round-9 reviewer findings on #218 reconciliation sweep

* fix(node): correct consumer-4 recipient test for call-invariant set

* fix(git): fail closed on non-UTF-8 tree listing in walk_tree_oids_inner (#218 round 10 P1)

walk_tree_oids_inner previously returned Ok(()) on a non-UTF-8
ls-tree -z listing, inserting only the tree OID and skipping
every child blob/tree OID. A direct tree ref (or an annotated
tag peeling to a tree) is valid Git input; git rev-list --objects
--all still enumerates the tree and every descendant, so the
keep-side removed the tree from the served set but passed the
child blob OIDs to pack-objects, exposing their bytes to an
anonymous clone.

Phase 1 already bails on the same input at the blob_paths walk;
this fix puts the tree-walk on the same fail-closed outcome.
The new fails_closed_on_non_utf8_tree_tip test pins the
invariant for both a direct refs/tags/direct-tree ref and an
annotated tag-of-tree (the two non-commit shapes round 9 added
tolerance for), confirming the walker bails rather than
returning Ok with a partial withheld set.

* fix(recon): run encrypted recovery even when public list is empty (#218 round 10 P1)

A path-scoped repo whose only reachable object is a direct blob
or direct tree ref yields an empty public list (the anonymous
public classifier removes the only object from the served set)
while withheld_blob_recipients_bounded still assigns that
object to the owner recovery set. The early `continue` on
object_list.is_empty() (and the second one after the
mid-pass refilter) suppressed encrypted recovery too, and a
lost or failed encrypted copy was never repaired.

Track empty-public-work as a flag and gate the public phase
on it. Encrypted phase 2 runs regardless. The backend enable
flags and `_pin_permit` move out of the gated block so phase
2 can read them when the public phase did not run.

* fix(db): rename migration v12 test to v32 (#218 round 10 P3)

The test name and its docstring/comments still say "v12", but
the cid-nullable migration is at v32 (line 1163). The test
itself is correct — it drives the real path — so this is
naming drift rather than dead coverage. A future reader
looking for the migration would otherwise chase a v12 that
no longer exists.

* fix(git): index OID membership before quadratic classifier scan (#218 round 10 P2)

all_object_paths was doing O(O×P) work in two places: the
catch-all cat-file branch checked `blob_set.iter().any(|o|
o == oid)` and `tree_set.iter().any(...)` for every reachable
object, and the phase 1 loop also did a per-commit unused
`git rev-parse <commit>^{tree}` probe whose result was
discarded (the root-tree OID is recomputed by
`root_tree_oids`).

On the 50k-object repos the sweep exists for, the quadratic
scan and the redundant per-commit child process can consume
each authorization deadline before any object reaches a
backend, producing a permanent hourly skip.

Maintain a separate `blob_oids` / `tree_oids` HashSet alongside
the path-pair sets so the membership check is O(1). Drop the
unused per-commit rev-parse probe.

* fix(git): bound tree walk by ls-tree invocation count, not just depth (#218 round 10 P2)

walk_tree_oids_inner stopped at MAX_TREE_WALK_DEPTH and the
shared deadline. A wide shallow tree reachable from N ref tips
or from multiple parents still spawned one ls-tree child
process per subtree well inside the depth cap, and the
wall-clock bound could not stop that — expiry was the only
stop. The ceil was wall-clock only.

Add a structural invocation cap (MAX_TREE_WALK_INVOCATIONS)
the walker fails closed at, and a memo of walked tree OIDs so
the same tree reachable from multiple ref tips is walked once.
Round 10 P2.

* fix(pinata): suppress push when DB record is not durable (#218 round 10 P2)

pin_new_objects pushed (sha, cid) on provider POST success
regardless of whether record_pinata_cid persisted the row.
A closed/failed DB write or an explicit-transaction source
record timeout meant the next sweep queried the DB, found no
row, and re-offered the same gap — with reconciliation having
counted a fill that never happened.

Track a `db_record_durable` flag. On hard failure of either
record_pinata_cid (non-timeout) or record_pin_source
(any failure, since it's a multi-statement transaction), set
the flag false and skip the `pinned.push`. The
`BoundedDbError::Elapsed` autocommit case stays as the
previous comment's "may have committed" reasoning: the push
fires so the reconcile still sees the fill. Round 10 P2.

* fix(node): serialize policy narrow with irreversible upload via in-process mutex (#218 round 10 P1)

The PolicyFence's epoch read was a one-way read: a visibility
narrow (rule insert, quarantine set) could commit between the
fence's epoch read and the upload's HTTP POST, and the
upload's DB-record INSERT's epoch check would see the new
epoch and reject — but the provider already accepted the
object. Encrypted envelopes had no fence at all.

Add a per-repo in-process mutex that PolicyFence acquires
at capture and releases at drop. The narrow paths
(set_repo_quarantine, set_visibility_rule,
remove_visibility_rule) acquire the same lock for the
duration of their DB transaction. A narrow therefore blocks
until the in-flight batch finishes, and an in-flight batch
waits for the narrow to commit before it even reads the
epoch.

Multi-process / multi-node is a known gap: a Postgres
advisory lock per repo would extend the same guarantee
across processes. Deferred to a follow-up.

* fix(pinata): update post-upload stalled record test for round 10 P2 contract

The previous test asserted that a successful upload with a
timed-out post-upload source record still returned the (sha,
cid) pair. That assertion pinned the buggy behavior the
reviewer called out: the reconcile would count a fill that
was not durable, then re-offer the same gap.

The test name and the `record_pin_source` arm of `pin_new_objects`
are unchanged; the assertion now reflects the new contract:
when the post-upload source record times out (its `mark_pin_sources_incomplete`
already sets the source set to incomplete, so the next
pass re-offers), the pair is suppressed so the reconcile
sees the gap as not-filled. Round 10 P2 follow-up.

* fix: clippy allow + rustfmt on round 10 commits

clippy::too_many_arguments on walk_tree_oids_inner (8 args after
the round 10 memo + invocation counter threading). rustfmt
re-ran on the if/else blocks added in the has_public_work
refactor (round 10 P1 #42). No behavior change.

* fix(node): address round-11 review findings on #218 reconciliation sweep

P1: remove the whole-batch policy mutex. PolicyFence no longer holds a
per-repo lock from capture through drop and the narrow paths
(set_visibility_rule, remove_visibility_rule, set_repo_quarantine) no
longer acquire it, so a visibility narrow commits immediately and the
batch aborts on its next is_current check instead of the narrow
blocking behind the sweep. Accepted residual is a single in-flight
object; the fenced DB record still refuses to land a raced row as
durable. encrypt_and_pin_stops_sealing_when_reader_removed_mid_batch
is green again. PolicyMutexes is deleted, resolving the key-leak P2
by removal.

P2: cover the round-10 has_public_work branch with
sweep_seals_withheld_blob_when_public_list_is_empty: a direct-blob-ref
repo yields an empty public list with a non-empty owner recovery set;
asserts no public gaps/fills, no cleartext pin, exactly one POST (the
seal envelope), and a recorded encrypted copy.

P3: peel-case isolation in the non-UTF-8 tree-tip test (delete the
direct-tree ref before the second call so the peel arm must run);
v12->v32 rename drift in the migration test (boundary, comments, and
the stale (2)->true doc line); accurate comments for the
pre-refilter flag and the post-refilter fall-through.

* fix(node): address merge-readiness findings on #218 reconciliation sweep

Reworks the sweep around explicit state-machine contracts instead of
adding conditionals: backend results carry last_attempted plus
confirmed durability separately (PinBatchOutcome), provider permits
scope to provider effects only, discovery pages by commit window with
a persisted cursor, legacy provenance migrates as unknown, and the
pin list advertises only validated raw local keys.

- Empty-path owner recovery without path rules: phase 2 drops the
  has_path_scoped_rule gate and runs on the actual recipient result,
  with a dedicated recovery recheck (no anonymous-listability
  requirement); matrix test over blob/tree/tag refs x zero/root
  rules, including recreate-after-delete.
- Pinata Elapsed is non-durable unless verify_pinata_record proves
  the exact row and fence epoch; loop test with row-locked repos.
- list_pins gates cid/local_cid on is_raw_cidv1 plus confirmed
  provenance; response-shape matrix uses real CIDs with a legacy
  row and GET /ipfs follow-through; docs match the nullable wire
  contract.
- Permits acquired per provider effect and dropped after; no nesting
  at pool size 1 (test renamed to the new shape).
- Scan discovery pages SCAN_COMMIT_WINDOW commits oldest-first with
  a node_state cursor (advance on drained/empty/attempted, delete on
  cover); refilters and phase-2 recipients run on the same window;
  walk-layer paging/invocation/union tests plus a 1050-commit
  cursor-to-completion sweep test.
- v35 migrates legacy rows as unknown (no TRUE backfill); v35 test
  asserts unknown + preserved Pinata history; sweep re-derivation
  test re-pins instead of filtering.
- Drained completions DELETE continuation rows (no tombstones);
  FK cascade on repo delete; row-count regression test.
- Audit: lru waiver gets an explicit retirement owner plus a weekly
  drift guard; revocation model recorded as a product decision on
  PolicyFence (prompt revocation, single-in-flight residual).

* fix(tests): configure git identity in bare clone for tag fixture

The windowed-union test creates its annotated tag in the bare clone,
which carries no config. It passed locally only via the developer's
global git identity; CI has none, so 'git tag -a' failed there.
Verified with an empty GIT_CONFIG_GLOBAL/SYSTEM.

* fix(tests): ignore pre-existing hung-git pid race on CI

read_object_bounded_returns_by_deadline_with_a_hung_git reads the
fake's pid file with no readiness handshake; a slow scheduler yields
'No such file or directory' (stable-only red, identical beta green).
Unrelated to #218 (store.rs untouched); re-enable with a readiness
signal from the fake.

* fix(node): address round-12 review findings on #218 reconciliation sweep

- P1 roots: admit structurally safe root trees at "/" into the allow
  set and candidates, so safe snapshots fully replicate; roots naming
  a withheld subtree stay excluded via the recursive structural check.
- P1 per-pass WalkBudget on sweep enumeration (entries / output bytes /
  git invocations) with fail-closed budget-exhaustion; serve paths keep
  the uncapped budget.
- P2 recovery lane: independent recovery cursor pages unlistable
  history across restarts instead of sharing the scan window.
- P2 gap-hold: failed gap queries hold discovery advance but not the
  healthy backend, via test-only failure seams.
- P2 nested tags: collect full tag chains so nested tag objects pin
  without an inner ref.
- P2 RepoStore: thread the Tigris-backed store into the sweep with an
  acquire timeout and shutdown select; restore cache-miss and skip
  missing.
- P2 Pinata source: keep confirmed pairs when the primary already
  covers the repo; suppress push for stalled post-upload records
  without coverage.
- Update large-history sweep expectation to blob+root+commit per round.

* fix(node): address consolidated review findings on #218 reconciliation sweep

P1: wire WalkBudgetExceeded through the sweep cursor logic. Both the
public scan and the recovery walk now dispatch the typed
budget-exhaustion signal at the sweep boundary: a budget-hit window
advances past content the sweep refuses to classify partially
(scan cursor save/delete via the existing persist site; recovery
cursor likewise), while git/policy/panic/timeout failures keep the
cursor for retry. visibility_pack documents the obligation; the
match arms are the handler, with a one-line pattern note for future
typed walk errors.

P2: run encrypted recovery when the public scan fails. Scan
walk/panic/timeout arms now fall through with an empty public list
instead of continuing the whole repo iteration, so phase 2 runs on
the recovery lane's own cursor like the unlistable-repo path. The
scan cursor stays untouched for retry. Refilter/recheck-public
failures remain fail-closed whole-iteration continues by design.
Includes a repo-scoped test seam (process-global set, safe across
spawn_blocking and parallel tests via fresh UUIDs).

P2: bump policy_epoch only when remove_visibility_rule deletes a
row, mirroring set_repo_quarantine's rows-affected guard, so a
typo'd or double-deleted glob cannot abort fenced batches.

Also adds the repo-loop outcome matrix (every early exit × scan
cursor / recovery cursor / phase 1 / phase 2) and deny tests:
budget-hit advances both lane cursors with zero pins/seals
(real 250k-entry trip, early-trip fixture runs in ~30s), scan
failure still seals the recovery copy, delete-missing-rule leaves
the epoch unchanged. Each deny was verified by temporarily
restoring the old branch.

* fix(node): address jatmn review findings on reconciliation sweep re-home

* fix(node): reload persisted sweep cursor between passes

* fix(node,gl): pinata offset warn wording and boundary-safe pin timestamp

---------

Co-authored-by: Gravirei <[email protected]>
Co-authored-by: Kevin Codex <[email protected]>
2026-11-09 15:35:18 +00:00
c805e35f40 fix(node): bound GraphQL query cost (#552)
* fix(node): bound GraphQL query cost

* fix(node)!: bound repo names so emitted cursors always parse

The reposPage cursor embeds base64(JSON(owner_did, name)) while
parse_repo_cursor rejects cursors over 4096 bytes, but repo names were
unbounded on every write path. A single long-named visible repo made the
server emit an endCursor it then rejected as "invalid repository cursor",
dead-ending pagination for every caller past that row.

Bound names to MAX_REPO_NAME_LEN (100 bytes) at the create_repo and
fork_repo handlers via a shared validate_repo_name helper (bad charset is
still a 400), and reject overlong peer-supplied names in
Db::upsert_mirror_repo (the sync caller already drops the error and skips
the mirror).

Also in this round:
- docs/graphql-pagination.md states the 200 page-size maximum and corrects
  the over-budget remedy: a smaller limit does not reduce the reposPage or
  repos price, which is charged at the full MAX_VISIBLE_REPO_PAGE_SIZE
  scan regardless of limit.
- Tests pin refUpdates(limit: -1) clamping to zero rows, the
  parse_repo_cursor wrong-shape-JSON arm with its exact "invalid
  repository cursor" message, the longest-admissible-name cursor
  round-trip, and the mirror write-path rejection.

BREAKING CHANGE: the GraphQL `repos` field now returns an error instead of
the full list when more than MAX_VISIBLE_REPO_PAGE_SIZE (200) repositories
are visible; larger lists must use `reposPage` with `limit`/`after`.
Repository names are additionally limited to 100 bytes at creation, fork,
and mirror-registration time.

* fix(node): address beardthelion P3 findings on name bound and input messages

- Assert each rejection's own message in the reposPage invalid-inputs
  loop: the loop no longer accepts either message for every query, so a
  misattributed message (limit arm returning the cursor message) goes red.
- Pin the create/fork name-bound wiring through the handler: new test
  drives create_repo with an overlong name via test_state_lazy and asserts
  the 400; deleting the validate_repo_name call site would proceed to the
  lazy DB pool and fail with a connection error instead.
- Document the API-side vs disk-side validate_repo_name divergence on the
  new function, and tie the store's literal 100 to MAX_REPO_NAME_LEN so
  the two validators cannot drift apart on length.

---------

Co-authored-by: Marlowe <[email protected]>
Co-authored-by: cairn-intern <[email protected]>
2026-11-08 23:28:09 +00:00
a4026e84ff fix(node): honor pre-assigned task assignee on claim (#550)
* fix(node): honor pre-assigned task assignee on claim

Stop strangers from overwriting a reserved assignee_did (and receiving
the UCAN). Bind finish_task to the stored assignee; return 403 on
reserved steals; keep exact stored DID forms; normalize blank assignees;
pin production claim/finish SQL race guards with FOR UPDATE tests; trim
broadcast by_did; and tighten blank/SQL re-check regressions.

* test(git-remote-gitlawb): wait for shim before real_git_fetch on Windows

Windows CI can race: git fetch connects before the shim accept loop is
polling, yielding connection aborted (os error 10053) with 0 POSTs.
Probe the listener after spawn so multi-round and withheld-path tests
start only once the shim is ready.

* test(git-remote-gitlawb): retry transient Windows fetch connection errors

Windows CI can still abort localhost connections (os error 10053) after the
shim is ready. Retry fetch_with_helper up to three times on transient
connection failures before asserting.

* test(git-remote-gitlawb): revert Windows fetch workarounds

The retry-on-10053 and wait-for-shim probes papered over the accepted-socket
inheriting the listener's non-blocking state. #312 fixed that at the source
(normalize_accepted_stream) and is merged, so take the workarounds back out
and let the real fix carry the Windows lane.

* test(git-remote-gitlawb): pin the fetch retry's absence with an exact GET count

The POST assertions are floors, so a reintroduced fetch-level retry could
satisfy them with a second full fetch while masking the withheld shape.
A retry cannot avoid re-advertising, so count GET /info/refs in the shim
and assert exactly one per fetch in both the multi-round and withheld
tests. The merge from main also brings #312's normalize_accepted_stream
into this head, closing the round's rebase ask.

* fix(node): mirror assignee normalization across GraphQL task mutations

GraphQL createTask returned the pre-insert input, so a whitespace-only
assignee came back as a reservation the store never kept; complete and
fail compared the caller against the raw stored assignee where REST and
Db::finish_task trim; and task-event broadcasts reported the caller
string rather than the stored assignee form REST publishes. All three
paths now mirror the REST and DB contract: the create response is the
stored row, finish gates compare the trimmed stored assignee, and
by_did on claim/complete/fail broadcasts derives from the returned row.

---------

Co-authored-by: Ayush7614 <[email protected]>
Co-authored-by: Kevin Codex <[email protected]>
2026-11-08 21:38:53 +00:00
38bab4191b fix(node): version ref_certificates for forward compat (#26 split 3/4) (#548)
* fix(node): version ref_certificates for forward compat (#26 split 3/4)

Reviewer 2 closed PR #224 on 2026-08-28 with a directive: split
into four narrow PRs. This is Split PR 3 (certificate / CLI
compatibility). No findings are assigned to it; it is required
because the protocol/client work is independently reviewable and
should not remain coupled to post-receive recovery.

The wire-format problem: the ref-cert payload is a fixed 7-field
JSON blob the node signs. Future changes (additional pusher
attestations, content-binding fields, secondary signatures) would
either break every existing cert or be impossible to ship without
a flag day. Versioning the cert makes the format forward-compat
and explicit.

NEW MIGRATION v28
  ref_certificates gains `version INTEGER NOT NULL DEFAULT 1`.
  DEFAULT 1 means an existing row reads as v1 with no backfill
  needed; NOT NULL means a missing value is a hard error rather
  than a silent v0.

NEW FIELD ON RefCertificate
  `version: u32`. v1 is the pre-versioning 7-field payload
  (no `version` key in the signed JSON, so a v1 cert is
  byte-for-byte identical to today's cert). v2+ certs will
  include a `version` key and any new fields, both
  backwards-compatible: an old client reading a v2 cert sees the
  unknown `version` field and either ignores it (graceful) or
  refuses to verify (explicit).

API CHANGES
  - list_certs and get_cert now include `version` in the JSON
    response.
  - issue_ref_certificate sets `version: 1` on every new cert.

GL CHANGES
  - gl cert show reads the `version` field; missing field
    defaults to 1 (forward-compat with old servers). v1 certs
    go through the unchanged verify path. v2+ certs are
    explicitly rejected with an "upgrade the client" message
    rather than silently guessing the payload shape — a future
    client that supports v2 cannot accidentally treat a v1 cert
    as v2.

GOLDEN-FORMAT FIXTURE
  v1_payload_matches_frozen_canonical_form pins the v1 signing
  payload byte-for-byte. Reverting this to a v2 shape (adding a
  `version` key) is what the versioned format forbids: an old
  client reading a v1 cert must not see a `version` key in the
  signed JSON. The same byte-form is what gl's verify_signature
  reconstructs in crates/gl/src/cert.rs, so any drift between
  node and CLI breaks the test instead of silently rendering
  every real cert INVALID.

TESTS
  - 3 v1_payload_tests in crates/gitlawb-node/src/cert.rs:
    frozen-canonical-form, round-trip structural, default
    version is 1.
  - 2 new tests in crates/gl/src/cert.rs:
    missing_version_defaults_to_1_and_verifies,
    explicit_version_1_takes_v1_path.
  - 12 existing ref_certificate_tests still pass with the
    version column added.

Compiles clean, 1091 tests pass with 0 regressions, gl 4 cert
tests pass, clippy clean under -D warnings, fmt clean.

Cross-PR overlap (declared in the PR description):

  - #134 (anchors auth): independent. The cert API is unchanged
    in shape; auth gates apply as before.
  - #285 (advisory-lock session affinity): independent.
  - #306 (Content-Digest on signed requests): independent.
  - #314 (small-order Ed25519): independent. v1 certs verify
    through the same path, which already enforces the small-
    order check from #314 on the embedded node DID.
  - #324 (libp2p keypair persistence): independent.
  - #325 (gossip ref-update auth): independent. PR 1's signed
    HTTP push is the durable-intent producer; this PR owns the
    cert shape. The two PRs read and write the same cert table
    but do not step on each other (PR 1 calls
    issue_ref_certificate_idempotent; this PR keeps the
    deterministic cert id but does not change who calls what).
  - #382 (replication withheld-subtree trees): independent.

* fix(node,gl): address review feedback on cert version compat

- Drop the duplicate `version` ALTER from the v1 migration bundle; v28
  is now the sole owner of the column.
- Add a v28 upgrade-path test (v25/v26 pattern) that seeds a legacy
  pre-v28 ref_certificates row, drops the column + migration record,
  re-runs migrations, and asserts DEFAULT 1 + legacy row reads as v1.
- Extract a shared `v1_signing_payload` builder so the live signer
  and the frozen-vector test share one literal; a regression in the
  signer now fails the test.
- Tighten gl's `version` parsing: missing key → v1, present value
  must be a JSON integer that fits in u32 and equals 1. Null, strings,
  floats, overflow, and unknown versions all return Err instead of
  collapsing to v1 (Reviewer 2). Add a truth-table test pinning every
  cell of the parser.
- Extend the v1 forward-compat tests to actually round-trip through
  `verify_signature`, and add a regression test that pins the verdict
  branch rejects v2 even when the v1 signature would otherwise verify.

* fix(cert): address CI failures on PR #386

- cargo fmt --all (CI fmt check was red on the prior commit)
- v28 upgrade-path test: switch version_column_default from
  fetch_one to fetch_optional, so a pre-v28 database with the
  column dropped returns Ok(None) rather than RowNotFound. The
  precondition assertion is what detects a missing column;
  RowNotFound would have masked a regression.
- clippy `needless_borrow` on the v2 verdict-branch test:
  drop the leading `&` on `cert_json["signature"].as_str()`.

* fix(gl,node): make the v2 rejection and issuer anchor real, and test them as such

Round-2 findings, all three demonstrated by execution:

- cmd_show's Ok(v != 1) verdict arm was dead code (parse_cert_version
  admits only 1), and the test aimed at it copied the match instead of
  calling the command. The match now mirrors the parser's contract and
  two end-to-end mockito tests drive cmd_show itself: a version-2 cert
  with a valid v1 signature fails --verify with the version named.

- --verify no longer accepts the queried node's self-reported DID as a
  trust anchor: the node that served a forged cert can serve a matching
  self-report, and that fallback passed a wholly forged certificate with
  exit 0. --verify now requires --expect-node before claiming a trusted
  issuer, with a test proving the refusal (and the anchored positive
  control passing).

- The node-side version test asserted version == 1 on its own literal;
  it now cross-checks the issuing construction: a version-1 claim must
  verify against the v1 payload builder's bytes rebuilt from the cert's
  own fields, and the v1 payload must carry no version key.

* fix(node,gl): address round-2 reviewer findings on #26 split 3/4 cert compat

BREAKING CHANGE: gl cert show <id> --verify now requires --expect-node <did>; the node's self-reported DID is no longer trusted as a cert anchor.

* fix(node): bind cert v2 test to live issuer via sqlx::test

* fix(node,gl): address round-3 reviewer findings on #26 split 3 cert compat

- Stamp new certs v1 until v2 verifier ships; v2 issuance refused by shipped gl
- Add v2_signing_payload with version in signed bytes, downgrade test
- Verify signature in well-formed test, add DB read-back to issuer test
- Fix README/db comments and version-refusal message

* fix(gl): route cert list through error_for_status; correct cert show example

Round 4 reviewer findings on #26 split 3 cert compat:

- P2 (cert.rs:113): cmd_list called .json() straight through on the
  cert list endpoint and let a gated 404 (whose body lacks a
  `certificates` key) fall into the empty-list branch, printing "No
  ref certificates" with exit 0. Wire cmd_list through the same
  error_for_status() gate that cmd_show and resolve_cert_id already
  use, and pin the new behavior with a load-bearing test
  (`cmd_list_surfaces_gated_404_as_err`) that drives cmd_list end
  to end against a 404 mock. cmd_show is unchanged.

- P3 (README.md:314): the documented `gl cert show <id> --verify`
  example bound the certificate ID as the REPO positional, because
  the parser is `gl cert show [OPTIONS] <REPO> <ID>`. Insert the
  missing `<repo>` positional so the example matches the CLI
  parser.

The original BREAKING CHANGE footer in bb553b67 carried the same
shortened `gl cert show <id> --verify` example; this commit's
footer supersedes it with the canonical command, which is what
release-note scrapers read. The round-2 commit message itself is
left in history unchanged; the new footer is the one callers
should rely on.

BREAKING CHANGE: gl cert show <repo> <id> --verify requires --expect-node <did>; the queried node's self-reported DID is no longer accepted as a cert anchor.

* style(gl): fix fmt in cert list response parsing

* fix(gl): label version refusals distinctly and clean cert show help

A cert declaring an unsupported version printed INVALID, which reads as
a failed Ed25519 check even though verify_signature never ran. The
display now prints UNSUPPORTED VERSION for version refusals and keeps
INVALID for real signature failures. The Show help no longer cites
review rounds or commit history.

* docs(node): align RefCertificate.version rustdoc with the signed-bytes v2 contract

---------

Co-authored-by: Gravirei <[email protected]>
Co-authored-by: Kevin Codex <[email protected]>
2026-11-08 21:38:02 +00:00
d84c445f94 fix(gl): derive View: URL from node instead of hardcoding gitlawb.com (#370) (#549)
* fix(gl): derive View: URL from node instead of hardcoding gitlawb.com (#370)

gl repo create and gl mirror print a View: link that was unconditionally
pointed at gitlawb.com, which 404s for repos on self-hosted nodes.

Make the node advertise an optional web_url on GET / (sourced from the
new GITLAWB_WEB_URL env var). The CLI fetches GET / after repo creation
and only prints View: when the node supplies web_url. Nodes without the
config skip the line entirely, which is better than printing a broken link.

* fix(gl): validate web_url response and trim trailing slashes (#370)

Address review feedback:
- Check GET / response status before parsing; only omit View: on
  success responses that lack web_url
- Trim trailing '/' from web_url to avoid double-slash in the URL
- Skip the line when web_url is empty after trimming

* fix: address review feedback on View-URL PR (#370)

- Extract fetch_node_web_url() helper for testability; add 6 unit tests
  covering: web_url present, trailing-slash trim, absent, empty string,
  whitespace-only, and server error
- Reject empty/whitespace-only GITLAWB_WEB_URL at boot in Config::validate()
- Trim whitespace from web_url in the CLI helper (not just trailing slashes)
- Document GITLAWB_WEB_URL in .env.example alongside GITLAWB_PUBLIC_URL
- Mirror uses shared fetch_node_web_url() helper

* fix: validate web_url format and surface node info failures (#370)

Node side: GITLAWB_WEB_URL must now parse as an absolute http(s) URL —
scheme-less hosts and other garbage fail at boot instead of serving a
View link no browser can follow.

CLI side: a non-success GET / status or a malformed advertised web_url
now warns on stderr (with the status/value) instead of silently dropping
the View link; successful responses lacking the field still omit it
quietly, as self-hosted nodes without a web front-end are expected to.

* fix: reject query/fragment web_urls and warn on node info errors (#370)

Both web_url validators now reject URLs carrying a query string or
fragment: the CLI builds View links by string-appending /{owner}/{repo},
so "?a=1" would swallow the repo path into the query. The CLI also no
longer swallows transport-level failures on GET / — a refused or failed
connection warns on stderr like the other degraded-link paths.

* fix(gl): cap node info body and reject control-byte web_urls (#370)

Treat GET / like every other caller-chosen node reply: bound the body
read at 8 KiB and parse from the cap (peer.rs precedent). A web_url
containing control or bidi characters — which passes URL parsing but
would reach the terminal verbatim through the View: line — is now
rejected as malformed, with a defanged preview in the warning.

* chore: bump h2 to 0.4.19 for RUSTSEC-2026-0258

h2 0.4.13 is vulnerable to unbounded empty DATA frames (remote DoS);
0.4.16+ carries the fix. Lockfile-only bump, no manifest change.

* fix(gl): close merge regressions on #370 branch and pin View: with integration test

The a6113fd8 merge onto #370 left the workspace red: a missing closing
`}` in `web_url_rejects_non_absolute_or_non_browser_urls` so the next
test nested inside it (config.rs), a duplicate `url = "2"` workspace
dep (Cargo.toml, one from the redirect-rule PR, one from #370), and
`fetch_node_web_url` derefing `CappedBody` as `&str` instead of
`.text` (repo.rs, regression in the same merge). All three are
straightforward to repair; the brace was the only one blocking CI.

The new `cmd_create_view_url_tracks_node_advertisement` integration
test drives the real `gl repo create` binary against a mockito node
that answers both `POST /api/v1/repos` and `GET /`, and asserts the
printed `View:` line tracks the advertised web_url — present with
the advertised origin when set, absent when the node omits it. The
helper-level `fetch_node_web_url_*` tests would still pass if
`cmd_create` reverted to a hardcoded URL; this one would not, so it
is the load-bearing check for #370.

* fix(gl,node): normalize stored web_url and reject userinfo (#370 review)

Two P2s from the #370 review addressed at the configuration boundary.

`Config::validate()` now rewrites `self.web_url` to the canonical
`url::Url::to_string()` after `validate_web_url` succeeds, so the
regular and degraded `GET /` handlers advertise the same string that
was actually validated instead of the operator-typed one. The CLI
trims a second time in `fetch_node_web_url`, which hid the divergence
on the View: line, but any other consumer parsing `web_url` directly
would see a non-canonical base. `validate` is now `&mut self` to
carry the write; the one non-test caller already binds `mut` and the
two test bindings are updated to match.

Both `validate_web_url` (node) and `fetch_node_web_url` (CLI) now
reject URLs with a non-empty username or a password: the configured
value is published on public `GET /` and reprinted on the View: line,
so an accidentally configured `https://user:[email protected]`
would land in public metadata, terminal scrollback, and CI logs.
New regression tests cover user-only and user+password forms on both
sides, plus a normalization test that the stored value matches the
canonical form (with and without operator padding).

* fix(gl,node): classify capped node-info bodies and reject encoded bidi in web_url

An oversized or half-read GET / body previously fell through JSON parse
failure into the same silent-None arm as a node with no web front-end.
The classify step now consults the capped body's truncated and
read_failed flags and warns before omitting the View link.

web_url validation on both sides also scanned only literal bytes for
control and bidi characters, so a percent-encoded reordering character
(%E2%80%AE) passed boot validation and CLI ingest even though a browser
decodes it into a bidi spoof. Both validators now check the decoded host
and path (INV-6). Also fixes the control-byte test, which served invalid
JSON and passed without reaching the check it pinned.

---------

Co-authored-by: Gravirei <[email protected]>
Co-authored-by: Gravirei <[email protected]>
2026-11-08 18:54:05 +00:00
beardthelionandAyush7614 42cc893fb3 fix(gl): make doctor exit non-zero on Fail-class checks (#357) (#547)
* fix(gl): make doctor exit non-zero on Fail-class checks (#357)

gl doctor printed per-row status and 'Some checks failed' prose, but its
exit code was always 0. A diagnostic whose exit status cannot express
failure is a trap waiting for the first 'gl doctor && ...' to land
somewhere it should not (#357).

Three changes:

1. Re-tier the 'GITLAWB_NODE unset' check from Fail to Warn. The CLI's
   --node flag defaults to https://node.gitlawb.com (PUBLIC_NODE) and the
   CLI works fine without the env var, so an unset env is advisory, not
   a failure. Without this re-tier the obvious 'return non-zero on any
   failure' change would have flipped gl doctor to exit 1 on a stock
   working install, which is exactly the regression the issue warns
   against.

2. Route the exit-code decision through a new has_failures(&[Check])
   helper: exit 1 if and only if at least one row is Fail-class.
   Warn-class rows (iCaptcha offline, version drift, shell-alias
   shadowing, GITLAWB_NODE unset) keep the process at exit 0 because
   those conditions are still printed to the user and the obvious
   '&& pipeline' pattern stays valid. Use std::process::exit(1) rather
   than returning Err so anyhow's error frame does not duplicate the
   user-facing summary that already prints.

3. Three unit tests pin the new predicate against the three regimes:
   all-Ok, warn-only, and a single Fail row tripping the exit. The
   warn-only case is the regression guard for the re-tier: if anyone
   flips GITLAWB_NODE back to Fail, exit_predicate_is_false_for_warn_only
   fails too, so the two halves of the fix cannot drift apart.

Verified end-to-end against the public gitlawb node:
  * unset GITLAWB_NODE, missing identity/registration  -> exit 1
  * unset GITLAWB_NODE, full healthy install           -> exit 0 (warn)
  * missing git-remote-gitlawb                         -> exit 1

* fix(gl): pin doctor exit wiring and GITLAWB_NODE Warn tier

- Extract GITLAWB_NODE env classification into gitlawb_node_env_check()
  so unit tests can pin the Warn tier directly; the previous
  has_failures-only tests built their own Check::warn rows and could not
  see a re-tier back to Fail.

- Add three unit tests for the extracted helper (unset/empty is Warn,
  set is Pass, loopback is Pass).

- Add integration probe crates/gl/tests/doctor_exit.rs that drives the
  real binary with a temp --dir and --node http://127.0.0.1:1 and asserts
  exit 1. Deleting the std::process::exit(1) gate keeps the unit suite
  green but breaks this probe, closing the gap reviewer noted.

Co-authored-by: review fix for #391

* fix(gl): align GITLAWB_NODE operator copy with PUBLIC_NODE

* fix(gl): dial the public node when GITLAWB_NODE is empty

Check 3 groups an empty env with unset (Warn), but clap still binds
args.node to "" and the connectivity probe dialed it verbatim, so a
GITLAWB_NODE="" install showed a Fail row and exited 1. effective_node_url
normalizes empty or whitespace-only input to PUBLIC_NODE before the
probe so empty matches unset on check 4.

---------

Co-authored-by: Ayush7614 <[email protected]>
2026-11-08 04:42:15 +00:00
0189990ac9 fix(node): bound federated repository aggregation (#546)
* fix(node): bound federated repository aggregation

* fix(node): propagate federated peer truncation

* fix(node): Bound federation latency and preserve truncated peer pages.

* fix(node): complete federation review follow-ups

Register the bounded peer-query fixture in the writer ledger, name federation bounds, and share the state-owned limiter across routers and the cleanup sweep. Cover exhausted-bucket rejection before database access.

* fix(node): pin federated response contract end to end (#466 review)

Address beardthelion's P2: the federated envelope was only exercised at
unit level, so hardcoding "truncated": false, deleting the failed-peer
truncated flag, or dropping node_did from enrich_federated_repo each left
all nine federated tests green.

Add four route tests driving build_router and asserting on the real HTTP
200 body:
- envelope fields (count, nodes_queried, truncated) and local annotations
  (node_url, node_did, local) for a seeded public repo
- truncated == true when the peer table overflows MAX_FEDERATED_PEERS
  (unreachable peer URLs, no live peer needed)
- truncated == true on the failed-fetch path with the table under the bound
- peer annotations (node_url, node_did, local: false) via a mockito peer

* fix: strengthen federated response bound tests per review

- db/mod.rs: add writer-ledger entries and disposition docs for the
  three federated route tests touching the peers table
- api/repos.rs: seed the envelope test repo under a non-node owner DID
  so node_did must equal the actual node DID; pin the outbound peer
  query to exact limit=200&offset=0; make the peer-table-overflow test
  use a reachable stub for every peer so the count bound is the only
  possible reason for truncated=true, and assert nodes_queried

---------

Co-authored-by: euxarist <[email protected]>
Co-authored-by: Marlowe <[email protected]>
2026-11-08 02:24:53 +00:00
beardthelion 61d8a4f8f1 fix(core): compute created-skew in checked arithmetic (#358) (#545)
created= is attacker-supplied i64 with no range clamp. now - created
overflows on i64::MIN (panic in debug, wrap in release), and the wrap
case made abs() return i64::MIN, which is not > 300, so a skew of 2^63
seconds read as fresh. Treat an unrepresentable difference as the
largest possible skew.
2026-11-07 23:24:17 +00:00
b33562f1a9 fix(node): bound Identify-derived Kademlia addresses (#544)
* fix(node): bound Identify-derived Kademlia addresses.

* fix(node): preserve refreshed Identify addresses.

* fix(node): Log rejected known-peer addresses for diagnosis.

* fix(node): bound Identify report processing and refresh eviction

Avoid retaining peers with empty or rejected address reports, bound canonicalization to a fixed input prefix, and move refreshed addresses behind stale addresses in global eviction order. Add regressions for each bound and below-cap growth.

* fix(node): address CodeRabbit majors on Identify address bounds

- Reject Identify address admissions once the global address budget
  is full instead of evicting another peer's entry. Cross-peer
  eviction let a flood of new identities push honest peers'
  addresses out of the book and, when the evicted address was their
  last one, out of the Kademlia routing table. Per-peer eviction of
  the reporting peer's own addresses is unchanged.
- Re-add refreshed Identify addresses to Kademlia, not just newly
  added ones. Kademlia may have dropped a retained address after a
  failed dial or never kept it for a full k-bucket, and a TTL
  refresh alone would never re-admit it.

* fix(node): address #462 review findings from beardthelion

- remove the write-only insertion_order bookkeeping (field,
  remove_insertion_token, and its call sites) and collapse the repeated
  peers.get/get_mut lookups in IdentifyAddressBook::update into a single
  entry binding
- drop a peer row at the end of update() when every candidate was
  rejected, so zero-address rows no longer linger until the expiry tick
- add a regression test proving per-peer eviction still applies at full
  global budget (fails if the global check is hoisted above eviction)
- extract the explicit-address retention predicate into the tested
  is_explicit_address helper used by both Kademlia removal sites

* fix(node): pin row-removal and expiry guards in Identify address book

Address beardthelion's round-two P3 findings on #462:

- Add test that a fully-rejected fresh peer at full global budget
  leaves no peers row (pins the empty-row drop in update()).
- Add test that update() past IDENTIFY_ADDRESS_TTL reports the stale
  address in changes.removed (pins expiry reporting).
- Add test that expire() drops rows emptied by expiry (pins the
  sibling drop in expire_peer).
- Distinguish per-peer from cross-peer eviction in the saturation
  test: insert a sentinel peer before the victim and assert its
  address survives, so a global-oldest eviction policy would fail.

* fix(node): pin refresh predicate negative direction per beardthelion round 3

---------

Co-authored-by: euxarist <[email protected]>
Co-authored-by: Marlowe <[email protected]>
Co-authored-by: cairn-intern <[email protected]>
2026-11-07 22:49:33 +00:00
beardthelionandGravirei d3845ced69 fix(node): gate /ipfs/pins and /arweave/anchors behind authentication (#121) (#543)
* fix(node): gate /ipfs/pins and /arweave/anchors behind authentication (#134)

Add auth rejection to list_pins and list_anchors handlers. The pin index
spans the entire node and would expose metadata for every object ever
pushed; anonymous callers must not see it.

- Require AuthenticatedDid extension in both handlers, return 401 when absent
- Add server.rs regression test for anonymous rejection through build_router
- Fix closed-pool tests to pass authenticated requests (test 503 path)

* fix(node): enforce authentication on /api/v1/ipfs/pins and /api/v1/arweave/anchors routes

* fix(node): format arweave routes for consistency in build_router function

* fix(node): gate scoped /api/v1/arweave/anchors reads on repository visibility

Closes the review follow-up on #134: a signed non-reader could pass the
auth-layer check on `?repo=<owner>/<name>` and obtain private-repo anchor
metadata (ref names, old/new SHAs, CIDs, irys tx ids). The auth check
proved only that the signature was valid; it did not bind the request to
the canonical repo-read policy.

* Parse the slug through `validate_repo_slug` (same helper the sync path
  uses) and call `authorize_repo_read` before the SQL query. Missing
  repos, quarantined mirrors, and signed non-readers all collapse to the
  standard 404 (`repo_not_found`).
* The unscoped global listing stays auth-only (#121 narrowing; the wider
  permissionless-identity enumeration is the #136 class, out of scope).
* Clamp the limit to `[0, 200]` so `?limit=-1` no longer reaches Postgres
  as `LIMIT -1`.

Five new `build_router` regressions cover the full production contract
through the real `optional_signature` middleware: scoped unsigned 401,
scoped owner 200, scoped non-reader 404 with no anchor metadata leaked
in the body, scoped missing-repo 404 indistinguishable from non-reader,
and `?limit=-1` clamped to an empty 200.

* fix(node): keep /api/v1/ipfs/pins out of the CID resolver's per-IP flood brake

Follow-up to #134: the previous reordering put `/api/v1/ipfs/pins` and
`/ipfs/{cid}` under the same `rate_limit_by_ip` + `IpRateLimiter` layer
stack, so both routes shared the resolver's flood brake bucket. That
bucket is documented as the once-per-request brake for `GET /ipfs/{cid}`
because the resolver can drive bounded-but-expensive repo walks. The
pin listing is a single `list_pinned_cids()` call and has no reason to
share the bucket. Sharing meant /ipfs/{cid} traffic could exhaust the
bucket and 429 the pins endpoint, and signed pin polling could exhaust
the bucket and 429 legitimate CID reads.

Compose `ipfs_routes` from two sub-routers with independent policies,
then merge:
  * `/ipfs/{cid}` keeps `optional_signature` + `rate_limit_by_ip` + the
    `IpRateLimiter` extension. Same behavior as before #134.
  * `/api/v1/ipfs/pins` gets only `optional_signature`. Auth-required
    (handler denies with 401), but no rate limit — it has nothing to
    flood.

Two new `build_router` regressions exercise the bucket independence
through `TrustedProxy::XForwardedFor`:
  * 5 signed pin requests against a size-2 CID bucket all 200, then a
    `/ipfs/{cid}` from the same IP also 200 (pins did not debit the
    bucket).
  * A `/ipfs/{cid}` exhausts a size-1 CID bucket (200, then 429), then
    a signed pin from the same IP is 200 (the bucket does not gate
    pins).

All 65 tests in `server::tests::` + `test_support::tests::ipfs_*`
still pass, including the existing 429 tests for the CID resolver.

* fix(node): normalize scoped anchor SQL filter to stored repo slug

After authorize_repo_read succeeds, the SQL filter used the raw ?repo=
string (format!("{owner}/{name}")). Anchor rows are stored with
normalize_owner_key(owner_did)/name (short key), so a caller using
?repo=did🔑.../name passed authz but got anchors: [] — a false
empty page that broke callers that legitimately pass the full DID
form. Build the filter from record.owner_did normalized and
record.name instead, matching the shape the seeder uses.

* All three findings addressed on bug_fix_2:

  P2 — signed_scoped_anchors_full_did_form_matches_stored_slug_through_build_router now seeds
  a second repo under a different owner and asserts irys-other-repo-tx is absent. Negative
  control: flipping the production filter to None made the test go RED with count: 2 showing
  the cross-repo leak.

  P3 — let short = owner_did.split(':').next_back().unwrap() replaced with let short =
  crate::db::normalize_owner_key(&owner_did).to_string(). Reader and writer now share one
  convention.

  P3 — PR #134 description updated: pins bullet now describes the actual pre-merge layering
  (and cites the axum-layer-vs-merge-pitfall memory entry), and round 2's stored-slug
  normalization is added as its own bullet with the regression's coverage.

  5/5 anchor-related tests pass, cargo fmt --check clean.

* fix(node): pass SignatureBodyLimit state to optional_signature layers

* test(node): complete listing-denial regressions for the new gates

---------

Co-authored-by: Gravirei <[email protected]>
2026-11-07 21:46:53 +00:00
beardthelionandGravirei bac49f150a fix(node): owner-gate create_task repo_id against hosted repos (#541)
* fix(node): owner-gate create_task repo_id against hosted repos (#496)

* fix(node): address review on #496 gate — opaque lookup errors, no-persist asserts, GraphQL arm pins, honest oracle comment

* fix(node): address P3 review on #496 gate — shared denial text, honest oracle wording, db-fault pins

- Give the REST and GraphQL copies of the create_task owner gate one denial
  sentence: add REPO_OWNER_ONLY_MESSAGE, used by require_repo_owner and the
  REST forbidden() envelope, and pin it from both surface tests.
- Carry the corrected one-bit-oracle wording into the GraphQL gate comment and
  the REST test docstring (drop the "oracle-free" claim).
- Fault-inject the gate's repo lookups: drop machine_id (get_repo_by_id) for
  REST and GraphQL, and quarantined (is_repo_quarantined) for REST, asserting
  the fixed opaque db_error envelope and no schema text.

* docs(node): stop claiming caller-scoped task reads in #496 gate comments

The create_task gate is a write gate. Unknown/quarantined repo ids are stored
verbatim, and the REST/GraphQL task-list paths are caller-independent on this
base, so the comments must not imply those ids are read-scoped to the
delegator/assignee. Describe only the enforced ownership check; note the
one-bit oracle split without asserting a read contract.

* fix(node): mirror AppError's 503 split and opaque INSERT in create_task

- db_error now sends pool-class failures to 503 db_unavailable like
  AppError::Db (db_unavailable made pub(crate)); query faults stay the opaque
  500. Unit tests cover the split and context-wrapped sqlx errors.
- Route the create_task INSERT through db_error instead of serializing
  e.to_string(); a REST fault-injection test pins the opaque envelope.
- The deny/allow matrix reads back the unknown and quarantined tasks to prove
  the id is stored verbatim, and drops no-leak asserts on the fixed forbidden
  envelope that could never carry the repo id.

* test(node): pin stored-verbatim repo_id on GraphQL gate arms

The REST deny/allow matrix reads each accepted task back and asserts the
stored repo_id is the supplied id; the GraphQL unknown/quarantined arms only
asserted no errors, so a GraphQL regression that dropped, normalized, or
rebound the id stayed green. Read those tasks back by the mutation's returned
id and assert the stored label matches.

* test(node): pin createTask repo-ownership marker in the GraphQL authz fence

---------

Co-authored-by: Gravirei <[email protected]>
2026-11-07 21:31:50 +00:00
beardthelionandGravirei b4a3122500 fix(core): reject zero threshold, canonicalize did:key, document replay (#509) (#540)
* fix(core): reject zero threshold, canonicalize did:key, document replay (#509)

* test(core): drop error echoes from assert messages flagged by CodeQL

---------

Co-authored-by: Gravirei <[email protected]>
2026-11-07 15:48:05 +00:00
beardthelionandeuxarist 0cdb4bcc7c fix(api,sync): Delimit positional arguments in git clone mirror invocations (#539)
* fix(api,sync): Delimit positional arguments in git clone mirror invocations

Pass "--" before positional path/URL arguments in Command invocations
for "git clone --mirror" across repos.rs, sync.rs, and gl mirror.rs.

Fixes #374

* style(gl): Format args array in mirror.rs

Fix cargo fmt line-length break for git clone --mirror arguments.

Refs #376

* fix(node): Pin the git clone delimiter against remote-argument injection

The delimiter change had no test, so nothing stopped it being reverted as
cosmetic. What it guards is real: git parses a remote beginning with a
dash as an option, and --upload-pack=<cmd> is one git hands to a shell.

No path reaches that today. Peer URLs are gated by is_public_http_url, so
the origin clone_repo composes can only begin with http:// or https://;
fork paths are joins under repos_dir; and clap rejects a leading-dash
positional in gl mirror. The delimiter is what removes git's dependence
on those three unrelated gates staying correct.

Assert git reports the whole argument as a repository it cannot find,
which is only true once the delimiter forces it to be a path.
Undelimited, git consumes it as an option and fails against the
destination instead, never naming the injected string, so the assertion
is red before the fix and green after.

Refs #374

* fix(sync,gl): Delimit positional arguments in fetch_repo and gl clone

Carry the positional argument delimiter into fetch_repo when setting the
remote origin URL and into setup_partial_clone across all clone branches.
Correct the error string quoted in the clone regression test docstring.

Refs #374

* test(gl,node): Pin the argv delimiter on every git sink this PR changed

The delimiter contract is duplicated across five direct `git` argv
constructions, but only `sync::clone_repo` had a test that went red when
`--` was deleted. Removing or misplacing the delimiter in any other sink
left the suite green, so the hardening rested on review rather than on
the tests.

Each changed sink now has a subprocess-facing regression driven through
real git: the three `setup_partial_clone` shapes (plain, `--branch`, and
the sparse/promisor arm), `fetch_repo`'s `remote set-url`, `gl mirror`,
and the fork clone.

Two of these could not carry the obvious assertion:

- `git_global` and `git_run` both format failures as
  `git {args:?} failed: {stderr}`, so the injected value is already in
  the message via the debug-printed argv. Asserting that the error
  merely contains it passes with `--` deleted. The clone tests assert on
  git's own single-quoted `repository '<value>'` instead, which only
  appears when git read the value as a repository.
- `remote set-url` does not fail on a delimited option-shaped value, it
  stores it. The `fetch_repo` test asserts the stored URL rather than an
  error: undelimited, git exits 129 with `unknown option` and leaves the
  previous URL in place.

`gl mirror` and `fork_repo` needed their argv extracted to be testable.
`mirror::run` loads a keypair and contacts a node before reaching the
clone, and bails with a message that interpolates the source whether or
not the delimiter is present. The extraction keeps the production
`.status()` call, so a large mirror clone still streams git's progress
to the terminal instead of being captured.

Every one of the five was confirmed red with its delimiter removed and
green with it restored.

Refs #374

* fix(gl): Route the last three clone warnings through emit_warning

`emit_warning` sanitizes a line before writing it, which strips ANSI
escape and bidi characters from untrusted text. Three warning sites in
the blob-recovery paths still used a bare `eprintln!` while interpolating
`oid`, which comes from node JSON or an Arweave gateway manifest, so
those escapes reached the user's terminal raw.

Refs #374

* style(gl): elide the lifetime in mirror_clone_args

* test(gl,node): pin LC_ALL=C on git-subprocess stderr assertions

---------

Co-authored-by: euxarist <[email protected]>
2026-11-07 12:38:49 +00:00
0e5a1608e2 feat(node): persist upstream mirror configuration and transition state (#538)
* feat(node): persist upstream mirror transition state

Origin-Session: local-d6a143 | Codex | 12 prompts

* fix(node): reject null mirror transition state

Origin-Session: local-d6a143 | Codex | 14 prompts

* fix(node): address mirror state review findings

Origin-Session: local-d6a143 | Codex | 16 prompts

* fix(node): close mirror review blockers

Origin-Session: local-d6a143 | Codex | 20 prompts

* fix(node): align peer URL validation message

Origin-Session: local-d6a143 | Codex | 20 prompts

* fix(node): close remaining mirror review findings

Origin-Session: local-d6a143 | Codex | 21 prompts

* test(node): track the Db pool field change in the mirror-state migration tests

* fix(node): revalidate stored webhook URLs at delivery and cover the subdomain repeated-dot case

* docs(node): document the invalid_url webhook delivery outcome

---------

Co-authored-by: KRATOS <[email protected]>
Co-authored-by: Kevin Codex <[email protected]>
2026-11-07 03:50:15 +00:00
beardthelionandGravirei 37947208c5 docs: state that GITLAWB_ENFORCE_OWNER_PUSH defaults to true (#537)
* docs: state that GITLAWB_ENFORCE_OWNER_PUSH defaults to true

PR #330 flipped the code default to true and updated .env.example, the
README env-var table, the AWS guide, the macOS app, and docs/RUN-A-NODE.md,
but two prose mentions in operator-facing docs still describe the
opposite. SECURITY.md line 57 ("Repository write authorization defaults"
section) and README.md line 69 (Known limitations) both say the flag
"defaults to `false`", which contradicts the live runtime and the rest
of the same files. Reviewer 2 closed #224 with this as the P2 finding
to fix in the config-and-policy split.

The correct text already exists in the README env-var table; the prose
restatements were missed. Bring the prose into line with the code.

- SECURITY.md: rewrite the section to lead with the default being true
  and describe `false` as the explicit rolling-upgrade override.
- README.md: rewrite the known-limitations bullet to state the default
  is true and the `false` opt-out is a rolling-upgrade only.
- Add `crates/gitlawb-node/tests/owner_push_default_docs.rs`: a
  regression guard that reads the four surfaces (clap default, .env.example,
  README known-limitations prose, README env-var table, SECURITY.md) and
  asserts each one says the default is `true`. Reverting either doc edit
  turns one named test red; flipping `default_value_t` back to `false`
  turns the env example test red. Each surface is checked independently
  so a stale doc lies alone.

No code change. No new migration. No dependency change. Compiles clean,
clippy clean under `-D warnings`.

Refs: closes the P2 finding from the review of #224.

* fix(docs): clarify default value consistency for GITLAWB_ENFORCE_OWNER_PUSH

* docs(readme): drop the default-open roadmap line and the em-dash punctuation

---------

Co-authored-by: Gravirei <[email protected]>
2026-11-07 00:38:41 +00:00
beardthelion 5ed3381a6b fix(gl): truncate node-supplied display strings at a char boundary (#350) (#536)
* fix(gl): truncate node-supplied display strings at a char boundary (#350)

Every timestamp/id/SHA the CLI prints was cut with &s[..N] or
&s[..N.min(s.len())]. The first panics on a short value, the second
still panics when byte N is inside a multi-byte char. A shared
text::truncate cuts at a char boundary and covers all sites, including
two the issue did not name (node events, agent did) and the same shape
in bounty, changelog, ipfs pin listing, and the merge-commit print.

* fix(gl): bound truncate by bytes, not chars (#460)

Address PR review feedback: char_indices().nth(max) capped characters,
so a multibyte char near the cut could exceed the byte limit. Start at
max.min(len) and walk back to the nearest char boundary instead.

* fix(gl): bound the bounty leaderboard did suffix at a char boundary
2026-11-06 20:38:05 +00:00
beardthelion 9b915b09a7 fix(attest): bound did:key method-id length before base58 decode (#360) (#535)
gitlawb-core added the same guard in 73bf132c; the second copy of the
parser in gitlawb-attest never got it, so an untrusted signer string of
arbitrary length reaches a decoder whose cost is quadratic in input.
2026-11-06 18:14:06 +00:00
77ac1037cb fix(icaptcha-client): parse GITLAWB_ICAPTCHA_INSECURE as truthy only (#227) (#533)
* fix(icaptcha-client): parse GITLAWB_ICAPTCHA_INSECURE as truthy only

Presence checks treated =0/=false/empty as enabled. Only explicit 1/true
now enable loopback HTTP trust relaxation (#227).

* fix(icaptcha-client): address #246 review (trim test, fmt, lock)

Pin trim with a whitespace truthy case, recover from a poisoned
ICAPTCHA_ENV_LOCK, document trim + warn visibility, and cargo fmt.

* fix(icaptcha-client): do not log raw ICAPTCHA_INSECURE value

The non-truthy warn path previously recorded the full trimmed env
value. That is arbitrary deployment input; report only that the
setting is invalid (#246 review).

* fix(icaptcha-client): pure insecure flag helpers for #246 review

Warn on present non-Unicode GITLAWB_ICAPTCHA_INSECURE (fail closed).
Pass insecure mode into is_https_with / resolve_solver_url_with so
tests no longer mutate process-global env (removes InsecureEnv race).

* fix(icaptcha-client): cover real env read + fix warn spacing

Add a serialized test that drives icaptcha_insecure_enabled() against
the process env so presence-check / typo / always-true mutations fail.
Collapse the mid-sentence space run in the non-truthy warning (#246).

* fix(icaptcha-client): put the non-truthy warning on one literal line

---------

Co-authored-by: Ayush7614 <[email protected]>
Co-authored-by: Kevin Codex <[email protected]>
2026-11-06 17:26:16 +00:00
db98b3957a fix(attest): make verify_all lenient under RequireAll (#359) (#532)
* fix(attest): make verify_all lenient under RequireAll (#359)

Registry::verify_all propagated every per-attestation error via ?, so a
single malformed or unknown attestation aborted the whole batch. Under
Policy::RequireAll that is a DoS vector: any third party can attach an
attestation, so an unrelated attacker/spam/v1 could block an otherwise
valid cert.

Under RequireAll, drop entries that fail to verify instead of aborting;
the required_types check remains the gate and only counts fully_verified
entries, so a required type present only as a malformed attestation still
fails with RequiredMissing. AcceptKnown and RejectUnknown stay strict.

Adds tests for both: a malformed extra is dropped without aborting, and a
malformed-only required type is still rejected.

Closes #359

* test(attest): lock the strict batch policies with an error-propagation test

AcceptKnown and RejectUnknown were only exercised per-attestation, so a
cleanup applying RequireAll's filter_map(Result::ok) shape to the strict
arms would fail open with every test green. Drive both policies through
verify_all with a wrong-cert-hash member and assert the batch surfaces
the underlying error. Also correct the RequireAll doc: unknown valid
types are retained as unverified, not dropped — only real Err paths are.

---------

Co-authored-by: andreolf <[email protected]>
Co-authored-by: Kevin Codex <[email protected]>
2026-11-06 16:54:42 +00:00
beardthelionandAyush7614 8c9e4adc2d fix(node): canonicalize and 404-on-no-match in remove_label (#344) (#530)
* fix(node): canonicalize and 404-on-no-match in remove_label (#344)

add_label stored labels trimmed and lowercased, but remove_label passed the
path segment through as-is, so a label stored as 'bug' did not match
DELETE /repos/{owner}/{repo}/labels/Bug. Worse, remove_label's DB call
discarded the row count and the handler always returned 200 with
{removed: true}, including when the delete matched nothing — the
no-op-rendered-as-success shape the rest of the API rejects.

Move trim+lowercase and charset/length validation into a shared
canonicalize_label helper used by both handlers, and have
db::remove_label return the affected row count so the handler can return
404 when no label was deleted. add_label's behavior is unchanged
(idempotent semantics preserved); the only externally observable change is
that mixed-case and surrounding-whitespace path segments now match the
stored label, and a delete that matches no row returns 404 instead of
{removed: true}.

Tests:
  * remove_label_canonicalizes_and_404s_on_no_match exercises the
    mixed-case path, the trimmed path, the empty-after-trim rejection,
    the disallowed-charset rejection, and the no-op-returns-404 case.
  * remove_label_denies_non_owner re-pins the 403 owner-gated shape
    (matches AGENTS.md: owner-gated mutations return 403 after the repo
    lookup, distinct from the not-found shape).

* test(node): pin len>50 arm in remove_label canonicalize

The helper's length branch had no coverage — 51 chars must 400 and
50 chars must pass validation (404 when absent). This pins the arm
lifted into canonicalize_label and decouples the no-match 404 from the
double-delete above, as requested in review of #390.

---------

Co-authored-by: Ayush7614 <[email protected]>
2026-11-06 15:08:34 +00:00
beardthelion 8eedb935c3 fix(node): gate bounty mutations on repo read before status checks (#529)
* fix(node): gate id-keyed bounty mutations on repo read before status checks

submit, approve, cancel, and dispute on /api/v1/bounties/{id}/* looked the
bounty up and ran status and participant checks without gating on the
bounty's repo read visibility. A signed stranger holding an id got a
distinguishable 400/403 on a private-repo bounty where an absent id gets
404, so the routes were an existence and lifecycle oracle.

Insert the same authorize_repo_read block get_bounty and claim_bounty
already carry, right after the lookup, so a denied caller sees the
not-found shape either way. Participant checks are unchanged; on a public
repo a stranger still reaches them.

Closes #341

* Address PR review feedback (#448)

- assert the exact not_found body shape for both held and absent bounty
  ids in the mutation-gate helper, so a denial carrying lifecycle or
  participant detail fails the test
2026-11-06 15:06:01 +00:00
beardthelion 47c6e6ce22 fix(node): fail pin_git_object when the add response carries no Hash (#528)
* fix(node): fail pin_git_object when the add response carries no Hash

A 200 from /api/v0/add without a parseable Hash fell back to the
caller-computed raw CID, recording an address the backend never
confirmed stored. For payloads above the backend's chunk threshold
(262144 bytes on Kubo) that raw CID is not even a stored block, so
every later /api/v0/cat on it fails permanently: get_encrypted_blob
errors on every fetch and mirrors retry the envelope on every sync.

A Hash-less 200 is now an add error carrying a truncated body excerpt.
The delaying_endpoint test fixture, which relied on the fallback by
returning an empty 200 body, now answers with a conformant NDJSON line.

* fix(node): reject a non-CID Hash in the add response

Address PR review feedback (#453): a string-valued Hash that does not
parse as a CID would be recorded into encrypted_blobs.cid and fail on
every later cat, the same failure class as a missing Hash. Validate the
returned Hash before accepting it and cover the malformed case with a
regression test.

* test(node): return real CIDs from the add-response test mocks

The pin_git_object CID validation rejects non-CID Hash strings, and a
dozen test_support fixtures answered 200 with placeholder values like
"bafyprovider" or "x". Replace them with a valid CID so the fixtures
model a healthy backend; no test asserts on the literal value.

* fix(node): bound the invalid Hash echoed in the add error
2026-11-06 06:32:13 +00:00
beardthelionandMystic 317a1550c2 fix(node): scope bounty aggregates to anonymously-listable repos (#477) (#527)
* fix(#477): scope bounty aggregates to anonymously-listable repos

bounty_stats and agent_bounty_stats ran unfiltered aggregates over all
bounties, leaking private-repo bounty activity to anonymous callers.

Mirror the stats() pattern from server.rs (#104):

1. Batch-load all deduped repos + visibility rules (2 SQL round-trips)
2. Filter to listable_at_root(rules, is_public, owner_did, None)
3. Pass visible (owner, name) pairs into SQL aggregates via EXISTS/unnest
   with owner key normalization on both sides (BOUNTY_OWNER_CASE_SQL).
4. Normalize claimant DID in agent_bounty_stats_visible and group by
   normalized claimant in bounty_leaderboard_visible (BOUNTY_CLAIMANT_CASE_SQL).
5. Remove orphaned unfiltered aggregate methods to pass dead_code lints.
6. In tests, seed completed bounty claimant state via the real claim/approve
   path, and verify root-rule-decided visibility overrides.

Aggregates are intentionally anonymous-scoped for all callers (matching the
#104 stats pattern).

Fail-closed: DB errors collapse the visible set to empty, so all counts
return 0 — an under-count never leaks existence.

Fixes #477

* style(node): apply rustfmt to the bounty visibility change

* test(node): pin bounty aggregate visibility for mirror rows

* docs(node): record why mirror rows are handled in the bounty visibility filter

* perf(node): keep the claimant index usable in per-agent bounty stats

* fix(node): return DB errors from bounty stats instead of zero counts

---------

Co-authored-by: Mystic <[email protected]>
2026-11-06 03:06:15 +00:00
JATMNandClaude Opus 5.5 30637193a6 fix(node): resolve issue ids from git refnames, not request input (#526)
* fix(node): resolve issue ids from git refnames, not request input

resolve_issue_id answered an exact match by echoing the request-path id
back to callers, so get_issue and close_issue spliced caller input into
`git cat-file blob refs/gitlawb/issues/<id>`. #521 restricted the input
to [A-Za-z0-9-], but the value handed to git was still the request
string (CodeQL rust/command-line-injection #83).

Resolve exact and prefix lookups through one for-each-ref listing and
return the refname git reports. An exact refname still wins over longer
ids that share it as a prefix; unique-prefix, ambiguous, not-found, and
revision-syntax behavior is unchanged.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix(node): keep a dangling exact issue ref from winning over longer ids

for-each-ref lists a ref even when its blob is missing, so the exact-id
precedence added in the previous commit picked a dangling exact ref over a
readable longer sibling: resolve_issue_id returned the dangling id where
main reported an ambiguous prefix, get_issue then returned None, and
close_issue hit its expect().

Let the exact id win only when `git cat-file -e` finds its object, run on
git's own refname so no request input returns to the argv. Otherwise fall
through to the unchanged prefix rules, matching main for every layout.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

---------

Co-authored-by: Claude Opus 5.5 <[email protected]>
2026-11-06 03:04:20 +00:00
Kevin Codex 6b0763ac2a fix(api): make the push-event cursor opaque and bound to its repository
The cursor was the table-global bigserial, handed to clients verbatim and
accepted back as any non-negative integer, then passed straight into a
per-repo seq > $1. Two things followed from that, and both were silent.

A value issued by a DIFFERENT repository — or by this one before a
restore — is an ordinary larger number here, so the query matched
nothing, the surface answered 200 with an empty page and echoed the value
back, and the subscriber sat permanently past history it had never
received with nothing anywhere reading as an error. And because
repo_push_events is written by every repository on the node, the gaps
between one repository's cursors measured how much the others pushed in
between, private ones included.

The cursor is now an opaque token carrying a version, a binding to
(node DID, repository), and the id of the last event served. The binding
is a hash and not a MAC, because there is nothing to forge: the position
inside is a row id resolved against this repository's own rows anyway.
What it buys is that a foreign cursor is refused by shape, with a message
naming which mistake was made, instead of being applied to a repository
it was never issued for. An id that resolves to nothing — a token kept
across a restore that lost the row — is a 400 that names the recovery,
not an empty page the poller reads as being up to date.

The global sequence still orders the walk internally, which is what keeps
the page cheap, but it no longer leaves the node. The position is the row
id, which this surface already publishes in every event, so the token
discloses nothing new.

The two properties an earlier round established are kept, in their new
shape: next_cursor is never null and never moves backwards, and the start
of history is an expressible position rather than an absent one, so a
first poll of a repository with no events still hands back a token the
endpoint accepts back rather than a value a persisting poller would
misread.
2026-08-31 21:56:23 +08:00
Kevin Codex 8652b86d33 fix(api): apply a push's side effects only to the refs git accepted
receive-pack exits zero when the process ran, not when the push applied.
A non-fast-forward, a hook rejection or a failed update comes back as an
ng pkt-line inside a response this handler forwarded without reading, so
every side effect keyed on the refs parsed from the REQUEST fired for
refs that were refused: the pull request on that branch had its stored
head moved to a commit this node does not have, and the catch-up poll
surface handed subscribers a SHA that resolves to nothing.

The report is now parsed and only the refs it names with ok are used.
unpack fail accepts nothing, since the pack never landed. A report that
does not parse at all is inconclusive and keeps every declared ref: a
client that never requested report-status is told nothing about
individual refs, and reading that silence as rejection would permanently
drop the events and the head update for a push git really did accept,
which is the failure the poll surface exists to prevent.

The parser is ported from PR #384, which introduced it for this same
problem, so the two branches converge on one definition of "accepted"
rather than growing two. Whichever lands second should drop its copy. It
diverges by three lines, and they are load-bearing: git double wraps the
report when the client negotiates side-band-64k, which git push over
smart HTTP does. The observed framing from git 2.50.1 is an outer
pkt-line carrying a band byte whose payload is itself a pkt-line stream,
so one pass of strip_sideband leaves "000eunpack ok" as the first line,
the parse fails, and every ref falls back to inconclusive — the parser
answering "cannot tell" for exactly the pushes it exists to classify. A
second pass peels the inner layer and is a no-op on the single-wrapped
shape. The tests carry the real captured byte layout, both wrappings, so
this is pinned rather than assumed.

Both writes also moved inside the repository write lock. That lock orders
the ref updates of two concurrent pushes; released before these ran, the
pair was an unordered race, and B's unconditional UPDATE landing before
A's leaves the stored head — which the rollup prefers over its fallback —
pinned to a commit the branch has moved past. record_push_events has the
same inversion by a different route, since latest_push_sha_for_ref reads
the highest seq and seq order is insert order. There is no per-row
version to make the UPDATE conditional on, so the ordering has to come
from the lock. Holding it across two batched statements on an already
open pool, on a path that just ran a subprocess over the whole pack, is
the cheaper half of that trade. A source guard pins the position, because
this is a property of where the calls sit.

Separately, the 10,000-ref bound was checked only after the parser had
scanned the entire request and kept three heap strings per valid
pkt-line. The git routes raise the body limit to GITLAWB_MAX_PACK_BYTES —
2 GB by default — so a signed caller could force the whole scan and the
whole allocation and only then be told 400: the work the cap exists to
refuse, done in full before the refusal. The parser stops one past the
cap, which is still enough for bound_declared_refs to refuse on, and the
message no longer reports a count that would now be a floor rather than
the request's real one.
2026-08-31 21:56:07 +08:00
Kevin Codex a4e2b9f3f3 fix(status): bind a stored claim to the body its signature covered
A claim is provenance, so the signature stored beside it has to reach the
bytes stored beside it. It did not. The signing string covers the body
only through a content-digest line, and require_signature rebuilds the
covered values from the request: with no Content-Digest header the
covered digest is the empty string on the signing side and the verifying
side alike, the Ed25519 check passes over method, path and nothing else,
and the digest-versus-body comparison was skipped for want of a header.
An owner could sign that once and post any body under it. The row that
produced carried a valid signature, a signing string that verifies under
the owner's key, and a request_body the signing string says nothing
about — history nobody can re-verify, which is the one thing the row is
for.

The write path now recomputes the digest of the bytes it is about to
persist and requires the signing string to carry exactly that line, which
is the write-time form of the check re_verify already ran against a
stored row. Line-exact rather than a substring search: build_signing_string
emits one component per line, and a contains would also be satisfied by
the digest appearing inside another component's value.

This is deliberately not the only line of defence, and deliberately not
the middleware's. PR #306 requires the header in require_signature and
closes the same hole one layer earlier for every signed route; that
belongs there, because no other route's correctness should depend on this
handler. Neither substitutes for the other. Without #306 the empty digest
is still accepted everywhere else; without this, a later relaxation of the
middleware resumes writing unverifiable history in silence. Nothing here
touches require_signature, so the two do not collide.

The route's advertised 8 KiB body bound was also a bound on what gets
stored rather than on what gets buffered: require_signature collects and
hashes the whole body before any handler runs, so by the time the handler
measured it the allocation had happened, and concurrent requests
multiplied exactly the cost the number was written to prevent. A
RequestBodyLimitLayer now carries the same figure at the transport,
outside the auth layers and inside the rate limiters, matching the shape
the git routes already use. The handler keeps its own check: it is the one
that holds for a handler reached by any other route table.

The hand-injected test material had to become coherent with the request
it rides on, because a stand-in built over other bytes now takes every
bare-router test down the refusal branch. It is built from the body it
accompanies and carries a real content-digest line.
2026-08-31 21:55:46 +08:00
beardthelion 61033af261 fix(auth): carry the request body only where a route persists it
The signature middleware handed every signed request a second handle to
the buffered body, and only the status write reads it. The field is
optional now and populated behind a marker the persisting route group
applies.

Be accurate about what that saves, because the first reading of this
finding overstated it and the plan was corrected before implementation.
The push handler takes its body as an extractor, so axum consumes the
request and drops every extension at that point; the extra handle lived
across the remaining middleware chain, not the handler's lifetime. The
whole pack really is pinned for the whole receive-pack, but by the
handler's own parameter, which is pre-existing and untouched here.

Layer order is the failure mode worth guarding. A marker applied inside
the auth layers is never seen by the middleware that reads it, so the
body silently stops being captured while every test stays green. The
mutation that proves this leaves the layer present on the same group and
only moves it inside, and the production-router test still reddens. The
handler also refuses an absent body rather than storing an empty column,
so a lost marker fails loudly at the one place it matters.

Four comments claimed a timestamp or a uuid tiebreak decided ordering
when the code orders on the sequence. Three were named in review; the
fourth turned up next to them, calling the timestamp format load-bearing
for a cursor that no longer reads it. The mirror-dedup comments that
correctly document timestamp ordering were checked and left alone.

The shared scrape helper advanced by one byte to stop an end anchor
matching at position zero, which reports a missing anchor for any region
starting on a multibyte character. It advances by a character now. Not
reachable today since every anchor is ASCII, but five guards share this
helper and one that silently finds nothing is worse than one that fails.
2026-08-08 11:07:10 -05:00
beardthelion 5445dda3f5 fix(db): stop the push cursor skipping a concurrently committed row
The sequence is a bigserial, so Postgres allocates it at INSERT and makes
it visible at COMMIT, and nextval does not roll back. Two overlapping
writes to one repo could therefore commit out of sequence order: one
allocates, another allocates and commits, a poller reads the later row
and advances, and the earlier row then becomes visible behind a cursor
that has already passed it. Never delivered, no error, no gap signal,
which is the one thing the catch-up surface exists to prevent.

This is the second time this branch has traded one ordering hazard for
another. The cursor moved off a wall clock because stamping and
committing disagree; it moved onto a sequence where allocating and
committing disagree.

The insert now runs in an explicit transaction that takes a per-repo
advisory lock, so the lock releases at commit and no writer can allocate
until the previous row is visible. The wait is bounded, because the lock
sits on a pooled connection on the inline push path and an unbounded
wait would turn one busy repo into pool exhaustion for unrelated
requests. Three attempts at a one second timeout leaves headroom under
the pool's own acquire timeout.

That bound needs the caller to stop swallowing failures. It warned and
continued, so any error the lock introduced would silently drop those
events forever while the push had already returned success, reaching the
same outcome by another route. It now retries once and then logs at
error level naming the repo, the refs, and the SHAs that were lost.

The defect is pinned by a test that owns both transaction boundaries
rather than a barrier. A barrier cannot reproduce it: before this change
the write was a single autocommit statement, so allocate and commit were
one round trip from the client and the interleaving window was never
open to a caller of the public API.
2026-08-08 10:07:47 -05:00
beardthelion d3074a6c86 fix(status): make a replayed write idempotent and stop dropping refs
Both defects came from an independent cross-model review, and neither
was visible to the reviewers that share this session's model.

A signed write was protected only by clock skew: no nonce, no
idempotency key, no record of signatures already seen. So a captured
request replayed inside the skew window inserted a new row with a new
sequence number, and because the projection takes the latest claim per
producer and context, a stale success outranked the failure that
superseded it. The append-only design is what made this reversible
rather than merely duplicative: a replay that only duplicated a row
would be harmless, a replay that earns a fresh sequence flips the
answer. An earlier reviewer saw the replay window and concluded
append-only bounded it, which had the mechanism exactly backwards.

An exact replay is now idempotent instead of an error. The row carries a
digest over the signature, its input, and the body, a unique index
enforces it in the database rather than a check-then-insert race, and a
repeat returns the original claim. Erroring would have punished the
legitimate case this design otherwise handles worst: a client whose
response was lost retrying the same signed request and concluding its
report failed when it had succeeded.

The fan-out cap was mine, added while fixing unbounded work, and it
truncated at 256 refs after receive-pack had already accepted them. A
push whose pull request source branch sat past that point left the head
permanently stale with no push event, announced only by a log line git
had already contradicted. The writers chunk now, so nothing accepted is
dropped, and the bound that remains refuses before the accept rather
than after: chunking bounds a statement, not a request, and the work
that scales with ref count sits upstream anyway, one protection query
per ref before the service runs and one certificate and webhook per ref
after, with body limits disabled on the git routes.

Removing the cap also exposed a quadratic branch dedupe it had been
keeping cheap; that is a map now.
2026-08-08 07:21:54 -05:00
beardthelion 612d9d2dbe fix(api): order the poll cursor on a sequence and batch the push writes
The push-event cursor paged on an application-stamped wall clock, so a
row stamped later could commit earlier and a poller past that point
would never see the earlier row. A clock step backwards widens it. It
now pages on a database-assigned sequence, the same decision
status_claims already made, so the two surfaces do not disagree about
what ordering means. The rollup's branch resolve had the same flaw and
is fixed with it.

The receive-pack path issued one round trip per ref for each of two
writers, sequentially, on the user's push, with nothing bounding the ref
count. Both are single statements now and the per-push fan-out is
capped, with a warning when it truncates rather than silent loss. The
head update batches through a VALUES join, with last-write-wins dedup on
duplicate branches: git will not produce them, but a join over
duplicates picks arbitrarily where the loop it replaced was
deterministic.

The cursor was unvalidated. A malformed value produced a wrong page
instead of a refusal, limit zero returned a page with a null cursor, and
a poller persisting that cursor silently restarted from the beginning of
history. It is validated, the limit is clamped, and the returned cursor
never moves backwards. This changes the query parameter names, which is
a breaking wire change on an endpoint that has never shipped; every
consumer was checked and only the route registration reads them.

Three copies of the source-scraping parser became one shared helper.
That refactor exposed a real gap: the empty-region mutation reddened
four of the five guards, but the gossip-containment guard passed on an
empty region because it only asserts a must-not. It now asserts its scan
covered what it claims to cover, so a helper bug cannot make it
vacuously green.

status.rs is split into status/mod.rs and status/tests.rs, production
logic unchanged. Four things read the old path and all were retargeted,
including 22 mutation targets across four specs, each re-confirmed to
still match exactly once.
2026-08-08 07:21:54 -05:00
beardthelion 806f9027c0 fix(status): make the caps hold and the projection mean one identity
Five reviewers independently flagged the capped insert. It counted then
inserted inside one transaction with a docstring promising a concurrent
writer could not slip past, which READ COMMITTED does not deliver, and
every cap test was sequential so deleting the transaction would have
kept them green. A per-repo advisory lock now serializes writers on the
bound they are both testing. The new test runs eight writers through a
barrier at cap minus one: without the lock it left 4, 5, and 5 rows
against a cap of 3 on consecutive runs, so the window was real.

The projection compared producer_did as a raw column while the handler
stored whatever spelling the signer used, so one owner writing as
did:key:X and as X produced two entries for one context and the
superseded claim kept voting. Identity is now canonicalized at write
time, and the read filter is a single equality rather than a set.

That fix had a trap. Applying normalize_owner_key directly, as the
finding suggested, collapses to the bare key, which is not a parseable
DID, and would have quietly undone the re-verifiability the previous
commit established. canonical_did canonicalizes to the full did:key
form instead and is written as a function of normalize_owner_key so the
equivalence with did_matches still holds exactly, including the case
where a naive prefix-add would merge two distinct identities.

With that in place the ported copy of the DID collapse in the status
module is gone rather than relocated, and did_matches itself now
delegates, so the Rust gate, the stored column, and the SQL all trace to
one definition.

The projection also selected the signature columns it never renders,
which the previous commit made worse by adding the request body. It has
its own narrow type now, enforced by a source-read test.

The per-repo cap counted for all time with nothing pruning the table, so
a repo that reached it was permanently closed while 429 told clients to
keep retrying. It is a rolling window now, which still bounds the
fan-out a caller-chosen SHA allows, and makes the refusal true.

Also folded two count queries that scanned the same rows into one.
2026-08-08 07:21:54 -05:00
beardthelion ad71f79e6d fix(status): make a stored claim actually re-verifiable
The row stored the RFC 9421 signing string under a column called
signed_payload. That string covers the body only through the content
digest, so it proved someone signed a request with some digest but not
that this claim row was what they signed. The whole append-only argument
rests on a claim staying verifiable after the request is gone, so the
row now carries the request body as well, in a column that says so, and
a test re-verifies a claim from the row alone: body to digest, digest
into the signing string, signature over that string under the producer's
key. Its negative mutates the stored state and shows the same procedure
refuses.

Migration v24 is amended in place rather than stacked, since it has
never shipped.

The persisted material had no size bound and is caller-influenced, so
each field is now bounded. The write response returned the whole claim,
so the wire carried the stored bytes as a JSON integer array; it now
returns a client-facing type and StatusClaim no longer derives
Serialize, so the leak cannot reappear by accident.

Carrying the body as Bytes rather than a Vec matters: the middleware
runs on every signed request including receive-pack POSTs bounded at
2 GB, so copying would have doubled peak memory on every push to serve
a field only the status write reads.
2026-08-08 07:21:54 -05:00
beardthelion 90b8294c2b fix(api): resolve the rollup head from push events, not pinned CIDs
Two review findings.

The fallback read branch_cids, whose only production writer sits inside
a Pinata pin-success branch. With no Pinata configured that table is
never written, so on a default node the fallback could never resolve and
every open pull request without a stored head answered head_resolved
false forever. It now reads repo_push_events, which the receive-pack
path writes unconditionally for every ref update, with both predicates
and the ordering in SQL against the existing keyset index. Still one
database read per request, still no repository acquire.

This is better, not complete: repo_push_events only carries pushes that
land after this ships, so a pull request whose branch was last pushed
before deployment still will not resolve until it is pushed again. There
is no backfill.

The best-effort persist propagated its error, so a transient database
failure turned a rollup that had already resolved its answer into a 500.
It now catches and logs, matching the sibling writes on the push path.
The test asserts the persist genuinely fails before driving the read, so
it cannot pass vacuously.

Also corrected a doc comment that claimed the resolve branch is never
taken twice for one pull request. Only the persist is once-per-pull-
request; the resolve attempt repeats on every read until it succeeds.
The accompanying test pins the corrected wording rather than a behavior
change, since the old code counted the same way.

The resolve counter is process-global while sqlx test databases are
per-test, so unguarded tests were inflating the counts guarded tests
assert on. The guard now covers every test that triggers a resolve.
2026-08-07 18:16:53 -05:00
beardthelion db3681e48d feat(api): add the catch-up poll surface for missed push webhooks
Webhook delivery is spawned, fire-and-forget, and never retried, so a
checker whose endpoint was unreachable during a push simply never learns
about the commit. Rather than add retry machinery and an unbounded queue
pointed at an owner-supplied URL, delivery reliability becomes a
read-side property: a push records an event, and a subscriber walks
forward from a cursor to find every commit a webhook would have told it
about.

Containment is the load-bearing constraint. The row goes into
repo_push_events, written only by the receive-pack path and read only by
this repo-scoped, gate-checked handler. It must never go into
received_ref_updates, which the unauthenticated global feed also reads;
writing local pushes there would publish private-repo pushes on an
anonymous surface. A test pushes to a private repo and asserts nothing
appears on that feed, and retargeting the write reddens it.

The cursor emits a canonical Z-form timestamp with fixed sub-second
width rather than the default rfc3339. The default emits a +00:00
offset, and + decodes to a space in a query string, so a poller echoing
the cursor back re-read the same page forever. The fixed width also
makes the TEXT column's lexicographic order match time order for the
keyset predicate. Found by seeding a timestamp collision, not by
reading.

The scenario tests drive the recorder directly, so a source-scrape test
pins its call site in the push handler, the same way the stored-head
update is pinned. Removing that call was verified to turn it red.
2026-08-07 16:29:48 -05:00
beardthelion 7be9f08b3d feat(api): add the pull request head rollup
GET /api/v1/repos/{owner}/{repo}/pulls/{number}/status answers for the
pull request's head. Both read surfaces now call one extracted
projection, so "the rollup uses the same projection as the commit read"
holds by construction rather than by two implementations agreeing today.

A pull request row stores branch names, so the head has to come from
somewhere. Preference order is the stored head, then a database-backed
branch lookup for an open pull request that has none. No path in this
module acquires the repository or lists refs from disk: that would
download the whole repository from object storage on a cold node, and
this read group carries no rate limiter. A source-read test enforces
that, and asserts its own scan still covers the handler so it cannot
shrink to vacuously passing.

The read-side persist is self-limiting by SQL, not by handler ordering:
the update is conditioned on the head being absent and the pull request
open, so a push landing mid-request cannot be rolled back to a staler
tip and a closed pull request is never back-filled. Reading the same
pull request twice resolves the branch once.

That property is invisible in the response. A variant that re-resolved
on every call and then still preferred the stored head would return
byte-identical bodies across the whole suite, so the test counts the
work done rather than the results emitted, and injecting exactly that
variant reddens only the counter assertion.

An unresolvable head is not a fifth wire state. The state field stays
inside the four values and the condition rides in head_resolved beside
the pull request's own state, so the two cases differ on that boolean
alone and both are tested.
2026-08-07 15:46:00 -05:00
beardthelion 6663fed064 feat(api): serve the combined commit status behind the visibility gate
GET /api/v1/repos/{owner}/{repo}/commits/{sha}/status returns the
projection over the claim history, computed per read rather than
materialized. A materialized index gated only at write time is how this
codebase previously kept serving a repo's slug, owner, branches, and
SHAs after it was made private, so the regression test writes a claim
while public, tightens visibility, and asserts an anonymous read finds
no trace of it.

Three outcomes stay distinct, which is the point of the unit. A caller
who cannot read the repo gets the repo's own not-found, compared byte
for byte against a missing repo so the endpoint is not an existence
oracle. A commit nobody has reported on gets 200 with the explicit
pending zero-count body, asserted exactly so no client can render it as
green. A failed lookup propagates as a 500 carrying the db_error code,
never as an empty success; nothing on this path uses ok-or-default.

Latest-per-context is decided by the database-assigned sequence, not by
the timestamp and not by the uuid. The ordering test seeds the winning
row with the lexically smaller id and a far older timestamp, so it fails
under either wrong key rather than passing by luck.

The projection filters to claims authorized by the current owner, so an
ownership transfer drops the prior owner's claims from the answer while
the history keeps them. That filter is a set-membership test inside the
query, which is the constraint the delegated-capability follow-on
inherits. It expands the owner DID into its equivalent forms, and a test
runs that expansion against did_matches over a ten-case matrix so the
two cannot drift apart silently.
2026-08-07 14:53:53 -05:00
beardthelion ac31868205 feat(api): add the owner-gated commit status write path
POST /api/v1/repos/{owner}/{repo}/statuses/{sha} appends a status claim.
Authorization runs read-visibility first, then owner, and the order is
the security property: authorize_repo_read denies a quarantined repo
before the visibility gate and answers with the repo's own not-found, so
a caller who cannot read the repo cannot learn it exists. Loading the
repo and comparing the owner would answer 403 there and turn the
endpoint into an existence oracle. That case is tested directly, since
it is what separates a correct implementation from a plausible one.

Three caps, all evaluated in the insert transaction and all refusing
with 429: per producer and context, distinct contexts per commit, and
total rows per repo. The per-tuple cap alone bounds nothing, because the
context string and the SHA are both caller-chosen and the SHA is never
existence-checked. The route group also carries the per-DID and per-IP
limiters that the creation routes use; the plain write group carries
neither and this endpoint appends on every call.

Wider than the endpoint: require_signature now attaches the verified
RFC 9421 material as a request extension, so a handler can persist what
the signature actually covered. Rebuilding the canonical string in the
handler would store a string nobody verified, and none of it survives
the request. The write fails closed with a 500 when that material is
absent, rather than storing an empty payload, because a claim nobody can
re-verify is not history the later attestation substrate can adopt.

Nine injected-defect mutations confirm the gates are load-bearing rather
than decorative, including that removing the route group's merge leaves
a group that compiles, lints, and does not exist at runtime.
2026-08-07 14:16:38 -05:00
beardthelion af17831429 feat(pulls): track the source branch head on open pull requests
The rollup needs an honest target commit, and a pull request row stores
only branch names. Resolving the branch at read time is racy (the SHA
moves between the resolve and the rollup) and gives a merged or
deleted-branch pull request no answer at all, since the merge commit is
computed and never persisted.

The receive-pack path now sets head_commit for open pull requests whose
source branch matches an updated ref, one statement keyed on repo,
branch, and open status. Excluding closed and merged rows from the WHERE
clause is what freezes the value. The merge path stamps the source head
it actually consumed, which covers a push landing between the handler
reading the row and committing the merge.

A ref arriving on the push path is a full refs/heads/ name while
source_branch stores the bare branch, so the normalisation is tested
directly; a silently non-matching WHERE clause would make this a no-op
that still reads green. Branch deletions are skipped rather than stored,
since 40 zeros is a target no commit resolves to.

The scenario tests drive the update helper directly, because a real
receive-pack POST needs a pack file. That proves the mechanism and says
nothing about the sink, so a source-scrape test pins the call site in
the push handler. Removing that call was verified to turn it red.
2026-08-07 13:17:32 -05:00
beardthelion 04263dbd49 feat(db): add migration v24 for status claims, PR head, and push events
Schema for the commit-status surface, all three objects in one versioned
entry rather than three: an append-only status_claims table, a nullable
head_commit column on pull_requests, and repo_push_events for the
catch-up poll surface.

Version 24 rather than 18 because origin/pr-173 and
origin/fix/issue-135-ipfs-cid-tree-gate both claim through 23, and the
migration runner keys only on version without comparing the recorded
name, so a collision is a silent full skip rather than an error.

status_claims orders on a database-assigned bigserial seq, not the uuid
id. A uuid v4 is random, so using it as the timestamp tiebreak would let
a retried pending claim beat a success roughly half the time when both
land in the same rfc3339 tick.

Each claim stores the producer's RFC 9421 signature headers and the
canonical signed bytes. Append-only preserves rows, but only stored
signature material preserves provenance, and a claim nobody can verify
after the request is gone is not history a signed-attestation substrate
can adopt later.

The table is repo_push_events, not push_events: the latter already
exists in v1 for agent trust scoring, so a second CREATE TABLE under
IF NOT EXISTS is a silent no-op whose follow-on index then fails.

The upgrade-path test is the load-bearing one. A sqlx::test provisions
an empty database and runs the whole array, so the fresh-chain tests
stay green even when DDL is appended to an already-applied entry.
2026-08-07 12:07:35 -05:00
130 changed files with 80950 additions and 5975 deletions

No files matched your search

+31
View File
@@ -21,6 +21,25 @@ ignore = [
# the build, at which point it becomes a real reachable advisory. # the build, at which point it becomes a real reachable advisory.
"RUSTSEC-2023-0071", # rsa Marvin attack (no fix; not linked in our build) "RUSTSEC-2023-0071", # rsa Marvin attack (no fix; not linked in our build)
# lru 0.16.3 (use-after-free in pop()). Reachable via alloy. The
# lockfile holds only 0.16.3 — the aws-sdk-s3 path that once pulled
# 0.12.5 no longer does. The RUSTSEC-2026-0253 advisory covers this
# range, so the ignore stays load-bearing until a fixed 0.16.x lands;
# REMOVE when the locked version is fixed or lru leaves the tree.
# Retirement owner: node maintainers, enforced by the weekly
# audit-schedule drift guard ("Fail if lru moved while its ignore
# remains"), which fails if either locked version moves or lru
# leaves the tree. No separate tracking issue exists; file one
# against the lru upgrade if this waiver needs discussion outside
# the schedule job.
"RUSTSEC-2026-0253",
#
# Round-3 P2 (reviewer 2): the previous version of this file
# ignored RUSTSEC-2026-0258 for h2 0.4.13, but the lockfile
# already has h2 0.4.18 (verified 2026-08-31: cargo tree -p h2
# shows 0.4.18) — the fix is in. The ignore was stale and would
# have silently re-accepted the advisory if h2 dropped back to
# 0.4.13. Removed.
# crypto-bigint 0.5.5 (secret-dependent branches in operations meant # crypto-bigint 0.5.5 (secret-dependent branches in operations meant
# to be constant-time). REACHABLE accepted risk, on two paths: # to be constant-time). REACHABLE accepted risk, on two paths:
# - alloy-signer -> elliptic-curve 0.13 -> crypto-bigint 0.5: the k256 # - alloy-signer -> elliptic-curve 0.13 -> crypto-bigint 0.5: the k256
@@ -39,4 +58,16 @@ ignore = [
# (verified 2026-11-03). REMOVE only when Cargo.lock has no crypto-bigint # (verified 2026-11-03). REMOVE only when Cargo.lock has no crypto-bigint
# below 0.7.6: `grep -A1 'name = "crypto-bigint"' Cargo.lock`. # below 0.7.6: `grep -A1 'name = "crypto-bigint"' Cargo.lock`.
"RUSTSEC-2026-0387", # crypto-bigint timing variability (no compatible fix) "RUSTSEC-2026-0387", # crypto-bigint timing variability (no compatible fix)
# libsqlite3-sys 0.30.1 (bundled SQLite WAL uninitialized-memory disclosure,
# CVE-2026-119375). Present in Cargo.lock because sqlx resolves sqlx-sqlite
# unconditionally; sqlx is built postgres-only
# (crates/gitlawb-node/Cargo.toml), so sqlx-sqlite and its bundled SQLite C
# source are never compiled or linked (`cargo tree -i sqlx-sqlite` prints
# nothing) and there is no reachable WAL open to attack. The fix requires
# libsqlite3-sys >= 0.39, but sqlx-sqlite 0.8.6 pins ^0.30.1; a compatible
# bump needs a sqlx upgrade, not a lockfile refresh. REMOVE this ignore if
# sqlx is ever built with the `sqlite` feature or any other libsqlite3-sys
# consumer enters the build.
"RUSTSEC-2026-0459", # libsqlite3-sys bundled SQLite CVE (not linked in our build)
] ]
+69 -4
View File
@@ -6,8 +6,16 @@
# Path to the node's Ed25519 keypair PEM file. Must name a key file, not a # Path to the node's Ed25519 keypair PEM file. Must name a key file, not a
# directory: a trailing slash, a final `.` or `..`, and `..` traversal are # directory: a trailing slash, a final `.` or `..`, and `..` traversal are
# refused so the path cannot retarget a parent. A `~/` value must stay under # refused so the path cannot retarget a parent. A `~/` value must stay under
# home after expansion. Bare `identity.pem` is still legal. # home after expansion, and the path must be absolute: its parent is also
# the delegation store that `gl ucan import` writes and `git-remote-gitlawb`
# reads, and the two do not share a working directory, so a relative path
# sends them to different places. Bare `identity.pem` under the default
# config dir is still legal.
# Generate with: gl identity new # Generate with: gl identity new
# Leaving it unset uses ~/.gitlawb/identity.pem, which is the shared default
# for both `gl` and `git-remote-gitlawb`. The bare-filename form is for the
# node's own service config; client tooling should inherit the default or use
# an absolute path.
GITLAWB_KEY=/data/keys/identity.pem GITLAWB_KEY=/data/keys/identity.pem
# Path to the node's persistent libp2p identity key file. Must include a # Path to the node's persistent libp2p identity key file. Must include a
@@ -39,6 +47,12 @@ GITLAWB_KEY=/data/keys/identity.pem
# Publicly reachable URL of this node (used in peer announcements) # Publicly reachable URL of this node (used in peer announcements)
GITLAWB_PUBLIC_URL=https://your-node.example.com GITLAWB_PUBLIC_URL=https://your-node.example.com
# Base URL for the web view of repos on this node (used by `gl` to print a
# working View: link after repo creation). Omit for nodes with no web front-end.
# Distinct from GITLAWB_PUBLIC_URL: that is API reachability for peers;
# this is browser reachability for humans.
# GITLAWB_WEB_URL=https://gitlawb.com
# ── Server ──────────────────────────────────────────────────────────────── # ── Server ────────────────────────────────────────────────────────────────
GITLAWB_HOST=0.0.0.0 GITLAWB_HOST=0.0.0.0
GITLAWB_PORT=7545 GITLAWB_PORT=7545
@@ -146,9 +160,10 @@ GITLAWB_REQUIRE_SIGNED_PEER_WRITES=false
# push to every repository, private ones included. On by default. # push to every repository, private ones included. On by default.
# #
# Set to false only for a rolling upgrade whose pushers are not yet the repo # Set to false only for a rolling upgrade whose pushers are not yet the repo
# owner. Note that delegated and CI keys count as non-owners: UCAN git/push is # owner. A holder of an owner-rooted, time-bounded git/push UCAN naming that
# verified but not yet honored for authorization, so they cannot push while this # repository can push while this is on — see docs/RUN-A-NODE.md for issuing and
# is on. See docs/RUN-A-NODE.md. # importing one. A CI key with no such delegation counts as a non-owner and
# cannot push.
GITLAWB_ENFORCE_OWNER_PUSH=true GITLAWB_ENFORCE_OWNER_PUSH=true
# Comma-separated libp2p multiaddrs. # Comma-separated libp2p multiaddrs.
@@ -320,6 +335,19 @@ GITLAWB_IPFS_RESOLVE_BUDGET_SECS=10
# from the concurrency caps above). 0 disables. Default 600. # from the concurrency caps above). 0 disables. Default 600.
GITLAWB_IPFS_RATE_LIMIT=600 GITLAWB_IPFS_RATE_LIMIT=600
# Per-client-IP rate limit for the anonymous task read routes
# (GET /api/v1/tasks, GET /api/v1/tasks/{id}), in requests per hour.
# Both are publicly reachable (optional_signature). GET /api/v1/tasks runs
# collect_visible_tasks and returns a visibility-filtered page;
# GET /api/v1/tasks/{id} runs get_visible_task and returns an opaque 404 when
# the task is hidden or missing. These reads can require task, repository,
# and visibility-rule queries even when no task is returned, so the brake
# bounds the cost of anonymous probes. Keyed on the resolved client IP via
# GITLAWB_TRUSTED_PROXY. 0 disables. Default: 1200 (a list page
# followed by per-task reads is a normal client pattern, so this sits above
# the /ipfs budget). The GraphQL/WS task-read brake shares this budget.
GITLAWB_TASK_READ_RATE_LIMIT=1200
# ── Creation rate limiting (repo/agent/issue/PR flood brake) ────────────── # ── Creation rate limiting (repo/agent/issue/PR flood brake) ──────────────
# Max creation requests (POST /api/v1/repos, /api/register, fork, issues, # Max creation requests (POST /api/v1/repos, /api/register, fork, issues,
# pulls) per client IP per hour, in addition to the per-DID limit. The per-DID # pulls) per client IP per hour, in addition to the per-DID limit. The per-DID
@@ -328,6 +356,14 @@ GITLAWB_IPFS_RATE_LIMIT=600
# brake caps a single-source flood. Uses GITLAWB_TRUSTED_PROXY below to resolve # brake caps a single-source flood. Uses GITLAWB_TRUSTED_PROXY below to resolve
# the client IP. 0 disables. Default 120. # the client IP. 0 disables. Default 120.
GITLAWB_CREATE_RATE_LIMIT=120 GITLAWB_CREATE_RATE_LIMIT=120
# Per-DID creation requests allowed inside the window below (repo create,
# issue create, PR create, register, fork). 0 is rejected at startup: it would
# silently disable the limiter. Default 10.
GITLAWB_RATE_LIMIT_MAX=10
# Window the per-DID creation count above applies over, in seconds. 0 is
# rejected at startup: every timestamp would prune immediately, disabling the
# limiter. Default 3600.
GITLAWB_RATE_LIMIT_WINDOW_SECS=3600
# ── Peer-sync rate limiting (per client IP, uses GITLAWB_TRUSTED_PROXY below) ─ # ── Peer-sync rate limiting (per client IP, uses GITLAWB_TRUSTED_PROXY below) ─
# /api/v1/peers/announce and /api/v1/sync/notify accept unsigned requests from # /api/v1/peers/announce and /api/v1/sync/notify accept unsigned requests from
@@ -353,6 +389,35 @@ GITLAWB_TRUSTED_PROXY=
# Enable automatic background sync from known peers # Enable automatic background sync from known peers
GITLAWB_AUTO_SYNC=false GITLAWB_AUTO_SYNC=false
# ── External upstream mirroring ───────────────────────────────────────────
# Default-off scheduled fetch worker for repositories configured as INBOUND.
# Enabling requires GITLAWB_ENFORCE_OWNER_PUSH=true; the node refuses to start
# otherwise. The worker updates branches and tags atomically while preserving
# Gitlawb-owned review/internal refs. Enabling also requires Git >= 2.37 on
# the host for http.curloptResolve DNS pinning; the node refuses to start
# when the installed Git is older.
GITLAWB_UPSTREAM_MIRROR_ENABLED=false
# Pause after each complete scan. Must be 5..=86400 seconds.
GITLAWB_UPSTREAM_MIRROR_INTERVAL_SECS=300
# Deadline for the external fetch child only; the preflight probes run on
# their own short bound. Must be 1..=3153600000 seconds.
GITLAWB_UPSTREAM_MIRROR_FETCH_TIMEOUT_SECS=600
# Keyset page size; every page is processed before the next scan delay.
GITLAWB_UPSTREAM_MIRROR_PAGE_SIZE=100
# ── Reconciliation sweep ─────────────────────────────────────────────────
# Periodic durability sweep: re-derives the public pin set and the withheld-blob
# recovery set each hour and fills gaps so a dropped replication job never means
# data loss. Defaults to true; set to false to disable the sweep even when a pin
# backend (IPFS/Pinata) is configured.
#
# Phase-capability matrix:
# - Public pin repair: IPFS-only, Pinata-only, or both (requires the
# respective backend to be configured).
# - Encrypted recovery repair: requires local IPFS (GITLAWB_IPFS_API).
# Pinata-only nodes reconcile public pins only; encrypted recovery
# reconciliation is not performed.
GITLAWB_RECONCILIATION_SWEEP=true
# ── iCaptcha proof-of-intelligence gate ─────────────────────────────────── # ── iCaptcha proof-of-intelligence gate ───────────────────────────────────
# Optional gate on create_repo + register: require callers to present an # Optional gate on create_repo + register: require callers to present an
# iCaptcha proof (X-ICaptcha-Proof header) earned at icaptcha.gitlawb.com. # iCaptcha proof (X-ICaptcha-Proof header) earned at icaptcha.gitlawb.com.
+4
View File
@@ -0,0 +1,4 @@
name: "Gitlawb CodeQL config"
paths-ignore:
- "**/tests/**"
- "**/test_support/**"
+24
View File
@@ -86,3 +86,27 @@ jobs:
exit 1 exit 1
fi fi
echo "No drift: ignore list is consistent with the pinned hickory-proto version." echo "No drift: ignore list is consistent with the pinned hickory-proto version."
# Drift guard: the RUSTSEC-2026-0253 (lru) ignore is only valid while
# the lockfile holds exactly the version it was written for (0.16.3
# via alloy). Any move — an upgrade that may carry the upstream fix,
# or removal of the dependency — must revisit the ignore. Retirement
# owner: node maintainers, via this weekly job (no separate tracking
# issue; see the waiver comment in .cargo/audit.toml).
- name: Fail if lru moved while its ignore remains
run: |
set -euo pipefail
versions="$(grep -A1 'name = "lru"' Cargo.lock | grep '^version' | cut -d'"' -f2 | sort -u | tr '\n' ' ')"
echo "lru versions in Cargo.lock: ${versions:-not present}"
lru_ignore_present=false
if grep -q 'RUSTSEC-2026-0253' .cargo/audit.toml; then
lru_ignore_present=true
fi
# Drift = the lockfile no longer matches what the ignore assumes:
# the version moved (possible fix) or lru left the tree
# (dead ignore). Both must fail for explicit re-triage.
if [ "$lru_ignore_present" = true ] && [ "${versions}" != "0.16.3 " ]; then
echo "::error::lru versions are [${versions:-absent from Cargo.lock}] but the RUSTSEC-2026-0253 ignore is still in .cargo/audit.toml. Re-triage the waiver against the advisory's fixed range and either drop the ignore or update the pinned set."
exit 1
fi
echo "No drift: lru ignore is consistent with the pinned versions."
+100
View File
@@ -0,0 +1,100 @@
name: CodeQL
# Advanced-setup replacement for CodeQL default setup. Default setup analyzes
# every detected language on every PR, including Swift (macos-app/), which
# autobuilds on a macOS runner for ~13 minutes per PR and has hung for 55 on
# main; that single job was the longest check on most PRs. Here a PR analyzes
# only the languages it touches (scripts/ci-surfaces.sh), while push to main
# and the weekly schedule analyze every language, so main's code-scanning
# baseline stays complete.
#
# Requires CodeQL default setup to be DISABLED (Settings -> Code security ->
# Code scanning): GitHub rejects advanced-setup uploads while it is enabled.
#
# A PR that skips a language may show "configuration not found" on the CodeQL
# check for that language; that is informational and expected.
on:
pull_request:
branches: [main]
push:
branches: [main]
schedule:
- cron: "17 4 * * 1"
workflow_dispatch:
permissions:
contents: read
concurrency:
group: codeql-${{ github.ref }}
cancel-in-progress: ${{ github.event_name == 'pull_request' }}
jobs:
changes:
name: changed languages
runs-on: ubuntu-latest
timeout-minutes: 5
outputs:
codeql: ${{ steps.surfaces.outputs.codeql }}
matrix: ${{ steps.surfaces.outputs.codeql_matrix }}
steps:
- name: Check out repository
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
persist-credentials: false
fetch-depth: 2
- name: Map changed paths to languages
id: surfaces
env:
EVENT_NAME: ${{ github.event_name }}
run: |
set -euo pipefail
if [ "$EVENT_NAME" = "pull_request" ]; then
# -z: literal paths; plain --name-only C-quotes unusual names.
git diff -z --name-only --no-renames HEAD^1 HEAD > changed-paths
tr '\0' '\n' < changed-paths >&2
scripts/ci-surfaces.sh < changed-paths > surfaces.out
else
scripts/ci-surfaces.sh --all > surfaces.out
fi
cat surfaces.out
grep -E '^(codeql|codeql_matrix)=' surfaces.out >> "$GITHUB_OUTPUT"
analyze:
name: Analyze (${{ matrix.language }})
needs: changes
if: needs.changes.outputs.codeql == 'true'
runs-on: ${{ matrix.runner }}
# Normal runs take <=14 min (Swift autobuild is the longest); cap a hung
# autobuild instead of letting it hold a runner for an hour.
timeout-minutes: 30
permissions:
contents: read
security-events: write
actions: read
packages: read
strategy:
fail-fast: false
matrix: ${{ fromJSON(needs.changes.outputs.matrix) }}
steps:
- name: Check out repository
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
persist-credentials: false
- name: Initialize CodeQL
uses: github/codeql-action/init@9b3254085494b55a0f7c4c2b3d2d52cc712ce370 # v4.39.1
with:
languages: ${{ matrix.language }}
build-mode: ${{ matrix.build-mode }}
- name: Autobuild
if: matrix.build-mode == 'autobuild'
uses: github/codeql-action/autobuild@9b3254085494b55a0f7c4c2b3d2d52cc712ce370 # v4.39.1
- name: Perform CodeQL Analysis
uses: github/codeql-action/analyze@9b3254085494b55a0f7c4c2b3d2d52cc712ce370 # v4.39.1
with:
category: "/language:${{ matrix.language }}"
+108 -3
View File
@@ -30,8 +30,62 @@ env:
RUST_BACKTRACE: 1 RUST_BACKTRACE: 1
jobs: jobs:
# Decide which checks this event needs. On pull_request, each job below runs
# only when the PR touches the surface it validates, and the cargo test /
# release-build jobs are scoped to the touched crates plus their reverse
# dependencies (scripts/ci-surfaces.sh). Every other event (push to main,
# merge_group, the release flow's workflow_dispatch) runs the full suite, so
# main and the merge queue keep whole-workspace coverage and main-red-alert
# still sees a complete run. A skipped job reports as "skipped", not green.
changes:
name: changed surfaces
runs-on: ubuntu-latest
timeout-minutes: 5
outputs:
rust: ${{ steps.surfaces.outputs.rust }}
rust_packages: ${{ steps.surfaces.outputs.rust_packages }}
windows: ${{ steps.surfaces.outputs.windows }}
windows_packages: ${{ steps.surfaces.outputs.windows_packages }}
docker: ${{ steps.surfaces.outputs.docker }}
audit: ${{ steps.surfaces.outputs.audit }}
core_deps: ${{ steps.surfaces.outputs.core_deps }}
release_versioning: ${{ steps.surfaces.outputs.release_versioning }}
release_tag: ${{ steps.surfaces.outputs.release_tag }}
steps:
- name: Check out repository
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
persist-credentials: false
# The pull_request checkout is the test-merge commit; depth 2 brings
# its first parent (the base tip) so the diff is exactly this PR.
fetch-depth: 2
# Every job's gate comes from the mapper, so pin its mapping first; a
# failure here fails this job and the run is red, not silently narrowed.
- name: Test the surface mapper
run: scripts/test-ci-surfaces.sh
- name: Map changed paths to surfaces
id: surfaces
env:
EVENT_NAME: ${{ github.event_name }}
run: |
set -euo pipefail
if [ "$EVENT_NAME" = "pull_request" ]; then
# -z: literal paths; plain --name-only C-quotes unusual names.
git diff -z --name-only --no-renames HEAD^1 HEAD > changed-paths
tr '\0' '\n' < changed-paths >&2
scripts/ci-surfaces.sh < changed-paths > surfaces.out
else
scripts/ci-surfaces.sh --all > surfaces.out
fi
cat surfaces.out
cat surfaces.out >> "$GITHUB_OUTPUT"
fmt-clippy: fmt-clippy:
name: fmt + clippy name: fmt + clippy
needs: changes
if: needs.changes.outputs.rust == 'true'
runs-on: ubuntu-latest runs-on: ubuntu-latest
timeout-minutes: 30 timeout-minutes: 30
steps: steps:
@@ -57,6 +111,8 @@ jobs:
release-tag-validation: release-tag-validation:
name: release tag validation name: release tag validation
needs: changes
if: needs.changes.outputs.release_tag == 'true'
runs-on: ubuntu-latest runs-on: ubuntu-latest
timeout-minutes: 5 timeout-minutes: 5
steps: steps:
@@ -86,6 +142,8 @@ jobs:
test: test:
name: test (${{ matrix.toolchain }}) name: test (${{ matrix.toolchain }})
needs: changes
if: needs.changes.outputs.rust == 'true'
runs-on: ubuntu-latest runs-on: ubuntu-latest
timeout-minutes: 45 timeout-minutes: 45
strategy: strategy:
@@ -130,10 +188,27 @@ jobs:
- name: cargo test - name: cargo test
env: env:
DATABASE_URL: postgres://postgres:postgres@localhost:5432/gitlawb_test DATABASE_URL: postgres://postgres:postgres@localhost:5432/gitlawb_test
run: cargo test --locked --workspace # `--workspace`, or `-p <crate>...` for the crates a PR touches plus
# every crate that depends on them.
PACKAGES: ${{ needs.changes.outputs.rust_packages }}
run: |
# shellcheck disable=SC2086 # PACKAGES is a word list by design
cargo test --locked $PACKAGES
# The real-node deny harness is behind the `test-harness` feature (so its
# spawn surface never compiles into the production binary), which means
# `--workspace` above skips it. Run it explicitly so the trust-boundary
# regression cases (INV-1/INV-2/INV-8) execute on every PR. Uses the same
# Postgres service; `#[sqlx::test]` provisions an isolated DB per test.
- name: cargo test (real-node deny harness)
env:
DATABASE_URL: postgres://postgres:postgres@localhost:5432/gitlawb_test
run: cargo test --locked -p gitlawb-node --features test-harness --lib --test deny_harness
build-release: build-release:
name: build --release name: build --release
needs: changes
if: needs.changes.outputs.rust == 'true'
runs-on: ubuntu-latest runs-on: ubuntu-latest
timeout-minutes: 45 timeout-minutes: 45
steps: steps:
@@ -153,10 +228,20 @@ jobs:
key: release key: release
- name: cargo build --release - name: cargo build --release
run: cargo build --locked --release --workspace env:
PACKAGES: ${{ needs.changes.outputs.rust_packages }}
run: |
# shellcheck disable=SC2086 # PACKAGES is a word list by design
cargo build --locked --release $PACKAGES
audit: audit:
name: cargo audit name: cargo audit
# cargo audit reads only Cargo.lock (+ .cargo/audit.toml), so a PR that
# leaves both unchanged cannot change its verdict; a newly published
# advisory against main is caught by the push-to-main run and the weekly
# Scheduled Audit instead of turning every unrelated PR red.
needs: changes
if: needs.changes.outputs.audit == 'true'
runs-on: ubuntu-latest runs-on: ubuntu-latest
timeout-minutes: 30 timeout-minutes: 30
steps: steps:
@@ -216,6 +301,8 @@ jobs:
msrv: msrv:
name: MSRV (Rust 1.91) name: MSRV (Rust 1.91)
needs: changes
if: needs.changes.outputs.rust == 'true'
runs-on: ubuntu-latest runs-on: ubuntu-latest
timeout-minutes: 30 timeout-minutes: 30
steps: steps:
@@ -239,6 +326,12 @@ jobs:
docker-build: docker-build:
name: Docker build smoke test name: Docker build smoke test
# Runs when the image's own inputs change: the Dockerfile, the manifests
# its dependency layer copies, the lockfile, bootstrap-peers.json. A
# source-only Rust change is already compiled by `build --release` (stable)
# and `MSRV` (1.91, the image's toolchain); push to main still runs this.
needs: changes
if: needs.changes.outputs.docker == 'true'
runs-on: ubuntu-latest runs-on: ubuntu-latest
timeout-minutes: 30 timeout-minutes: 30
steps: steps:
@@ -255,6 +348,8 @@ jobs:
test-windows: test-windows:
name: test (windows, non-blocking) name: test (windows, non-blocking)
needs: changes
if: needs.changes.outputs.windows == 'true'
runs-on: windows-latest runs-on: windows-latest
timeout-minutes: 45 timeout-minutes: 45
# Non-blocking, mirroring the informational `beta` lane above. We ship `gl` # Non-blocking, mirroring the informational `beta` lane above. We ship `gl`
@@ -282,7 +377,13 @@ jobs:
key: windows key: windows
- name: cargo test (shipped Windows crates) - name: cargo test (shipped Windows crates)
run: cargo test --locked -p gl -p git-remote-gitlawb shell: bash
env:
# `-p gl -p git-remote-gitlawb`, or whichever of the two a PR affects.
PACKAGES: ${{ needs.changes.outputs.windows_packages }}
run: |
# shellcheck disable=SC2086 # PACKAGES is a word list by design
cargo test --locked $PACKAGES
# gitlawb-core is embedded by every consumer (gl, git-remote-gitlawb, the node # gitlawb-core is embedded by every consumer (gl, git-remote-gitlawb, the node
# daemon), so it must stay lean. This gate fails if gitlawb-core's normal # daemon), so it must stay lean. This gate fails if gitlawb-core's normal
@@ -292,6 +393,8 @@ jobs:
# cannot do. Regen instructions live in the allowlist header. # cannot do. Regen instructions live in the allowlist header.
core-deps-purity: core-deps-purity:
name: gitlawb-core dependency purity name: gitlawb-core dependency purity
needs: changes
if: needs.changes.outputs.core_deps == 'true'
runs-on: ubuntu-latest runs-on: ubuntu-latest
timeout-minutes: 15 timeout-minutes: 15
steps: steps:
@@ -324,6 +427,8 @@ jobs:
# not its author knew this gate existed. # not its author knew this gate existed.
release-versioning: release-versioning:
name: release versioning coverage name: release versioning coverage
needs: changes
if: needs.changes.outputs.release_versioning == 'true'
runs-on: ubuntu-latest runs-on: ubuntu-latest
timeout-minutes: 15 timeout-minutes: 15
steps: steps:
+1 -1
View File
@@ -40,5 +40,5 @@ When a node denies a request, surface the denial to the user. Never render a den
## Toolchain and CI reality ## Toolchain and CI reality
- MSRV is Rust 1.91 (`rust-version` in the workspace `Cargo.toml`). The MSRV CI job runs `cargo check --workspace --all-targets` on exactly 1.91, so check that an API you rely on is stable there; the full test suite runs on stable. - MSRV is Rust 1.91 (`rust-version` in the workspace `Cargo.toml`). The MSRV CI job runs `cargo check --workspace --all-targets` on exactly 1.91, so check that an API you rely on is stable there; the full test suite runs on stable.
- The PR workflow runs these jobs on every PR: `cargo fmt --check`, `cargo clippy --workspace --all-targets -- -D warnings`, `cargo test --workspace` (against Postgres), a release build, `cargo audit`, the MSRV check, and a Docker build. The `main` branch ruleset currently requires one approving review but does not require these checks to pass before merge; a failing job should block review approval in practice, but nothing enforces that mechanically today. - The PR workflow's jobs are `cargo fmt --check`, `cargo clippy --workspace --all-targets -- -D warnings`, `cargo test` (against Postgres), a release build, `cargo audit`, the MSRV check, and a Docker build. On a PR, `scripts/ci-surfaces.sh` runs each job only when the PR touches its inputs, and scopes `cargo test` and the release build to the touched crates plus every crate that depends on them. Root `Cargo.toml`/`Cargo.lock`/`.cargo/` changes run the Rust jobs (fmt, clippy, tests, release build, MSRV) on the whole workspace; every other job still runs only when its own inputs change. Edits to the CI wiring itself (`pr-checks.yml`, `codeql.yml`, `scripts/ci-surfaces.sh`), push to `main`, and the merge queue run every job on the whole workspace. Edits to any other workflow run none of these jobs on a PR; CodeQL only analyzes them as Actions code. Run `cargo test --workspace` locally regardless. The same script limits CodeQL (`codeql.yml`) to the languages a PR touches. The `main` branch ruleset currently requires one approving review but does not require these checks to pass before merge; a failing job should block review approval in practice, but nothing enforces that mechanically today.
- Conventional commit titles (`feat:`, `fix:`, `docs:`, ...) are required by convention and drive releases, but no CI job checks them; a bad title fails review, not the pipeline. - Conventional commit titles (`feat:`, `fix:`, `docs:`, ...) are required by convention and drive releases, but no CI job checks them; a bad title fails review, not the pipeline.
+1 -1
View File
@@ -81,7 +81,7 @@ time once updated.
**Requirements:** **Requirements:**
- Rust stable (≥ 1.91) — install via [rustup](https://rustup.rs) - Rust stable (≥ 1.91) — install via [rustup](https://rustup.rs)
- PostgreSQL — required for the node. Use the bundled `docker-compose.yml` for local dev. - PostgreSQL 16+ — required for the node (it uses `pg_input_is_valid`, which exists only on PostgreSQL 16 and later). Use the bundled `docker-compose.yml` for local dev.
- Docker (optional, for full-stack local testing) - Docker (optional, for full-stack local testing)
**Environment variables:** **Environment variables:**
Generated
+15 -5
View File
@@ -2208,7 +2208,7 @@ version = "3.1.1"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "faf9468729b8cbcea668e36183cb69d317348c2e08e994829fb56ebfdfbaac34" checksum = "faf9468729b8cbcea668e36183cb69d317348c2e08e994829fb56ebfdfbaac34"
dependencies = [ dependencies = [
"windows-sys 0.61.2", "windows-sys 0.48.0",
] ]
[[package]] [[package]]
@@ -2701,7 +2701,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7ab67060fc6b8ef687992d439ca0fa36e7ed17e9a0b16b25b601e8757df720de" checksum = "7ab67060fc6b8ef687992d439ca0fa36e7ed17e9a0b16b25b601e8757df720de"
dependencies = [ dependencies = [
"data-encoding", "data-encoding",
"syn 2.0.117", "syn 1.0.109",
] ]
[[package]] [[package]]
@@ -3473,10 +3473,13 @@ name = "git-remote-gitlawb"
version = "0.7.1" version = "0.7.1"
dependencies = [ dependencies = [
"anyhow", "anyhow",
"chrono",
"dirs",
"gitlawb-core", "gitlawb-core",
"libc", "libc",
"mockito", "mockito",
"reqwest", "reqwest",
"serde_json",
"tempfile", "tempfile",
"tracing", "tracing",
"tracing-subscriber", "tracing-subscriber",
@@ -3542,6 +3545,7 @@ dependencies = [
"axum", "axum",
"base64 0.22.1", "base64 0.22.1",
"bytes", "bytes",
"chacha20poly1305",
"chrono", "chrono",
"cid", "cid",
"clap", "clap",
@@ -3563,7 +3567,9 @@ dependencies = [
"libp2p-swarm", "libp2p-swarm",
"mockito", "mockito",
"multiaddr", "multiaddr",
"percent-encoding",
"prometheus", "prometheus",
"rand 0.8.6",
"reqwest", "reqwest",
"serde", "serde",
"serde_json", "serde_json",
@@ -3580,6 +3586,7 @@ dependencies = [
"tracing", "tracing",
"tracing-subscriber", "tracing-subscriber",
"unicode-normalization", "unicode-normalization",
"url",
"uuid", "uuid",
"zeroize", "zeroize",
"zstd", "zstd",
@@ -3598,7 +3605,9 @@ dependencies = [
"ed25519-dalek 2.2.0", "ed25519-dalek 2.2.0",
"gitlawb-core", "gitlawb-core",
"icaptcha-client", "icaptcha-client",
"libc",
"mockito", "mockito",
"percent-encoding",
"reqwest", "reqwest",
"serde", "serde",
"serde_json", "serde_json",
@@ -3607,6 +3616,7 @@ dependencies = [
"tokio", "tokio",
"tracing", "tracing",
"tracing-subscriber", "tracing-subscriber",
"url",
"urlencoding", "urlencoding",
"uuid", "uuid",
] ]
@@ -3642,9 +3652,9 @@ dependencies = [
[[package]] [[package]]
name = "h2" name = "h2"
version = "0.4.18" version = "0.4.19"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "839c0e8a181239723652be9062bb56ca5bf5f64011f73b623f6f4fc59086a228" checksum = "ef8e5e5a340588f4452631496976cf8636d4a7ecf600239fdc27615d2530bc16"
dependencies = [ dependencies = [
"atomic-waker", "atomic-waker",
"bytes", "bytes",
@@ -7949,7 +7959,7 @@ version = "0.1.11"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c2a7b1c03c876122aa43f3020e6c3c3ee5c05081c9a00739faf7503aeba10d22" checksum = "c2a7b1c03c876122aa43f3020e6c3c3ee5c05081c9a00739faf7503aeba10d22"
dependencies = [ dependencies = [
"windows-sys 0.61.2", "windows-sys 0.48.0",
] ]
[[package]] [[package]]
+3 -1
View File
@@ -54,8 +54,10 @@ uuid = { version = "1", features = ["v4"] }
# http client # http client
reqwest = { version = "0.12", features = ["blocking", "json", "multipart", "rustls-tls"], default-features = false } reqwest = { version = "0.12", features = ["blocking", "json", "multipart", "rustls-tls"], default-features = false }
# URL parsing (what reqwest::Url re-exports, so the shared redirect predicate can # URL parsing (what reqwest::Url re-exports, so the shared redirect predicate can
# take a parsed URL without pulling reqwest into gitlawb-core) # take a parsed URL without pulling reqwest into gitlawb-core; also used for the
# absolute-browser-url validation on GITLAWB_WEB_URL in the node and CLI)
url = "2" url = "2"
percent-encoding = "2"
# HMAC # HMAC
hmac = "0.12" hmac = "0.12"
+4
View File
@@ -26,11 +26,15 @@ COPY crates/icaptcha-client/Cargo.toml crates/icaptcha-client/
# is what let it sit dead from the moment gl gained a path dependency on # is what let it sit dead from the moment gl gained a path dependency on
# icaptcha-client. A failure here should stop the build, not silently turn every # icaptcha-client. A failure here should stop the build, not silently turn every
# image build into a cold one. # image build into a cold one.
# gitlawb-node declares BOTH a [[bin]] and a [lib], so it needs both stubs:
# cargo resolves every declared target before it fetches anything, and a
# missing src/lib.rs aborts the layer with a target resolution error.
RUN mkdir -p crates/gitlawb-core/src crates/gitlawb-node/src crates/gl/src \ RUN mkdir -p crates/gitlawb-core/src crates/gitlawb-node/src crates/gl/src \
crates/git-remote-gitlawb/src crates/gitlawb-attest/src crates/icaptcha-client/src && \ crates/git-remote-gitlawb/src crates/gitlawb-attest/src crates/icaptcha-client/src && \
echo 'fn main() {}' > crates/gitlawb-node/src/main.rs && \ echo 'fn main() {}' > crates/gitlawb-node/src/main.rs && \
echo 'fn main() {}' > crates/gl/src/main.rs && \ echo 'fn main() {}' > crates/gl/src/main.rs && \
echo 'fn main() {}' > crates/git-remote-gitlawb/src/main.rs && \ echo 'fn main() {}' > crates/git-remote-gitlawb/src/main.rs && \
echo '' > crates/gitlawb-node/src/lib.rs && \
echo '' > crates/gitlawb-core/src/lib.rs && \ echo '' > crates/gitlawb-core/src/lib.rs && \
echo '' > crates/gitlawb-attest/src/lib.rs && \ echo '' > crates/gitlawb-attest/src/lib.rs && \
echo '' > crates/icaptcha-client/src/lib.rs && \ echo '' > crates/icaptcha-client/src/lib.rs && \
+121
View File
@@ -0,0 +1,121 @@
# Gitlawb Wire Protocol Specification (v1, alpha)
This document describes the wire behavior a second implementation can interop
with **today**. Anything the node does not ship yet is explicitly marked
*planned*; nothing below is aspirational unless it says so.
## 1. Identity & DIDs
- **Key type:** Ed25519.
- **Method (alpha):** `did:key` only. Resolution is deterministic public-key
extraction from the multicodec prefix (`z6Mk…`, multicodec `0xed01`).
- Signed requests presenting any other DID method are rejected; the node's
auth layer answers with a hint that only `did:key` is supported in alpha.
- *Planned:* `did:web` and `did:gitlawb` resolution. Clients MUST NOT assume
either is accepted anywhere yet.
## 2. Authentication (RFC 9421)
- **Transport:** production deployments should run the node behind TLS
(typically a reverse proxy). The node itself has no scheme check: it verifies
RFC 9421 headers on whatever transport the request arrives over, so nothing
at the signature layer refuses a cleartext request.
- **HTTP signatures:** `Signature-Input` covers `@method`, `@path`, and
`content-digest`:
`sig1=("@method" "@path" "content-digest");keyid="did:key:…";alg="ed25519";created=<unix>`
All three components are mandatory: a `Signature-Input` missing one is
rejected as `incomplete_signature`.
- **`@path` value:** the node binds `@path` to the path *and* query string of
the request, not to the path-only value RFC 9421 defines for `@path`. A
second implementation must sign the request exactly as it goes on the
wire, e.g. `GET /zOwner/myrepo.git/info/refs?service=git-upload-pack`
including the `?service=…` part (see §5 for the owner and `.git`
normalization).
- **Signer identity:** `keyid` is the actor DID (`did:key` only, see §1).
- **Integrity:** `Content-Digest` is SHA-256 over the request body and is
verified against it. Every signed request MUST send the header: a signed
request without it is refused `400 missing_content_digest` before routing.
On a body-less request the header still carries the SHA-256 digest of the
empty body (`sha-256=:47DEQpj8HBSa+/TImW+5JCeuQeRkm5NMpJWZG3hSuFU=:`),
it is never omitted.
## 3. Proof of Intelligence (iCaptcha)
- **Mode:** controlled by `ICAPTCHA_MODE` = `off` | `shadow` | `enforce`,
**default `off`**. A node with the gate off never demands proof.
- **Scope:** the gate applies to a small set of write endpoints (agent
registration and repo create/fork), not to all writes.
- **Wire shape:** the client-supplied header is `x-icaptcha-proof` — that is
the only iCaptcha header a node reads from a request. When the gate is
`enforce` and a verifier public key is loaded, a missing or invalid proof
answers `403 icaptcha_proof_required` and sets `x-icaptcha-url` and
`x-icaptcha-level` **on the response** (also mirrored as JSON fields) so
the client can go solve a challenge and retry. Fail-safe: `enforce` with
no public key loaded (unset and unfetchable) stays inert and admits the
request rather than rejecting every request.
## 4. Ref-Update Certificate
*Planned:* the `gitlawb/ref-update/v1` schema below is defined in code, but no
shipped endpoint serves or verifies these certificates today; nothing in
sections 2-3 produces one on the wire. It is documented here so a second
implementation shares the type vocabulary ahead of the wire surface.
The hash fields reserve 64 hex characters for the SHA-256 object format; the
repositories the node serves are sha1 with 40-hex OIDs (see §5), so the field
width is a forward reservation, not what a current push produces.
Two distinct certificate-shaped payloads exist; this section documents the
**core signing schema**, which is what `gitlawb/ref-update/v1` names:
- **Type string:** `gitlawb/ref-update/v1`.
- **Body (canonical JSON, the signed bytes):** `type`, `repo` (the
repository DID), `ref_name`, `from` (previous hash, 64 hex chars,
all-zeros for a new ref), `to` (target hash, 64 hex chars), `seq`
(monotonically increasing, replay prevention), `timestamp` (RFC 3339),
`nonce`.
- **Signatures:** a list of `{signer: did:key, sig: base64url-unpadded
Ed25519}` entries appended outside the signed body; thresholds are counted
over distinct signers that verify.
The node's REST API separately serves **push receipts** — a different,
node-issued JSON payload attesting that a push was processed. Receipts are
not `gitlawb/ref-update/v1` documents and are not interchangeable with them;
they are documented with the node's HTTP API, not here.
## 5. Storage & Git Transport
- **Object format:** bare repositories are created with
`--object-format=sha1`; production OIDs are 40 hex chars. *Planned:*
SHA-256 object format.
- **Path normalization:** `{repo}` may carry one trailing `.git` suffix,
stripped before lookup (so `myrepo.git` resolves to `myrepo`; a repo
literally named `foo.git` addresses as `foo.git.git`). `{owner}` may be the
full `did:key:…` string or the bare multibase segment the `did:key:` prefix
implies; both resolve to the same owner. None of this changes `@path`:
sign the request path and query exactly as sent.
- **Smart-HTTP endpoints:**
- `GET /{owner}/{repo}/info/refs?service=git-upload-pack` — fetch
advertisement.
- `GET /{owner}/{repo}/info/refs?service=git-receive-pack` — push
advertisement.
The `?service=` parameter is mandatory and takes exactly those two
values; a missing parameter is a `400`, not an omitted-service default.
- `POST /{owner}/{repo}/git-upload-pack` — clone/fetch. Request and
response bodies are pkt-line framed; the result content type is
`application/x-git-upload-pack-result`.
- `POST /{owner}/{repo}/git-receive-pack` — push. **Authentication is
mandatory**: an unsigned push is refused with `401`. Same pkt-line
framing; the result content type is
`application/x-git-receive-pack-result`.
Signed requests on this path follow the RFC 9421 header rules in §2.
§2's transport note applies here too: the verifier has no scheme check,
so nothing at the signature layer requires this push to arrive over TLS.
- **Fetch vs push auth split:** fetch-side requests against a withheld or
private repo answer `404` (existence is not confirmed to unauthorized
callers), while the push path answers `401` for missing/invalid
signatures. A second implementation must not treat the two symmetrically.
- **Content addressing:** the node records IPFS CID metadata per branch and
serves it over REST (the repo `refs` endpoint maps branch heads to CIDs).
*Planned:* IPNS-backed branch pointers; IPNS is not part of the shipped
wire contract.
+13 -8
View File
@@ -53,6 +53,7 @@ Good today:
- Local or Docker node startup. - Local or Docker node startup.
- Postgres-backed repo metadata. - Postgres-backed repo metadata.
- Bounded GraphQL queries with [repository pagination](docs/graphql-pagination.md).
- Bare git repository storage. - Bare git repository storage.
- Git smart-HTTP clone/fetch/push. - Git smart-HTTP clone/fetch/push.
- RFC 9421-signed writes. - RFC 9421-signed writes.
@@ -66,10 +67,10 @@ Good today:
Known limitations: Known limitations:
- Repository write authorization is not secure by default: `GITLAWB_ENFORCE_OWNER_PUSH` defaults to `false` for compatibility, so a valid HTTP Signature identifies a pusher but does not enforce owner-only pushes. - Repository write authorization is one switch: `GITLAWB_ENFORCE_OWNER_PUSH`, which defaults to `true`. With it off — rolling upgrades only — a valid HTTP Signature identifies a pusher but does not authorize one, so every signed caller may push everywhere.
- UCAN proof chains are validated when supplied, but UCAN capabilities are not consulted by write authorization and the root issuer is not independently trust-anchored. UCANs therefore do not yet grant scoped collaborator access. - UCAN capabilities are consulted on the push path only. There, the chain's root issuer is anchored to the repository owner, so an owner-rooted, time-bounded delegation that names the repository grants scoped collaborator access for pushing to it, for the `git/push`, `*`, or `repo/admin` actions. A `*` *resource* is refused: a delegation must name the repository it applies to, so it cannot grow to cover repos created later. Push-only cuts the other way too: a delegation grants no read access, so on a private repository the delegate must also be a reader (`gl visibility set`) — the two gates a push crosses are described in [`docs/RUN-A-NODE.md`](docs/RUN-A-NODE.md). The rest is unchanged: there is no revocation path, `nb` constraints are refused rather than interpreted, and no other route — reads, pull requests, issues, agents — consults capabilities at all.
- Agent lifecycle revocation is not enforced by HTTP Signature authorization; do not rely on removing or revoking an agent record to block a compromised signer. - Agent lifecycle revocation is not enforced by HTTP Signature authorization; do not rely on removing or revoking an agent record to block a compromised signer.
- Read visibility is not a blanket data-classification boundary: task, IPFS-pin, and Arweave-anchor listings are not repository-gated; withheld path names can be visible to a root reader; and later visibility changes cannot retract content already announced or externally anchored. - Task list/get reads (REST and GraphQL) are repository/task-gated and omit `ucan_token`. Slash-form `repo_id` rows (mirror-hosted tasks) can appear on list for the delegator or trimmed assignee, but `GET /api/v1/tasks/{id}` and GraphQL `task` fail closed for every caller because there is no local repository backing to serve. Read visibility is not a blanket data-classification boundary: IPFS-pin and Arweave-anchor listings are not repository-gated; withheld path names can be visible to a root reader; and later visibility changes cannot retract content already announced or externally anchored.
- Peer writes are signed by upgraded nodes, but strict signed-peer enforcement is opt-in during rolling upgrades. - Peer writes are signed by upgraded nodes, but strict signed-peer enforcement is opt-in during rolling upgrades.
- GraphQL mutations require a signed identity (a verified signer bound to the acting DID), are limited to the agent-task queue, and are covered by a CI completeness fence that requires signer acquisition and actor binding on every registered mutation; repo writes are not exposed over GraphQL and remain REST-only. - GraphQL mutations require a signed identity (a verified signer bound to the acting DID), are limited to the agent-task queue, and are covered by a CI completeness fence that requires signer acquisition and actor binding on every registered mutation; repo writes are not exposed over GraphQL and remain REST-only.
- Pull-request review comments do not yet have threaded line-level anchors, and merges do not enforce approval requirements. - Pull-request review comments do not yet have threaded line-level anchors, and merges do not enforce approval requirements.
@@ -311,7 +312,8 @@ metadata local disk / optional S3
| DID | A user, agent, or node identity derived from an Ed25519 public key. | | DID | A user, agent, or node identity derived from an Ed25519 public key. |
| HTTP Signature | RFC 9421 signature proving control of the DID key for write requests. | | HTTP Signature | RFC 9421 signature proving control of the DID key for write requests. |
| Ref certificate | Signed record of a ref update. Useful for audit and replication. | | Ref certificate | Signed record of a ref update. Useful for audit and replication. |
| UCAN | Delegation token for future capability-based workflows. | | Ref certificate versioning | The wire format is versioned; new certificates are currently issued as v1. `gl cert show <repo> <id> --verify` requires `--expect-node <did>`, and the queried node's self-reported DID is **not** accepted as a trust anchor. v2 is reserved for a future payload shape that carries the version inside the signed bytes; this client refuses to verify v2+ rather than guessing. |
| UCAN | Capability token. An owner delegates `git/push` on a repo to another DID; the node honors it when the proof chain roots at that owner. See [`docs/RUN-A-NODE.md`](docs/RUN-A-NODE.md). |
| Peer announce | Node-to-node HTTP announcement of DID + public URL. | | Peer announce | Node-to-node HTTP announcement of DID + public URL. |
| Gossipsub | libp2p topic for ref-update events. | | Gossipsub | libp2p topic for ref-update events. |
| Smart HTTP | Standard git protocol over HTTP for clone/fetch/push. | | Smart HTTP | Standard git protocol over HTTP for clone/fetch/push. |
@@ -399,16 +401,17 @@ Important node settings:
| `GITLAWB_P2P_BOOTSTRAP` | Comma-separated libp2p multiaddrs. | | `GITLAWB_P2P_BOOTSTRAP` | Comma-separated libp2p multiaddrs. |
| `GITLAWB_P2P_KEY` | Path to the node's persistent libp2p identity key file, which fixes the PeerId across restarts. Must name a key file inside a directory, such as `/data/keys/p2p.key` or `./keys/p2p.key`. Two kinds of problem are handled differently. A value that cannot name a securable key file at all (a bare filename, `..` traversal, a trailing `/`, the filesystem root, or a `~/` path that escapes home) is refused before the node binds, so the process exits. Anything wrong with the storage itself (a symlinked or non-directory parent, an unsafe ancestor or working directory, an unreadable or malformed key) leaves p2p off while HTTP keeps serving; the node logs it at error level as `p2p_identity_key_load_failed` and sets the `gitlawb_p2p_identity_key_load_failed` metric to 1. Note that `/health` still reports healthy in that state, so alert on the log event or the metric rather than on the health check. Correcting the storage and restarting restores the same PeerId. On Unix a new key is created `0600` inside a `0700` directory. A key directory that grants access beyond its owner is tightened to `0700` on start; one that is closed too far is refused with a `chmod 700` remedy rather than widened, since a directory locked down on purpose is an operator decision. Rotate the key only when the node logs that the directory was group- or world-writable (or the key file itself was group- or world-readable). An ordinary `0755` directory is tightened too, but others could not read a `0600` key or replace its directory entry. An existing key file is never modified: one with group or other bits set is rejected (`chmod 600`), and so is one this node cannot read. On other platforms no permissions are enforced. Default `~/.gitlawb/p2p.key`; point it at a persistent volume when running in a container. | | `GITLAWB_P2P_KEY` | Path to the node's persistent libp2p identity key file, which fixes the PeerId across restarts. Must name a key file inside a directory, such as `/data/keys/p2p.key` or `./keys/p2p.key`. Two kinds of problem are handled differently. A value that cannot name a securable key file at all (a bare filename, `..` traversal, a trailing `/`, the filesystem root, or a `~/` path that escapes home) is refused before the node binds, so the process exits. Anything wrong with the storage itself (a symlinked or non-directory parent, an unsafe ancestor or working directory, an unreadable or malformed key) leaves p2p off while HTTP keeps serving; the node logs it at error level as `p2p_identity_key_load_failed` and sets the `gitlawb_p2p_identity_key_load_failed` metric to 1. Note that `/health` still reports healthy in that state, so alert on the log event or the metric rather than on the health check. Correcting the storage and restarting restores the same PeerId. On Unix a new key is created `0600` inside a `0700` directory. A key directory that grants access beyond its owner is tightened to `0700` on start; one that is closed too far is refused with a `chmod 700` remedy rather than widened, since a directory locked down on purpose is an operator decision. Rotate the key only when the node logs that the directory was group- or world-writable (or the key file itself was group- or world-readable). An ordinary `0755` directory is tightened too, but others could not read a `0600` key or replace its directory entry. An existing key file is never modified: one with group or other bits set is rejected (`chmod 600`), and so is one this node cannot read. On other platforms no permissions are enforced. Default `~/.gitlawb/p2p.key`; point it at a persistent volume when running in a container. |
| `GITLAWB_BOOTSTRAP_DISABLE_SEEDS` | Disable embedded seed peers for isolated dev/test networks. | | `GITLAWB_BOOTSTRAP_DISABLE_SEEDS` | Disable embedded seed peers for isolated dev/test networks. |
| `GITLAWB_REQUIRE_SIGNED_PEER_WRITES` | Require peer writes to prove their DID on every transport: RFC 9421 signatures on the announce/sync routes, and an Ed25519 payload signature on gossip ref-update events. Defaults to `false` during the staged rollout below. Upgrade every gossip publisher before enabling, or their updates are dropped with no error on their side. | | `GITLAWB_REQUIRE_SIGNED_PEER_WRITES` | Require signed peer announce/sync writes. Defaults to `false` during the staged rollout below. |
| `GITLAWB_ENFORCE_OWNER_PUSH` | Require the authenticated pusher to be the repo owner on `git-receive-pack`. **Defaults to `true`.** A `did:key` signature is authentication, not authorization — anyone can mint a key and sign — so with this off every signed caller may push to every repository, private ones included. Delegated and CI keys count as non-owners: a UCAN `git/push` capability is verified but not yet honored for authorization, so they cannot push while this is on. Set `false` only for a rolling upgrade; see [`docs/RUN-A-NODE.md`](docs/RUN-A-NODE.md). | | `GITLAWB_ENFORCE_OWNER_PUSH` | Require the pusher on `git-receive-pack` to be the repo owner, or to present an owner-rooted, time-bounded UCAN `git/push` capability naming the repository (`X-Ucan`). **Defaults to `true`.** A `did:key` signature is authentication, not authorization — anyone can mint a key and sign — so with this off every signed caller may push to every repository, private ones included. A CI or delegated key with no such delegation counts as a non-owner and cannot push while this is on. The delegation clears the push gate only; on a private repository the delegate must also be a reader. Set `false` only for a rolling upgrade; see [`docs/RUN-A-NODE.md`](docs/RUN-A-NODE.md). |
| `GITLAWB_AUTO_SYNC` | Enable automatic sync from known peers. | | `GITLAWB_AUTO_SYNC` | Enable automatic sync from known peers. |
| `GITLAWB_RECONCILIATION_SWEEP` | Enable the hourly durability sweep that re-pins/backstops missing objects (default `true`; disabled when no IPFS/Pinata backend is configured). Public pin repair runs against any configured backend (IPFS, Pinata, or both). Encrypted recovery repair requires local IPFS (`GITLAWB_IPFS_API`); Pinata-only nodes reconcile public pins only. |
| `GITLAWB_MAX_PACK_BYTES` | Max git pack body size for smart-HTTP routes. | | `GITLAWB_MAX_PACK_BYTES` | Max git pack body size for smart-HTTP routes. |
| `GITLAWB_GIT_SERVICE_TIMEOUT_SECS` | Max seconds a served git upload-pack, receive-pack, or `info/refs` advertisement may run before it is aborted (504). Default 600. Also bounds the withheld-blob classification walk (on both the upload-pack serve and receive-pack replication paths) and the push-side pin-candidate discovery (`rev-list` / `cat-file`), each reaped via process-group teardown at the deadline. On the path-scoped upload-pack path the classification walk and the pack serve share ONE deadline, so this value bounds their combined duration rather than granting each stage a full budget: a walk that consumes it leaves the serve nothing and the clone gets a 504. Serving large path-scoped repos may therefore need a higher value than they did when each stage was budgeted separately. Accepted range is 1 to 3153600000 (100 years), since the node derives deadlines from this value and a larger one cannot be represented. | | `GITLAWB_GIT_SERVICE_TIMEOUT_SECS` | Max seconds a served git upload-pack, receive-pack, or `info/refs` advertisement may run before it is aborted (504). Default 600. Also bounds the withheld-blob classification walk (on both the upload-pack serve and receive-pack replication paths) and the push-side pin-candidate discovery (`rev-list` / `cat-file`), each reaped via process-group teardown at the deadline. On the path-scoped upload-pack path the classification walk and the pack serve share ONE deadline, so this value bounds their combined duration rather than granting each stage a full budget: a walk that consumes it leaves the serve nothing and the clone gets a 504. Serving large path-scoped repos may therefore need a higher value than they did when each stage was budgeted separately. Accepted range is 1 to 3153600000 (100 years), since the node derives deadlines from this value and a larger one cannot be represented. |
| `GITLAWB_GIT_ACQUIRE_TIMEOUT_SECS` | Max seconds the storage-acquisition phase (Tigris HEAD/GET, push advisory-lock) of a served git op may run before the request is shed with a 503, separate from the git-run timeout. The concurrency permit is released on expiry so a stalled backend cannot pin the pool. Default 30. | | `GITLAWB_GIT_ACQUIRE_TIMEOUT_SECS` | Max seconds the storage-acquisition phase (Tigris HEAD/GET, push advisory-lock) of a served git op may run before the request is shed with a 503, separate from the git-run timeout. The concurrency permit is released on expiry so a stalled backend cannot pin the pool. Default 30. |
| `GITLAWB_MAX_CONCURRENT_GIT_OPS` | Max concurrent served git READ ops (upload-pack and its `info/refs` advertisement) across all callers; over-cap sheds a 503 + Retry-After. Anonymous reads draw from this pool, so pair it with `GITLAWB_MAX_CONCURRENT_READS_PER_CALLER`. Pushes and the receive-pack advertisement have their own pools, so a read flood cannot shed an authenticated push. Default 128. | | `GITLAWB_MAX_CONCURRENT_GIT_OPS` | Max concurrent served git READ ops (upload-pack and its `info/refs` advertisement) across all callers; over-cap sheds a 503 + Retry-After. Anonymous reads draw from this pool, so pair it with `GITLAWB_MAX_CONCURRENT_READS_PER_CALLER`. Pushes and the receive-pack advertisement have their own pools, so a read flood cannot shed an authenticated push. Default 128. |
| `GITLAWB_MAX_CONCURRENT_GIT_PUSHES` | Max concurrent `git-receive-pack` POST operations, in a pool separate from the read pool. The anon receive-pack `info/refs` advertisement runs in a third pool of the same size, disjoint from both, so an advertisement flood cannot shed a push either. Two per-source push caps are derived from this value (`/8`, floor 1) and have no env var of their own. Over-cap sheds a 503 + Retry-After. Default 32. | | `GITLAWB_MAX_CONCURRENT_GIT_PUSHES` | Max concurrent `git-receive-pack` POST operations, in a pool separate from the read pool. The anon receive-pack `info/refs` advertisement runs in a third pool of the same size, disjoint from both, so an advertisement flood cannot shed a push either. Two per-source push caps are derived from this value (`/8`, floor 1) and have no env var of their own. Over-cap sheds a 503 + Retry-After. Default 32. |
| `GITLAWB_MAX_CONCURRENT_READS_PER_CALLER` | Max concurrent read ops a single caller may hold, so one caller cannot monopolize the read pool. Keyed on the resolved source IP, never the DID, and only as granular as `GITLAWB_TRUSTED_PROXY`: left unset, a node behind an edge or NAT keys every caller on the edge IP and this collapses to one global cap. Default 16. | | `GITLAWB_MAX_CONCURRENT_READS_PER_CALLER` | Max concurrent read ops a single caller may hold, so one caller cannot monopolize the read pool. Keyed on the resolved source IP, never the DID, and only as granular as `GITLAWB_TRUSTED_PROXY`: left unset, a node behind an edge or NAT keys every caller on the edge IP and this collapses to one global cap. Default 16. |
| `GITLAWB_MAX_CONCURRENT_PIN_TASKS` | Max post-push pin loops (IPFS + Pinata) running concurrently across all repos. This caps how many loops RUN at once, not how much object-id list memory the node retains: on the local IPFS path a loop parked waiting for a permit still holds its full list. Do not size memory from this knob alone. A loop over cap waits, never drops a pin. Default 8. | | `GITLAWB_MAX_CONCURRENT_PIN_TASKS` | Max post-push pin loops (IPFS + Pinata) running concurrently across all repos. This caps how many loops RUN at once, not how much object-id list memory the node retains: on the local IPFS path a loop parked waiting for a permit still holds its full list. Do not size memory from this knob alone. A loop over cap waits, never drops a pin. Up to two loops run per push that starts its own encrypt/pin task when both IPFS and Pinata are configured; a push coalesced onto an in-flight task adds at most one warm loop. See [docs/STORAGE-AND-PINNING.md](docs/STORAGE-AND-PINNING.md) for the two-tier model, dedup, and parked-list memory. Default 8. |
| `GITLAWB_REPO_LEASE_MAX_WAITERS` | Max pushes parked at once waiting for the same repo's write lease. Each waiter pins its buffered pack body, so this bounds that memory for a hot repo; past the cap the newest push sheds a 503 + Retry-After instead of queueing. Pushes to other repos are unaffected, and the lease holder is not counted. Default 8. | | `GITLAWB_REPO_LEASE_MAX_WAITERS` | Max pushes parked at once waiting for the same repo's write lease. Each waiter pins its buffered pack body, so this bounds that memory for a hot repo; past the cap the newest push sheds a 503 + Retry-After instead of queueing. Pushes to other repos are unaffected, and the lease holder is not counted. Default 8. |
| `GITLAWB_MAX_CONCURRENT_IPFS_WALKS` | Max concurrent `GET /ipfs/{cid}` visibility walks across all callers (own pool, disjoint from the served-git pools); over-cap sheds 503. Default 32. | | `GITLAWB_MAX_CONCURRENT_IPFS_WALKS` | Max concurrent `GET /ipfs/{cid}` visibility walks across all callers (own pool, disjoint from the served-git pools); over-cap sheds 503. Default 32. |
| `GITLAWB_IPFS_WALK_PER_SOURCE` | Max concurrent `/ipfs` walks a single source IP may hold. Default 4. | | `GITLAWB_IPFS_WALK_PER_SOURCE` | Max concurrent `/ipfs` walks a single source IP may hold. Default 4. |
@@ -419,6 +422,7 @@ Important node settings:
| `GITLAWB_IPFS_REQUEST_BUDGET_SECS` | Absolute wall-clock budget for one admitted `/ipfs/{cid}` request's acquire+walk lifetime. Per-stage clamps bound the acquire and walk stages to the remaining budget, and no stage starts once it is exhausted; the scan then stops with a retryable 503. The object-type probe and content-read `cat-file` subprocesses are budget-checked before starting and each also run under their own deadline (the lesser of `GITLAWB_GIT_SERVICE_TIMEOUT_SECS` and the remaining budget), reaped via process-group teardown, so a hung `cat-file` cannot hold the request's walk slot past it. One hang path is still unbounded: the probe's object-store readability check is a plain filesystem sweep with nothing to reap, so a wedged filesystem can hold the slot past the deadline. Default 600. Accepted range is 1 to 3153600000 (100 years), since the node derives a deadline from this value and a larger one cannot be represented. | | `GITLAWB_IPFS_REQUEST_BUDGET_SECS` | Absolute wall-clock budget for one admitted `/ipfs/{cid}` request's acquire+walk lifetime. Per-stage clamps bound the acquire and walk stages to the remaining budget, and no stage starts once it is exhausted; the scan then stops with a retryable 503. The object-type probe and content-read `cat-file` subprocesses are budget-checked before starting and each also run under their own deadline (the lesser of `GITLAWB_GIT_SERVICE_TIMEOUT_SECS` and the remaining budget), reaped via process-group teardown, so a hung `cat-file` cannot hold the request's walk slot past it. One hang path is still unbounded: the probe's object-store readability check is a plain filesystem sweep with nothing to reap, so a wedged filesystem can hold the slot past the deadline. Default 600. Accepted range is 1 to 3153600000 (100 years), since the node derives a deadline from this value and a larger one cannot be represented. |
| `GITLAWB_IPFS_RESOLVE_BUDGET_SECS` | Shorter budget for the pre-walk CID resolve inside an admitted `/ipfs/{cid}` request: the lookup that maps the requested CID to its git oid(s), which runs while the scarce walk admission is already held. A well-formed CID with no pin row does no probe and no walk work, so without this it could hold a walk slot for the whole request budget while nothing walked, and enough such requests shed every real retrieval at admission. The effective deadline is the lesser of this and the remaining request budget, so a value above `GITLAWB_IPFS_REQUEST_BUDGET_SECS` degrades to the request budget. Only the resolve is on this clock; walk and probe work stay on the request budget, so a slow but progressing scan is never shed by it. Default 10. Accepted range is 1 to 3153600000 (100 years). | | `GITLAWB_IPFS_RESOLVE_BUDGET_SECS` | Shorter budget for the pre-walk CID resolve inside an admitted `/ipfs/{cid}` request: the lookup that maps the requested CID to its git oid(s), which runs while the scarce walk admission is already held. A well-formed CID with no pin row does no probe and no walk work, so without this it could hold a walk slot for the whole request budget while nothing walked, and enough such requests shed every real retrieval at admission. The effective deadline is the lesser of this and the remaining request budget, so a value above `GITLAWB_IPFS_REQUEST_BUDGET_SECS` degrades to the request budget. Only the resolve is on this clock; walk and probe work stay on the request budget, so a slow but progressing scan is never shed by it. Default 10. Accepted range is 1 to 3153600000 (100 years). |
| `GITLAWB_IPFS_RATE_LIMIT` | Max `/ipfs/{cid}` requests per client IP per hour (route flood brake). 0 disables. Default 600. | | `GITLAWB_IPFS_RATE_LIMIT` | Max `/ipfs/{cid}` requests per client IP per hour (route flood brake). 0 disables. Default 600. |
| `GITLAWB_TASK_READ_RATE_LIMIT` | Max anonymous task-read requests (`GET /api/v1/tasks`, `GET /api/v1/tasks/{id}`) per client IP per hour, sharing the budget with the GraphQL/WS task-read brake. Uses `GITLAWB_TRUSTED_PROXY` to resolve the client IP. 0 disables. Default 1200 (above the `/ipfs` budget to allow list pages followed by per-task reads). |
| `GITLAWB_TIGRIS_BUCKET` | Optional S3/Tigris shared repo storage bucket. | | `GITLAWB_TIGRIS_BUCKET` | Optional S3/Tigris shared repo storage bucket. |
| `GITLAWB_PINATA_JWT` | Optional Pinata/IPFS warm-storage pinning. | | `GITLAWB_PINATA_JWT` | Optional Pinata/IPFS warm-storage pinning. |
| `GITLAWB_IRYS_URL` | Optional Irys/Arweave permanent anchoring. | | `GITLAWB_IRYS_URL` | Optional Irys/Arweave permanent anchoring. |
@@ -474,6 +478,7 @@ Read:
- [`docs/RUN-A-NODE.md`](docs/RUN-A-NODE.md) - [`docs/RUN-A-NODE.md`](docs/RUN-A-NODE.md)
- [`docs/ECONOMICS.md`](docs/ECONOMICS.md) - [`docs/ECONOMICS.md`](docs/ECONOMICS.md)
- [`docs/STORAGE-AND-PINNING.md`](docs/STORAGE-AND-PINNING.md)
Use a dedicated low-balance operator wallet. Do not use a treasury wallet as the heartbeat key. Use a dedicated low-balance operator wallet. Do not use a treasury wallet as the heartbeat key.
@@ -553,7 +558,7 @@ Short-term priorities:
2. Add Docker and installer smoke tests. 2. Add Docker and installer smoke tests.
3. Improve operator docs and `gl doctor` checks. 3. Improve operator docs and `gl doctor` checks.
4. Harden peer writes and publish the signed-peer rollout plan. 4. Harden peer writes and publish the signed-peer rollout plan.
5. Close default-open write authorization and wire trusted UCAN delegation into repository permissions. 5. Wire trusted UCAN delegation into repository write authorization.
6. Add threaded, line-level pull-request discussions and enforce approval requirements on merges. 6. Add threaded, line-level pull-request discussions and enforce approval requirements on merges.
7. Add metrics for pushes, fetches, pack sizes, peer sync, failed auth, and webhooks. 7. Add metrics for pushes, fetches, pack sizes, peer sync, failed auth, and webhooks.
+11 -11
View File
@@ -26,7 +26,7 @@ We will acknowledge receipt within 48 hours and aim to release a fix within 14 d
**Content addressing** **Content addressing**
- Every git object is content-addressed via CIDv1 (SHA-256) - Every git object is content-addressed via CIDv1 (SHA-256)
- Tamper-evident by construction — a modified object changes its CID - Tamper-evident by construction: a modified object changes its CID
**UCAN token validation** **UCAN token validation**
- Bootstrap UCAN tokens are issued at registration. - Bootstrap UCAN tokens are issued at registration.
@@ -35,8 +35,8 @@ We will acknowledge receipt within 48 hours and aim to release a fix within 14 d
- Capability grants are not yet consulted by repository write authorization; see the limitations below. - Capability grants are not yet consulted by repository write authorization; see the limitations below.
**Smart contracts (Base Sepolia testnet)** **Smart contracts (Base Sepolia testnet)**
- `GitlawbDIDRegistry` — on-chain DID → document registry - `GitlawbDIDRegistry`: on-chain DID → document registry
- `GitlawbNameRegistry` — human name → DID registry - `GitlawbNameRegistry`: human name → DID registry
- Both auditable on-chain, no admin keys - Both auditable on-chain, no admin keys
--- ---
@@ -54,9 +54,9 @@ We will acknowledge receipt within 48 hours and aim to release a fix within 14 d
These are documented limitations of the current live release. They should be prioritized without breaking existing nodes during rolling upgrades. These are documented limitations of the current live release. They should be prioritized without breaking existing nodes during rolling upgrades.
### Repository write authorization defaults ### Repository write authorization defaults
- `git-receive-pack` verifies HTTP Signatures, but `GITLAWB_ENFORCE_OWNER_PUSH` defaults to `false` for compatibility during rollout. - `git-receive-pack` verifies HTTP Signatures and, with the default configuration, also requires the pusher DID to be the repository owner: `GITLAWB_ENFORCE_OWNER_PUSH` defaults to `true`.
- **Impact:** With the default setting, a valid signature authenticates the pusher but does not require that DID to be the repository owner. - **Impact:** With the default setting, a valid signature authenticates the pusher, and the pusher must be the repository owner. Authentication is not authorization on its own, since anyone can mint a `did:key` and sign, so owner enforcement is the only thing that prevents every signed caller from pushing to every repository, including private ones.
- **Mitigation:** Set `GITLAWB_ENFORCE_OWNER_PUSH=true` on nodes where owner-only pushes are required. Confirm that every legitimate pusher uses the owner DID before enabling it. - **Mitigation:** `GITLAWB_ENFORCE_OWNER_PUSH=false` is the explicit rolling-upgrade compatibility override. Disable it only for the duration of an upgrade whose pushers are not yet the repo owner, then turn it back on. Delegated and CI keys count as non-owners: a UCAN `git/push` capability is verified but not yet honored for authorization, so they cannot push while this is on.
### UCAN delegation and revocation ### UCAN delegation and revocation
- The middleware validates a supplied UCAN's complete proof chain, but a root token is accepted without an independently trusted issuer anchor. `Ucan::can` is not yet used by write handlers, so a UCAN does not grant scoped repository access. - The middleware validates a supplied UCAN's complete proof chain, but a root token is accepted without an independently trusted issuer anchor. `Ucan::can` is not yet used by write handlers, so a UCAN does not grant scoped repository access.
@@ -73,7 +73,7 @@ These are documented limitations of the current live release. They should be pri
### Private repository reads ### Private repository reads
- Repository and path-scoped visibility checks are enforced for repository API and Git content reads. A denied whole-repository or root read returns the same 404 shape as a missing repository, so the denial does not reveal private-repository existence. - Repository and path-scoped visibility checks are enforced for repository API and Git content reads. A denied whole-repository or root read returns the same 404 shape as a missing repository, so the denial does not reveal private-repository existence.
- Sparse-clone support exposes withheld path globs to callers who may read the repository root. Do not put sensitive information in withheld path names. - Sparse-clone support exposes withheld path globs to callers who may read the repository root. Do not put sensitive information in withheld path names.
- `GET /api/v1/tasks`, `/api/v1/ipfs/pins`, and `/api/v1/arweave/anchors` are not repository-gated. Task records include a UCAN token; pin and anchor listings expose object and ref metadata. - Task list/get reads (`GET /api/v1/tasks`, `GET /api/v1/tasks/{id}`, and GraphQL `tasks` / `task` over HTTP and WebSocket) are repository/task-gated through shared visibility collectors and omit `ucan_token`. Slash-form `repo_id` rows (mirror-hosted tasks) can appear on list for the delegator or trimmed assignee, but `GET /api/v1/tasks/{id}` and GraphQL `task` fail closed for every caller because there is no local repository backing to serve. All of those transports share the same per-IP `GITLAWB_TASK_READ_RATE_LIMIT` budget. `/api/v1/ipfs/pins` and `/api/v1/arweave/anchors` are not repository-gated; pin and anchor listings expose object and ref metadata.
- Changing a repository's visibility controls future serving, but cannot retract ref metadata or configured external pins and anchors already announced while the repository was public. Do not push secrets to an announceable repository. - Changing a repository's visibility controls future serving, but cannot retract ref metadata or configured external pins and anchors already announced while the repository was public. Do not push secrets to an announceable repository.
- **Impact:** Visibility policies protect the repository and Git content routes they gate, not every metadata endpoint or previously published content. - **Impact:** Visibility policies protect the repository and Git content routes they gate, not every metadata endpoint or previously published content.
- **Remaining boundary:** This read control does not address the independent write-authorization and UCAN-delegation limitations described above. - **Remaining boundary:** This read control does not address the independent write-authorization and UCAN-delegation limitations described above.
@@ -119,10 +119,10 @@ These are documented limitations of the current live release. They should be pri
## Threat Model ## Threat Model
Twigpine is designed to be secure against: Twigpine is designed to be secure against:
- **Unauthenticated writes** — HTTP Signature auth on protected write endpoints; see Known Limitations for owner and capability authorization gaps - **Unauthenticated writes:** HTTP Signature auth on protected write endpoints; see Known Limitations for owner and capability authorization gaps
- **Tampered git objects** — CIDv1 content addressing detects modification - **Tampered git objects:** CIDv1 content addressing detects modification
- **Identity spoofing** — DIDs derived from public keys, unforgeable without the private key - **Identity spoofing:** DIDs derived from public keys, unforgeable without the private key
- **Centralized takedown** — no single point of control; data on IPFS + Arweave - **Centralized takedown:** no single point of control; data on IPFS + Arweave
Twigpine is **not yet** designed to defend against: Twigpine is **not yet** designed to defend against:
- A compromised node operator (node operators are trusted for their own node) - A compromised node operator (node operators are trusted for their own node)
+9 -1
View File
@@ -1,7 +1,7 @@
{ {
"$comment": "Canonical seed list for the Twigpine network. Merged with GITLAWB_BOOTSTRAP_PEERS at startup. PRs to add public nodes welcome.", "$comment": "Canonical seed list for the Twigpine network. Merged with GITLAWB_BOOTSTRAP_PEERS at startup. PRs to add public nodes welcome.",
"version": 1, "version": 1,
"updated": "2026-06-30", "updated": "2026-10-02",
"peers": [ "peers": [
{ {
"name": "gitlawb", "name": "gitlawb",
@@ -42,6 +42,14 @@
"http_url": "https://rapybus-gitlawb.com", "http_url": "https://rapybus-gitlawb.com",
"p2p_multiaddr": null, "p2p_multiaddr": null,
"added": "2026-05-16" "added": "2026-05-16"
},
{
"name": "moonwake-ledger",
"operator": "Moonwake Ledger",
"did": "did:key:z6Mkh9AZ46Z3RkGZfFCom5vzX2535f54d6G2o2ZGkCqjwW8K",
"http_url": "https://node.moonwakeledger.com",
"p2p_multiaddr": null,
"added": "2026-10-02"
} }
] ]
} }
+9
View File
@@ -14,8 +14,17 @@ path = "src/main.rs"
gitlawb-core = { path = "../gitlawb-core", features = ["redirect"] } gitlawb-core = { path = "../gitlawb-core", features = ["redirect"] }
anyhow = { workspace = true } anyhow = { workspace = true }
reqwest = { workspace = true } reqwest = { workspace = true }
# Reading the node's DID from `GET /` so a delegated push can address its
# invocation to the right executor.
serde_json = { workspace = true }
# The invocation carries the delegation's expiry, so an unbounded write capability
# is never minted; converting the stored i64 timestamp needs chrono.
chrono = { workspace = true }
tracing = { workspace = true } tracing = { workspace = true }
tracing-subscriber = { workspace = true } tracing-subscriber = { workspace = true }
# Home-directory lookup for the shared GITLAWB_KEY resolver in gitlawb-core,
# which takes `home` as an argument so core's dependency allowlist stays lean.
dirs = "5"
[dev-dependencies] [dev-dependencies]
mockito = "1" mockito = "1"
File diff suppressed because it is too large. Load diff
@@ -137,6 +137,11 @@ enum ShimMode {
struct Shim { struct Shim {
base_url: String, base_url: String,
posts: Arc<AtomicUsize>, posts: Arc<AtomicUsize>,
/// Advertisement (GET /info/refs) count. One `git fetch` performs exactly
/// one advertisement, so tests assert this is EXACTLY one: any fetch-level
/// retry that comes back re-advertises and goes red on the spot, which the
/// lower-bound POST assertions alone cannot see (#275 round 6).
gets: Arc<AtomicUsize>,
stop: Arc<AtomicBool>, stop: Arc<AtomicBool>,
handle: Option<std::thread::JoinHandle<()>>, handle: Option<std::thread::JoinHandle<()>>,
} }
@@ -166,15 +171,17 @@ fn start_shim(repo: PathBuf, mode: ShimMode) -> Shim {
let addr = listener.local_addr().unwrap(); let addr = listener.local_addr().unwrap();
let base_url = format!("http://{addr}"); let base_url = format!("http://{addr}");
let posts = Arc::new(AtomicUsize::new(0)); let posts = Arc::new(AtomicUsize::new(0));
let gets = Arc::new(AtomicUsize::new(0));
let stop = Arc::new(AtomicBool::new(false)); let stop = Arc::new(AtomicBool::new(false));
let posts_t = posts.clone(); let posts_t = posts.clone();
let gets_t = gets.clone();
let stop_t = stop.clone(); let stop_t = stop.clone();
let handle = std::thread::spawn(move || { let handle = std::thread::spawn(move || {
while !stop_t.load(Ordering::SeqCst) { while !stop_t.load(Ordering::SeqCst) {
match listener.accept() { match listener.accept() {
Ok((stream, _)) => { Ok((stream, _)) => {
handle_conn(stream, &repo, mode, &posts_t); handle_conn(stream, &repo, mode, &posts_t, &gets_t);
} }
Err(ref e) if e.kind() == std::io::ErrorKind::WouldBlock => { Err(ref e) if e.kind() == std::io::ErrorKind::WouldBlock => {
std::thread::sleep(Duration::from_millis(5)); std::thread::sleep(Duration::from_millis(5));
@@ -187,6 +194,7 @@ fn start_shim(repo: PathBuf, mode: ShimMode) -> Shim {
Shim { Shim {
base_url, base_url,
posts, posts,
gets,
stop, stop,
handle: Some(handle), handle: Some(handle),
} }
@@ -212,7 +220,13 @@ fn normalize_accepted_stream(stream: &TcpStream) {
stream.set_read_timeout(Some(Duration::from_secs(30))).ok(); stream.set_read_timeout(Some(Duration::from_secs(30))).ok();
} }
fn handle_conn(stream: TcpStream, repo: &Path, mode: ShimMode, posts: &AtomicUsize) { fn handle_conn(
stream: TcpStream,
repo: &Path,
mode: ShimMode,
posts: &AtomicUsize,
gets: &AtomicUsize,
) {
normalize_accepted_stream(&stream); normalize_accepted_stream(&stream);
let mut reader = BufReader::new(stream); let mut reader = BufReader::new(stream);
@@ -263,6 +277,7 @@ fn handle_conn(stream: TcpStream, repo: &Path, mode: ShimMode, posts: &AtomicUsi
receive_pack(repo, false, &body), receive_pack(repo, false, &body),
) )
} else if method == "GET" && target.contains("/info/refs") { } else if method == "GET" && target.contains("/info/refs") {
gets.fetch_add(1, Ordering::SeqCst);
// v0 advertisement, wrapped exactly as the node's info_refs does. // v0 advertisement, wrapped exactly as the node's info_refs does.
let adv = upload_pack(repo, true, b""); let adv = upload_pack(repo, true, b"");
let mut wrapped = pkt(b"# service=git-upload-pack\n"); let mut wrapped = pkt(b"# service=git-upload-pack\n");
@@ -1007,6 +1022,7 @@ fn shim_answers_a_connection_that_arrives_non_blocking() {
let server = std::thread::spawn(move || { let server = std::thread::spawn(move || {
let posts = AtomicUsize::new(0); let posts = AtomicUsize::new(0);
let gets = AtomicUsize::new(0);
let stream = loop { let stream = loop {
match listener.accept() { match listener.accept() {
Ok((s, _)) => break s, Ok((s, _)) => break s,
@@ -1021,7 +1037,7 @@ fn shim_answers_a_connection_that_arrives_non_blocking() {
// the assertions below run on every platform. // the assertions below run on every platform.
stream.set_nonblocking(true).unwrap(); stream.set_nonblocking(true).unwrap();
accepted_tx.send(()).unwrap(); accepted_tx.send(()).unwrap();
handle_conn(stream, &repo, ShimMode::Normal, &posts); handle_conn(stream, &repo, ShimMode::Normal, &posts, &gets);
}); });
let mut client = TcpStream::connect(addr).unwrap(); let mut client = TcpStream::connect(addr).unwrap();
@@ -1085,6 +1101,17 @@ fn real_git_multi_round_fetch_completes() {
posts >= 2, posts >= 2,
"fixture did not force multi-round negotiation (observed {posts} POST(s)); the bridging path was not exercised" "fixture did not force multi-round negotiation (observed {posts} POST(s)); the bridging path was not exercised"
); );
// Exactly one advertisement: the POST bound above is a floor, so a
// reintroduced fetch-level retry could satisfy it by running a second full
// fetch instead of multi-round negotiation. Re-advertising is what a
// second fetch cannot avoid, so this is the committed guard on the
// retry's absence.
let gets = shim.gets.load(Ordering::SeqCst);
assert_eq!(
gets, 1,
"expected exactly one advertisement GET per fetch; {gets} means a \
fetch-level retry is back"
);
// The fetched tip is present and the clone's object graph is intact. // The fetched tip is present and the clone's object graph is intact.
let server_head = String::from_utf8(git(&server, &["rev-parse", "HEAD"])).unwrap(); let server_head = String::from_utf8(git(&server, &["rev-parse", "HEAD"])).unwrap();
@@ -1177,6 +1204,17 @@ fn real_git_withheld_shaped_first_post() {
"helper hung on a withheld-shaped response (should forward-and-terminate, not deadlock). stderr:\n{stderr}" "helper hung on a withheld-shaped response (should forward-and-terminate, not deadlock). stderr:\n{stderr}"
); );
// Exactly one advertisement, pass or fail: the POST assertions below are
// floors, so a reintroduced fetch-level retry could mask the withheld
// shape behind a second full fetch. A retry cannot avoid re-advertising,
// so this is the committed guard on its absence.
let gets = shim.gets.load(Ordering::SeqCst);
assert_eq!(
gets, 1,
"expected exactly one advertisement GET per fetch; {gets} means a \
fetch-level retry is back"
);
if out.status.success() { if out.status.success() {
// Real git accepted the mid-negotiation pack: the withheld multi-round // Real git accepted the mid-negotiation pack: the withheld multi-round
// path works end to end with no extra handling. // path works end to end with no extra handling.
+54
View File
@@ -504,6 +504,23 @@ mod tests {
assert!(matches!(err, Error::Did(_))); assert!(matches!(err, Error::Did(_)));
} }
#[test]
fn verify_rejects_oversized_method_id_before_decoding() {
let sk = fresh();
let mut att = dummy_attestation(&sk, sample_cert_hash());
// A well-formed prefix on an absurdly long id must fail the length
// bound, not the decoder: base58 decode cost is quadratic in input
// length, so reaching it at all is the defect.
att.signer = format!("did:key:z{}", "1".repeat(256));
let err = att.verify_signature(sample_cert_hash()).unwrap_err();
// Match the bound's own message: any other Did error (a decode or
// length mismatch downstream) means the oversized input was decoded.
match err {
Error::Did(m) => assert!(m.contains("too long"), "unexpected: {m}"),
e => panic!("expected Error::Did, got {e:?}"),
}
}
#[test] #[test]
fn verify_rejects_bad_signature_base64() { fn verify_rejects_bad_signature_base64() {
let sk = fresh(); let sk = fresh();
@@ -522,6 +539,43 @@ mod tests {
assert!(matches!(err, Error::Signature(_))); assert!(matches!(err, Error::Signature(_)));
} }
/// The identity-point forgery must be rejected: the shared attestation
/// verifier is a cert-bound provenance gate, so accepting the weak-key
/// signature would let anyone mint a forged attestation that verifies.
/// Strict verification rejects small-order public keys and R (the identity
/// point here), which ordinary verification does not.
#[test]
fn verify_rejects_identity_point_forgery() {
let cert_hash = sample_cert_hash();
let mut att = dummy_attestation(&fresh(), cert_hash);
// Public key A = identity point (0,1); signature R = identity, S = 0.
// The equation `[S]B = R + [k]A` then holds for any k and any message.
let identity = [
1u8, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0,
];
let mut buf = Vec::with_capacity(34);
buf.extend_from_slice(&ED25519_MULTICODEC);
buf.extend_from_slice(&identity);
att.signer = format!(
"did:key:{}",
multibase::encode(multibase::Base::Base58Btc, &buf)
);
let mut sig = [0u8; 64];
sig[..32].copy_from_slice(&identity);
att.sig = B64U.encode(sig);
let err = att.verify_signature(cert_hash).unwrap_err();
match err {
Error::Did(msg) => assert!(
msg.contains("small-order"),
"identity-point signer must be refused as a small-order key, got: {msg}"
),
other => panic!("expected Error::Did for an identity-point signer, got {other:?}"),
}
}
/// A payload that happens to contain a `cert_hash` field of its own does /// A payload that happens to contain a `cert_hash` field of its own does
/// not interfere with the outer binding: the attestation envelope's /// not interfere with the outer binding: the attestation envelope's
/// `cert_hash` is the only field consulted by `verify_signature`, and the /// `cert_hash` is the only field consulted by `verify_signature`, and the
+127 -11
View File
@@ -150,23 +150,48 @@ impl Registry {
} }
/// Verify a batch, then enforce `RequireAll`. /// Verify a batch, then enforce `RequireAll`.
///
/// Under [`Policy::RequireAll`] this never short-circuits: an attestation
/// that fails to verify — bad signature, cert-hash mismatch, or malformed
/// payload, the actual `Err` paths of [`Registry::verify`] — is dropped
/// from the result rather than aborting the batch. A valid attestation of
/// a type with NO registered verifier is not one of those: under
/// `RequireAll` it verifies to `Ok` with `fully_verified: false` and stays
/// in the result, preserving the distinction between *untrusted* and
/// *invalid* (see `require_all_is_lenient_on_unknown_types_in_the_batch`).
/// Any third party can attach an attestation, so a hard error here would
/// let an unrelated `attacker/spam/v1` deny-of-service the whole cert (the
/// DoS vector the module docstring warns about). The `required_types`
/// check below is the real gate, and it only counts `fully_verified`
/// entries — so a required type represented solely by a malformed
/// attestation still fails with [`Error::RequiredMissing`].
///
/// Under `AcceptKnown` and `RejectUnknown` the batch stays strict: the
/// first verification error is surfaced to the caller.
pub fn verify_all( pub fn verify_all(
&self, &self,
attestations: &[Attestation], attestations: &[Attestation],
expected_cert_hash: [u8; 32], expected_cert_hash: [u8; 32],
) -> Result<Vec<VerifiedAttestation>> { ) -> Result<Vec<VerifiedAttestation>> {
let mut verified = Vec::with_capacity(attestations.len()); if self.policy != Policy::RequireAll {
for a in attestations { let mut verified = Vec::with_capacity(attestations.len());
verified.push(self.verify(a, expected_cert_hash)?); for a in attestations {
verified.push(self.verify(a, expected_cert_hash)?);
}
return Ok(verified);
} }
if self.policy == Policy::RequireAll {
for required in &self.required_types { // RequireAll: drop entries that fail to verify instead of aborting.
let present = verified let verified: Vec<VerifiedAttestation> = attestations
.iter() .iter()
.any(|v| &v.type_ == required && v.fully_verified); .filter_map(|a| self.verify(a, expected_cert_hash).ok())
if !present { .collect();
return Err(Error::RequiredMissing(required.clone())); for required in &self.required_types {
} let present = verified
.iter()
.any(|v| &v.type_ == required && v.fully_verified);
if !present {
return Err(Error::RequiredMissing(required.clone()));
} }
} }
Ok(verified) Ok(verified)
@@ -324,6 +349,38 @@ mod tests {
.any(|v| v.type_ == "other/v1" && !v.fully_verified)); .any(|v| v.type_ == "other/v1" && !v.fully_verified));
} }
/// The strict policies must surface a batch member's verification error
/// through `verify_all`, not launder it into a partial result. This binds
/// the POLICY BOUNDARY itself: only the `RequireAll` arm may drop failing
/// entries, and an apparently harmless cleanup that applies its
/// `filter_map(Result::ok)` shape to the strict arms would silently turn
/// the fail-closed contracts into fail-open — with every per-attestation
/// `verify` test still green, because none of them go through the batch
/// path. The failing entry is a wrong-cert-hash attestation (signed over a
/// different hash than the batch expects), a genuine `Err` under every
/// policy.
#[test]
fn strict_policies_propagate_a_batch_members_error() {
let sk = fresh();
let cert_hash = sample_hash();
let mut wrong_hash = sample_hash();
wrong_hash[0] ^= 0xff;
for policy in [Policy::AcceptKnown, Policy::RejectUnknown] {
let mut reg = Registry::new().with_policy(policy);
reg.register(DemoVerifier);
let good = signed_demo(&sk, cert_hash, "ok");
let bad = signed_demo(&sk, wrong_hash, "ok");
let err = reg
.verify_all(&[good, bad], cert_hash)
.expect_err("a strict policy must fail the whole batch");
assert!(
matches!(err, Error::Signature(_) | Error::CertHashMismatch),
"expected the underlying verification error, got {err:?}"
);
}
}
/// A required type that is present but unverified (no verifier registered /// A required type that is present but unverified (no verifier registered
/// even though the type appears in `required_types`) must fail /// even though the type appears in `required_types`) must fail
/// `verify_all` with `RequiredMissing`. Catches a misconfiguration where /// `verify_all` with `RequiredMissing`. Catches a misconfiguration where
@@ -341,6 +398,65 @@ mod tests {
assert!(matches!(err, Error::RequiredMissing(t) if t == "demo/v1")); assert!(matches!(err, Error::RequiredMissing(t) if t == "demo/v1"));
} }
/// A cert-hash that does not match `sample_hash()`, so an attestation
/// signed against it fails `verify_signature` when the batch is checked
/// against `sample_hash()`.
fn other_hash() -> [u8; 32] {
let mut h = sample_hash();
h[0] ^= 0xff;
h
}
/// Under `RequireAll`, a malformed attestation (here: signed against the
/// wrong cert hash, so it fails signature verification) attached alongside a
/// valid required one must not abort the batch. The junk is dropped and the
/// cert still verifies. Before the fix, `verify_all` propagated the error
/// via `?` and the whole batch failed — the DoS vector the docstring warns
/// about.
#[test]
fn require_all_drops_a_malformed_extra_and_still_accepts() {
let sk = fresh();
let cert_hash = sample_hash();
let mut reg = Registry::new()
.with_policy(Policy::RequireAll)
.require_types(["demo/v1"]);
reg.register(DemoVerifier);
// Signed against `other_hash()`, so it will not verify under `cert_hash`.
let malformed = signed_demo(&sk, other_hash(), "boom");
let real = signed_demo(&sk, cert_hash, "ok");
let verified = reg
.verify_all(&[malformed, real], cert_hash)
.expect("a malformed extra must not abort the batch under RequireAll");
assert_eq!(verified.len(), 1, "the malformed entry should be dropped");
assert!(verified
.iter()
.any(|v| v.type_ == "demo/v1" && v.fully_verified));
}
/// Leniency must not open a hole: if the *only* attestation of a required
/// type is malformed, dropping it leaves the required type absent and
/// `verify_all` must still fail with `RequiredMissing`. Guards against a fix
/// that skips errors so eagerly it lets an unverified required type pass.
#[test]
fn require_all_rejects_when_only_a_malformed_required_type_is_present() {
let sk = fresh();
let cert_hash = sample_hash();
let mut reg = Registry::new()
.with_policy(Policy::RequireAll)
.require_types(["demo/v1"]);
reg.register(DemoVerifier);
// The sole demo/v1 is bound to the wrong cert hash → fails verification.
let malformed_required = signed_demo(&sk, other_hash(), "ok");
let err = reg
.verify_all(&[malformed_required], cert_hash)
.unwrap_err();
assert!(matches!(err, Error::RequiredMissing(t) if t == "demo/v1"));
}
#[test] #[test]
fn payload_check_failure_rejects() { fn payload_check_failure_rejects() {
let sk = fresh(); let sk = fresh();
+381 -11
View File
@@ -5,6 +5,28 @@
//! //!
//! The schema is frozen at v1. All fields are mandatory for forward compatibility. //! The schema is frozen at v1. All fields are mandatory for forward compatibility.
//! Nodes that receive a certificate with an unknown version MUST reject it. //! Nodes that receive a certificate with an unknown version MUST reject it.
//!
//! ## Replay
//!
//! A certificate proves *who* asserted a ref update and *what* they asserted.
//! It does not prove the assertion is fresh. [`RefUpdateBody::seq`] and
//! [`RefUpdateBody::nonce`] are inside the signed bytes (see
//! [`RefUpdateBody::to_signing_bytes`]), so a verifier can trust the values,
//! but deciding whether a nonce is being reused or a `seq` is going backwards
//! needs state.
//!
//! This crate is deliberately stateless — it is embedded in both nodes and
//! clients — so replay is **not** enforced here and cannot be. Ownership of
//! the check sits with the verifier that accepts and persists certificates,
//! which must:
//!
//! - reject a `nonce` it has already accepted for the same `(repo, ref_name)`;
//! - reject a `seq` that does not advance past the last accepted `seq` for
//! that `(repo, ref_name)`.
//!
//! Until such a verifier exists, replay of ref-update certificates is not
//! enforced anywhere in this workspace. This contract exists so the first one
//! inherits the requirement instead of re-deriving it.
use std::collections::HashSet; use std::collections::HashSet;
@@ -19,6 +41,30 @@ use crate::{Error, Result};
/// The certificate type discriminant. Always `"gitlawb/ref-update/v1"`. /// The certificate type discriminant. Always `"gitlawb/ref-update/v1"`.
pub const CERT_TYPE: &str = "gitlawb/ref-update/v1"; pub const CERT_TYPE: &str = "gitlawb/ref-update/v1";
/// Exact url-safe unpadded base64 length of a 64-byte Ed25519 signature.
/// The tolerant threshold scan requires this canonical unpadded length before
/// attempting decode: the length gate is a DoS bound that keeps an
/// attacker-sized sig string from costing allocation before the signature is
/// rejected. A longer string is not meaningless (the same 64 bytes can appear
/// in padded or non-url-safe encodings) but no in-tree signer produces one,
/// so the scan skips non-canonical entries rather than decoding them.
const ENCODED_SIG_LEN: usize = 86;
/// Hard ceiling on signature entries a certificate may carry into the
/// tolerant threshold scan. A legitimate certificate holds one entry per
/// maintainer (a re-sign after key rotation may briefly leave a second), so
/// this is far above any real signer set, while turning the scan's worst case
/// from "one curve verification per attacker-appended entry" into a small
/// constant. Rejection is loud (`Err`), not a silent prefix scan: scanning
/// only the first N entries would let junk placed ahead of the real
/// signatures starve them out of the window, which is the #349 denial again
/// in positional form. An attacker positioned to inflate the list past this
/// ceiling can already suppress the certificate outright. The loud
/// rejection does deny a legitimately over-long certificate; that is an
/// explicit, detectable denial above the cap rather than a silent window
/// junk placement could exploit.
const MAX_SIGNATURE_ENTRIES: usize = 64;
/// A signature on a ref-update certificate. /// A signature on a ref-update certificate.
#[derive(Debug, Clone, Serialize, Deserialize)] #[derive(Debug, Clone, Serialize, Deserialize)]
pub struct CertSignature { pub struct CertSignature {
@@ -41,11 +87,19 @@ pub struct RefUpdateBody {
pub from: String, pub from: String,
/// The new commit hash (SHA-256 hex). /// The new commit hash (SHA-256 hex).
pub to: String, pub to: String,
/// Monotonically increasing sequence number. Prevents replay. /// Monotonically increasing sequence number, scoped to `(repo, ref_name)`.
///
/// Signed, but not enforced by this crate: core holds no state. The
/// verifier that persists certificates must reject a `seq` that does not
/// advance for the same repo and ref. See the module docs on replay.
pub seq: u64, pub seq: u64,
/// RFC 3339 timestamp of this update. /// RFC 3339 timestamp of this update.
pub timestamp: DateTime<Utc>, pub timestamp: DateTime<Utc>,
/// Random nonce for deduplication. /// Random nonce, unique per certificate.
///
/// Signed, but not enforced by this crate: core holds no state. The
/// verifier that persists certificates must reject a nonce it has already
/// accepted for the same repo and ref. See the module docs on replay.
pub nonce: String, pub nonce: String,
} }
@@ -108,9 +162,15 @@ impl RefUpdateCert {
Ok(()) Ok(())
} }
/// Verify all signatures on this certificate. /// Verify all signatures on this certificate, failing closed.
/// ///
/// Returns the list of DIDs whose signatures are valid. /// On success returns the list of DIDs whose signatures are valid. This is
/// strict: it returns an error if *any* signature entry is malformed or
/// invalid. Because signature entries live outside the signed body (anyone
/// can append one without key material), do not use this to make a
/// threshold decision on an untrusted certificate — a single appended junk
/// entry would reject the whole cert. Use [`Self::satisfies_threshold`],
/// which ignores invalid entries and counts only the valid signers.
pub fn verify_all(&self) -> Result<Vec<Did>> { pub fn verify_all(&self) -> Result<Vec<Did>> {
use base64::{engine::general_purpose::URL_SAFE_NO_PAD, Engine}; use base64::{engine::general_purpose::URL_SAFE_NO_PAD, Engine};
let signing_bytes = self.body.to_signing_bytes()?; let signing_bytes = self.body.to_signing_bytes()?;
@@ -139,12 +199,65 @@ impl RefUpdateCert {
/// from the provided set of authorized maintainer DIDs. /// from the provided set of authorized maintainer DIDs.
/// ///
/// Counts distinct signer DIDs, not signature entries: a repeated /// Counts distinct signer DIDs, not signature entries: a repeated
/// signature from the same maintainer counts once. /// signature from the same maintainer counts once. Malformed and invalid
/// signature entries are skipped, not fatal — entries live outside the
/// signed body, so any third party can append junk without key material,
/// and short-circuiting on the first bad one (as [`Self::verify_all`]
/// does) would let that junk deny an otherwise valid certificate (#349).
///
/// Tolerance must not become an unbounded verification path (#349 review
/// round 2): entries are attacker-extensible, so the work done for them is
/// bounded BEFORE the loop, not discovered inside it:
///
/// - a list over [`MAX_SIGNATURE_ENTRIES`] is rejected outright (see the
/// constant for why loud rejection beats a silent prefix scan);
/// - an encoded signature is length-gated before base64 work — a 64-byte
/// Ed25519 signature is exactly [`ENCODED_SIG_LEN`] characters in
/// url-safe unpadded base64, so an oversized blob never allocates;
/// - entries whose signer is not in `maintainers`, or is already counted,
/// are skipped on a string compare, before any decode or curve work —
/// so even within the cap, junk naming strangers costs no crypto;
/// - the scan returns as soon as the threshold is met.
pub fn satisfies_threshold(&self, maintainers: &[Did], threshold: usize) -> Result<bool> { pub fn satisfies_threshold(&self, maintainers: &[Did], threshold: usize) -> Result<bool> {
let valid = self.verify_all()?; use base64::{engine::general_purpose::URL_SAFE_NO_PAD, Engine};
let distinct_signers: HashSet<&Did> = if threshold == 0 {
valid.iter().filter(|d| maintainers.contains(d)).collect(); return Err(Error::RefCert("threshold must be at least 1".to_string()));
Ok(distinct_signers.len() >= threshold) }
if self.signatures.len() > MAX_SIGNATURE_ENTRIES {
return Err(Error::RefCert(format!(
"certificate carries {} signature entries (max {})",
self.signatures.len(),
MAX_SIGNATURE_ENTRIES
)));
}
let signing_bytes = self.body.to_signing_bytes()?;
let mut counted: HashSet<&Did> = HashSet::new();
for cert_sig in &self.signatures {
if counted.contains(&cert_sig.signer) || !maintainers.contains(&cert_sig.signer) {
continue;
}
if cert_sig.sig.len() != ENCODED_SIG_LEN {
continue;
}
let Ok(vk) = cert_sig.signer.to_verifying_key() else {
continue;
};
let Ok(sig_bytes_vec) = URL_SAFE_NO_PAD.decode(&cert_sig.sig) else {
continue;
};
let Ok(sig_bytes) = <[u8; 64]>::try_from(sig_bytes_vec) else {
continue;
};
if verify(&vk, &signing_bytes, &sig_bytes).is_ok() {
counted.insert(&cert_sig.signer);
if counted.len() >= threshold {
return Ok(true);
}
}
}
Ok(counted.len() >= threshold)
} }
/// Validate the certificate structure (not signatures). /// Validate the certificate structure (not signatures).
@@ -377,8 +490,151 @@ mod tests {
assert!(!cert.satisfies_threshold(&maintainers, 2).unwrap()); assert!(!cert.satisfies_threshold(&maintainers, 2).unwrap());
} }
/// An attacker can append junk signature entries to a cert (they live
/// outside the signed body, so no key material is needed). `satisfies_threshold`
/// must ignore them and still count the real signers — before the fix,
/// `verify_all`'s `?` on the junk made the whole check error out, denying a
/// valid 2-of-2 cert.
#[test] #[test]
fn threshold_zero_is_always_satisfied() { fn satisfies_threshold_ignores_appended_junk_signatures() {
let kp1 = Keypair::generate();
let kp2 = Keypair::generate();
let mut cert = RefUpdateCert::new(
kp1.did(),
"refs/heads/main".to_string(),
dummy_hash('0'),
dummy_hash('a'),
1,
&kp1,
)
.unwrap();
cert.countersign(&kp2).unwrap();
// Junk sits AHEAD of the real entries so the scan must actually skip
// it: appended last, the threshold-2 early return at the second real
// signature would fire before any junk is examined and the test would
// stay green even if the skip logic never ran.
cert.signatures.insert(
0,
CertSignature {
signer: kp1.did(),
sig: "!!!not-base64!!!".to_string(),
},
);
// Valid base64 but only 3 bytes — fails the 64-byte length check.
cert.signatures.insert(
0,
CertSignature {
signer: kp2.did(),
sig: "AAAA".to_string(),
},
);
// Exactly ENCODED_SIG_LEN of valid base64 decoding to garbage —
// exercises the decode+verify path before the skip, not just the
// length gate.
cert.signatures.insert(
0,
CertSignature {
signer: kp1.did(),
sig: "A".repeat(ENCODED_SIG_LEN),
},
);
let maintainers = vec![kp1.did(), kp2.did()];
assert!(cert.satisfies_threshold(&maintainers, 2).unwrap());
}
/// A junk entry that names a real maintainer DID but carries a signature
/// that does not verify must not be counted — otherwise appending garbage
/// under a maintainer's DID could forge a threshold.
#[test]
fn satisfies_threshold_does_not_count_a_forged_maintainer_signature() {
let kp1 = Keypair::generate();
let kp2 = Keypair::generate();
let mut cert = RefUpdateCert::new(
kp1.did(),
"refs/heads/main".to_string(),
dummy_hash('0'),
dummy_hash('a'),
1,
&kp1,
)
.unwrap();
// Well-formed 64-byte signature from kp2, but over unrelated bytes, so
// it fails verification against the cert body.
cert.signatures.push(CertSignature {
signer: kp2.did(),
sig: kp2.sign_b64(b"unrelated bytes"),
});
let maintainers = vec![kp1.did(), kp2.did()];
// Only kp1 actually signed the body: 2-of-2 must fail, 1-of-2 holds.
assert!(!cert.satisfies_threshold(&maintainers, 2).unwrap());
assert!(cert.satisfies_threshold(&maintainers, 1).unwrap());
}
/// The tolerant scan must not be an unbounded verification path: a list
/// past MAX_SIGNATURE_ENTRIES is rejected loudly before any curve work,
/// while a list at the cap still verifies normally.
#[test]
fn satisfies_threshold_rejects_an_oversized_signature_list() {
let kp = Keypair::generate();
let mut cert = RefUpdateCert::new(
kp.did(),
"refs/heads/main".to_string(),
dummy_hash('0'),
dummy_hash('a'),
1,
&kp,
)
.unwrap();
let junk = || CertSignature {
signer: Keypair::generate().did(),
sig: "AAAA".to_string(),
};
while cert.signatures.len() < MAX_SIGNATURE_ENTRIES {
cert.signatures.push(junk());
}
let maintainers = vec![kp.did()];
// At the cap: still fine, the real signature counts.
assert!(cert.satisfies_threshold(&maintainers, 1).unwrap());
// One past the cap: loud rejection, no tolerant scan.
cert.signatures.push(junk());
let err = cert.satisfies_threshold(&maintainers, 1).unwrap_err();
assert!(err.to_string().contains("signature entries"));
}
/// An entry whose encoded signature is not the exact base64 length of a
/// 64-byte Ed25519 signature is skipped before any base64 decode — an
/// attacker-sized sig string must cost neither allocation nor curve work,
/// and must not affect the verdict.
#[test]
fn satisfies_threshold_skips_an_oversized_encoded_signature() {
let kp1 = Keypair::generate();
let kp2 = Keypair::generate();
let mut cert = RefUpdateCert::new(
kp1.did(),
"refs/heads/main".to_string(),
dummy_hash('0'),
dummy_hash('a'),
1,
&kp1,
)
.unwrap();
// Names a real maintainer, valid base64 alphabet, but megabytes long:
// the length gate must refuse it without decoding.
cert.signatures.push(CertSignature {
signer: kp2.did(),
sig: "A".repeat(2 * 1024 * 1024),
});
let maintainers = vec![kp1.did(), kp2.did()];
assert!(cert.satisfies_threshold(&maintainers, 1).unwrap());
assert!(!cert.satisfies_threshold(&maintainers, 2).unwrap());
}
#[test]
fn threshold_zero_is_rejected() {
let kp = Keypair::generate(); let kp = Keypair::generate();
let cert = RefUpdateCert::new( let cert = RefUpdateCert::new(
kp.did(), kp.did(),
@@ -389,6 +645,120 @@ mod tests {
&kp, &kp,
) )
.unwrap(); .unwrap();
assert!(cert.satisfies_threshold(&[], 0).unwrap());
// Zero must never read as "no signatures required", even with an empty
// maintainer set.
let err = cert.satisfies_threshold(&[], 0).unwrap_err();
assert!(
err.to_string().contains("threshold must be at least 1"),
"zero must be refused as an invalid threshold"
);
}
/// The threshold is checked before any signature work, so a zero is
/// reported as a threshold error even when the certificate's signatures
/// are also invalid. Otherwise a broken certificate would let a caller
/// mistake the failure for a legitimate verification result.
#[test]
fn threshold_zero_is_rejected_before_signature_verification() {
let kp = Keypair::generate();
let other = Keypair::generate();
let mut cert = RefUpdateCert::new(
kp.did(),
"refs/heads/main".to_string(),
dummy_hash('0'),
dummy_hash('a'),
1,
&kp,
)
.unwrap();
cert.signatures[0].sig = other.sign_b64(b"unrelated bytes");
let err = cert.satisfies_threshold(&[kp.did()], 0).unwrap_err();
assert!(
err.to_string().contains("threshold must be at least 1"),
"the invalid threshold must win over the invalid signature"
);
}
/// `seq` and `nonce` are the verifier's replay inputs, so they have to be
/// inside the signed body; tapping either one must invalidate the
/// certificate rather than leave it verifying with different values.
#[test]
fn replay_fields_are_covered_by_the_signature() {
let kp = Keypair::generate();
let cert = RefUpdateCert::new(
kp.did(),
"refs/heads/main".to_string(),
dummy_hash('0'),
dummy_hash('a'),
7,
&kp,
)
.unwrap();
cert.verify_all().expect("baseline must verify");
let mut bumped = cert.clone();
bumped.body.seq = 8;
assert!(
matches!(bumped.verify_all(), Err(Error::SignatureInvalid)),
"changing seq must invalidate the signature"
);
let mut renamed = cert.clone();
renamed.body.nonce = "different-nonce".to_string();
assert!(
matches!(renamed.verify_all(), Err(Error::SignatureInvalid)),
"changing nonce must invalidate the signature"
);
}
/// A signature minted by a real key must not verify when the signer is
/// spelled in another multibase alphabet. Otherwise the same identity
/// would satisfy a maintainer allowlist under several DID strings.
///
/// Base64url is deliberate: its 47-character method-id fits under the
/// pre-existing length bound, so this test reaches the canonical-prefix
/// rule and fails on the pre-fix code instead of being refused for length.
#[test]
fn verify_all_rejects_a_non_canonical_did_spelling() {
let kp = Keypair::generate();
let mut cert = RefUpdateCert::new(
kp.did(),
"refs/heads/main".to_string(),
dummy_hash('0'),
dummy_hash('a'),
1,
&kp,
)
.unwrap();
let mut prefixed = vec![0xed, 0x01];
prefixed.extend_from_slice(&kp.verifying_key().to_bytes());
cert.signatures[0].signer = format!(
"did:key:{}",
multibase::encode(multibase::Base::Base64Url, &prefixed)
)
.parse()
.expect("method is still `key`");
let err = cert
.verify_all()
.expect_err("a non-canonical did:key spelling must not resolve");
assert!(
err.to_string().contains("base58btc"),
"rejection must come from the canonical-prefix rule, not the length guard"
);
assert!(
!matches!(
cert.satisfies_threshold(&[cert.signatures[0].signer.clone()], 1),
Ok(true)
),
"the alternate spelling must not satisfy a threshold"
);
// The canonical spelling of the same key is unaffected.
cert.signatures[0].signer = kp.did();
assert!(cert.satisfies_threshold(&[kp.did()], 1).unwrap());
} }
} }
+158 -3
View File
@@ -20,6 +20,19 @@ use crate::{Error, Result};
/// Value: 0xed01 encoded as a varint. /// Value: 0xed01 encoded as a varint.
const ED25519_MULTICODEC: &[u8] = &[0xed, 0x01]; const ED25519_MULTICODEC: &[u8] = &[0xed, 0x01];
/// The only multibase prefix a `did:key` method-specific id may carry.
///
/// The DID:key spec encodes Ed25519 keys as base58btc, which is multibase
/// `z`. `multibase::decode` happily accepts every other alphabet, and each
/// spelling of the same key bytes would produce a distinct DID string while
/// gates, allowlists, and database rows compare DIDs as strings — one
/// identity would read as several. Rejecting the alternatives here keeps the
/// mapping from `did:key` to key single-valued for every caller of
/// [`Did::to_verifying_key`] (HTTP signatures, UCAN, ref certs, peer
/// announces). `gitlawb-attest` enforces the same rule for attestation
/// signers.
const DID_KEY_MULTIBASE_PREFIX: char = 'z';
/// A Decentralized Identifier. /// A Decentralized Identifier.
#[derive(Debug, Clone, PartialEq, Eq, Hash, Serialize, Deserialize)] #[derive(Debug, Clone, PartialEq, Eq, Hash, Serialize, Deserialize)]
#[serde(transparent)] #[serde(transparent)]
@@ -71,8 +84,14 @@ impl Did {
/// Resolve the Ed25519 verifying key from a `did:key`. /// Resolve the Ed25519 verifying key from a `did:key`.
/// ///
/// Returns `Err` if the DID is not a `did:key`, the key bytes are malformed, /// Only the canonical base58btc (`z`) spelling is accepted. Other
/// or the key is a small-order (weak) point. /// multibase encodings decode to the same key but spell a different DID
/// string, so accepting them would let one identity appear under several
/// identifiers to string-comparing gates and allowlists.
///
/// Returns `Err` if the DID is not a `did:key`, uses a non-canonical
/// multibase prefix, the key bytes are invalid, or the key is a
/// small-order (weak) point.
pub fn to_verifying_key(&self) -> Result<VerifyingKey> { pub fn to_verifying_key(&self) -> Result<VerifyingKey> {
if !self.is_did_key() { if !self.is_did_key() {
return Err(Error::InvalidDid(format!( return Err(Error::InvalidDid(format!(
@@ -94,9 +113,27 @@ impl Did {
)); ));
} }
let (_, bytes) = // Cheap prefix gate before the decode, then the base check on what
// multibase actually parsed. The first keeps a clearly non-canonical id
// off the decoder. The second is belt-and-braces — multibase maps a
// leading `z` only to `Base58Btc` — but it states the requirement
// directly instead of relying on that mapping, and `gitlawb-attest`
// makes the same pair of checks.
if !self.method_id().starts_with(DID_KEY_MULTIBASE_PREFIX) {
return Err(Error::InvalidDid(
"did:key must use base58btc (z-prefix)".to_string(),
));
}
let (base, bytes) =
multibase::decode(self.method_id()).map_err(|e| Error::InvalidDid(e.to_string()))?; multibase::decode(self.method_id()).map_err(|e| Error::InvalidDid(e.to_string()))?;
if base != Base::Base58Btc {
return Err(Error::InvalidDid(
"did:key must use base58btc (z-prefix)".to_string(),
));
}
if !bytes.starts_with(ED25519_MULTICODEC) { if !bytes.starts_with(ED25519_MULTICODEC) {
return Err(Error::InvalidDid( return Err(Error::InvalidDid(
"not an ed25519 multicodec key".to_string(), "not an ed25519 multicodec key".to_string(),
@@ -389,6 +426,124 @@ mod tests {
did.to_verifying_key().expect("a real did:key must resolve"); did.to_verifying_key().expect("a real did:key must resolve");
} }
/// `multibase::decode` accepts every alphabet, so the same Ed25519 key can
/// be spelled as several `did:key` strings. Only base58btc is canonical;
/// the rest must be refused so one identity cannot appear as several to
/// gates and allowlists that compare DID strings.
#[test]
fn to_verifying_key_rejects_non_base58btc_multibase() {
use ed25519_dalek::SigningKey;
let signing = SigningKey::from_bytes(&[9u8; 32]);
let mut prefixed = ED25519_MULTICODEC.to_vec();
prefixed.extend_from_slice(&signing.verifying_key().to_bytes());
let canonical = Did::from_verifying_key(&signing.verifying_key());
assert_eq!(
canonical.to_verifying_key().unwrap().to_bytes(),
signing.verifying_key().to_bytes(),
"base58btc is the accepted spelling"
);
// These spellings fit inside the method-id bound, so they reach the
// canonical-prefix check and must be refused there.
for base in [Base::Base64Url, Base::Base32Lower] {
let alternative = format!("did:key:{}", multibase::encode(base, &prefixed));
let did: Did = alternative.parse().expect("method is still `key`");
assert_ne!(
did.as_str(),
canonical.as_str(),
"the same key must not compare equal under another spelling"
);
let err = did
.to_verifying_key()
.expect_err("non-canonical multibase must be refused");
assert!(
matches!(err, Error::InvalidDid(_)),
"must be an InvalidDid, got {err:?}"
);
assert!(
err.to_string().contains("base58btc"),
"the error must name the canonical encoding: {err}"
);
}
// A base16 spelling of the same key is longer than the method-id
// bound, so it is refused by the length guard before the prefix check.
// Either way it must not resolve.
for base in [Base::Base16Lower, Base::Base16Upper] {
let alternative = format!("did:key:{}", multibase::encode(base, &prefixed));
let did: Did = alternative.parse().expect("method is still `key`");
assert!(
matches!(did.to_verifying_key(), Err(Error::InvalidDid(_))),
"a base16 spelling of a real key must not resolve"
);
}
}
/// base58flickr (`Z`) is the non-bitcoin sibling alphabet. Its spelling of
/// a real key fits under the length bound, so it must be refused for the
/// prefix, not merely for its size.
#[test]
fn to_verifying_key_rejects_a_base58flickr_spelling() {
use ed25519_dalek::SigningKey;
let signing = SigningKey::from_bytes(&[11u8; 32]);
let mut prefixed = ED25519_MULTICODEC.to_vec();
prefixed.extend_from_slice(&signing.verifying_key().to_bytes());
let did: Did = format!(
"did:key:{}",
multibase::encode(Base::Base58Flickr, &prefixed)
)
.parse()
.expect("method is still `key`");
let err = did
.to_verifying_key()
.expect_err("base58flickr must be refused");
assert!(
err.to_string().contains("base58btc"),
"the error must name the canonical encoding: {err}"
);
}
/// An empty method-specific id must be refused by the prefix rule without
/// panicking on a missing first character.
#[test]
fn to_verifying_key_rejects_an_empty_method_id() {
let did: Did = "did:key:".parse().expect("method is still `key`");
let err = did
.to_verifying_key()
.expect_err("an empty method id must be refused");
assert!(
err.to_string().contains("base58btc"),
"the error must name the canonical encoding: {err}"
);
}
/// A canonical base58btc spelling is not enough on its own: the bytes must
/// also carry the ed25519-pub multicodec. This pins the check that stops a
/// key of another type from resolving as an Ed25519 key.
#[test]
fn to_verifying_key_rejects_base58btc_with_a_wrong_multicodec() {
use ed25519_dalek::SigningKey;
let signing = SigningKey::from_bytes(&[12u8; 32]);
let mut buf = vec![0x12, 0x00]; // sha2-256, not ed25519-pub
buf.extend_from_slice(&signing.verifying_key().to_bytes());
let did: Did = format!("did:key:{}", multibase::encode(Base::Base58Btc, &buf))
.parse()
.expect("method is still `key`");
let err = did
.to_verifying_key()
.expect_err("a non-ed25519 multicodec must be refused");
assert!(
err.to_string().contains("ed25519"),
"the error must name the expected multicodec: {err}"
);
}
/// A `did:key` can encode a small-order (weak) Ed25519 point. Such a key /// A `did:key` can encode a small-order (weak) Ed25519 point. Such a key
/// satisfies the verification equation for any message and, converted to /// satisfies the verification equation for any message and, converted to
/// Montgomery form for X25519, yields the all-zero shared secret, so an /// Montgomery form for X25519, yields the all-zero shared secret, so an
+20
View File
@@ -566,6 +566,26 @@ mod tests {
assert!(sig.check_created().is_ok()); assert!(sig.check_created().is_ok());
} }
#[test]
fn extreme_created_timestamps_reject_without_wrapping() {
let kp = Keypair::generate();
let did = kp.did();
let parse = |created: i64| {
let sig_input =
format!(r#"sig1=("@method");keyid="{did}";alg="ed25519";created={created}"#);
HttpSignature::parse(&sig_input, "sig1=:AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA:").unwrap()
};
// i64::MIN used to panic in debug (subtraction overflow) and wrap to a
// passing skew in release; now - 2^63 wrapped to i64::MIN, whose abs()
// wrapped back to itself and read as fresh. Both must reject.
for created in [i64::MIN, i64::MAX, Utc::now().timestamp() - i64::MAX] {
assert!(
parse(created).check_created().is_err(),
"created={created} must not pass the freshness gate"
);
}
}
#[test] #[test]
fn parse_error_missing_sig1_prefix() { fn parse_error_missing_sig1_prefix() {
let err = HttpSignature::parse( let err = HttpSignature::parse(
+40
View File
@@ -77,6 +77,14 @@ impl Keypair {
} }
/// Verify an Ed25519 signature. /// Verify an Ed25519 signature.
///
/// Strict verification: rejects small-order `R` and small-order public keys
/// (the identity point, and any point of low order). Ordinary `verify` accepts
/// a signature forged with the identity point as the public key plus
/// `R = identity, S = 0`, which verifies for *any* message. `identity::verify`
/// is the shared primitive behind HTTP request authentication, UCANs, and
/// certificates, so weak-key acceptance is an authentication bypass, not a
/// malleability nuance.
pub fn verify(verifying_key: &VerifyingKey, msg: &[u8], sig_bytes: &[u8; 64]) -> Result<()> { pub fn verify(verifying_key: &VerifyingKey, msg: &[u8], sig_bytes: &[u8; 64]) -> Result<()> {
let sig = Signature::from_bytes(sig_bytes); let sig = Signature::from_bytes(sig_bytes);
verifying_key verifying_key
@@ -208,6 +216,38 @@ mod tests {
); );
} }
/// The identity-point forgery: with public key A = identity, R = identity,
/// and S = 0, the equation `[S]B = R + [k]A` holds for every message,
/// because `[k]·identity = identity`. Ordinary (non-strict) Ed25519
/// verification accepts it, so the shared `verify` primitive must use
/// strict verification, which rejects small-order R and public keys.
#[test]
fn verify_rejects_identity_point_forgery() {
use ed25519_dalek::Verifier;
// The identity point (0,1) compresses to y = 1 with sign bit 0.
let identity = [
1u8, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0,
];
let vk = VerifyingKey::from_bytes(&identity).expect("identity point is on the curve");
let mut sig_bytes = [0u8; 64];
sig_bytes[..32].copy_from_slice(&identity);
let msg = b"arbitrary message the key owner never signed";
// Prove the forged signature satisfies the ordinary verification
// equation, so the strict check below is what actually defends the
// boundary (not a signature that was already invalid everywhere).
assert!(
vk.verify(msg, &Signature::from_bytes(&sig_bytes)).is_ok(),
"identity-point forgery must satisfy ordinary verification (this is why strict is needed)"
);
assert!(
verify(&vk, msg, &sig_bytes).is_err(),
"strict verification must reject the identity-point forgery"
);
}
#[test] #[test]
fn verify_rejects_weak_key_signature() { fn verify_rejects_weak_key_signature() {
// Regression guard for strict verification: a signature forged under a // Regression guard for strict verification: a signature forged under a
+299
View File
@@ -0,0 +1,299 @@
//! Where the identity key and the delegation store live.
//!
//! `gl` and `git-remote-gitlawb` have to agree on this. `gl ucan import` writes a
//! delegation to `<dir>/delegations/`, and the helper reads it back from the same
//! place when it builds the `X-Ucan` header on push. If the two resolve
//! `GITLAWB_KEY` differently the push goes out with no header and the node refuses
//! the delegate, with nothing on either side to say why — so the rules live here
//! once, in the crate both binaries already depend on, rather than being written
//! twice and drifting.
//!
//! The home directory is a parameter rather than something this module looks up.
//! `gitlawb-core` is embedded by every consumer and is held to an explicit
//! dependency allowlist (`ci/gitlawb-core-allowed-deps.txt`); the callers already
//! carry `dirs`, so taking `home` keeps the rules shared without widening core's
//! tree. It also makes every case below testable against a fixed home.
use std::ffi::OsStr;
use std::path::{Component, Path, PathBuf};
use crate::{Error, Result};
/// Environment variable naming the identity PEM.
pub const KEY_ENV: &str = "GITLAWB_KEY";
/// Name of the PEM file inside the identity directory.
pub const KEY_FILE_NAME: &str = "identity.pem";
/// Directory under the home directory used when `GITLAWB_KEY` is unset.
pub const DEFAULT_DIR_NAME: &str = ".gitlawb";
/// Absolute path of the identity PEM: `$GITLAWB_KEY`, else
/// `<home>/.gitlawb/identity.pem`.
///
/// An empty `GITLAWB_KEY` counts as unset. That is what a shell leaves behind for
/// `FOO=` and for an unset variable expanded into a wrapper script, and reading it
/// as a path would resolve against the process working directory instead.
pub fn identity_key_path(home: Option<&Path>) -> Result<PathBuf> {
// `var_os`, not `var`: `var` folds a non-UTF-8 value into the same `Err` as
// unset, so an operator whose key path is not valid UTF-8 would silently get
// the default directory rather than theirs — or an error naming the real
// problem.
match std::env::var_os(KEY_ENV) {
Some(raw) if !raw.is_empty() => resolve_key_value(Path::new(&raw), home),
_ => Ok(require_home(home)?
.join(DEFAULT_DIR_NAME)
.join(KEY_FILE_NAME)),
}
}
/// The directory holding `identity.pem` and `delegations/` — the parent of
/// [`identity_key_path`].
pub fn identity_dir(home: Option<&Path>) -> Result<PathBuf> {
let key = identity_key_path(home)?;
key.parent().map(Path::to_path_buf).ok_or_else(|| {
Error::Key(format!(
"{KEY_ENV} has no parent directory: {}",
key.display()
))
})
}
/// The home directory, demanded only where the value being resolved needs it.
///
/// An absolute `GITLAWB_KEY` never needs home, so requiring it up front would
/// discard a perfectly good key on a host where `dirs::home_dir()` returns `None`
/// — no `HOME` and no passwd entry, which is an ordinary container shape. The
/// helper would then push unsigned, blaming the home directory for a setting the
/// operator had configured correctly.
fn require_home(home: Option<&Path>) -> Result<&Path> {
home.ok_or_else(|| {
Error::Key(format!(
"could not determine the home directory, which is needed to resolve this \
{KEY_ENV} value. Set {KEY_ENV} to an absolute path to avoid needing it."
))
})
}
/// Apply the `GITLAWB_KEY` rules to a raw value.
///
/// Split out from [`identity_key_path`] so the rules can be tested without setting
/// a process-global environment variable, which would make the tests race.
fn resolve_key_value(raw: &Path, home: Option<&Path>) -> Result<PathBuf> {
let path = expand_tilde(raw, home)?;
if !path.is_absolute() {
return Err(Error::Key(format!(
"{KEY_ENV} must be an absolute path (got {}). It also determines where \
delegations are stored, and `gl` and `git-remote-gitlawb` do not share a \
working directory, so a relative path sends them to different stores.",
raw.display()
)));
}
if path.parent().is_none() {
return Err(Error::Key(format!(
"{KEY_ENV} must name the key file, not the filesystem root (got {}). \
Point it at the PEM, e.g. ~/{DEFAULT_DIR_NAME}/{KEY_FILE_NAME}.",
raw.display()
)));
}
Ok(path)
}
/// Expand a leading `~/`, and only that.
///
/// `~user` is shell syntax this does not implement; leaving its `~` in place makes
/// it fail the absolute-path check with a message that names the real problem,
/// which beats resolving it somewhere the operator did not ask for. A bare `~` or
/// `~/` is refused outright: it names a directory where a file is required, and
/// expanding it to the home directory would put the delegation store beside the
/// home directory rather than inside it, since the store is the key's *parent*.
///
/// Matched on the first path component rather than on a string prefix. That is
/// what lets the value stay an `OsStr` end to end: the helper's old
/// `str::strip_prefix("~/")` needed a `String` first, which is why it reached for
/// `env::var` and folded every non-UTF-8 path into "unset".
fn expand_tilde(path: &Path, home: Option<&Path>) -> Result<PathBuf> {
let mut components = path.components();
match components.next() {
Some(Component::Normal(first)) if first == OsStr::new("~") => {
let rest = components.as_path();
if rest.as_os_str().is_empty() {
return Err(Error::Key(format!(
"{KEY_ENV} must name the key file, not a directory (got {}). \
Point it at the PEM, e.g. ~/{DEFAULT_DIR_NAME}/{KEY_FILE_NAME}.",
path.display()
)));
}
Ok(require_home(home)?.join(rest))
}
_ => Ok(path.to_path_buf()),
}
}
#[cfg(test)]
mod tests {
use super::*;
/// A home that is absolute on the host running the tests. `/home/op` is not
/// absolute on Windows — it has a root but no prefix — so a shared literal
/// would make the absolute-path assertions test the wrong thing there.
fn home() -> PathBuf {
if cfg!(windows) {
PathBuf::from(r"C:\Users\op")
} else {
PathBuf::from("/home/op")
}
}
#[test]
fn absolute_value_is_taken_verbatim() {
let raw = home().join("data").join("keys").join(KEY_FILE_NAME);
assert_eq!(resolve_key_value(&raw, Some(&home())).unwrap(), raw);
}
#[test]
fn tilde_slash_expands_to_the_home_directory() {
let resolved = resolve_key_value(Path::new("~/keys/identity.pem"), Some(&home())).unwrap();
assert_eq!(resolved, home().join("keys").join(KEY_FILE_NAME));
}
/// The shell-style spelling of the default resolves to the default. Worth
/// pinning: this is the form an operator gets by copying a path out of their
/// shell, and the two binaries used to reach it by different routes.
#[test]
fn the_tilde_spelling_of_the_default_resolves_to_the_default() {
let resolved =
resolve_key_value(Path::new("~/.gitlawb/identity.pem"), Some(&home())).unwrap();
assert_eq!(resolved, home().join(DEFAULT_DIR_NAME).join(KEY_FILE_NAME));
}
/// A relative value resolves against the working directory, and `gl` and
/// `git-remote-gitlawb` do not share one: the import would land where the helper
/// never looks.
#[test]
fn relative_values_are_refused() {
for raw in ["identity.pem", "keys/identity.pem", "./keys/identity.pem"] {
assert!(
resolve_key_value(Path::new(raw), Some(&home())).is_err(),
"{raw} is relative and must be refused"
);
}
}
/// `~user` is shell syntax, not a path, and a bare `~` names a directory where
/// a file is required. Refused rather than guessed at.
#[test]
fn unsupported_tilde_forms_are_refused() {
for raw in ["~", "~/", "~someone/keys/identity.pem"] {
assert!(
resolve_key_value(Path::new(raw), Some(&home())).is_err(),
"{raw} must be refused rather than resolved"
);
}
}
/// The root has no parent, so the delegation store would have nowhere to go.
#[test]
fn the_filesystem_root_is_refused() {
assert!(resolve_key_value(Path::new("/"), Some(&home())).is_err());
}
/// The whole point of `var_os`: a non-UTF-8 value must reach the rules rather
/// than being folded into "unset" by `var`. Byte 0xFF is not valid UTF-8 in any
/// position, so this value is unreachable through `env::var`.
#[cfg(unix)]
#[test]
fn non_utf8_values_reach_the_rules() {
use std::ffi::OsString;
use std::os::unix::ffi::OsStringExt;
let relative = OsString::from_vec(b"keys/\xFF/identity.pem".to_vec());
assert!(
resolve_key_value(Path::new(&relative), Some(&home())).is_err(),
"a non-UTF-8 relative path must be refused, not silently defaulted"
);
let mut absolute = OsString::from("/data/");
absolute.push(OsString::from_vec(vec![0xFF]));
absolute.push("/identity.pem");
let resolved = resolve_key_value(Path::new(&absolute), Some(&home())).unwrap();
assert_eq!(resolved.as_os_str(), absolute.as_os_str());
}
/// Unset and empty both mean "use the default", and the two accessors must stay
/// consistent: the directory is the parent of the key, never a sibling of it.
/// The process environment is global, so the two cases share one test and one
/// lock rather than racing each other.
#[test]
fn unset_and_empty_both_select_the_default_directory() {
use std::sync::Mutex;
static LOCK: Mutex<()> = Mutex::new(());
let _guard = LOCK.lock().unwrap_or_else(|e| e.into_inner());
let restore = std::env::var_os(KEY_ENV);
std::env::remove_var(KEY_ENV);
let unset = (
identity_key_path(Some(&home())),
identity_dir(Some(&home())),
);
std::env::set_var(KEY_ENV, "");
let empty = (
identity_key_path(Some(&home())),
identity_dir(Some(&home())),
);
match restore {
Some(v) => std::env::set_var(KEY_ENV, v),
None => std::env::remove_var(KEY_ENV),
}
for (label, (key, dir)) in [("unset", unset), ("empty", empty)] {
assert_eq!(
key.unwrap(),
home().join(DEFAULT_DIR_NAME).join(KEY_FILE_NAME),
"{label} key path"
);
assert_eq!(
dir.unwrap(),
home().join(DEFAULT_DIR_NAME),
"{label} directory"
);
}
}
}
#[cfg(test)]
mod no_home_tests {
use super::*;
/// An absolute key needs no home directory. Demanding one up front discarded a
/// correctly-configured `GITLAWB_KEY` on any host where `dirs::home_dir()`
/// returns `None` — no `HOME` and no passwd entry, an ordinary container shape —
/// and the helper then pushed unsigned while blaming the home directory.
#[test]
fn an_absolute_key_resolves_without_a_home_directory() {
let raw = if cfg!(windows) {
r"C:\data\keys\identity.pem"
} else {
"/data/keys/identity.pem"
};
let resolved = resolve_key_value(Path::new(raw), None)
.expect("an absolute key must not need a home directory");
assert_eq!(resolved, PathBuf::from(raw));
assert_eq!(resolved.parent().unwrap(), Path::new(raw).parent().unwrap());
}
/// The forms that genuinely need a home still say so, rather than resolving
/// somewhere arbitrary.
#[test]
fn the_forms_that_need_a_home_report_its_absence() {
let err = resolve_key_value(Path::new("~/keys/identity.pem"), None)
.expect_err("a ~/ path cannot resolve without a home directory");
assert!(
err.to_string().contains("home directory"),
"the error must name the missing home directory, got: {err}"
);
}
}
+1
View File
@@ -5,6 +5,7 @@ pub mod encrypt;
pub mod error; pub mod error;
pub mod http_sig; pub mod http_sig;
pub mod identity; pub mod identity;
pub mod identity_path;
// `url` is the one dependency here that drags a tail (idna, then the icu // `url` is the one dependency here that drags a tail (idna, then the icu
// crates), and gitlawb-core is allowlisted to stay embeddable. Every client that // crates), and gitlawb-core is allowlisted to stay embeddable. Every client that
// needs this predicate already parses URLs, so they opt in and nothing else // needs this predicate already parses URLs, so they opt in and nothing else
+20
View File
@@ -30,10 +30,30 @@ pub fn is_bidi_format(c: char) -> bool {
) )
} }
/// Both halves of INV-6 in one call: drop every `Cc` control (which defangs
/// ANSI and OSC escapes) and every bidi/format control that
/// [`is_bidi_format`] names. What remains is safe to write to a terminal.
///
/// Length is not this function's business: a caller that shows the text
/// decides how much of it to show.
pub fn strip_terminal_controls(s: &str) -> String {
s.chars()
.filter(|c| !c.is_control() && !is_bidi_format(*c))
.collect()
}
#[cfg(test)] #[cfg(test)]
mod tests { mod tests {
use super::*; use super::*;
#[test]
fn strip_removes_escapes_and_reordering_but_keeps_text() {
let hostile = "ok\u{1b}[31m red\u{1b}]0;title\u{7}\u{202E}dlrow\u{202C} \u{0627}\u{200D}";
let clean = strip_terminal_controls(hostile);
assert_eq!(clean, "ok[31m red]0;titledlrow \u{0627}\u{200D}");
assert!(!clean.chars().any(|c| c.is_control() || is_bidi_format(c)));
}
#[test] #[test]
fn strips_every_reordering_code_point() { fn strips_every_reordering_code_point() {
// Asserted individually so a partial predicate (e.g. one that covers the // Asserted individually so a partial predicate (e.g. one that covers the
+785 -29
View File
@@ -16,6 +16,19 @@ use crate::did::Did;
use crate::identity::Keypair; use crate::identity::Keypair;
use crate::{Error, Result}; use crate::{Error, Result};
/// Proof chains deeper than this fail closed, in every walk over `prf`:
/// [`Ucan::verify_chain`], [`Ucan::chain_lifetime_is_bounded`] and
/// [`Ucan::chain_grants_push_to`] all stop here. Nothing this codebase mints is
/// longer than owner → agent → node. The bound exists for the hand-built token:
/// the node verifies the chain on the `X-Ucan` header of any signed request, and
/// nothing else says how deep a sender may nest `prf`. In practice not very
/// deep — a proof is a JSON string inside the next link's JSON, so each level
/// re-escapes the one beneath it and the encoding roughly doubles per link
/// (eight links is about 11 KiB, thirteen about 270 KiB) — but a walk should
/// refuse on its own terms rather than count on the encoding running the sender
/// out of room first.
pub const MAX_CHAIN_DEPTH: usize = 8;
/// A UCAN capability: what resource the token grants access to. /// A UCAN capability: what resource the token grants access to.
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq)] #[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq)]
pub struct Capability { pub struct Capability {
@@ -49,11 +62,32 @@ impl Capability {
/// action field and `repo/admin` in the parent's action position act as /// action field and `repo/admin` in the parent's action position act as
/// wildcards that cover any delegated value; wildcards on `self` carry no /// wildcards that cover any delegated value; wildcards on `self` carry no
/// special meaning. /// special meaning.
///
/// Constraints (`nb`) participate, and conservatively:
///
/// | parent | child | verdict |
/// |---|---|---|
/// | none | anything | attenuated — adding constraints narrows |
/// | some | identical | attenuated |
/// | some | different | refused — narrowing is unprovable without semantics |
/// | some | none | refused — dropping constraints widens |
///
/// The last row is the one that matters. Ignoring `nb` here let a holder of a
/// constrained capability re-delegate the same resource and action with the
/// constraints removed, and the chain still verified — so a consumer that
/// refuses constrained capabilities at the leaf saw an unconstrained one and
/// granted it. Since `nb` has no interpreted semantics yet, "different" cannot
/// be shown to be narrower and is refused with it.
pub fn is_attenuated_by(&self, parent: &Capability) -> bool { pub fn is_attenuated_by(&self, parent: &Capability) -> bool {
let resource_ok = parent.with == self.with || parent.with == "*"; let resource_ok = parent.with == self.with || parent.with == "*";
let action_ok = let action_ok =
parent.can == self.can || parent.can == "*" || parent.can == caps::REPO_ADMIN; parent.can == self.can || parent.can == "*" || parent.can == caps::REPO_ADMIN;
resource_ok && action_ok let constraints_ok = match (&parent.constraints, &self.constraints) {
(None, _) => true,
(Some(p), Some(c)) => p == c,
(Some(_), None) => false,
};
resource_ok && action_ok && constraints_ok
} }
} }
@@ -71,6 +105,157 @@ pub mod caps {
pub const REPO_ADMIN: &str = "repo/admin"; pub const REPO_ADMIN: &str = "repo/admin";
} }
/// Rules shared by every boundary that decides whether a UCAN can push.
///
/// Four places answer "can this token push to this repository?": `gl ucan
/// delegate` and the MCP `ucan_delegate` tool when a token is issued, `gl ucan
/// import` when one is stored, `git-remote-gitlawb` when it mints an invocation,
/// and the node when it authorizes the push. For three review rounds each of them
/// carried its own copy of the answer, and every round found a token that one
/// boundary accepted and the next refused: a constrained grant that imported and
/// then authorized nothing; a wildcard proof the helper narrowed and the node
/// rejected; a resource whose owner the leaf named but the chain root did not.
///
/// The rules live here, once, in the crate all four already depend on. A boundary
/// may add checks of its own — the node anchors the chain root to a repository
/// record it holds independently, import binds the audience to the local key —
/// but the definition of a *usable push capability* is not one of them.
pub mod push {
use super::{caps, Capability, Ucan};
/// The `did:key` representation rule: `did:key:z6Mk…` and bare `z6Mk…` are
/// the same identity. Mirror rows store the bare form; canonical rows and
/// every token store the full form.
///
/// Collapses representation only within `did:key`. `did:web` and
/// `did:gitlawb` share the base58 space, so a trailing-segment compare would
/// treat `did:key:X` and `did:gitlawb:X` as equal; after stripping the prefix,
/// a value that still contains `:` is a non-key DID and matches nothing bare.
pub fn did_key_eq(a: &str, b: &str) -> bool {
if a == b {
return true;
}
fn key_id(d: &str) -> &str {
d.strip_prefix("did:key:").unwrap_or(d)
}
let (ka, kb) = (key_id(a), key_id(b));
!ka.contains(':') && !kb.contains(':') && ka == kb
}
/// The actions that authorize a push: `git/push` itself, the action wildcard,
/// and `repo/admin`, which covers it.
pub fn is_push_action(can: &str) -> bool {
can == caps::GIT_PUSH || can == "*" || can == caps::REPO_ADMIN
}
/// `gitlawb://repos/<owner>/<repo>` → `(owner, repo)`, or `None` for any other
/// shape. Exactly two non-empty segments: an owner DID never contains `/`, so
/// a third segment, a trailing slash, or an empty half is malformed rather than
/// something to interpret.
pub fn parse_repo_resource(with: &str) -> Option<(&str, &str)> {
let rest = with.strip_prefix("gitlawb://repos/")?;
let (owner, repo) = rest.split_once('/')?;
if owner.is_empty() || repo.is_empty() || repo.contains('/') {
return None;
}
Some((owner, repo))
}
/// The canonical repository name a capability or URL names: at most one
/// trailing `.git` stripped, the same normalization the node's smart-HTTP
/// routes apply (`smart_http_repo_name`). A git remote carries the suffix;
/// `RepoRecord.name` does not, so every boundary that compares a capability
/// to a record must go through the same strip or a `…/myrepo.git` capability
/// names a repository that never exists.
///
/// At-most-once, not repeatedly: a repository literally named `foo.git`
/// (creatable via the peer mirror path, which skips API name validation) is
/// addressed as `foo.git.git`, and stripping twice would send it to `foo`.
pub fn canonical_repo_name(repo: &str) -> &str {
repo.strip_suffix(".git").unwrap_or(repo)
}
/// A push-class capability whose resource is the wildcard. Refused at
/// issuance, at import, by the helper, and by the node: a delegation's scope is
/// fixed when it is issued, and `*` cannot say which repositories it covered
/// at that moment.
pub fn is_push_wildcard(with: &str, can: &str) -> bool {
with == "*" && is_push_action(can)
}
pub use super::MAX_CHAIN_DEPTH;
impl Capability {
/// Push-class, unconstrained, and naming exactly this repository.
///
/// Unconstrained because `nb` has no semantics yet: an owner who wrote
/// constraints meant to restrict, and honouring the capability while
/// ignoring them would grant more than was intended. The owner segment is
/// compared with [`did_key_eq`], so a token issued against the full DID
/// matches a mirror row keyed on the bare form.
///
/// The capability's repo segment goes through [`canonical_repo_name`]
/// while `repo` (a `RepoRecord.name`, already canonical) does not: a
/// delegation written `…/myrepo.git` authorizes `myrepo`, the same
/// mapping the smart-HTTP URL applies.
pub fn grants_push_to(&self, owner: &str, repo: &str) -> bool {
self.constraints.is_none()
&& is_push_action(&self.can)
&& parse_repo_resource(&self.with)
.is_some_and(|(o, r)| did_key_eq(o, owner) && canonical_repo_name(r) == repo)
}
}
impl Ucan {
/// Whether this chain — the leaf **and every proof behind it** — carries a
/// capability that [`Capability::grants_push_to`] this repository.
///
/// Every link, not just the leaf. [`Capability::is_attenuated_by`] accepts
/// a concrete child under a `*` parent, so a leaf that names the repository
/// can sit on a proof that names every repository the root owns — including
/// ones created after the delegation was issued. Checking the leaf alone
/// let exactly that through.
///
/// This establishes scope, not trust. It does not verify signatures,
/// expiry, or audience; call [`Ucan::verify_chain`] first, and anchor the
/// root it returns to something held independently of the token.
///
/// Fails closed on anything it cannot vouch for: a proof that does not
/// decode, more than one proof per link, or a chain deeper than
/// [`MAX_CHAIN_DEPTH`].
pub fn chain_grants_push_to(&self, owner: &str, repo: &str) -> bool {
self.chain_grants_push_to_at(owner, repo, 0)
}
fn chain_grants_push_to_at(&self, owner: &str, repo: &str, depth: usize) -> bool {
if depth >= MAX_CHAIN_DEPTH {
return false;
}
if !self
.payload
.att
.iter()
.any(|c| c.grants_push_to(owner, repo))
{
return false;
}
// `verify_chain` refuses more than one proof per link for the same
// reason: two proofs mean two roots, and nothing says which authorized
// what.
if self.payload.prf.len() > 1 {
return false;
}
match self.payload.prf.first() {
None => true,
Some(token) => match Ucan::decode(token) {
Ok(proof) => proof.chain_grants_push_to_at(owner, repo, depth + 1),
Err(_) => false,
},
}
}
}
}
/// The UCAN payload (what gets signed). /// The UCAN payload (what gets signed).
#[derive(Debug, Clone, Serialize, Deserialize)] #[derive(Debug, Clone, Serialize, Deserialize)]
pub struct UcanPayload { pub struct UcanPayload {
@@ -146,6 +331,37 @@ impl Ucan {
} }
} }
/// Whether every link in this chain carries a finite `exp`.
///
/// `exp` is optional in the format, and [`Self::is_expired`] reports `false`
/// when it is absent — so a link without one never expires. With no revocation
/// mechanism, a chain containing such a link is a permanent grant: a leaked
/// token cannot be withdrawn, and the issuer's only remedy is to rotate the
/// identity the resource is keyed on.
///
/// Consumers that turn a UCAN into write authority should require this. It is
/// deliberately not enforced inside [`Self::verify_chain`], because a
/// non-expiring token is well-formed and may be perfectly appropriate for a
/// read-only or advisory capability; whether an unbounded grant is acceptable
/// is the consumer's policy, not the format's.
///
/// Fails closed past [`MAX_CHAIN_DEPTH`], like every other walk over `prf`.
pub fn chain_lifetime_is_bounded(&self) -> bool {
self.chain_lifetime_is_bounded_at(0)
}
fn chain_lifetime_is_bounded_at(&self, depth: usize) -> bool {
if depth >= MAX_CHAIN_DEPTH {
return false;
}
if self.payload.exp.is_none() {
return false;
}
self.payload.prf.iter().all(|token| {
Self::decode(token).is_ok_and(|proof| proof.chain_lifetime_is_bounded_at(depth + 1))
})
}
/// Check if this UCAN's not-before time is in the future (token not yet valid). /// Check if this UCAN's not-before time is in the future (token not yet valid).
pub fn is_before_valid(&self) -> bool { pub fn is_before_valid(&self) -> bool {
if let Some(nbf) = self.payload.nbf { if let Some(nbf) = self.payload.nbf {
@@ -248,8 +464,33 @@ impl Ucan {
/// 3. Check the proof is not expired /// 3. Check the proof is not expired
/// 4. Recursively verify the proof's own chain /// 4. Recursively verify the proof's own chain
/// ///
/// A UCAN with no proofs (root capability) passes trivially. /// A UCAN with no proofs is its own root, so it returns its own issuer.
pub fn verify_chain(&self) -> Result<()> { ///
/// **This establishes internal consistency, not trust.** `did:key` is
/// self-certifying, so anyone can mint a keypair and produce a chain that
/// verifies. A caller making an authorization decision MUST compare the
/// returned root against an identity it trusts for some reason outside this
/// token — a repo owner, a configured value, a registry lookup. Discarding
/// the return value is only correct when the caller is checking that a token
/// is well-formed and deliberately does not care who issued it.
///
/// The walk stops at [`MAX_CHAIN_DEPTH`] and fails closed there. The node
/// runs this on the `X-Ucan` header of any signed request before it asks
/// what the chain grants, so this is the first place sender-chosen depth is
/// felt: each link costs a signature check and a decode of everything
/// beneath it. The scope walk was bounded and this one was not, which made
/// that bound decorative — a deep chain met the unbounded walk first.
pub fn verify_chain(&self) -> Result<Did> {
self.verify_chain_at(0)
}
fn verify_chain_at(&self, depth: usize) -> Result<Did> {
if depth >= MAX_CHAIN_DEPTH {
return Err(Error::Ucan(format!(
"proof chain deeper than {MAX_CHAIN_DEPTH} links is not accepted"
)));
}
// First verify our own signature // First verify our own signature
self.verify_signature()?; self.verify_signature()?;
@@ -261,34 +502,44 @@ impl Ucan {
return Err(Error::Ucan("token is not yet valid".to_string())); return Err(Error::Ucan("token is not yet valid".to_string()));
} }
for proof_token in &self.payload.prf { if self.payload.prf.len() > 1 {
let proof = Self::decode(proof_token) return Err(Error::Ucan(
.map_err(|e| Error::Ucan(format!("failed to decode proof: {e}")))?; "multi-proof chains are not supported: more than one proof means \
more than one root, and which root authorized a given capability \
// The proof's audience must be this UCAN's issuer is ambiguous"
if proof.payload.aud != self.payload.iss { .to_string(),
return Err(Error::Ucan(format!( ));
"proof chain broken: proof audience {} does not match issuer {}",
proof.payload.aud, self.payload.iss
)));
}
// Every delegated capability must be covered by the proof (attenuation).
for cap in &self.payload.att {
let covered = proof.payload.att.iter().any(|p| cap.is_attenuated_by(p));
if !covered {
return Err(Error::Ucan(format!(
"capability attenuation violated: '{}' on '{}' not covered by proof",
cap.can, cap.with
)));
}
}
// Verify the proof's signature and chain recursively
proof.verify_chain()?;
} }
Ok(()) let Some(proof_token) = self.payload.prf.first() else {
// No proofs: this token is its own root.
return Ok(self.payload.iss.clone());
};
let proof = Self::decode(proof_token)
.map_err(|e| Error::Ucan(format!("failed to decode proof: {e}")))?;
// The proof's audience must be this UCAN's issuer
if proof.payload.aud != self.payload.iss {
return Err(Error::Ucan(format!(
"proof chain broken: proof audience {} does not match issuer {}",
proof.payload.aud, self.payload.iss
)));
}
// Every delegated capability must be covered by the proof (attenuation).
for cap in &self.payload.att {
let covered = proof.payload.att.iter().any(|p| cap.is_attenuated_by(p));
if !covered {
return Err(Error::Ucan(format!(
"capability attenuation violated: '{}' on '{}' not covered by proof",
cap.can, cap.with
)));
}
}
// Recurse; the root of the proof's chain is the root of ours.
proof.verify_chain_at(depth + 1)
} }
} }
@@ -664,4 +915,509 @@ mod tests {
delegated.verify_chain().unwrap(); delegated.verify_chain().unwrap();
} }
#[test]
fn verify_chain_returns_the_root_issuer_of_a_delegated_chain() {
// owner -> agent (delegation), agent -> node (invocation).
// The root is the owner: that is the identity the whole chain rests on,
// and the only one a caller can meaningfully anchor a trust decision to.
let owner = Keypair::generate();
let agent = Keypair::generate();
let node = Keypair::generate();
let caps_vec = vec![Capability::new("gitlawb://repos/zowner/r", caps::GIT_PUSH)];
let delegation =
Ucan::issue(&owner, agent.did(), caps_vec.clone(), None).expect("issue delegation");
let invocation = Ucan::delegate(&agent, node.did(), caps_vec, None, &delegation)
.expect("wrap invocation");
assert_eq!(
invocation.verify_chain().expect("chain must verify"),
owner.did(),
"the root issuer is the owner who started the chain, not the agent presenting it"
);
}
/// A chain is only bounded if EVERY link is. An unbounded link anywhere makes
/// the whole grant permanent, because `is_expired` reports false for it and
/// there is no revocation path to withdraw it.
#[test]
fn chain_lifetime_is_bounded_requires_an_expiry_on_every_link() {
let owner = Keypair::generate();
let agent = Keypair::generate();
let node = Keypair::generate();
let cap = || vec![Capability::new("gitlawb://repos/zowner/r", caps::GIT_PUSH)];
let hour = Utc::now() + chrono::Duration::hours(1);
let bounded_root = Ucan::issue(&owner, agent.did(), cap(), Some(hour)).expect("issue");
let unbounded_root = Ucan::issue(&owner, agent.did(), cap(), None).expect("issue");
assert!(
Ucan::delegate(&agent, node.did(), cap(), Some(hour), &bounded_root)
.expect("wrap")
.chain_lifetime_is_bounded(),
"both links finite"
);
assert!(
!Ucan::delegate(&agent, node.did(), cap(), None, &bounded_root)
.expect("wrap")
.chain_lifetime_is_bounded(),
"the leaf has no expiry, so the grant never lapses"
);
assert!(
!Ucan::delegate(&agent, node.did(), cap(), Some(hour), &unbounded_root)
.expect("wrap")
.chain_lifetime_is_bounded(),
"a bounded leaf cannot rescue an unbounded proof: the holder can always \
mint a fresh leaf from it"
);
assert!(
!unbounded_root.chain_lifetime_is_bounded(),
"a self-issued token with no expiry is itself unbounded"
);
}
/// A three-link chain: owner -> lead -> agent, which is the real shape of an
/// org delegating to a team lead who delegates to a CI identity.
///
/// Every other chain here is depth two, where the immediate proof IS the root —
/// so nothing distinguishes recursing to the true root from simply returning the
/// proof's issuer. Both `assert_eq!` and `assert_ne!` below are load-bearing:
/// without the second, returning the middle issuer would still satisfy a test
/// that only checked "not the leaf".
#[test]
fn verify_chain_walks_past_the_immediate_proof_to_the_true_root() {
let owner = Keypair::generate();
let lead = Keypair::generate();
let agent = Keypair::generate();
let node = Keypair::generate();
let cap = || vec![Capability::new("gitlawb://repos/zowner/r", caps::GIT_PUSH)];
let root = Ucan::issue(&owner, lead.did(), cap(), None).expect("owner -> lead");
let mid = Ucan::delegate(&lead, agent.did(), cap(), None, &root).expect("lead -> agent");
let leaf = Ucan::delegate(&agent, node.did(), cap(), None, &mid).expect("agent -> node");
let found = leaf.verify_chain().expect("a three-link chain must verify");
assert_eq!(
found,
owner.did(),
"the root is the owner who started the chain, two hops up"
);
assert_ne!(
found,
lead.did(),
"returning the immediate proof's issuer is not walking to the root"
);
}
#[test]
fn verify_chain_returns_self_as_root_for_a_self_issued_token() {
// A token with no proofs roots at its own issuer. This is what makes a
// self-minted token useless: the caller compares this against the repo
// owner and it will only ever match when the presenter IS the owner.
let agent = Keypair::generate();
let node = Keypair::generate();
let ucan =
Ucan::issue(&agent, node.did(), vec![Capability::new("*", "*")], None).expect("issue");
assert_eq!(
ucan.verify_chain().expect("a root token still verifies"),
agent.did(),
"a self-minted token roots at the minter, however permissive its capabilities"
);
}
/// Stripping `nb` is a widening, and a widening must fail attenuation.
///
/// Without this, a constrained delegation is trivially escalated: the holder
/// re-delegates the same resource and action with the constraints removed,
/// `verify_chain` accepts the chain because attenuation only compared `with`
/// and `can`, and a consumer that refuses constrained capabilities at the leaf
/// (as the node's push gate does) then sees an unconstrained one and grants it.
/// Guarding only the leaf guards the wrong end of the chain.
#[test]
fn verify_chain_rejects_a_child_that_strips_the_parents_constraints() {
let owner = Keypair::generate();
let agent = Keypair::generate();
let node = Keypair::generate();
let constrained = Capability::new("gitlawb://repos/zowner/r", caps::GIT_PUSH)
.with_constraints(serde_json::json!({ "refs": ["refs/heads/feat/*"] }));
let delegation = Ucan::issue(&owner, agent.did(), vec![constrained], None)
.expect("issue constrained delegation");
// Same resource, same action, constraints dropped.
let widened = Capability::new("gitlawb://repos/zowner/r", caps::GIT_PUSH);
let forged =
Ucan::delegate(&agent, node.did(), vec![widened], None, &delegation).expect("wrap");
let err = forged
.verify_chain()
.expect_err("dropping the parent's constraints must fail attenuation");
assert!(
err.to_string().contains("attenuation"),
"the failure must name attenuation, got: {err}"
);
}
#[test]
fn verify_chain_accepts_a_child_that_keeps_the_parents_constraints() {
let owner = Keypair::generate();
let agent = Keypair::generate();
let node = Keypair::generate();
let nb = serde_json::json!({ "refs": ["refs/heads/feat/*"] });
let constrained = Capability::new("gitlawb://repos/zowner/r", caps::GIT_PUSH)
.with_constraints(nb.clone());
let delegation =
Ucan::issue(&owner, agent.did(), vec![constrained.clone()], None).expect("issue");
let invocation =
Ucan::delegate(&agent, node.did(), vec![constrained], None, &delegation).expect("wrap");
assert_eq!(
invocation
.verify_chain()
.expect("an unchanged constraint must verify"),
owner.did()
);
}
#[test]
fn an_unconstrained_parent_still_allows_a_child_to_add_constraints() {
// Adding `nb` narrows, which is always a legal attenuation.
let owner = Keypair::generate();
let agent = Keypair::generate();
let node = Keypair::generate();
let open = Capability::new("gitlawb://repos/zowner/r", caps::GIT_PUSH);
let delegation = Ucan::issue(&owner, agent.did(), vec![open], None).expect("issue");
let narrowed = Capability::new("gitlawb://repos/zowner/r", caps::GIT_PUSH)
.with_constraints(serde_json::json!({ "refs": ["refs/heads/main"] }));
let invocation =
Ucan::delegate(&agent, node.did(), vec![narrowed], None, &delegation).expect("wrap");
assert_eq!(
invocation.verify_chain().expect("narrowing must verify"),
owner.did()
);
}
#[test]
fn verify_chain_rejects_a_multi_proof_chain() {
// Two proofs mean two roots, and nothing says which root authorized a
// given capability. Returning either one would be unsound, so refuse.
let owner_a = Keypair::generate();
let owner_b = Keypair::generate();
let agent = Keypair::generate();
let node = Keypair::generate();
let caps_vec = vec![Capability::new("gitlawb://repos/zowner/r", caps::GIT_PUSH)];
let proof_a = Ucan::issue(&owner_a, agent.did(), caps_vec.clone(), None).expect("issue a");
let proof_b = Ucan::issue(&owner_b, agent.did(), caps_vec.clone(), None).expect("issue b");
// `delegate` only ever writes one proof, so build the two-proof payload by hand.
let payload = UcanPayload {
ucan: "1.0.0".to_string(),
iss: agent.did(),
aud: node.did(),
att: caps_vec,
exp: None,
nbf: None,
prf: vec![
proof_a.encode().expect("encode a"),
proof_b.encode().expect("encode b"),
],
};
let signing_bytes = serde_json::to_vec(&payload).expect("serialize payload");
let s = agent.sign_b64(&signing_bytes);
let multi = Ucan { payload, s };
let err = multi
.verify_chain()
.expect_err("a two-proof chain must be refused");
assert!(
err.to_string().contains("multi-proof"),
"the error must name the reason, got: {err}"
);
}
}
#[cfg(test)]
mod push_scope_tests {
use super::push::*;
use super::{caps, Capability, Ucan};
use crate::identity::Keypair;
fn hour() -> chrono::DateTime<chrono::Utc> {
chrono::Utc::now() + chrono::Duration::hours(1)
}
#[test]
fn did_key_eq_collapses_representation_only_within_did_key() {
assert!(did_key_eq("did:key:z6MkAbc", "z6MkAbc"));
assert!(did_key_eq("z6MkAbc", "did:key:z6MkAbc"));
assert!(did_key_eq("did:key:z6MkAbc", "did:key:z6MkAbc"));
assert!(!did_key_eq("did:key:z6MkAbc", "did:key:z6MkXyz"));
// A bare id must never match across methods: `did:gitlawb` shares the
// base58 space with `did:key`.
assert!(!did_key_eq("did:gitlawb:z6MkAbc", "z6MkAbc"));
assert!(!did_key_eq("did:key:z6MkAbc", "did:gitlawb:z6MkAbc"));
assert!(!did_key_eq("did:web:example.com", "example.com"));
}
#[test]
fn parse_repo_resource_requires_exactly_two_segments() {
assert_eq!(
parse_repo_resource("gitlawb://repos/did:key:z6Mk/r"),
Some(("did:key:z6Mk", "r"))
);
for bad in [
"*",
"",
"gitlawb://repos/",
"gitlawb://repos/owner",
"gitlawb://repos/owner/",
"gitlawb://repos//r",
"gitlawb://repos/owner/r/extra",
"https://repos/owner/r",
] {
assert_eq!(parse_repo_resource(bad), None, "{bad:?}");
}
}
#[test]
fn is_push_wildcard_is_the_resource_wildcard_with_a_push_action() {
for can in [caps::GIT_PUSH, "*", caps::REPO_ADMIN] {
assert!(is_push_wildcard("*", can), "{can}");
}
assert!(!is_push_wildcard("*", caps::GIT_FETCH));
assert!(!is_push_wildcard("*", caps::PR_OPEN));
assert!(!is_push_wildcard("gitlawb://repos/o/r", caps::GIT_PUSH));
}
#[test]
fn a_capability_grants_push_only_when_concrete_unconstrained_and_push_class() {
let owner = "did:key:z6MkOwner";
let ok = Capability::new("gitlawb://repos/did:key:z6MkOwner/r", caps::GIT_PUSH);
assert!(ok.grants_push_to(owner, "r"));
// Bare and full owner forms are the same identity.
assert!(ok.grants_push_to("z6MkOwner", "r"));
assert!(!Capability::new("*", caps::GIT_PUSH).grants_push_to(owner, "r"));
assert!(
!Capability::new("gitlawb://repos/did:key:z6MkOwner/r", caps::GIT_FETCH)
.grants_push_to(owner, "r")
);
assert!(
!Capability::new("gitlawb://repos/did:key:z6MkOwner/other", caps::GIT_PUSH)
.grants_push_to(owner, "r")
);
assert!(
!Capability::new("gitlawb://repos/did:key:z6MkOther/r", caps::GIT_PUSH)
.grants_push_to(owner, "r")
);
assert!(
!Capability::new("gitlawb://repos/did:key:z6MkOwner/r", caps::GIT_PUSH)
.with_constraints(serde_json::json!({"max_bytes": 1}))
.grants_push_to(owner, "r")
);
for can in ["*", caps::REPO_ADMIN] {
assert!(
Capability::new("gitlawb://repos/did:key:z6MkOwner/r", can)
.grants_push_to(owner, "r"),
"{can} covers git/push"
);
}
}
#[test]
fn a_git_suffixed_capability_names_the_canonical_repo() {
let owner = "did:key:z6MkOwner";
// A git remote carries `.git`; the record does not. The capability
// matches the repository the URL would reach, so a delegation pasted
// from the remote still authorizes.
assert!(Capability::new(
"gitlawb://repos/did:key:z6MkOwner/myrepo.git",
caps::GIT_PUSH
)
.grants_push_to(owner, "myrepo"));
// At most one suffix is stripped: a repo literally named `foo.git`
// (reachable as `foo.git.git` over smart HTTP) is a different
// repository from `foo`, and a single `.git` must not resolve to it.
assert!(Capability::new(
"gitlawb://repos/did:key:z6MkOwner/foo.git.git",
caps::GIT_PUSH
)
.grants_push_to(owner, "foo.git"));
assert!(
!Capability::new("gitlawb://repos/did:key:z6MkOwner/foo.git", caps::GIT_PUSH)
.grants_push_to(owner, "foo.git")
);
}
/// The chain walk, at every depth: a wildcard anywhere denies; concrete
/// everywhere grants. The grandparent case is the one a single-level check
/// misses, because the immediate proof already names the repo.
#[test]
fn chain_grants_push_only_when_every_link_names_the_repo() {
let owner = Keypair::generate();
let a = Keypair::generate();
let b = Keypair::generate();
let node = Keypair::generate();
let owner_s = owner.did().to_string();
let res = format!("gitlawb://repos/{owner_s}/r");
let concrete = || vec![Capability::new(&res, caps::GIT_PUSH)];
let wild = || vec![Capability::new("*", caps::GIT_PUSH)];
// Single self-issued link.
assert!(Ucan::issue(&owner, a.did(), concrete(), Some(hour()))
.unwrap()
.chain_grants_push_to(&owner_s, "r"));
assert!(!Ucan::issue(&owner, a.did(), wild(), Some(hour()))
.unwrap()
.chain_grants_push_to(&owner_s, "r"));
// Two links: concrete leaf on a wildcard proof must deny.
let wild_proof = Ucan::issue(&owner, a.did(), wild(), Some(hour())).unwrap();
let leaf = Ucan::delegate(&a, node.did(), concrete(), Some(hour()), &wild_proof).unwrap();
assert!(
leaf.verify_chain().is_ok(),
"attenuation accepts it — that is why we walk"
);
assert!(!leaf.chain_grants_push_to(&owner_s, "r"));
// Three links: wildcard grandparent, concrete parent, concrete leaf.
let gp = Ucan::issue(&owner, a.did(), wild(), Some(hour())).unwrap();
let parent = Ucan::delegate(&a, b.did(), concrete(), Some(hour()), &gp).unwrap();
let leaf = Ucan::delegate(&b, node.did(), concrete(), Some(hour()), &parent).unwrap();
assert!(leaf.verify_chain().is_ok());
assert!(
!leaf.chain_grants_push_to(&owner_s, "r"),
"a wildcard two links up must deny"
);
// Three links, all concrete.
let gp = Ucan::issue(&owner, a.did(), concrete(), Some(hour())).unwrap();
let parent = Ucan::delegate(&a, b.did(), concrete(), Some(hour()), &gp).unwrap();
let leaf = Ucan::delegate(&b, node.did(), concrete(), Some(hour()), &parent).unwrap();
assert!(leaf.chain_grants_push_to(&owner_s, "r"));
// And the bare owner form is the same repo.
assert!(leaf.chain_grants_push_to(owner_s.strip_prefix("did:key:").unwrap(), "r"));
// But not another repo.
assert!(!leaf.chain_grants_push_to(&owner_s, "other"));
}
/// Fail closed on anything the walk cannot vouch for.
#[test]
fn chain_walk_fails_closed_on_undecodable_or_multi_proofs_or_excess_depth() {
let owner = Keypair::generate();
let a = Keypair::generate();
let owner_s = owner.did().to_string();
let res = format!("gitlawb://repos/{owner_s}/r");
let cap = || vec![Capability::new(&res, caps::GIT_PUSH)];
let mut broken = Ucan::issue(&owner, a.did(), cap(), Some(hour())).unwrap();
broken.payload.prf = vec!["not a ucan".into()];
assert!(!broken.chain_grants_push_to(&owner_s, "r"));
let good = Ucan::issue(&owner, a.did(), cap(), Some(hour())).unwrap();
let mut two = Ucan::issue(&owner, a.did(), cap(), Some(hour())).unwrap();
two.payload.prf = vec![good.encode().unwrap(), good.encode().unwrap()];
assert!(!two.chain_grants_push_to(&owner_s, "r"));
// Deeper than MAX_CHAIN_DEPTH, all concrete: refused on depth alone.
let mut cur = Ucan::issue(&owner, a.did(), cap(), Some(hour())).unwrap();
let mut signer = a;
for _ in 0..MAX_CHAIN_DEPTH {
let next = Keypair::generate();
cur = Ucan::delegate(&signer, next.did(), cap(), Some(hour()), &cur).unwrap();
signer = next;
}
assert!(
!cur.chain_grants_push_to(&owner_s, "r"),
"depth bound must fail closed"
);
}
}
/// Every walk over `prf` stops at the same depth. The round-ten P1: the scope
/// walk was bounded while `verify_chain` and `chain_lifetime_is_bounded` were
/// not, and the node runs `verify_chain` on an attacker-supplied `X-Ucan` header
/// before the bounded walk ever sees it, so the bound protected nothing.
#[cfg(test)]
mod chain_depth_tests {
use super::{caps, Capability, Ucan, MAX_CHAIN_DEPTH};
use crate::identity::Keypair;
fn hour() -> chrono::DateTime<chrono::Utc> {
chrono::Utc::now() + chrono::Duration::hours(1)
}
/// A well-formed, fully signed, expiring chain of exactly `links` links, every
/// one naming the owner's repository, so the only thing any walk can object
/// to is how long it is. Returns the leaf and the owner it roots at.
fn concrete_chain(links: usize) -> (Ucan, Keypair) {
assert!(links >= 1);
let owner = Keypair::generate();
let owner_s = owner.did().to_string();
let res = format!("gitlawb://repos/{owner_s}/r");
let cap = || vec![Capability::new(&res, caps::GIT_PUSH)];
let mut signer = Keypair::generate();
let mut cur = Ucan::issue(&owner, signer.did(), cap(), Some(hour())).unwrap();
for _ in 1..links {
let next = Keypair::generate();
cur = Ucan::delegate(&signer, next.did(), cap(), Some(hour()), &cur).unwrap();
signer = next;
}
(cur, owner)
}
#[test]
fn a_chain_at_the_bound_passes_every_walk() {
let (leaf, owner) = concrete_chain(MAX_CHAIN_DEPTH);
let owner_s = owner.did().to_string();
assert_eq!(
leaf.verify_chain()
.expect("a chain at the bound must verify"),
owner.did()
);
assert!(leaf.chain_lifetime_is_bounded());
assert!(leaf.chain_grants_push_to(&owner_s, "r"));
}
#[test]
fn one_link_past_the_bound_fails_every_walk_closed() {
let (leaf, owner) = concrete_chain(MAX_CHAIN_DEPTH + 1);
let owner_s = owner.did().to_string();
let err = leaf
.verify_chain()
.expect_err("verify_chain must refuse a chain past the bound");
assert!(
err.to_string().contains("deeper than"),
"the refusal must say depth was the reason, not a fake signature failure: {err}"
);
assert!(
!leaf.chain_lifetime_is_bounded(),
"an unwalkable chain cannot be vouched for as bounded"
);
assert!(!leaf.chain_grants_push_to(&owner_s, "r"));
}
/// A chain well past the bound is refused from the top, before any per-link
/// work. Only a few links past it, deliberately: a proof is embedded as a JSON
/// string inside the next link's JSON, so every level re-escapes the one
/// inside it and the encoding roughly doubles per link — eight links is about
/// 10 KiB, sixteen is megabytes, and a chain of "a few hundred levels" cannot
/// physically be encoded, let alone sent as a header. The bound is still
/// what stops the reachable dozen-odd levels from each costing a decode of
/// everything beneath them.
#[test]
fn a_chain_well_past_the_bound_is_refused_not_walked() {
let (leaf, _) = concrete_chain(MAX_CHAIN_DEPTH + 4);
let err = leaf.verify_chain().expect_err("must refuse");
assert!(err.to_string().contains("deeper than"), "{err}");
assert!(!leaf.chain_lifetime_is_bounded());
}
} }
+33 -2
View File
@@ -6,10 +6,28 @@ edition.workspace = true
rust-version.workspace = true rust-version.workspace = true
license.workspace = true license.workspace = true
[lib]
name = "gitlawb_node"
path = "src/lib.rs"
[[bin]] [[bin]]
name = "gitlawb-node" name = "gitlawb-node"
path = "src/main.rs" path = "src/main.rs"
# Exposes the `test_harness` spawn surface (src/test_harness.rs) to the
# real-node deny-harness integration crate. Off by default so the production
# binary never compiles test-only boot code. Enable with
# `cargo test -p gitlawb-node --features test-harness`.
[features]
test-harness = ["dep:tempfile"]
# Only builds when `test-harness` is enabled; otherwise skipped (not an error),
# so `cargo test --workspace` without the feature stays green.
[[test]]
name = "deny_harness"
path = "tests/deny_harness.rs"
required-features = ["test-harness"]
[dependencies] [dependencies]
gitlawb-core = { path = "../gitlawb-core" } gitlawb-core = { path = "../gitlawb-core" }
ed25519-dalek = { workspace = true } ed25519-dalek = { workspace = true }
@@ -23,6 +41,8 @@ tracing = { workspace = true }
tracing-subscriber = { workspace = true } tracing-subscriber = { workspace = true }
chrono = { workspace = true } chrono = { workspace = true }
uuid = { workspace = true } uuid = { workspace = true }
url = { workspace = true }
percent-encoding = { workspace = true }
axum = { version = "0.8", features = ["http1", "http2", "json", "ws"] } axum = { version = "0.8", features = ["http1", "http2", "json", "ws"] }
async-graphql = { version = "7", features = ["chrono", "uuid", "tracing"] } async-graphql = { version = "7", features = ["chrono", "uuid", "tracing"] }
async-graphql-axum = "7" async-graphql-axum = "7"
@@ -38,6 +58,7 @@ cid = { workspace = true }
hex = { workspace = true } hex = { workspace = true }
sha2 = { workspace = true } sha2 = { workspace = true }
hmac = { workspace = true } hmac = { workspace = true }
chacha20poly1305 = "0.10"
http-body-util = "0.1" http-body-util = "0.1"
tokio-util = { version = "0.7", features = ["io"] } tokio-util = { version = "0.7", features = ["io"] }
dirs-next = "2" dirs-next = "2"
@@ -73,11 +94,21 @@ alloy = { version = "1", default-features = false, features = [
"network", "network",
"rpc-types-eth", "rpc-types-eth",
] } ] }
rand = { workspace = true }
libp2p-dns = { version = "0.45.0", features = ["tokio"] } libp2p-dns = { version = "0.45.0", features = ["tokio"] }
# Optional: only needed by the test-harness feature's lib modules
# (test_git_shim, test_harness); dev-deps are invisible to lib code.
tempfile = { version = "3", optional = true }
[dev-dependencies] [dev-dependencies]
mockito = "1" mockito = "1"
tempfile = "3" tempfile = "3"
# `test-util` for `#[tokio::test(start_paused = true)]`: lets the deadline tests assert # test-util enables tokio::test(start_paused): the delayed claim re-sweep
# on which timer fired instead of on wall clock. # test drives the real CLAIM_SWEEP_MIN_AGE-plus-margin sleep on paused time.
tokio = { workspace = true, features = ["test-util"] } tokio = { workspace = true, features = ["test-util"] }
# Used by the deny-harness integration crate (tests/deny_harness.rs): the
# #[sqlx::test] macro for an ephemeral per-test pool, and reqwest as the real
# HTTP client that drives deny paths over the socket.
sqlx = { version = "0.8", features = ["postgres", "runtime-tokio-rustls", "macros", "migrate"] }
reqwest = { workspace = true }
hex = "0.4"
+56 -9
View File
@@ -1,12 +1,13 @@
//! GET /api/v1/arweave/anchors — list Arweave ref-update anchors. //! GET /api/v1/arweave/anchors — list Arweave ref-update anchors.
use axum::{ use axum::{
extract::{Query, State}, extract::{Extension, Query, State},
Json, Json,
}; };
use serde::Deserialize; use serde::Deserialize;
use crate::error::Result; use crate::db::normalize_owner_key;
use crate::error::{AppError, Result};
use crate::state::AppState; use crate::state::AppState;
#[derive(Debug, Deserialize)] #[derive(Debug, Deserialize)]
@@ -21,17 +22,62 @@ fn default_limit() -> i64 {
} }
/// GET /api/v1/arweave/anchors /// GET /api/v1/arweave/anchors
///
/// `?repo=<owner>/<name>` returns anchors for one repository and binds the
/// request to the canonical read-authorization path used by every other
/// repo-scoped read: the same `authorize_repo_read` helper, with the caller
/// identity attached. Missing repositories, quarantined mirrors, and signed
/// non-readers all collapse to the standard `404` (no existence oracle). The
/// unscoped listing is auth-only — the #121 contract — and does not admit
/// visibility filtering, which is the #136 stale-index class and explicitly
/// out of scope for this auth slice.
pub async fn list_anchors( pub async fn list_anchors(
State(state): State<AppState>, State(state): State<AppState>,
Query(q): Query<ListAnchorsQuery>, Query(q): Query<ListAnchorsQuery>,
auth: Option<Extension<crate::auth::AuthenticatedDid>>,
) -> Result<Json<serde_json::Value>> { ) -> Result<Json<serde_json::Value>> {
let limit = q.limit.min(200); // The route's `optional_signature` layer is permissive (it admits unsigned
// Bare `?` so connection-class sqlx failures downcast to `AppError::Db` and // legacy callers) so this in-handler check is the actual gate. A signed
// map to 503 `db_unavailable` (not 500 via `.map_err(AppError::Internal)`) (#251). // non-reader still passes this check and proceeds to the authz step below.
let anchors = state let caller = auth.as_ref().map(|e| e.0 .0.as_str());
.db if caller.is_none() {
.list_arweave_anchors(q.repo.as_deref(), limit) return Err(AppError::Unauthorized(
.await?; "authentication required for anchor listing".into(),
));
}
// Clamp before the SQL query. A negative or non-numeric limit must not
// reach Postgres as `LIMIT -1` (which it rejects as a db error / 500) — the
// contract is the same default ceiling as the listing page itself.
let limit = q.limit.clamp(0, 200);
let anchors = if let Some(repo_slug) = q.repo.as_deref() {
// Parse the user-supplied "owner/name" through the same slug validator
// the sync path uses, so a malformed query is a 400, not a 500.
let (owner, name) = match crate::git::repo_store::validate_repo_slug(repo_slug) {
Ok(parts) => parts,
Err(e) => return Err(AppError::BadRequest(format!("invalid ?repo: {e}"))),
};
// Read gate. Returns `RepoNotFound` (→ 404) indistinguishably for
// missing repos, quarantined mirrors, and signed non-readers.
let (record, _rules) =
crate::api::authorize_repo_read(&state, owner, name, caller, "/").await?;
// Build the SQL filter from the canonical stored slug, NOT from the
// user-supplied `?repo=` string. Anchor rows are written as
// `{normalize_owner_key(owner_did)}/{name}` (the short form), so a
// request that passes authz with the full `did:key:…/name` form would
// match zero rows and return a false empty page. The gate already
// verified the repo is readable; `record.owner_did` is the canonical
// identity and `record.name` is the stored name.
let stored_slug = format!("{}/{}", normalize_owner_key(&record.owner_did), record.name);
state
.db
.list_arweave_anchors(Some(&stored_slug), limit)
.await?
} else {
// No scope → no repo-read decision. Auth-only.
state.db.list_arweave_anchors(None, limit).await?
};
Ok(Json(serde_json::json!({ Ok(Json(serde_json::json!({
"anchors": anchors, "anchors": anchors,
@@ -60,6 +106,7 @@ mod closed_pool_tests {
.oneshot( .oneshot(
Request::builder() Request::builder()
.uri("/api/v1/arweave/anchors") .uri("/api/v1/arweave/anchors")
.extension(crate::auth::AuthenticatedDid("did:key:test".into()))
.body(axum::body::Body::empty()) .body(axum::body::Body::empty())
.unwrap(), .unwrap(),
) )
+123 -29
View File
@@ -285,11 +285,22 @@ pub async fn submit_bounty(
Path(id): Path<String>, Path(id): Path<String>,
Json(req): Json<SubmitBountyRequest>, Json(req): Json<SubmitBountyRequest>,
) -> Result<Json<BountyRecord>> { ) -> Result<Json<BountyRecord>> {
let bounty = state let not_found = || AppError::NotFound(format!("bounty {id} not found"));
.db
.get_bounty(&id) let bounty = state.db.get_bounty(&id).await?.ok_or_else(not_found)?;
.await?
.ok_or_else(|| AppError::NotFound(format!("bounty {id} not found")))?; if crate::api::authorize_repo_read(
&state,
&bounty.repo_owner,
&bounty.repo_name,
Some(auth.0.as_str()),
"/",
)
.await
.is_err()
{
return Err(not_found());
}
if bounty.status != "claimed" { if bounty.status != "claimed" {
return Err(AppError::BadRequest(format!( return Err(AppError::BadRequest(format!(
@@ -325,11 +336,22 @@ pub async fn approve_bounty(
Path(id): Path<String>, Path(id): Path<String>,
Json(req): Json<ApproveBountyRequest>, Json(req): Json<ApproveBountyRequest>,
) -> Result<Json<BountyRecord>> { ) -> Result<Json<BountyRecord>> {
let bounty = state let not_found = || AppError::NotFound(format!("bounty {id} not found"));
.db
.get_bounty(&id) let bounty = state.db.get_bounty(&id).await?.ok_or_else(not_found)?;
.await?
.ok_or_else(|| AppError::NotFound(format!("bounty {id} not found")))?; if crate::api::authorize_repo_read(
&state,
&bounty.repo_owner,
&bounty.repo_name,
Some(auth.0.as_str()),
"/",
)
.await
.is_err()
{
return Err(not_found());
}
if bounty.status != "submitted" { if bounty.status != "submitted" {
return Err(AppError::BadRequest(format!( return Err(AppError::BadRequest(format!(
@@ -372,11 +394,22 @@ pub async fn cancel_bounty(
Extension(auth): Extension<AuthenticatedDid>, Extension(auth): Extension<AuthenticatedDid>,
Path(id): Path<String>, Path(id): Path<String>,
) -> Result<Json<BountyRecord>> { ) -> Result<Json<BountyRecord>> {
let bounty = state let not_found = || AppError::NotFound(format!("bounty {id} not found"));
.db
.get_bounty(&id) let bounty = state.db.get_bounty(&id).await?.ok_or_else(not_found)?;
.await?
.ok_or_else(|| AppError::NotFound(format!("bounty {id} not found")))?; if crate::api::authorize_repo_read(
&state,
&bounty.repo_owner,
&bounty.repo_name,
Some(auth.0.as_str()),
"/",
)
.await
.is_err()
{
return Err(not_found());
}
if bounty.status != "open" { if bounty.status != "open" {
return Err(AppError::BadRequest(format!( return Err(AppError::BadRequest(format!(
@@ -407,11 +440,22 @@ pub async fn dispute_bounty(
Extension(auth): Extension<AuthenticatedDid>, Extension(auth): Extension<AuthenticatedDid>,
Path(id): Path<String>, Path(id): Path<String>,
) -> Result<Json<BountyRecord>> { ) -> Result<Json<BountyRecord>> {
let bounty = state let not_found = || AppError::NotFound(format!("bounty {id} not found"));
.db
.get_bounty(&id) let bounty = state.db.get_bounty(&id).await?.ok_or_else(not_found)?;
.await?
.ok_or_else(|| AppError::NotFound(format!("bounty {id} not found")))?; if crate::api::authorize_repo_read(
&state,
&bounty.repo_owner,
&bounty.repo_name,
Some(auth.0.as_str()),
"/",
)
.await
.is_err()
{
return Err(not_found());
}
if bounty.status != "claimed" && bounty.status != "submitted" { if bounty.status != "claimed" && bounty.status != "submitted" {
return Err(AppError::BadRequest(format!( return Err(AppError::BadRequest(format!(
@@ -456,21 +500,68 @@ pub async fn dispute_bounty(
Ok(Json(updated)) Ok(Json(updated))
} }
/// Resolve the set of `(owner_did, name)` pairs for repos an anonymous caller
/// can list. Mirrors the `stats()` pattern in `server.rs` (#104): batch-load
/// all deduped repos, batch-load their visibility rules, keep only those that
/// pass `listable_at_root`. Pure I/O after the two DB round-trips — no
/// per-repo authorization queries.
///
/// Both bounty stats handlers mount behind `optional_signature` (see
/// `bounty_read_routes` in server.rs), so the caller is always `None` for
/// anonymous access. Every caller, signed or not, currently receives the
/// anonymous-scoped aggregates (matching the #104 pattern); per-caller
/// visibility can be threaded through here when that work lands.
///
/// A database error is returned to the caller, never collapsed to an empty
/// set: an empty set would render as HTTP 200 with zero counts, which a client
/// reads as confirmed absence of activity.
async fn visible_repo_pairs(state: &AppState) -> Result<Vec<(String, String)>> {
let rows = state.db.list_all_repos_deduped().await?;
let ids: Vec<String> = rows.iter().map(|r| r.id.clone()).collect();
let rules_by_repo = state.db.list_visibility_rules_for_repos(&ids).await?;
let pairs = rows
.iter()
.filter(|r| {
let rules = rules_by_repo.get(&r.id).map(Vec::as_slice).unwrap_or(&[]);
// mirror-rows-handled: a mirror with a canonical twin is deduped to the
// canonical row and its rules; a mirror-only repo counts as public, the
// same answer authorize_repo_read and /stats give. Pinned by
// bounty_stats_mirror_rows_follow_canonical_twin_or_count_when_alone.
crate::visibility::listable_at_root(rules, r.is_public, &r.owner_did, None)
})
.map(|r| {
(
crate::db::normalize_owner_key(&r.owner_did).to_string(),
r.name.clone(),
)
})
.collect();
Ok(pairs)
}
/// GET /api/v1/bounties/stats /// GET /api/v1/bounties/stats
///
/// Aggregates are restricted to anonymously-listable repos so private-repo
/// bounty activity is not exposed (#477). The visible-repo set is resolved
/// once per request via `visible_repo_pairs` (two SQL round-trips), then
/// passed into the filtered aggregate queries.
pub async fn bounty_stats(State(state): State<AppState>) -> Result<Json<BountyStatsResponse>> { pub async fn bounty_stats(State(state): State<AppState>) -> Result<Json<BountyStatsResponse>> {
let open = state.db.count_bounties_by_status("open").await.unwrap_or(0); let visible = visible_repo_pairs(&state).await?;
let open = state
.db
.count_bounties_by_status_visible("open", &visible)
.await?;
let claimed = state let claimed = state
.db .db
.count_bounties_by_status("claimed") .count_bounties_by_status_visible("claimed", &visible)
.await .await?;
.unwrap_or(0);
let completed = state let completed = state
.db .db
.count_bounties_by_status("completed") .count_bounties_by_status_visible("completed", &visible)
.await .await?;
.unwrap_or(0);
let leaders = state.db.bounty_leaderboard(10).await.unwrap_or_default(); let leaders = state.db.bounty_leaderboard_visible(10, &visible).await?;
let leaderboard = leaders let leaderboard = leaders
.into_iter() .into_iter()
.map(|(did, cnt, total)| AgentBountyEntry { .map(|(did, cnt, total)| AgentBountyEntry {
@@ -489,11 +580,14 @@ pub async fn bounty_stats(State(state): State<AppState>) -> Result<Json<BountySt
} }
/// GET /api/v1/agents/{did}/bounties /// GET /api/v1/agents/{did}/bounties
///
/// Per-agent earnings restricted to anonymously-listable repos (#477).
pub async fn agent_bounty_stats( pub async fn agent_bounty_stats(
State(state): State<AppState>, State(state): State<AppState>,
Path(did): Path<String>, Path(did): Path<String>,
) -> Result<Json<serde_json::Value>> { ) -> Result<Json<serde_json::Value>> {
let (count, total) = state.db.agent_bounty_stats(&did).await.unwrap_or((0, 0)); let visible = visible_repo_pairs(&state).await?;
let (count, total) = state.db.agent_bounty_stats_visible(&did, &visible).await?;
Ok(Json(serde_json::json!({ Ok(Json(serde_json::json!({
"did": did, "did": did,
"completed_bounties": count, "completed_bounties": count,
+2
View File
@@ -52,6 +52,7 @@ pub async fn list_certs(
"node_did": c.node_did, "node_did": c.node_did,
"signature": c.signature, "signature": c.signature,
"issued_at": c.issued_at, "issued_at": c.issued_at,
"version": c.version,
}) })
}) })
.collect(); .collect();
@@ -92,5 +93,6 @@ pub async fn get_cert(
"node_did": cert.node_did, "node_did": cert.node_did,
"signature": cert.signature, "signature": cert.signature,
"issued_at": cert.issued_at, "issued_at": cert.issued_at,
"version": cert.version,
}))) })))
} }
+10 -31
View File
@@ -6,29 +6,24 @@ use axum::Json;
use crate::auth::AuthenticatedDid; use crate::auth::AuthenticatedDid;
use crate::error::{AppError, Result}; use crate::error::{AppError, Result};
use crate::state::AppState; use crate::state::AppState;
use crate::visibility::{visibility_check, Decision};
/// GET /api/v1/repos/{owner}/{repo}/encrypted-blobs /// GET /api/v1/repos/{owner}/{repo}/encrypted-blobs
/// Returns [{oid, cid}] for every encrypted blob in the repo, to any caller who /// Returns [{oid, cid}] for every encrypted blob in the repo, to any caller who
/// can read the repo. Not recipient-scoped: recipient identities are not stored, /// can read the repo. Not recipient-scoped: recipient identities are not stored,
/// so access control here is repo readability and decryption is gated by the /// so access control here is repo readability and decryption is gated by the
/// envelope crypto (only a real recipient can open an envelope). /// envelope crypto (only a real recipient can open an envelope).
///
/// Quarantined repos are opaque 404 via [`crate::api::authorize_repo_read`] —
/// same as issues/changelogs — so a public-but-quarantined mirror cannot leak
/// its encrypted blob index.
pub async fn list_encrypted_blobs( pub async fn list_encrypted_blobs(
State(state): State<AppState>, State(state): State<AppState>,
auth: Option<Extension<AuthenticatedDid>>, auth: Option<Extension<AuthenticatedDid>>,
Path((owner, repo)): Path<(String, String)>, Path((owner, repo)): Path<(String, String)>,
) -> Result<Json<serde_json::Value>> { ) -> Result<Json<serde_json::Value>> {
let record = state
.db
.get_repo(&owner, &repo)
.await?
.ok_or_else(|| AppError::RepoNotFound(format!("{owner}/{repo}")))?;
let caller = auth.as_ref().map(|e| e.0 .0.as_str()); let caller = auth.as_ref().map(|e| e.0 .0.as_str());
let rules = state.db.list_visibility_rules(&record.id).await?; let (record, _rules) =
if visibility_check(&rules, record.is_public, &record.owner_did, caller, "/") == Decision::Deny crate::api::authorize_repo_read(&state, &owner, &repo, caller, "/").await?;
{
return Err(AppError::RepoNotFound(format!("{owner}/{repo}")));
}
let rows = state.db.list_all_encrypted_blobs(&record.id).await?; let rows = state.db.list_all_encrypted_blobs(&record.id).await?;
let blobs: Vec<_> = rows let blobs: Vec<_> = rows
.into_iter() .into_iter()
@@ -45,17 +40,9 @@ pub async fn get_encrypted_blob(
auth: Option<Extension<AuthenticatedDid>>, auth: Option<Extension<AuthenticatedDid>>,
Path((owner, repo, oid)): Path<(String, String, String)>, Path((owner, repo, oid)): Path<(String, String, String)>,
) -> Result<Vec<u8>> { ) -> Result<Vec<u8>> {
let record = state
.db
.get_repo(&owner, &repo)
.await?
.ok_or_else(|| AppError::RepoNotFound(format!("{owner}/{repo}")))?;
let caller = auth.as_ref().map(|e| e.0 .0.as_str()); let caller = auth.as_ref().map(|e| e.0 .0.as_str());
let rules = state.db.list_visibility_rules(&record.id).await?; let (record, _rules) =
if visibility_check(&rules, record.is_public, &record.owner_did, caller, "/") == Decision::Deny crate::api::authorize_repo_read(&state, &owner, &repo, caller, "/").await?;
{
return Err(AppError::RepoNotFound(format!("{owner}/{repo}/{oid}")));
}
let cid = state let cid = state
.db .db
.encrypted_blob_cid(&record.id, &oid) .encrypted_blob_cid(&record.id, &oid)
@@ -81,17 +68,9 @@ pub async fn replicate_encrypted_blobs(
auth: Option<Extension<AuthenticatedDid>>, auth: Option<Extension<AuthenticatedDid>>,
Path((owner, repo)): Path<(String, String)>, Path((owner, repo)): Path<(String, String)>,
) -> Result<Json<serde_json::Value>> { ) -> Result<Json<serde_json::Value>> {
let record = state
.db
.get_repo(&owner, &repo)
.await?
.ok_or_else(|| AppError::RepoNotFound(format!("{owner}/{repo}")))?;
let caller = auth.as_ref().map(|e| e.0 .0.as_str()); let caller = auth.as_ref().map(|e| e.0 .0.as_str());
let rules = state.db.list_visibility_rules(&record.id).await?; let (record, _rules) =
if visibility_check(&rules, record.is_public, &record.owner_did, caller, "/") == Decision::Deny crate::api::authorize_repo_read(&state, &owner, &repo, caller, "/").await?;
{
return Err(AppError::RepoNotFound(format!("{owner}/{repo}")));
}
let rows = state.db.list_all_encrypted_blobs(&record.id).await?; let rows = state.db.list_all_encrypted_blobs(&record.id).await?;
let blobs: Vec<_> = rows let blobs: Vec<_> = rows
.into_iter() .into_iter()
File diff suppressed because it is too large. Load diff
+444 -9
View File
@@ -2130,14 +2130,77 @@ async fn gate_and_serve(
/// GET /api/v1/ipfs/pins /// GET /api/v1/ipfs/pins
/// ///
/// Returns all CIDs that have been pinned to the local IPFS node from git /// Returns all CIDs that have been pinned from git objects received via push.
/// objects received via push. Each entry includes the git SHA-256 hex, the /// Each entry includes the git SHA-256 hex and the timestamp when it was
/// CIDv1 string, and the timestamp when it was pinned. /// pinned. The wire contract is nullable by design:
pub async fn list_pins(State(state): State<AppState>) -> Result<Json<serde_json::Value>> { ///
/// - `cid` / `local_cid` — node-local resolver keys: the raw CIDv1 the
/// `GET /ipfs/{cid}` endpoint serves. Non-null ONLY when the stored value
/// parses as a raw CIDv1 AND writer-owned local provenance confirms the
/// bytes went through this node's IPFS daemon. Otherwise null, even when
/// the database holds some other CID-shaped string for the row.
/// - `pinata_cid` — the provider identifier Pinata returned (dag-pb/UnixFS,
/// not a node resolver key). The only usable identifier for remote-only
/// rows; `gl` consumers must read this field, never `cid`, for those.
/// - `local_pinned` — `true` iff the local-IPFS writer path
/// (`record_pinned_cid_with_source`) pushed the bytes into the local
/// daemon. Writer-owned; never inferred from CID shape. A row can carry
/// `local_pinned = true` with a null `cid` when its stored key predates
/// the raw-key contract (legacy provider key awaiting repair).
/// - `pinata_pinned` — `true` iff the row has a non-null `pinata_cid`.
///
/// Rows with neither a local nor a Pinata CID are omitted so the response
/// only contains rows with at least one backend. Remote-only rows stay
/// visible through `pinata_cid`/`pinata_pinned`; nothing is hidden.
///
/// #218 review P2: the response surfaces writer-owned provenance so a
/// `gl` consumer can distinguish local-only, remote-only, and dual rows
/// without re-inferring semantics from nullability.
pub async fn list_pins(
State(state): State<AppState>,
auth: Option<Extension<crate::auth::AuthenticatedDid>>,
) -> Result<Json<serde_json::Value>> {
if auth.is_none() {
return Err(crate::error::AppError::Unauthorized(
"authentication required for pin listing".into(),
));
}
// Bare `?` so connection-class sqlx failures downcast to `AppError::Db` and // Bare `?` so connection-class sqlx failures downcast to `AppError::Db` and
// map to 503 `db_unavailable` (not 500 via `.map_err(AppError::Internal)`) (#251). // map to 503 `db_unavailable` (not 500 via `.map_err(AppError::Internal)`) (#251).
let pins = state.db.list_pinned_cids().await?; let pins = state.db.list_pinned_cids().await?;
let pins: Vec<serde_json::Value> = pins
.into_iter()
.filter(|p| p.cid.is_some() || p.pinata_cid.is_some())
.map(|p| {
// A stored `cid` is advertised as a node-local resolver key
// only when it passes the raw-CIDv1 contract AND local
// provenance is confirmed. The historical `cid` column has
// held more than one namespace (Kubo dag-pb, Pinata CIDv0),
// and the `/ipfs/{cid}` resolver recomputes the raw CID
// from object bytes and rejects anything else — so a
// legacy provider key in `cid`/`local_cid` would promise a
// local object the node immediately refuses to serve.
// Such rows stay listed (with their provenance flags) but
// carry no local key until repair rewrites them.
let local_key = match &p.cid {
Some(c) if gitlawb_core::cid::is_raw_cidv1(c) && p.local_ipfs_provenance => {
Some(c.clone())
}
_ => None,
};
serde_json::json!({
"sha256_hex": p.sha256_hex,
"cid": local_key.clone(),
"local_cid": local_key,
"pinata_cid": p.pinata_cid,
"local_pinned": p.local_ipfs_provenance,
"pinata_pinned": p.pinata_cid.is_some(),
"pinned_at": p.pinned_at,
})
})
.collect();
Ok(Json(serde_json::json!({ Ok(Json(serde_json::json!({
"pins": pins, "pins": pins,
"count": pins.len(), "count": pins.len(),
@@ -2426,6 +2489,7 @@ mod closed_pool_tests {
.oneshot( .oneshot(
Request::builder() Request::builder()
.uri("/api/v1/ipfs/pins") .uri("/api/v1/ipfs/pins")
.extension(crate::auth::AuthenticatedDid("did:key:test".into()))
.body(axum::body::Body::empty()) .body(axum::body::Body::empty())
.unwrap(), .unwrap(),
) )
@@ -2450,6 +2514,369 @@ mod closed_pool_tests {
); );
} }
/// #218 review P2: the `list_pins` API response surfaces writer-owned
/// provenance (`local_pinned`, `pinata_pinned`) so a `gl` consumer
/// can distinguish local-only, remote-only, and dual rows without
/// re-inferring semantics from nullability. `cid`/`local_cid` carry a
/// value ONLY for rows whose stored key parses as a raw CIDv1 with
/// confirmed local provenance; anything else (Pinata-only rows,
/// legacy provider keys) exposes its usable identifier through
/// `pinata_cid` alone, and every non-null local key must resolve
/// through `GET /ipfs/{cid}`.
#[sqlx::test]
async fn list_pins_reports_writer_owned_provenance_for_all_shapes(pool: sqlx::PgPool) {
use sqlx::Row as _;
let state = crate::test_support::test_state(pool.clone()).await;
let db = &state.db;
// Raw resolver keys must be REAL raw CIDv1 strings: the handler
// gates `cid`/`local_cid` on `is_raw_cidv1`, so placeholder text
// would fail the gate for the wrong reason. Provider CIDs stay
// opaque Qm… strings (never syntax-checked, only echoed).
fn raw_cid_for(content: &[u8]) -> String {
gitlawb_core::cid::Cid::from_git_object_bytes(content).to_string()
}
// Two real blobs on disk in one public repo, so the local and
// dual rows below can be followed through `GET /ipfs/{cid}`.
// The resolver locates repos through `repo_store.acquire`, not
// the row's `disk_path`, so the recipe overwrites the acquired
// bare path with a real clone: workdir commit of both files,
// cloned --bare into the store path, oids via rev-parse, raw
// keys recomputed from the file bytes exactly as the serve
// path does.
fn run_git(args: &[&str], cwd: &std::path::Path) {
let out = std::process::Command::new("git")
.args(args)
.current_dir(cwd)
.output()
.expect("git runs");
assert!(
out.status.success(),
"git {args:?} failed: {}",
String::from_utf8_lossy(&out.stderr)
);
}
let content1 = b"pins follow-through one\n";
let content2 = b"pins follow-through two\n";
let tmp = tempfile::TempDir::new().unwrap();
let work = tmp.path().join("work-pins-list");
std::fs::create_dir_all(&work).unwrap();
std::fs::write(work.join("one.txt"), content1).unwrap();
std::fs::write(work.join("two.txt"), content2).unwrap();
run_git(
&["init", "-q", "--object-format=sha256", "-b", "main"],
&work,
);
run_git(&["config", "user.email", "t@t"], &work);
run_git(&["config", "user.name", "t"], &work);
run_git(&["add", "."], &work);
run_git(&["commit", "-qm", "seed"], &work);
let oid_of = |spec: &str| {
let out = std::process::Command::new("git")
.args(["rev-parse", spec])
.current_dir(&work)
.output()
.expect("git rev-parse runs");
assert!(out.status.success(), "rev-parse {spec} failed");
String::from_utf8_lossy(&out.stdout).trim().to_string()
};
let sha_local = oid_of("HEAD:one.txt");
let sha_dual = oid_of("HEAD:two.txt");
let raw1 = raw_cid_for(content1);
let raw4 = raw_cid_for(content2);
db.upsert_mirror_repo(
"zPinsListOwner",
"pins-list",
"/unused-pins-list",
None,
false,
)
.await
.unwrap();
let rec = db
.get_repo("zPinsListOwner", "pins-list")
.await
.unwrap()
.expect("mirror repo row");
let bare = state
.repo_store
.acquire(&rec.owner_did, &rec.name)
.await
.unwrap();
let _ = std::fs::remove_dir_all(&bare);
std::fs::create_dir_all(&bare).unwrap();
run_git(
&[
"clone",
"--bare",
"-q",
work.to_str().unwrap(),
bare.to_str().unwrap(),
],
tmp.path(),
);
// The combination table for what the response must carry:
//
// shape | local_pinned | pinata_pinned | local_cid | pinata_cid
// -------------------+--------------+---------------+-----------+-----------
// local-only | true | false | raw | null
// pinata-only (raw) | false | true | null | provider
// pinata-only (null) | false | true | null | raw(=provider)
// dual | true | true | raw | provider
// legacy provider key| true | false | null | null
//
// The pinata-only (raw) row keeps a genuine raw key in the
// column but must NOT surface it: provenance is unconfirmed,
// so no local field may carry it. The legacy row predates the
// raw-key contract (dag-pb key, provenance claimed) and must
// surface no local key until repair rewrites it — while the
// row itself (and its `local_pinned` flag) stays visible.
//
// Distinct shas (the table is keyed on sha256_hex) keep the
// rows independent.
// (1) local-only: real local pin via the production writer.
db.record_pinned_cid_with_source(&sha_local, &raw1, &rec.id)
.await
.unwrap();
// (2) pinata-only (raw != provider): the post-v27/v30 row shape
// produced by `record_pinata_cid`. The stored raw key is genuine
// but local provenance is unconfirmed, so neither local field
// may carry it; the provider field is the usable identifier.
let sha_pinata_raw = "sha_p2_pinata_only_distinct_cids";
let raw2 = raw_cid_for(b"pins pinata-only bytes\n");
let pinata2 = "QmPinataProviderCidForRawOnlyRow";
assert_ne!(raw2, pinata2);
db.record_pinata_cid(
sha_pinata_raw,
&raw2,
pinata2,
Some("repo-p2-pinata-raw"),
i64::MAX,
)
.await
.unwrap();
// (3) pinata-only (raw == provider): `record_pinata_cid` stores
// `cid = NULL` so the resolver key isn't aliased to a dag-pb
// provider CID that doesn't hash to the raw bytes (#173).
let sha_pinata_null = "sha_p2_pinata_only_null_cid";
let same = "QmPinataOnlyCidRawEqualsProvider";
db.record_pinata_cid(
sha_pinata_null,
same,
same,
Some("repo-p2-pinata-null"),
i64::MAX,
)
.await
.unwrap();
// (4) dual: local first (sets the flag and `cid`), then Pinata
// (sets `pinata_cid` and preserves `cid` and `local_ipfs_provenance`).
let pinata4 = "QmPinataProviderCidForDualRow";
db.record_pinned_cid_with_source(&sha_dual, &raw4, &rec.id)
.await
.unwrap();
db.record_pinata_cid(&sha_dual, &raw4, pinata4, Some(&rec.id), i64::MAX)
.await
.unwrap();
// (5) legacy provider key: a pre-fix row whose `cid` is a dag-pb
// provider identifier with claimed local provenance. Production
// writers can no longer produce this shape, so it is seeded with
// raw SQL — the point is that the projection must not trust column
// shape as proof of a raw local key.
let sha_legacy = "sha_p2_legacy_provider_key";
sqlx::query(
"INSERT INTO pinned_cids (sha256_hex, cid, pinned_at, pinata_cid, repo_id, local_ipfs_provenance)
VALUES ($1, $2, $3, NULL, $4, TRUE)",
)
.bind(sha_legacy)
.bind("QmLegacyProviderKeyPredatesRawContract")
.bind("2026-07-01T00:00:00Z")
.bind(&rec.id)
.execute(&pool)
.await
.unwrap();
// Hit the production handler end-to-end through a one-shot router
// request, so the test exercises the actual response shape
// (not a unit test on PinnedCidRecord fields).
let resp = Router::new()
.route("/api/v1/ipfs/pins", axum::routing::get(list_pins))
.layer(axum::Extension(crate::auth::AuthenticatedDid(
"did:key:zPinListTest".to_string(),
)))
.with_state(state.clone())
.oneshot(
axum::http::Request::builder()
.uri("/api/v1/ipfs/pins")
.body(axum::body::Body::empty())
.unwrap(),
)
.await
.unwrap();
assert_eq!(resp.status(), axum::http::StatusCode::OK);
let bytes = axum::body::to_bytes(resp.into_body(), usize::MAX)
.await
.expect("read body");
let v: Value = serde_json::from_slice(&bytes).expect("json body");
let pins = v
.get("pins")
.and_then(|p| p.as_array())
.expect("pins array");
assert_eq!(
pins.len(),
5,
"all five shapes must appear in the response (every row has at least one of cid/pinata_cid set)"
);
// Build a map sha -> pin object for ergonomic assertions.
let by_sha: std::collections::HashMap<String, &Value> = pins
.iter()
.map(|p| {
(
p.get("sha256_hex")
.and_then(|s| s.as_str())
.unwrap()
.to_string(),
p,
)
})
.collect();
// Helper: assert a single pin's fields.
let assert_pin = |sha: &str,
local_pinned: bool,
pinata_pinned: bool,
local_cid: Option<&str>,
pinata_cid: Option<&str>| {
let pin = by_sha.get(sha).unwrap_or_else(|| {
panic!(
"pin row for {sha} missing; got shas {:?}",
by_sha.keys().collect::<Vec<_>>()
)
});
assert_eq!(
pin.get("local_pinned").and_then(|v| v.as_bool()),
Some(local_pinned),
"{sha}: local_pinned mismatch",
);
assert_eq!(
pin.get("pinata_pinned").and_then(|v| v.as_bool()),
Some(pinata_pinned),
"{sha}: pinata_pinned mismatch",
);
assert_eq!(
pin.get("local_cid").and_then(|v| v.as_str()),
local_cid,
"{sha}: local_cid mismatch",
);
assert_eq!(
pin.get("pinata_cid").and_then(|v| v.as_str()),
pinata_cid,
"{sha}: pinata_cid mismatch",
);
// Round-3 P2: `cid` is the LOCAL resolver key, full
// stop. It is the raw CID when the row has a local pin
// (local-only or dual shape) and `None` for Pinata-only
// rows. The previous contract aliased the Pinata provider
// CID into `cid` for Pinata-only rows, which made
// `gl ipfs list` advertise a CID the node's own
// `/ipfs/{cid}` resolver cannot serve (404). The
// provenance split is exposed via `local_cid`,
// `pinata_cid`, and the boolean `local_pinned` /
// `pinata_pinned` flags.
let cid_value = pin.get("cid").cloned().unwrap_or(serde_json::Value::Null);
let expected_cid = local_cid
.map(|s| serde_json::Value::String(s.to_string()))
.unwrap_or(serde_json::Value::Null);
assert_eq!(cid_value, expected_cid, "{sha}: cid mismatch");
};
// (1) local-only
assert_pin(&sha_local, true, false, Some(raw1.as_str()), None);
// (2) pinata-only (raw != provider): the stored raw key is
// genuine but provenance is unconfirmed, so NO local field may
// carry it — remote-only state is visible through `pinata_cid`
// alone.
assert_pin(sha_pinata_raw, false, true, None, Some(pinata2));
// (3) pinata-only (raw == provider) — cid NULL, pinata_cid set
assert_pin(sha_pinata_null, false, true, None, Some(same));
// (4) dual
assert_pin(&sha_dual, true, true, Some(raw4.as_str()), Some(pinata4));
// (5) legacy provider key: no local field carries the dag-pb
// key, but the row and its claimed provenance stay visible so
// the repair gap is observable rather than hidden.
assert_pin(sha_legacy, true, false, None, None);
// Sanity: the `local_ipfs_provenance` column itself is what
// powers `local_pinned`, so the response field must agree
// with the database column. Reading directly avoids any
// confusion if the writer path changes.
let rows = sqlx::query("SELECT sha256_hex, local_ipfs_provenance FROM pinned_cids")
.fetch_all(&pool)
.await
.unwrap();
let mut db_provenance: std::collections::HashMap<String, bool> = Default::default();
for r in rows {
let sha: String = r.get("sha256_hex");
let p: bool = r.get("local_ipfs_provenance");
db_provenance.insert(sha, p);
}
for (sha, expected) in [
(sha_local.as_str(), true),
(sha_pinata_raw, false),
(sha_pinata_null, false),
(sha_dual.as_str(), true),
(sha_legacy, true),
] {
assert_eq!(
db_provenance.get(sha).copied(),
Some(expected),
"db column for {sha} does not match expected writer-owned provenance"
);
}
// Follow-through: every non-null local key in the response must
// resolve through `GET /ipfs/{cid}`. The local-only and dual rows
// above name real blobs on disk in a public repo, so both must
// serve their exact bytes; the Pinata-only and legacy rows carry
// no local key and have nothing to follow.
let follow_router = Router::new()
.route("/ipfs/{cid}", axum::routing::get(get_by_cid))
.with_state(state.clone());
for (raw, content) in [(&raw1, &content1[..]), (&raw4, &content2[..])] {
let resp = follow_router
.clone()
.oneshot(
axum::http::Request::builder()
.uri(format!("/ipfs/{raw}"))
.body(axum::body::Body::empty())
.unwrap(),
)
.await
.unwrap();
assert_eq!(
resp.status(),
axum::http::StatusCode::OK,
"advertised local key {raw} must resolve"
);
let body = axum::body::to_bytes(resp.into_body(), usize::MAX)
.await
.expect("read body");
assert_eq!(
body.as_ref(),
content,
"resolved bytes for {raw} must match the pinned blob"
);
}
}
/// #251 / CodeRabbit nit: cover `get_by_cid`'s DB-error conversion path — a /// #251 / CodeRabbit nit: cover `get_by_cid`'s DB-error conversion path — a
/// valid CID must still yield 503 on a closed pool. /// valid CID must still yield 503 on a closed pool.
#[sqlx::test] #[sqlx::test]
@@ -8358,6 +8785,7 @@ mod tests {
#[tokio::test] #[tokio::test]
async fn get_by_cid_per_source_cap_sheds_same_source_admits_other() { async fn get_by_cid_per_source_cap_sheds_same_source_admits_other() {
let mut state = crate::test_support::test_state_lazy(); let mut state = crate::test_support::test_state_lazy();
state.db.pool().close().await;
// Global pool has room; the per-source cap is 1. // Global pool has room; the per-source cap is 1.
state.git_ipfs_walk_semaphore = Arc::new(Semaphore::new(8)); state.git_ipfs_walk_semaphore = Arc::new(Semaphore::new(8));
state.git_ipfs_walk_per_caller = crate::rate_limit::PerCallerConcurrency::new(1, 100); state.git_ipfs_walk_per_caller = crate::rate_limit::PerCallerConcurrency::new(1, 100);
@@ -8384,16 +8812,23 @@ mod tests {
"a source at its per-source /ipfs walk cap must shed 503 with global capacity free" "a source at its per-source /ipfs walk cap must shed 503 with global capacity free"
); );
let bytes = axum::body::to_bytes(resp.into_body(), 1024).await.unwrap();
let body: serde_json::Value = serde_json::from_slice(&bytes).unwrap();
assert_eq!(body["error"], "overloaded");
// A DIFFERENT source is NOT shed by the per-source cap: it clears admission and // A DIFFERENT source is NOT shed by the per-source cap: it clears admission and
// proceeds (then errors on the lazy DB, which is not a 503). // proceeds to the closed DB, which has a distinct db_unavailable error code.
let resp = ipfs_router(state) let resp = ipfs_router(state)
.oneshot(get_cid(&cid, Some(other))) .oneshot(get_cid(&cid, Some(other)))
.await .await
.unwrap(); .unwrap();
assert_ne!( assert_eq!(resp.status(), StatusCode::SERVICE_UNAVAILABLE);
resp.status(), let bytes = axum::body::to_bytes(resp.into_body(), 1024).await.unwrap();
StatusCode::SERVICE_UNAVAILABLE, let body: serde_json::Value = serde_json::from_slice(&bytes).unwrap();
"a different source must not be shed by the per-source cap" assert_eq!(
body["error"],
crate::error::DB_UNAVAILABLE_CODE,
"a different source must clear admission and reach the closed database"
); );
} }
+75 -40
View File
@@ -238,49 +238,32 @@ pub async fn close_issue(
headers: axum::http::HeaderMap, headers: axum::http::HeaderMap,
crate::rate_limit::PeerAddr(peer): crate::rate_limit::PeerAddr, crate::rate_limit::PeerAddr(peer): crate::rate_limit::PeerAddr,
) -> Result<Json<serde_json::Value>> { ) -> Result<Json<serde_json::Value>> {
let record = state // READ-GATE before any snapshot work or rate charging, through the shared
.db // authorize_repo_read gate so quarantined repos are opaque here too. The
.get_repo(&owner, &repo) // author fallback below needs the issue blob, which needs the repo tree, so
.await? // authorship cannot be established without a download; but a caller who
.ok_or_else(|| AppError::RepoNotFound(format!("{owner}/{repo}")))?; // cannot even READ the repo must be stopped here, cheaply, before any
// Tigris transfer or extraction happens. Without this, any signed non-owner
// READ-GATE before any snapshot work or rate charging. The author fallback below // could issue parallel close requests for arbitrary issue ids and drive
// needs the issue blob, which needs the repo tree, so authorship cannot be established // unbounded downloads and blocking extraction (a disposable-identity DoS),
// without a download; but a caller who cannot even READ the repo must be // because the route has no other pre-authorization.
// stopped here, cheaply, before any Tigris transfer or extraction happens.
// Without this, any signed non-owner could issue parallel close requests for
// arbitrary issue ids and drive unbounded downloads and blocking extraction
// (a disposable-identity DoS), because the route has no other pre-authorization.
// //
// Rate limiting runs AFTER this gate so a 429 cannot distinguish a hidden repo // Rate limiting runs AFTER this gate so a 429 cannot distinguish a hidden
// from a missing one (INV-12 read-denial status contract). // repo from a missing one (INV-12 read-denial status contract).
// //
// mirror-rows-handled: a repo row synced from a peer is stored public and // mirror-rows-handled: a repo row synced from a peer is stored public and
// carries none of the owner's visibility rules, so for such a row this check // carries none of the owner's visibility rules, so for such a row this gate
// can only return allow. That is deliberate rather than overlooked, and it is // can only return allow. That is deliberate rather than overlooked, and it
// the same verdict every other read gate in this API reaches for one. Refusing // is the same verdict every other read gate in this API reaches for one.
// instead would deny the repo's real owner and the issue's real author on any // Refusing instead would deny the repo's real owner and the issue's real
// node whose only copy of the repo is a synced one, which is an ordinary state // author on any node whose only copy of the repo is a synced one, which is
// here, and it would not buy the protection it appears to, because a synced // an ordinary state here, and it would not buy the protection it appears
// row's recorded owner comes from the peer that sent it. The expensive work // to, because a synced row's recorded owner comes from the peer that sent
// this check guards is bounded ahead of it by the per-IP limiter and the read // it. The expensive work this check guards is bounded ahead of it by the
// pool slot taken below, and the authoritative owner-or-author decision still // per-IP limiter and the read pool slot taken below, and the authoritative
// runs below and again under the write lock. // owner-or-author decision still runs below and again under the write lock.
{ let (record, _rules) =
// mirror-rows-handled: synced mirror rows carry no owner rules; read gate only. crate::api::authorize_repo_read(&state, &owner, &repo, Some(auth.0.as_str()), "/").await?;
let rules = state.db.list_visibility_rules(&record.id).await?;
let caller = auth.0.as_str();
if crate::visibility::visibility_check(
&rules,
record.is_public,
&record.owner_did,
Some(caller),
"/",
) == crate::visibility::Decision::Deny
{
return Err(AppError::RepoNotFound(format!("{owner}/{repo}")));
}
}
// Per-IP flood brake, layered on the same shared limiter and trusted-proxy // Per-IP flood brake, layered on the same shared limiter and trusted-proxy
// policy as the push advertisement. The pre-lock snapshot downloads the // policy as the push advertisement. The pre-lock snapshot downloads the
@@ -502,6 +485,15 @@ mod tests {
.await .await
.expect("get_repo") .expect("get_repo")
.expect("repo exists"); .expect("repo exists");
// upsert_mirror_repo seeds the row quarantined, and the read gate now
// hides quarantined rows as opaque 404 before the owner-or-author check
// this test names. Release it so the stranger is refused by the 403, not
// by the quarantine short-circuit.
state
.db
.set_repo_quarantine(&record.id, false)
.await
.expect("release seed quarantine");
// An independent session holds the repo's write lock for the whole call. // An independent session holds the repo's write lock for the whole call.
let key = crate::git::repo_store::advisory_lock_key_for_test( let key = crate::git::repo_store::advisory_lock_key_for_test(
@@ -703,6 +695,15 @@ mod tests {
.await .await
.expect("get_repo") .expect("get_repo")
.expect("seeded repo exists"); .expect("seeded repo exists");
// upsert_mirror_repo seeds the admission flag as quarantined, but the
// gate now hides quarantined rows from every caller class. Release it
// so these tests exercise what they name: the owner-or-author check
// after the read gate, not the quarantine short-circuit.
state
.db
.set_repo_quarantine(&record.id, false)
.await
.expect("release seed quarantine");
let _ = owner_did; let _ = owner_did;
let path = state let path = state
.repo_store .repo_store
@@ -728,6 +729,40 @@ mod tests {
path path
} }
/// The read gate blocks the owner (and any caller) before the
/// owner-or-author checks: even the owner gets the opaque 404 on a
/// quarantined repo, and the seeded row's quarantined flag is left set
/// deliberately. The released-mirror variants above exercise the
/// owner-or-author path.
#[sqlx::test]
async fn quarantined_repo_close_is_opaque_404(pool: PgPool) {
let state = crate::test_support::test_state(pool.clone()).await;
let owner_did = "did:key:z6MkT1Owner";
state
.db
.upsert_mirror_repo("z6MkT1Owner", "tquar", "/unused", None, true)
.await
.expect("seed repo row");
let res = close_issue(
axum::extract::State(state.clone()),
axum::Extension(crate::auth::AuthenticatedDid(owner_did.to_string())),
axum::extract::Path((
"z6MkT1Owner".to_string(),
"tquar".to_string(),
"1".to_string(),
)),
axum::http::HeaderMap::new(),
crate::rate_limit::PeerAddr(Some("203.0.113.70:5000".parse().unwrap())),
)
.await;
assert!(
matches!(res, Err(crate::error::AppError::RepoNotFound(_))),
"a quarantined repo must 404 close_issue for every caller: {:?}",
res.ok().map(|_| "unexpected success")
);
}
/// INV-21(c) positive twin 1: the OWNER can still close. The reorder moved the /// INV-21(c) positive twin 1: the OWNER can still close. The reorder moved the
/// owner check above the lock, so this is the arm most likely to have broken, /// owner check above the lock, so this is the arm most likely to have broken,
/// and the deny test alone could not see it. /// and the deny test alone could not see it.
+28 -19
View File
@@ -14,14 +14,13 @@ pub struct LabelRequest {
pub label: String, pub label: String,
} }
/// POST /api/v1/repos/:owner/:repo/labels /// Canonicalize and validate a label string. `add_label` and `remove_label`
pub async fn add_label( /// must agree on the form stored in the DB: `add_label` already trims and
State(state): State<AppState>, /// lowercases before storing, so `remove_label` has to do the same to find
Extension(auth): Extension<AuthenticatedDid>, /// the row. Validation is shared so neither handler can drift on charset or
Path((owner, name)): Path<(String, String)>, /// length (#344).
Json(req): Json<LabelRequest>, fn canonicalize_label(raw: &str) -> std::result::Result<String, AppError> {
) -> Result<(StatusCode, Json<serde_json::Value>)> { let label = raw.trim().to_lowercase();
let label = req.label.trim().to_lowercase();
if label.is_empty() || label.len() > 50 { if label.is_empty() || label.len() > 50 {
return Err(AppError::BadRequest("label must be 1–50 characters".into())); return Err(AppError::BadRequest("label must be 1–50 characters".into()));
} }
@@ -33,12 +32,20 @@ pub async fn add_label(
"label must contain only alphanumeric characters, hyphens, and colons".into(), "label must contain only alphanumeric characters, hyphens, and colons".into(),
)); ));
} }
Ok(label)
}
let record = state /// POST /api/v1/repos/:owner/:repo/labels
.db pub async fn add_label(
.get_repo(&owner, &name) State(state): State<AppState>,
.await? Extension(auth): Extension<AuthenticatedDid>,
.ok_or_else(|| AppError::RepoNotFound(format!("{owner}/{name}")))?; Path((owner, name)): Path<(String, String)>,
Json(req): Json<LabelRequest>,
) -> Result<(StatusCode, Json<serde_json::Value>)> {
let label = canonicalize_label(&req.label)?;
let (record, _rules) =
crate::api::authorize_repo_read(&state, &owner, &name, Some(auth.0.as_str()), "/").await?;
crate::api::require_repo_owner(&record, &auth.0)?; crate::api::require_repo_owner(&record, &auth.0)?;
let added = state.db.add_label(&record.id, &label).await?; let added = state.db.add_label(&record.id, &label).await?;
@@ -59,14 +66,16 @@ pub async fn remove_label(
Extension(auth): Extension<AuthenticatedDid>, Extension(auth): Extension<AuthenticatedDid>,
Path((owner, name, label)): Path<(String, String, String)>, Path((owner, name, label)): Path<(String, String, String)>,
) -> Result<Json<serde_json::Value>> { ) -> Result<Json<serde_json::Value>> {
let record = state let label = canonicalize_label(&label)?;
.db
.get_repo(&owner, &name) let (record, _rules) =
.await? crate::api::authorize_repo_read(&state, &owner, &name, Some(auth.0.as_str()), "/").await?;
.ok_or_else(|| AppError::RepoNotFound(format!("{owner}/{name}")))?;
crate::api::require_repo_owner(&record, &auth.0)?; crate::api::require_repo_owner(&record, &auth.0)?;
state.db.remove_label(&record.id, &label).await?; let removed = state.db.remove_label(&record.id, &label).await?;
if !removed {
return Err(AppError::NotFound(format!("label '{label}'")));
}
Ok(Json(serde_json::json!({ "label": label, "removed": true }))) Ok(Json(serde_json::json!({ "label": label, "removed": true })))
} }
+125 -21
View File
@@ -22,6 +22,8 @@ pub mod replicas;
pub mod repos; pub mod repos;
pub mod resolve; pub mod resolve;
pub mod stars; pub mod stars;
pub mod status;
pub mod task_cursor;
pub mod tasks; pub mod tasks;
pub mod visibility; pub mod visibility;
pub mod webhooks; pub mod webhooks;
@@ -67,28 +69,32 @@ pub(crate) async fn authorize_repo_read(
/// representation only within `did:key`; never let a bare id match across methods — /// representation only within `did:key`; never let a bare id match across methods —
/// `did:web` / `did:gitlawb` share the base58 space with `did:key`, so a /// `did:web` / `did:gitlawb` share the base58 space with `did:key`, so a
/// trailing-segment compare would treat `did:key:X` and `did:gitlawb:X` as equal. /// trailing-segment compare would treat `did:key:X` and `did:gitlawb:X` as equal.
///
/// The collapse itself lives in exactly one place, [`crate::db::normalize_owner_key`],
/// which is also the function the stored `owner_did` / `authorizing_did` columns
/// are normalized through and which `OWNER_KEY_CASE_SQL` mirrors byte for byte.
/// Two identities match when they normalize to the same key; a second copy of the
/// rule here is how the Rust gate and the SQL filters would drift apart.
pub(crate) fn did_matches(a: &str, b: &str) -> bool { pub(crate) fn did_matches(a: &str, b: &str) -> bool {
if a == b { // One implementation, shared with `gl` and `git-remote-gitlawb` through core:
return true; // every boundary that compares a DID applies the same collapse rule.
} gitlawb_core::ucan::push::did_key_eq(a, b)
fn key_id(d: &str) -> &str {
d.strip_prefix("did:key:").unwrap_or(d)
}
let (ka, kb) = (key_id(a), key_id(b));
// After stripping `did:key:`, a value still containing ':' is a non-key full
// DID — do not let it match a bare `did:key` id.
!ka.contains(':') && !kb.contains(':') && ka == kb
} }
/// Client-facing denial shared by every repo-owner gate copy. `require_repo_owner`
/// renders it as `AppError::Forbidden`; a handler that emits its own
/// `(StatusCode, Json)` envelope instead of `AppError` (e.g.
/// `api::tasks::create_task`) must use this constant too, so two copies of one
/// gate cannot drift in observable output.
pub(crate) const REPO_OWNER_ONLY_MESSAGE: &str = "only the repo owner can perform this action";
/// 403 unless `caller` is the repo owner. Uses [`did_matches`] so the owner check /// 403 unless `caller` is the repo owner. Uses [`did_matches`] so the owner check
/// and the author check (close policy) share one normalization. /// and the author check (close policy) share one normalization.
pub(crate) fn require_repo_owner(record: &RepoRecord, caller: &str) -> Result<()> { pub(crate) fn require_repo_owner(record: &RepoRecord, caller: &str) -> Result<()> {
if did_matches(caller, &record.owner_did) { if did_matches(caller, &record.owner_did) {
Ok(()) Ok(())
} else { } else {
Err(AppError::Forbidden( Err(AppError::Forbidden(REPO_OWNER_ONLY_MESSAGE.into()))
"only the repo owner can perform this action".into(),
))
} }
} }
@@ -171,16 +177,28 @@ mod authz_guard {
"\nasync fn ", "\nasync fn ",
"\npub fn ", "\npub fn ",
"\nfn ", "\nfn ",
"#[cfg(test)]",
] ]
.iter() .iter()
.filter_map(|p| rest[1..].find(p).map(|i| i + 1)) .filter_map(|p| rest[1..].find(p).map(|i| i + 1))
.min() .min()
.unwrap_or(rest.len()); .unwrap_or(rest.len());
rest[..end] // Strip block comments too: a marker surviving only inside /* */ would
// otherwise satisfy a row with the real gate deleted.
// Line comments first: a `//` line containing `/*` must not open a
// block range or real code (and gate markers) gets stripped with it.
let mut body = rest[..end]
.lines() .lines()
.filter(|l| !l.trim_start().starts_with("//")) .filter(|l| !l.trim_start().starts_with("//"))
.collect::<Vec<_>>() .collect::<Vec<_>>()
.join("\n") .join("\n");
while let Some(open) = body.find("/*") {
match body[open..].find("*/").map(|i| open + i + 2) {
Some(close) => body.replace_range(open..close, ""),
None => break,
}
}
body
} }
#[test] #[test]
@@ -194,6 +212,7 @@ mod authz_guard {
let events = include_str!("events.rs"); let events = include_str!("events.rs");
let tasks = include_str!("tasks.rs"); let tasks = include_str!("tasks.rs");
let stars = include_str!("stars.rs"); let stars = include_str!("stars.rs");
let status = include_str!("status/mod.rs");
let protect = include_str!("protect.rs"); let protect = include_str!("protect.rs");
let visibility = include_str!("visibility.rs"); let visibility = include_str!("visibility.rs");
let profiles = include_str!("profiles.rs"); let profiles = include_str!("profiles.rs");
@@ -203,9 +222,14 @@ mod authz_guard {
// (source, handler, expected gate marker) // (source, handler, expected gate marker)
let rows: &[(&str, &str, &str)] = &[ let rows: &[(&str, &str, &str)] = &[
// Bucket A — owner-gate (require_repo_owner -> 403) // Bucket A — owner-gate (require_repo_owner -> 403), now behind
// the shared read gate first so a hidden repo 404s for every
// caller class before the owner check can answer 403.
(pulls, "merge_pr", "authorize_repo_read("),
(pulls, "merge_pr", "require_repo_owner("), (pulls, "merge_pr", "require_repo_owner("),
(webhooks, "create_webhook", "authorize_repo_read("),
(webhooks, "create_webhook", "require_repo_owner("), (webhooks, "create_webhook", "require_repo_owner("),
(webhooks, "delete_webhook", "authorize_repo_read("),
(webhooks, "delete_webhook", "require_repo_owner("), (webhooks, "delete_webhook", "require_repo_owner("),
// Bucket A/B hybrid — list_webhooks is read-visibility THEN owner: // Bucket A/B hybrid — list_webhooks is read-visibility THEN owner:
// authorize_repo_read 404s a non-reader of a private repo, then // authorize_repo_read 404s a non-reader of a private repo, then
@@ -214,10 +238,22 @@ mod authz_guard {
// (existence hiding) and require_repo_owner guards the owner half. // (existence hiding) and require_repo_owner guards the owner half.
(webhooks, "list_webhooks", "authorize_repo_read("), (webhooks, "list_webhooks", "authorize_repo_read("),
(webhooks, "list_webhooks", "require_repo_owner("), (webhooks, "list_webhooks", "require_repo_owner("),
// Same two-half shape as list_webhooks: authorize_repo_read runs
// first (a quarantined or unreadable repo gets the missing-repo
// not-found, never a 403 that would confirm existence), then
// require_repo_owner 403s a non-owner of a readable repo. Both halves
// are pinned, because dropping either changes what a stranger learns.
(status, "create_status", "authorize_repo_read("),
(status, "create_status", "require_repo_owner("),
(labels, "add_label", "authorize_repo_read("),
(labels, "add_label", "require_repo_owner("), (labels, "add_label", "require_repo_owner("),
(labels, "remove_label", "authorize_repo_read("),
(labels, "remove_label", "require_repo_owner("), (labels, "remove_label", "require_repo_owner("),
// Bucket A' — owner OR author (did_matches against the author) // Bucket A' — owner OR author (did_matches against the author),
// also behind the shared read gate first.
(pulls, "close_pr", "authorize_repo_read("),
(pulls, "close_pr", "did_matches("), (pulls, "close_pr", "did_matches("),
(issues, "close_issue", "authorize_repo_read("),
(issues, "close_issue", "did_matches("), (issues, "close_issue", "did_matches("),
// Bucket B — read-gate (authorize_repo_read) // Bucket B — read-gate (authorize_repo_read)
(pulls, "create_review", "authorize_repo_read("), (pulls, "create_review", "authorize_repo_read("),
@@ -240,8 +276,26 @@ mod authz_guard {
(protect, "list_protected_branches", "authorize_repo_read("), (protect, "list_protected_branches", "authorize_repo_read("),
(labels, "list_labels", "authorize_repo_read("), (labels, "list_labels", "authorize_repo_read("),
(events, "list_repo_events", "authorize_repo_read("), (events, "list_repo_events", "authorize_repo_read("),
// The catch-up poll surface gates before any push-event row is read,
// so a stranger cannot use it to learn that a private repo exists or
// is being pushed to.
(events, "list_repo_push_events", "authorize_repo_read("),
// The status read gates before any claim data is loaded, so its deny
// is the repo's own not-found rather than an existence oracle over
// which commits were reported on.
(status, "commit_status", "authorize_repo_read("),
// The rollup gates on the same helper before the pull request row is
// loaded, so a caller who cannot read the repo cannot learn which
// pull request numbers exist on it — and the fallback's branch
// resolve and head persist sit behind that same gate.
(status, "pull_request_status", "authorize_repo_read("),
// Bucket C — signer-self: the acting DID is matched/bound to auth.0 // Bucket C — signer-self: the acting DID is matched/bound to auth.0
(tasks, "create_task", "did_matches("), (tasks, "create_task", "did_matches("),
// #496: create_task is signer-self AND owner-when-repo-scoped — a
// repo_id naming a hosted repo admits tasks only from its owner.
// Both halves are pinned: the did_matches row guards the signer
// binding, this one the repo-ownership gate.
(tasks, "create_task", "require_repo_owner("),
(tasks, "claim_task", "did_matches("), (tasks, "claim_task", "did_matches("),
(tasks, "complete_task", "did_matches("), (tasks, "complete_task", "did_matches("),
(tasks, "fail_task", "did_matches("), (tasks, "fail_task", "did_matches("),
@@ -258,21 +312,66 @@ mod authz_guard {
(bounties, "dispute_bounty", "did_matches("), (bounties, "dispute_bounty", "did_matches("),
(replicas, "register_replica", "did_matches("), (replicas, "register_replica", "did_matches("),
(replicas, "unregister_replica", "replica_did = &auth.0"), (replicas, "unregister_replica", "replica_did = &auth.0"),
// PRE-GATED — already owner-gated, in-scope group; guard the gate itself // PRE-GATED — owner-gated behind the shared read gate; guard both
(protect, "protect_branch", "authorize_repo_read("),
(protect, "protect_branch", "did_matches("), (protect, "protect_branch", "did_matches("),
(protect, "unprotect_branch", "authorize_repo_read("),
(protect, "unprotect_branch", "did_matches("), (protect, "unprotect_branch", "did_matches("),
(visibility, "set_visibility", "authorize_repo_read("),
(visibility, "set_visibility", "require_owner("), (visibility, "set_visibility", "require_owner("),
(visibility, "remove_visibility", "authorize_repo_read("),
(visibility, "remove_visibility", "require_owner("), (visibility, "remove_visibility", "require_owner("),
(visibility, "list_visibility", "authorize_repo_read("),
(visibility, "list_visibility", "require_owner("), (visibility, "list_visibility", "require_owner("),
]; ];
// The visibility rows prove require_owner is CALLED; this proves the helper // Rows above prove each visibility handler calls authorize_repo_read and
// itself does DID-safe matching, not a raw/trailing-segment compare. // require_owner; this proves the helper itself does DID-safe matching.
assert!( assert!(
fn_body(visibility, "require_owner").contains("did_matches("), fn_body(visibility, "require_owner").contains("did_matches("),
"visibility::require_owner must use did_matches for DID-safe owner matching" "visibility::require_owner must use did_matches for DID-safe owner matching"
); );
// Order matters as much as presence: hoisting the owner/author check
// above the read gate would 403 a non-owner on a hidden repo, an
// existence oracle. Pin read-gate-then-owner on every handler that
// takes both; the owner marker differs per bucket (require_repo_owner
// for owner-only, require_owner for visibility, did_matches for the
// owner-or-author and protect paths).
for (src, func, owner_marker) in [
(visibility, "set_visibility", "require_owner("),
(visibility, "remove_visibility", "require_owner("),
(visibility, "list_visibility", "require_owner("),
(pulls, "merge_pr", "require_repo_owner("),
(webhooks, "list_webhooks", "require_repo_owner("),
(webhooks, "create_webhook", "require_repo_owner("),
(webhooks, "delete_webhook", "require_repo_owner("),
(labels, "add_label", "require_repo_owner("),
(labels, "remove_label", "require_repo_owner("),
(pulls, "close_pr", "did_matches("),
(issues, "close_issue", "did_matches("),
(protect, "protect_branch", "did_matches("),
(protect, "unprotect_branch", "did_matches("),
] {
let body = fn_body(src, func);
let gate = body.find("authorize_repo_read(").expect("gate marker");
let owner = body.find(owner_marker).expect("owner marker");
assert!(
gate < owner,
"{func} must run authorize_repo_read before {owner_marker}"
);
// The owner-or-author and protect handlers also carry
// require_repo_owner: a hoist of that call above the read gate is
// the same 403-vs-404 oracle, so pin it too where it exists.
if func == "close_pr" || func == "close_issue" {
let owner_gate = body.find("require_repo_owner(").expect("owner marker");
assert!(
gate < owner_gate,
"{func} must run authorize_repo_read before require_repo_owner("
);
}
}
// The CID read surface (#173) enforces its gate inside the shared // The CID read surface (#173) enforces its gate inside the shared
// `gate_and_serve`, which BOTH the provenance path and the legacy scan call, so // `gate_and_serve`, which BOTH the provenance path and the legacy scan call, so
// the markers must live there (the get_by_cid row above only proves delegation). // the markers must live there (the get_by_cid row above only proves delegation).
@@ -341,7 +440,11 @@ mod authz_guard {
( (
"create_task", "create_task",
"createTask", "createTask",
&["require_signer(ctx)?", "did_matches("], &[
"require_signer(ctx)?",
"did_matches(",
"require_repo_owner(",
],
), ),
( (
"claim_task", "claim_task",
@@ -1096,6 +1199,7 @@ mod tests { async fn t() { let _ = did_matches(x, y); } }
(include_str!("replicas.rs"), "replicas.rs"), (include_str!("replicas.rs"), "replicas.rs"),
(include_str!("repos.rs"), "repos.rs"), (include_str!("repos.rs"), "repos.rs"),
(include_str!("stars.rs"), "stars.rs"), (include_str!("stars.rs"), "stars.rs"),
(include_str!("status/mod.rs"), "status/mod.rs"),
(include_str!("visibility.rs"), "visibility.rs"), (include_str!("visibility.rs"), "visibility.rs"),
(include_str!("webhooks.rs"), "webhooks.rs"), (include_str!("webhooks.rs"), "webhooks.rs"),
]; ];
+220 -66
View File
@@ -70,6 +70,8 @@ pub struct PeerResponse {
pub reachable: bool, pub reachable: bool,
} }
pub(crate) const PUBLIC_HTTP_URL_REQUIREMENT: &str = "must be a public http(s) URL (no loopback, private, localhost, .localhost, .internal, or .local hosts)";
/// Extract an IPv4 address embedded in an IPv6 literal across the transition /// Extract an IPv4 address embedded in an IPv6 literal across the transition
/// formats that carry one: IPv4-mapped (`::ffff:a.b.c.d`), IPv4-compatible /// formats that carry one: IPv4-mapped (`::ffff:a.b.c.d`), IPv4-compatible
/// (`::a.b.c.d`), 6to4 (`2002:WWXX:YYZZ::/16`), and the NAT64 well-known prefix /// (`::a.b.c.d`), 6to4 (`2002:WWXX:YYZZ::/16`), and the NAT64 well-known prefix
@@ -82,7 +84,7 @@ pub struct PeerResponse {
/// sits at a prefix-length-dependent offset (RFC 6052 §2.2) — so they return /// sits at a prefix-length-dependent offset (RFC 6052 §2.2) — so they return
/// `None`. Any caller that needs them blocked must reject the wider /// `None`. Any caller that needs them blocked must reject the wider
/// `64:ff9b::/32` itself; `is_public_http_url` does this in its native-v6 arm. /// `64:ff9b::/32` itself; `is_public_http_url` does this in its native-v6 arm.
fn embedded_ipv4(v6: std::net::Ipv6Addr) -> Option<std::net::Ipv4Addr> { pub(crate) fn embedded_ipv4(v6: std::net::Ipv6Addr) -> Option<std::net::Ipv4Addr> {
if let Some(v4) = v6.to_ipv4_mapped().or_else(|| v6.to_ipv4()) { if let Some(v4) = v6.to_ipv4_mapped().or_else(|| v6.to_ipv4()) {
return Some(v4); return Some(v4);
} }
@@ -102,10 +104,120 @@ fn embedded_ipv4(v6: std::net::Ipv6Addr) -> Option<std::net::Ipv4Addr> {
None None
} }
/// Whether an IP literal is outside the local/private ranges that an
/// attacker-controlled outbound URL must never reach. Kept separate from URL
/// parsing so callers that resolve a DNS name themselves can apply the same
/// classification to every answer and then pin the approved addresses.
pub(crate) fn is_public_ip(ip: std::net::IpAddr) -> bool {
// Reject loopback/unspecified on the literal as given — catches `::1` and
// `::` before the IPv4-folding below (`::1`.to_ipv4() would otherwise map
// to a non-loopback `0.0.0.1`).
if ip.is_loopback() || ip.is_unspecified() {
return false;
}
// Fold any IPv6 literal that embeds an IPv4 address (mapped, compatible,
// 6to4, NAT64) down to that IPv4 so the v4 range checks catch
// loopback/private addresses smuggled in via an IPv6 encoding, then
// re-check loopback/unspecified.
let ip = match ip {
std::net::IpAddr::V6(v6) => embedded_ipv4(v6).map(std::net::IpAddr::V4).unwrap_or(ip),
v4 => v4,
};
if ip.is_loopback() || ip.is_unspecified() {
return false;
}
match ip {
std::net::IpAddr::V4(v4) => {
let o = v4.octets();
// RFC1918 private, link-local, CGNAT (100.64.0.0/10), or the
// RFC1122 "this host" block 0.0.0.0/8 (never a valid destination;
// 0.0.0.0 itself is already caught by the is_unspecified check).
if v4.is_private()
|| v4.is_link_local()
|| (o[0] == 100 && (o[1] & 0xc0) == 64)
|| o[0] == 0
{
return false;
}
}
std::net::IpAddr::V6(v6) => {
let s = v6.segments();
// fc00::/7 (unique-local), fe80::/10 (link-local), or fec0::/10
// (site-local, deprecated by RFC 3879; the link-local mask does
// not cover it).
if (s[0] & 0xfe00) == 0xfc00 || (s[0] & 0xffc0) == 0xfe80 || (s[0] & 0xffc0) == 0xfec0 {
return false;
}
// Any NAT64 address (64:ff9b::/32) that is not the cleanly
// decodable well-known /96 — e.g. the RFC 8215 local-use
// 64:ff9b:1::/48 — carries a translated target whose embedded v4
// sits at a prefix-length-dependent offset (RFC 6052 §2.2).
// Rather than risk a wrong decode across every prefix length we
// reject the whole NAT64 space here. The well-known /96 was already
// folded to its v4 above and never reaches this arm.
if s[0] == 0x0064 && s[1] == 0xff9b {
return false;
}
}
}
true
}
/// Mirror-upstream variant of [`is_public_ip`]: in addition to the shared
/// peer classification, rejects IANA special-purpose ranges that are
/// non-global, locally meaningful, or unsafe as owner-controlled outbound
/// fetch endpoints. For IP-literal upstreams, configure-time URL validation
/// and the fetch-time egress policy share this classifier, so a stored
/// literal can never fail the worker it is scheduled for. Hostname upstreams
/// are only shape-checked at configure time: each fetch resolves DNS and
/// applies this classifier to the answers, so a hostname whose answers turn
/// non-public still fails at fetch time.
pub(crate) fn mirror_upstream_ip_permitted(ip: std::net::IpAddr) -> bool {
if !is_public_ip(ip) {
return false;
}
let classified = match ip {
std::net::IpAddr::V6(v6) => embedded_ipv4(v6).map(std::net::IpAddr::V4).unwrap_or(ip),
_ => ip,
};
match classified {
std::net::IpAddr::V4(v4) => {
let [a, b, c, _] = v4.octets();
// IANA special-purpose ranges that are not globally reachable but
// may be routed inside labs, clouds, or operator networks. They are
// unsuitable for an owner-controlled outbound fetch even though
// `Ipv4Addr::is_private` does not classify them as RFC1918.
!(a >= 224
|| (a == 192 && b == 0 && c == 0)
|| (a == 192 && b == 0 && c == 2)
|| (a == 192 && b == 88 && c == 99)
|| (a == 198 && matches!(b, 18 | 19))
|| (a == 198 && b == 51 && c == 100)
|| (a == 203 && b == 0 && c == 113))
}
std::net::IpAddr::V6(v6) => {
let s = v6.segments();
// IANA special-purpose ranges that are non-global, locally
// meaningful, or unsafe to treat as ordinary forge endpoints.
// Reject the complete IETF protocol-assignment /23 rather than
// trying to maintain its narrow globally reachable exceptions;
// a public forge has no reason to depend on those anycast ranges.
!(v6.is_multicast()
|| (s[0] & 0xffc0) == 0xfec0
|| (s[0] == 0x0100 && s[1] == 0 && s[2] == 0 && s[3] == 0)
|| (s[0] == 0x0100 && s[1] == 0 && s[2] == 0 && s[3] == 1)
|| (s[0] == 0x2001 && (s[1] & 0xfe00) == 0)
|| (s[0] == 0x2001 && s[1] == 0x0db8)
|| (s[0] == 0x3fff && (s[1] & 0xf000) == 0)
|| s[0] == 0x5f00)
}
}
}
/// Whether a peer `http_url` is a public http(s) endpoint safe to register. /// Whether a peer `http_url` is a public http(s) endpoint safe to register.
/// Rejects non-http(s) schemes, loopback/unspecified/private/link-local IPs, /// Rejects non-http(s) schemes, loopback/unspecified/private/link-local IPs,
/// and `localhost` / `.local` / `.internal` hostnames. Used at announce time /// and `localhost` / `.localhost` / `.local` / `.internal` hostnames. Used at
/// and by the boot-time prune of already-poisoned rows. /// announce time and by the boot-time prune of already-poisoned rows.
pub fn is_public_http_url(raw: &str) -> bool { pub fn is_public_http_url(raw: &str) -> bool {
let url = match reqwest::Url::parse(raw) { let url = match reqwest::Url::parse(raw) {
Ok(u) => u, Ok(u) => u,
@@ -125,6 +237,7 @@ pub fn is_public_http_url(raw: &str) -> bool {
host.truncate(host.trim_end_matches('.').len()); host.truncate(host.trim_end_matches('.').len());
if host.is_empty() if host.is_empty()
|| host == "localhost" || host == "localhost"
|| host.ends_with(".localhost")
|| host.ends_with(".local") || host.ends_with(".local")
|| host.ends_with(".internal") || host.ends_with(".internal")
{ {
@@ -134,60 +247,9 @@ pub fn is_public_http_url(raw: &str) -> bool {
// before parsing as an IP. // before parsing as an IP.
let ip_candidate = host.trim_start_matches('[').trim_end_matches(']'); let ip_candidate = host.trim_start_matches('[').trim_end_matches(']');
if let Ok(ip) = ip_candidate.parse::<std::net::IpAddr>() { if let Ok(ip) = ip_candidate.parse::<std::net::IpAddr>() {
// Reject loopback/unspecified on the literal as given — catches `::1` // IP literals share the single DNS/literal classification in
// and `::` before the IPv4-folding below (`::1`.to_ipv4() would // `is_public_ip`; hostname-only rules stay local to this function.
// otherwise map to a non-loopback `0.0.0.1`). return is_public_ip(ip);
if ip.is_loopback() || ip.is_unspecified() {
return false;
}
// Fold any IPv6 literal that embeds an IPv4 address (mapped, compatible,
// 6to4, NAT64) down to that IPv4 so the v4 range checks catch
// loopback/private addresses smuggled in via an IPv6 encoding, then
// re-check loopback/unspecified.
let ip = match ip {
std::net::IpAddr::V6(v6) => embedded_ipv4(v6).map(std::net::IpAddr::V4).unwrap_or(ip),
v4 => v4,
};
if ip.is_loopback() || ip.is_unspecified() {
return false;
}
match ip {
std::net::IpAddr::V4(v4) => {
let o = v4.octets();
// RFC1918 private, link-local, CGNAT (100.64.0.0/10), or the
// RFC1122 "this host" block 0.0.0.0/8 (never a valid destination;
// 0.0.0.0 itself is already caught by the is_unspecified check).
if v4.is_private()
|| v4.is_link_local()
|| (o[0] == 100 && (o[1] & 0xc0) == 64)
|| o[0] == 0
{
return false;
}
}
std::net::IpAddr::V6(v6) => {
let s = v6.segments();
// fc00::/7 (unique-local), fe80::/10 (link-local), or fec0::/10
// (site-local, deprecated by RFC 3879 and so never a legitimate
// peer address, but still routable on networks that kept it).
if (s[0] & 0xfe00) == 0xfc00
|| (s[0] & 0xffc0) == 0xfe80
|| (s[0] & 0xffc0) == 0xfec0
{
return false;
}
// Any NAT64 address (64:ff9b::/32) that is not the cleanly
// decodable well-known /96 — e.g. the RFC 8215 local-use
// 64:ff9b:1::/48 — carries a translated target whose embedded v4
// sits at a prefix-length-dependent offset (RFC 6052 §2.2).
// Rather than risk a wrong decode across every prefix length we
// reject the whole NAT64 space here. The well-known /96 was
// already folded to its v4 above and never reaches this arm.
if s[0] == 0x0064 && s[1] == 0xff9b {
return false;
}
}
}
} else if !host.contains('.') { } else if !host.contains('.') {
// A name with no dot at all. The `.local` / `.internal` rules above are // A name with no dot at all. The `.local` / `.internal` rules above are
// suffix checks, so a single label slips both, and the resolver decides // suffix checks, so a single label slips both, and the resolver decides
@@ -260,9 +322,9 @@ pub async fn announce(
// and turn our outbound sync-notify fan-out into an SSRF probe — and bury // and turn our outbound sync-notify fan-out into an SSRF probe — and bury
// the real peers under junk so node-origin repos stop replicating. // the real peers under junk so node-origin repos stop replicating.
if !is_public_http_url(&req.http_url) { if !is_public_http_url(&req.http_url) {
return Err(AppError::BadRequest( return Err(AppError::BadRequest(format!(
"http_url must be a public http(s) URL (no loopback, private, or .internal/.local hosts)".into(), "http_url {PUBLIC_HTTP_URL_REQUIREMENT}"
)); )));
} }
// Reject self-announcements: a peer row whose http_url is our own public // Reject self-announcements: a peer row whose http_url is our own public
@@ -377,6 +439,7 @@ pub async fn trigger_sync(State(state): State<AppState>) -> Result<Json<serde_js
"refs/heads/main", "refs/heads/main",
"0000000000000000000000000000000000000000", "0000000000000000000000000000000000000000",
None, None,
None,
) )
.await; .await;
enqueued += 1; enqueued += 1;
@@ -412,6 +475,13 @@ pub struct NotifyRequest {
pub old_sha: Option<String>, pub old_sha: Option<String>,
#[serde(default)] #[serde(default)]
pub timestamp: Option<String>, pub timestamp: Option<String>,
/// The instant the origin push landed, for sync-lag measurement only —
/// mirrors RefUpdateEvent::lag_origin on the gossip transport. `timestamp`
/// stays publish semantics to match what gossip ingest stores on the feed
/// row; when an older sender omits this field the enqueue falls back to
/// `timestamp`, which was the pre-split approximation of land time.
#[serde(default)]
pub lag_origin: Option<String>,
#[serde(default)] #[serde(default)]
pub cert_id: Option<String>, pub cert_id: Option<String>,
/// Full owner DID — added in #144 for DID-aware feed gating. /// Full owner DID — added in #144 for DID-aware feed gating.
@@ -458,7 +528,14 @@ pub async fn notify_sync(
state state
.db .db
.enqueue_sync(&req.repo, &req.node_did, &req.ref_name, &req.new_sha, None) .enqueue_sync(
&req.repo,
&req.node_did,
&req.ref_name,
&req.new_sha,
None,
req.lag_origin.as_deref().or(req.timestamp.as_deref()),
)
.await?; .await?;
// Mirror the gossipsub-receive handler: insert the same record we'd // Mirror the gossipsub-receive handler: insert the same record we'd
@@ -541,6 +618,7 @@ mod tests {
fn rejects_loopback_private_and_internal() { fn rejects_loopback_private_and_internal() {
for bad in [ for bad in [
"http://localhost:7545", "http://localhost:7545",
"http://node.localhost:7545",
"http://127.0.0.1:5432/", "http://127.0.0.1:5432/",
"http://localhost:22/", "http://localhost:22/",
"http://0.0.0.0:7545", "http://0.0.0.0:7545",
@@ -612,6 +690,8 @@ mod tests {
fn rejects_deprecated_site_local_v6() { fn rejects_deprecated_site_local_v6() {
// fec0::/10, deprecated by RFC 3879. The link-local mask (0xffc0 == // fec0::/10, deprecated by RFC 3879. The link-local mask (0xffc0 ==
// 0xfe80) does not cover it: 0xfec0 & 0xffc0 is 0xfec0. // 0xfe80) does not cover it: 0xfec0 & 0xffc0 is 0xfec0.
assert!(!super::is_public_ip("fec0::1".parse().unwrap()));
assert!(!super::is_public_ip("feff:ffff::1".parse().unwrap()));
for bad in ["http://[fec0::1]/", "http://[feff:ffff::1]:7545"] { for bad in ["http://[fec0::1]/", "http://[feff:ffff::1]:7545"] {
assert!(!is_public_http_url(bad), "{bad:?} must be rejected"); assert!(!is_public_http_url(bad), "{bad:?} must be rejected");
} }
@@ -620,11 +700,14 @@ mod tests {
#[test] #[test]
fn rejects_repeated_trailing_root_dots() { fn rejects_repeated_trailing_root_dots() {
// Stripping a single root dot leaves `localhost.`, which matches // Stripping a single root dot leaves `localhost.`, which matches
// neither the equality nor the suffix check. The subdomain form // neither the equality nor the suffix check. The subdomain form now
// (`node.localhost..`) is not asserted here: `.localhost` is not yet a // exercises the `.localhost` suffix rejection together with the extra
// rejected suffix on this base, so that case belongs to whichever // trailing dot.
// change adds it rather than to this one. for bad in [
for bad in ["http://localhost../", "http://localhost...:7545"] { "http://localhost../",
"http://localhost...:7545",
"http://node.localhost../",
] {
assert!(!is_public_http_url(bad), "{bad:?} must be rejected"); assert!(!is_public_http_url(bad), "{bad:?} must be rejected");
} }
} }
@@ -1214,6 +1297,52 @@ mod tests {
); );
} }
#[sqlx::test]
async fn sync_notify_preserves_origin_timestamp_for_sync_lag(pool: PgPool) {
let state = test_state(pool).await;
let db = state.db.clone();
let peer_did = seed_peer(&state).await;
// Distinct publish and land instants: the feed row must carry publish
// semantics while only the queue row sees lag_origin.
let timestamp = "2026-08-16T00:00:00Z";
let lag_origin = "2026-08-15T23:59:00Z";
let body = serde_json::json!({
"repo": "z6Mkfoo/timestamped",
"ref_name": "refs/heads/main",
"new_sha": "0000000000000000000000000000000000000000",
"node_did": peer_did,
"timestamp": timestamp,
"lag_origin": lag_origin,
})
.to_string();
let router = crate::server::build_router(state);
let resp = router
.oneshot(unsigned_post(
"/api/v1/sync/notify",
&body,
"198.51.100.23:5000",
))
.await
.unwrap();
assert_eq!(resp.status(), StatusCode::OK);
let queued = db.dequeue_pending_syncs(100).await.unwrap();
assert_eq!(queued.len(), 1);
assert_eq!(queued[0].origin_timestamp.as_deref(), Some(lag_origin));
let feed = db
.list_ref_updates_keyset(Some("z6Mkfoo/timestamped"), 10, None)
.await
.unwrap();
assert_eq!(
feed.iter()
.find(|u| u.repo == "z6Mkfoo/timestamped")
.map(|u| u.timestamp.as_str()),
Some(timestamp),
"the feed row must keep publish semantics, matching gossip ingest"
);
}
#[sqlx::test] #[sqlx::test]
async fn sync_notify_unknown_peer_wins_over_malformed_slug(pool: PgPool) { async fn sync_notify_unknown_peer_wins_over_malformed_slug(pool: PgPool) {
// Ordering: the slug check sits after the peer gate, so an unknown peer // Ordering: the slug check sits after the peer gate, so an unknown peer
@@ -1582,6 +1711,31 @@ mod tests {
) )
} }
#[sqlx::test]
async fn announce_rejects_localhost_subdomains_with_an_accurate_error(pool: PgPool) {
let state = test_state(pool).await;
let did = Keypair::generate().did().to_string();
let resp = announce_only(state.clone())
.oneshot(announce_as(
&did,
&announce_body(&did, "https://node.localhost:7545"),
))
.await
.unwrap();
let (status, error, message) = status_and_error(resp).await;
assert_eq!(status, StatusCode::BAD_REQUEST);
assert_eq!(error, "bad_request");
assert_eq!(
message,
"http_url must be a public http(s) URL (no loopback, private, localhost, .localhost, .internal, or .local hosts)"
);
assert!(
snapshot(&state.db, &did).await.is_none(),
"a rejected .localhost announce must leave no row behind"
);
}
/// U4 scenario 1, and the first test the keyid branch has ever had: a caller /// U4 scenario 1, and the first test the keyid branch has ever had: a caller
/// who proved control of one DID must not announce another's. Kills /// who proved control of one DID must not announce another's. Kills
/// neutralizing the handler's keyid comparison, and kills demoting this /// neutralizing the handler's keyid comparison, and kills demoting this
+4 -10
View File
@@ -17,11 +17,8 @@ pub async fn protect_branch(
Extension(auth): Extension<AuthenticatedDid>, Extension(auth): Extension<AuthenticatedDid>,
Path((owner, repo, branch)): Path<(String, String, String)>, Path((owner, repo, branch)): Path<(String, String, String)>,
) -> Result<(StatusCode, Json<serde_json::Value>)> { ) -> Result<(StatusCode, Json<serde_json::Value>)> {
let record = state let (record, _rules) =
.db crate::api::authorize_repo_read(&state, &owner, &repo, Some(auth.0.as_str()), "/").await?;
.get_repo(&owner, &repo)
.await?
.ok_or_else(|| AppError::RepoNotFound(format!("{owner}/{repo}")))?;
// Only the repo owner can protect branches (DID-safe match, shared idiom). // Only the repo owner can protect branches (DID-safe match, shared idiom).
let caller = &auth.0; let caller = &auth.0;
@@ -51,11 +48,8 @@ pub async fn unprotect_branch(
Extension(auth): Extension<AuthenticatedDid>, Extension(auth): Extension<AuthenticatedDid>,
Path((owner, repo, branch)): Path<(String, String, String)>, Path((owner, repo, branch)): Path<(String, String, String)>,
) -> Result<Json<serde_json::Value>> { ) -> Result<Json<serde_json::Value>> {
let record = state let (record, _rules) =
.db crate::api::authorize_repo_read(&state, &owner, &repo, Some(auth.0.as_str()), "/").await?;
.get_repo(&owner, &repo)
.await?
.ok_or_else(|| AppError::RepoNotFound(format!("{owner}/{repo}")))?;
let caller = &auth.0; let caller = &auth.0;
if !crate::api::did_matches(caller, &record.owner_did) { if !crate::api::did_matches(caller, &record.owner_did) {
+211 -12
View File
@@ -76,6 +76,7 @@ pub async fn create_pr(
status: "open".to_string(), status: "open".to_string(),
merged_by_did: None, merged_by_did: None,
merged_at: None, merged_at: None,
head_commit: None,
created_at: now.clone(), created_at: now.clone(),
updated_at: now, updated_at: now,
}; };
@@ -95,6 +96,7 @@ pub async fn create_pr(
webhooks::fire_event( webhooks::fire_event(
std::sync::Arc::clone(&state.db), std::sync::Arc::clone(&state.db),
std::sync::Arc::clone(&state.http_client), std::sync::Arc::clone(&state.http_client),
std::sync::Arc::clone(&state.webhook_secret_key),
&record.id, &record.id,
"pull_request.opened", "pull_request.opened",
serde_json::json!({ serde_json::json!({
@@ -199,11 +201,8 @@ pub async fn merge_pr(
Extension(auth): Extension<AuthenticatedDid>, Extension(auth): Extension<AuthenticatedDid>,
Path((owner, name, number)): Path<(String, String, i64)>, Path((owner, name, number)): Path<(String, String, i64)>,
) -> Result<Json<serde_json::Value>> { ) -> Result<Json<serde_json::Value>> {
let record = state let (record, _rules) =
.db crate::api::authorize_repo_read(&state, &owner, &name, Some(auth.0.as_str()), "/").await?;
.get_repo(&owner, &name)
.await?
.ok_or_else(|| AppError::RepoNotFound(format!("{owner}/{name}")))?;
// Owner-only merge (N7). Merging writes the served tree, so this is the same // Owner-only merge (N7). Merging writes the served tree, so this is the same
// trust boundary as owner-only push; it subsumes branch protection (a // trust boundary as owner-only push; it subsumes branch protection (a
@@ -227,6 +226,17 @@ pub async fn merge_pr(
.map_err(|e| crate::api::repos::acquire_write_app_error(&e, &name))?; .map_err(|e| crate::api::repos::acquire_write_app_error(&e, &name))?;
let disk_path = guard.path().to_path_buf(); let disk_path = guard.path().to_path_buf();
let merger_did = auth.0; let merger_did = auth.0;
// The source head this merge is about to consume, read under the write lock.
// The PR row above was loaded before the lock, so a push landing in between
// already moved the branch and the row's own `head_commit` is stale; the
// value frozen on the merged PR has to be what was merged. `None` when the
// ref cannot be resolved, which leaves whatever the push path last stored.
let merged_source_head =
store::read_ref(&disk_path, &store::local_branch_ref(&pr.source_branch))
.ok()
.flatten();
let merge_result = store::merge_branch( let merge_result = store::merge_branch(
&disk_path, &disk_path,
&pr.target_branch, &pr.target_branch,
@@ -243,12 +253,16 @@ pub async fn merge_pr(
let merge_sha = merge_result.map_err(|e| AppError::Git(e.to_string()))?; let merge_sha = merge_result.map_err(|e| AppError::Git(e.to_string()))?;
state.db.merge_pr(&pr.id, &merger_did).await?; state
.db
.merge_pr(&pr.id, &merger_did, merged_source_head.as_deref())
.await?;
let _ = state.db.touch_repo(&record.id).await; let _ = state.db.touch_repo(&record.id).await;
webhooks::fire_event( webhooks::fire_event(
std::sync::Arc::clone(&state.db), std::sync::Arc::clone(&state.db),
std::sync::Arc::clone(&state.http_client), std::sync::Arc::clone(&state.http_client),
std::sync::Arc::clone(&state.webhook_secret_key),
&record.id, &record.id,
"pull_request.merged", "pull_request.merged",
serde_json::json!({ serde_json::json!({
@@ -274,11 +288,8 @@ pub async fn close_pr(
Extension(auth): Extension<AuthenticatedDid>, Extension(auth): Extension<AuthenticatedDid>,
Path((owner, name, number)): Path<(String, String, i64)>, Path((owner, name, number)): Path<(String, String, i64)>,
) -> Result<Json<serde_json::Value>> { ) -> Result<Json<serde_json::Value>> {
let record = state let (record, _rules) =
.db crate::api::authorize_repo_read(&state, &owner, &name, Some(auth.0.as_str()), "/").await?;
.get_repo(&owner, &name)
.await?
.ok_or_else(|| AppError::RepoNotFound(format!("{owner}/{name}")))?;
let pr = state let pr = state
.db .db
@@ -295,11 +306,41 @@ pub async fn close_pr(
)); ));
} }
state.db.close_pr(&pr.id).await?; // Same refusal merge_pr gives, before the head is resolved: a repeated close
// must not store the branch's current tip on a historical pull request.
if pr.status != "open" {
return Err(AppError::BadRequest(format!("PR is already {}", pr.status)));
}
// Freeze the head this close decided against, the same snapshot merge_pr
// takes on its path: a closed pull request's rollup serves only the stored
// head, so a PR closed before its first status read would report
// head_resolved=false forever. The recorded push for the branch is the
// canonical answer; the disk read covers a push whose event write failed.
// An unresolvable head keeps whatever the push path last stored.
let branch_ref = store::local_branch_ref(&pr.source_branch);
let closed_head = if pr.head_commit.is_some() {
None
} else {
state
.db
.latest_push_sha_for_ref(&record.id, &branch_ref)
.await
.ok()
.flatten()
.or_else(|| {
store::read_ref(std::path::Path::new(&record.disk_path), &branch_ref)
.ok()
.flatten()
})
.map(|sha| sha.to_ascii_lowercase())
};
state.db.close_pr(&pr.id, closed_head.as_deref()).await?;
webhooks::fire_event( webhooks::fire_event(
std::sync::Arc::clone(&state.db), std::sync::Arc::clone(&state.db),
std::sync::Arc::clone(&state.http_client), std::sync::Arc::clone(&state.http_client),
std::sync::Arc::clone(&state.webhook_secret_key),
&record.id, &record.id,
"pull_request.closed", "pull_request.closed",
serde_json::json!({ serde_json::json!({
@@ -350,6 +391,7 @@ pub async fn create_review(
webhooks::fire_event( webhooks::fire_event(
std::sync::Arc::clone(&state.db), std::sync::Arc::clone(&state.db),
std::sync::Arc::clone(&state.http_client), std::sync::Arc::clone(&state.http_client),
std::sync::Arc::clone(&state.webhook_secret_key),
&record.id, &record.id,
"pull_request.reviewed", "pull_request.reviewed",
serde_json::json!({ serde_json::json!({
@@ -439,6 +481,162 @@ pub async fn list_comments(
Ok(Json(serde_json::json!({ "comments": comments }))) Ok(Json(serde_json::json!({ "comments": comments })))
} }
#[cfg(test)]
mod tests {
use super::*;
use axum::body::Body;
use axum::http::{Method, StatusCode};
use axum::routing::post;
use axum::Router;
use std::process::Command;
use tower::ServiceExt;
const OWNER_DID: &str = "did:key:z6MkpTHR8VNsBxYAAWHut2Geadd9jSwuBV8xRoAnwWsdvktH";
const STALE_SHA: &str = "9999999999999999999999999999999999999999";
fn git(dir: &std::path::Path, args: &[&str]) {
let out = Command::new("git")
.args(args)
.current_dir(dir)
.env("GIT_AUTHOR_NAME", "t")
.env("GIT_AUTHOR_EMAIL", "[email protected]")
.env("GIT_COMMITTER_NAME", "t")
.env("GIT_COMMITTER_EMAIL", "[email protected]")
.output()
.expect("git runs");
assert!(
out.status.success(),
"git {args:?} failed: {}",
String::from_utf8_lossy(&out.stderr)
);
}
/// A bare repo with `main` and a `feature` branch one commit ahead.
/// Returns the bare path and the real `feature` tip.
fn bare_repo_with_feature(
root: &std::path::Path,
owner_slug: &str,
) -> (std::path::PathBuf, String) {
let work = root.join("work");
std::fs::create_dir_all(&work).unwrap();
git(&work, &["init", "-q", "-b", "main", "."]);
std::fs::write(work.join("a.txt"), "one").unwrap();
git(&work, &["add", "."]);
git(&work, &["commit", "-qm", "one"]);
git(&work, &["checkout", "-qb", "feature"]);
std::fs::write(work.join("b.txt"), "two").unwrap();
git(&work, &["add", "."]);
git(&work, &["commit", "-qm", "two"]);
git(&work, &["checkout", "-q", "main"]);
let bare_dir = root.join(owner_slug);
std::fs::create_dir_all(&bare_dir).unwrap();
let bare = bare_dir.join("demo.git");
git(
root,
&[
"clone",
"-q",
"--bare",
work.to_str().unwrap(),
bare.to_str().unwrap(),
],
);
let tip = Command::new("git")
.args(["rev-parse", "refs/heads/feature"])
.current_dir(&bare)
.output()
.unwrap();
let tip = String::from_utf8_lossy(&tip.stdout).trim().to_string();
assert_eq!(tip.len(), 40, "feature tip must resolve");
(bare, tip)
}
/// Merging freezes the stored head at the source commit the merge actually
/// consumed. The PR row is loaded before the write lock is taken, so a push
/// that lands in between leaves the row's own `head_commit` stale; seeding a
/// value that matches nothing on disk stands in for that race, and the
/// merged PR must still come out pointing at the real `feature` tip.
#[sqlx::test]
async fn merge_stamps_the_source_head_it_merged_over_a_racing_value(pool: sqlx::PgPool) {
let tmp = tempfile::tempdir().unwrap();
let owner_slug = OWNER_DID.replace([':', '/'], "_");
let (bare, feature_tip) = bare_repo_with_feature(tmp.path(), &owner_slug);
let mut state = crate::test_support::test_state(pool.clone()).await;
state.repo_store =
crate::git::repo_store::RepoStore::for_testing(tmp.path().to_path_buf(), pool);
let now = Utc::now();
let repo = crate::db::RepoRecord {
id: uuid::Uuid::new_v4().to_string(),
name: "demo".into(),
owner_did: OWNER_DID.into(),
description: None,
is_public: true,
default_branch: "main".into(),
created_at: now,
updated_at: now,
disk_path: bare.to_string_lossy().into_owned(),
forked_from: None,
machine_id: None,
};
state.db.create_repo(&repo).await.unwrap();
let pr = PullRequest {
id: Uuid::new_v4().to_string(),
repo_id: repo.id.clone(),
number: 1,
title: "add b".into(),
body: None,
author_did: OWNER_DID.into(),
source_branch: "feature".into(),
target_branch: "main".into(),
status: "open".into(),
merged_by_did: None,
merged_at: None,
head_commit: None,
created_at: now.to_rfc3339(),
updated_at: now.to_rfc3339(),
};
state.db.create_pr(&pr).await.unwrap();
// The stale value the racing push scenario leaves on the row.
state
.db
.set_open_pr_heads(&repo.id, "feature", STALE_SHA)
.await
.unwrap();
let db = state.db.clone();
let app = Router::new()
.route(
"/api/v1/repos/{owner}/{repo}/pulls/{number}/merge",
post(merge_pr),
)
.with_state(state);
let response = app
.oneshot(crate::test_support::signed_request_as(
OWNER_DID,
Method::POST,
"/api/v1/repos/z6MkpTHR8VNsBxYAAWHut2Geadd9jSwuBV8xRoAnwWsdvktH/demo/pulls/1/merge",
Body::empty(),
))
.await
.unwrap();
assert_eq!(response.status(), StatusCode::OK);
let merged = db.get_pr(&repo.id, 1).await.unwrap().unwrap();
assert_eq!(merged.status, "merged");
assert_eq!(
merged.head_commit,
Some(feature_tip),
"the merge must stamp the source head it consumed, not the racing value"
);
}
}
/// #173 F1 follow-up: merge holds no admission permit, so an exhausted write-lock /// #173 F1 follow-up: merge holds no admission permit, so an exhausted write-lock
/// pool must shed 503 + Retry-After. /// pool must shed 503 + Retry-After.
#[cfg(test)] #[cfg(test)]
@@ -482,6 +680,7 @@ mod lock_pool_shed_tests {
state state
.db .db
.create_pr(&PullRequest { .create_pr(&PullRequest {
head_commit: None,
id: Uuid::new_v4().to_string(), id: Uuid::new_v4().to_string(),
repo_id: repo_id.clone(), repo_id: repo_id.clone(),
number: 1, number: 1,
File diff suppressed because it is too large. Load diff
+665
View File
@@ -0,0 +1,665 @@
//! Commit status claims: the write path and the combined read.
//!
//! POST /api/v1/repos/:owner/:repo/statuses/:sha — append one claim (owner only)
//! GET /api/v1/repos/:owner/:repo/commits/:sha/status — the combined projection
//!
//! Claims are append-only: a producer reporting twice for the same context
//! leaves both rows and the visible status is a projection over the history.
//! Reporting twice means two REQUESTS, though. An exact repeat of one already
//! accepted is answered with 200 and the row it wrote, not a second row: the
//! projection elects the highest `seq`, so a replayed claim would not duplicate
//! a verdict but overturn the one that superseded it.
use axum::extract::{Extension, Path, State};
use axum::http::StatusCode;
use axum::Json;
use chrono::Utc;
use serde::{Deserialize, Serialize};
use uuid::Uuid;
use crate::auth::AuthenticatedDid;
use crate::db::StatusClaim;
use crate::error::{AppError, Result};
use crate::state::AppState;
/// Claims for one (repo, commit, producer, context). Generous for a real
/// producer; a producer past it is misbehaving.
const MAX_CLAIMS_PER_TUPLE: i64 = 100;
/// Distinct contexts for one (repo, commit). The context string is caller-chosen
/// and free-form, so the tuple cap alone bounds nothing.
const MAX_CONTEXTS_PER_COMMIT: i64 = 50;
/// Claim rows for one repo within the trailing window the db layer counts over.
/// The commit SHA is caller-chosen and never existence-checked, so a writer at
/// the two caps above can still fan out over fresh 40-hex strings without this
/// one. It bounds the RATE of that fan-out rather than a lifetime total, because
/// nothing prunes the table: a lifetime bound would close the surface on a repo
/// permanently while answering with a status every client retries.
const MAX_CLAIMS_PER_REPO_PER_WINDOW: i64 = 10_000;
#[derive(Deserialize)]
#[serde(deny_unknown_fields)]
pub struct CreateStatusRequest {
/// Payload format version. The signature covers these bytes, so the
/// field set is the signed format: `deny_unknown_fields` keeps a future
/// producer's claim from silently reading as this one, and `v` names the
/// version explicitly so a later revision has somewhere to live. Absent
/// means 1.
pub v: Option<u32>,
pub state: String,
pub context: String,
pub target_url: Option<String>,
pub description: Option<String>,
}
/// The wire state set (KTD-1): GitHub's four commit-status states, so absence
/// stays distinguishable without a fifth value no client understands.
const CLAIM_STATES: [&str; 4] = ["error", "failure", "pending", "success"];
/// Bounds on the signature material this write path persists verbatim. Every one
/// of the four is caller-influenced: the headers come off the request, the
/// signing string grows with the component list the caller's Signature-Input
/// chose, and the body is the caller's. Without a bound the claim log's row size
/// is set by whoever writes to it, so these are explicit and generous rather than
/// implicit and absent. A conforming `sign_request` produces roughly 100, 200 and
/// 400 bytes for the first three.
const MAX_SIGNATURE_CHARS: usize = 512;
const MAX_SIGNATURE_INPUT_CHARS: usize = 1024;
const MAX_SIGNING_STRING_CHARS: usize = 4096;
/// The body is a `CreateStatusRequest`, whose own fields are already capped at
/// roughly 3.3 KB in total by the three limits below; this leaves room for JSON
/// framing and nothing more.
///
/// Enforced twice, at two different moments, because the handler's check alone
/// is not the bound it advertises: `require_signature` buffers the whole body
/// before any handler runs, so by the time [`create_status`] measures it the
/// allocation has already happened. `build_router` applies the same number as a
/// `RequestBodyLimitLayer` at the transport, outside the auth layers, which is
/// what makes it a bound on what is ALLOCATED and not only on what is STORED.
/// The handler keeps its own check regardless — it is the one that holds for a
/// handler reached by any other route table, and it is what the material size
/// tests drive.
pub(crate) const MAX_REQUEST_BODY_BYTES: usize = 8192;
const MAX_CONTEXT_CHARS: usize = 255;
const MAX_TARGET_URL_CHARS: usize = 2048;
const MAX_DESCRIPTION_CHARS: usize = 1024;
/// The 201 body for an accepted claim: what the client needs to identify the row
/// it just wrote, and nothing else.
///
/// Deliberately not the stored [`StatusClaim`]. That struct carries the signature
/// material, and serializing it echoed the signature, the signing string and the
/// whole request body back on every write — the body as a JSON array of integers.
/// Write-time provenance belongs in the row, not on the wire, which is the same
/// line [`StatusEntry`] draws on the read side.
#[derive(Serialize)]
pub struct CreatedStatus {
pub id: String,
pub repo_id: String,
pub commit_sha: String,
pub state: String,
pub context: String,
pub target_url: Option<String>,
pub description: Option<String>,
pub producer_did: String,
pub created_at: String,
}
impl CreatedStatus {
fn from_claim(claim: StatusClaim) -> Self {
Self {
id: claim.id,
repo_id: claim.repo_id,
commit_sha: claim.commit_sha,
state: claim.state,
context: claim.context,
target_url: claim.target_url,
description: claim.description,
producer_did: claim.producer_did,
created_at: claim.created_at,
}
}
}
/// POST /api/v1/repos/:owner/:repo/statuses/:sha
pub async fn create_status(
State(state): State<AppState>,
Extension(auth): Extension<AuthenticatedDid>,
Path((owner, name, sha)): Path<(String, String, String)>,
material: Option<Extension<crate::auth::SignatureMaterial>>,
Json(req): Json<CreateStatusRequest>,
) -> Result<(StatusCode, Json<CreatedStatus>)> {
// Read-visibility first, then owner, and the order is the security property.
// authorize_repo_read denies a quarantined repo before the visibility gate and
// answers with the repo's own not-found, byte-identical to a missing repo, so
// a caller who cannot read the repo cannot learn it exists. Loading the repo
// and comparing the owner would answer 403 there, turning this endpoint into
// an existence oracle. require_repo_owner then 403s a non-owner of a repo the
// caller can read, where existence is not secret.
let (record, _rules) =
crate::api::authorize_repo_read(&state, &owner, &name, Some(&auth.0), "/").await?;
crate::api::require_repo_owner(&record, &auth.0)?;
let commit_sha = normalize_sha(&sha)?;
if req.v.is_some_and(|v| v != 1) {
return Err(AppError::BadRequest(
"unsupported status payload version".into(),
));
}
let claim_state = validate_state(&req.state)?;
let context = validate_context(&req.context)?;
let target_url = validate_target_url(req.target_url.as_deref())?;
let description = validate_description(req.description.as_deref())?;
// KTD-5: the signature material is captured here because the request carries
// the only copy, and `signing_string` is the exact byte sequence the
// middleware verified. Its absence means this handler was reached without
// `require_signature` — a server misconfiguration (a route group that lost
// its auth layer, a handler mounted somewhere new), not anything the client
// did, so it is a 500 and no row is written. Storing an empty payload instead
// would leave every claim from that moment on unverifiable with nothing going
// red, which is exactly the absence-renders-as-success shape.
let Some(Extension(material)) = material else {
return Err(AppError::Internal(anyhow::anyhow!(
"status claim write reached without verified signature material — \
the route is not behind require_signature"
)));
};
// The body is carried only behind the `PersistsSignedBody` marker, which
// this route's group applies outside the auth layers. Its absence means the
// marker did not reach the middleware, because the layer was dropped or
// reordered. That is the same class of misconfiguration as the branch above
// and gets the same answer. Storing an empty body instead would write claims
// that carry a signature over a digest of bytes nobody kept, which is
// exactly the case `stored_claim_re_verifies_and_a_tampered_row_does_not`
// exists to make impossible.
let Some(body) = material.body.clone() else {
return Err(AppError::Internal(anyhow::anyhow!(
"status claim write reached with signature material carrying no request body: \
the route lost its persist-body marker or applied it inside the auth layers"
)));
};
// Bounded before anything is written. The material is verified, which says
// the caller holds the key, not that what they signed is a reasonable size.
bound("signature", material.signature.len(), MAX_SIGNATURE_CHARS)?;
bound(
"signature_input",
material.signature_input.len(),
MAX_SIGNATURE_INPUT_CHARS,
)?;
bound(
"signing_string",
material.signing_string.len(),
MAX_SIGNING_STRING_CHARS,
)?;
bound("request body", body.len(), MAX_REQUEST_BODY_BYTES)?;
// After the bounds, so the scan below walks a signing string of known size.
require_body_bound_signature(&material, &body)?;
let digest = request_digest(&material, &body);
let (signature, signature_input, signing_string, request_body) = (
material.signature,
material.signature_input,
material.signing_string,
body.to_vec(),
);
let claim = StatusClaim {
id: Uuid::new_v4().to_string(),
// Ignored on insert: the database assigns the ordering key (KTD-3).
seq: 0,
repo_id: record.id.clone(),
commit_sha,
state: claim_state,
context,
target_url,
description,
// Both are the owner identity today (KTD-5); they split when delegated
// capabilities land.
producer_did: auth.0.clone(),
authorizing_did: auth.0,
signature,
signature_input,
signing_string,
request_body,
request_digest: digest,
created_at: Utc::now().to_rfc3339(),
};
let caps = crate::db::ClaimCaps {
per_tuple: MAX_CLAIMS_PER_TUPLE,
contexts_per_commit: MAX_CONTEXTS_PER_COMMIT,
per_repo_window: MAX_CLAIMS_PER_REPO_PER_WINDOW,
};
match state.db.insert_status_claim_capped(&claim, &caps).await? {
crate::db::ClaimInsert::Inserted(_) => {
Ok((StatusCode::CREATED, Json(CreatedStatus::from_claim(claim))))
}
// Already recorded, so 200 and the original row rather than 201 and a
// second one. The claim the caller gets back is the stored one, id
// included, which is what makes a retry safe to treat as success.
crate::db::ClaimInsert::AlreadyRecorded(existing) => {
Ok((StatusCode::OK, Json(CreatedStatus::from_claim(*existing))))
}
crate::db::ClaimInsert::CapExceeded(which) => Err(AppError::TooManyRequests(format!(
"claim limit reached for {which}"
))),
}
}
/// Refuse a claim whose signature does not cover the body about to be stored.
///
/// A claim IS provenance, so the signature stored beside it has to bind the
/// bytes stored beside it. It does not do that directly: `signing_string`
/// reaches the body only through a `content-digest` line. And a caller can sign
/// a content-digest the request then never carries. `require_signature` rebuilds
/// the covered values from the request, so an ABSENT `Content-Digest` header
/// makes the covered digest the empty string on the signing side and the
/// verifying side alike; the Ed25519 check passes over method, path and an empty
/// digest, and any body at all rides along. The row that write produces asserts
/// a provenance that cannot be re-verified: the signing string covers a digest
/// of bytes that are not the ones in `request_body`.
///
/// The predicate here is the write-time form of the one `re_verify` runs against
/// a stored row in the tests — recompute the digest of the bytes about to be
/// persisted and require the signing string to carry exactly that line — so what
/// this route accepts and what a third party can later re-verify are the same
/// question, asked at both ends.
///
/// This is a route property, checked where the claim becomes history, and it is
/// deliberately not the only line of defence: requiring the header in
/// `require_signature` closes the same hole one layer earlier and for every
/// signed route, and that change belongs there rather than here because no other
/// route's correctness should depend on the status handler. The two agree and
/// neither substitutes for the other — without the middleware's requirement the
/// empty digest is still accepted everywhere else, and without this one a later
/// relaxation of the middleware would silently resume writing unverifiable
/// history.
///
/// A 400, not a 500: the material is exactly what the client signed, and what
/// they signed does not cover what they sent.
fn require_body_bound_signature(
material: &crate::auth::SignatureMaterial,
body: &[u8],
) -> Result<()> {
let covered = format!(
"\"content-digest\": {}",
gitlawb_core::http_sig::compute_content_digest(body)
);
// Line-exact rather than a substring search: `build_signing_string` emits one
// component per line, and a `contains` would also be satisfied by the digest
// appearing INSIDE some other covered component's value, which proves
// nothing about what the signature covered.
if material.signing_string.lines().any(|line| line == covered) {
return Ok(());
}
Err(AppError::BadRequest(
"the signature does not cover this request body: send a Content-Digest header \
computed over the body and name \"content-digest\" among the signature's covered \
components"
.into(),
))
}
/// The identity of one signed write, as a hex sha-256.
///
/// All four inputs together, because a replay is the same bytes arriving twice:
/// the signature, the input that names what it covers, the canonical string it
/// was verified over, and the body that string covers through a content-digest.
/// The signature alone would nearly do (it covers the other three transitively),
/// but hashing what is actually stored means the digest describes the row rather
/// than a claim about it.
///
/// Each field is length-prefixed so no two different requests can concatenate to
/// the same bytes: without it, a signature ending in some prefix of the next
/// field would collide with the shorter signature that absorbed it.
///
/// This is not a nonce, and deliberately so. A nonce table needs pruning and a
/// pruning window is a second replay window; the claim row IS the record of what
/// was accepted, so uniqueness on it is self-maintaining.
fn request_digest(material: &crate::auth::SignatureMaterial, body: &[u8]) -> String {
use sha2::{Digest, Sha256};
let mut h = Sha256::new();
for part in [
material.signature.as_bytes(),
material.signature_input.as_bytes(),
material.signing_string.as_bytes(),
body,
] {
h.update((part.len() as u64).to_be_bytes());
h.update(part);
}
format!("{:x}", h.finalize())
}
/// One reported context in the combined response. Signature material is
/// deliberately absent: it is write-time provenance, not read-surface data.
#[derive(Serialize)]
pub struct StatusEntry {
pub state: String,
pub context: String,
pub target_url: Option<String>,
pub description: Option<String>,
pub producer_did: String,
pub created_at: String,
}
/// The combined commit status. `state` never leaves the four-value set (KTD-1),
/// so absence is carried by `total_count` 0 with the pending state rather than a
/// fifth value, and `reported_only` (R19) says out loud that the state covers the
/// contexts that reported, not every check a caller expected.
#[derive(Serialize)]
pub struct CombinedStatus {
pub state: String,
pub sha: String,
pub total_count: usize,
pub statuses: Vec<StatusEntry>,
pub reported_only: bool,
}
/// GET /api/v1/repos/:owner/:repo/commits/:sha/status
///
/// The auth extension is optional and MUST be last in the extractor list: the
/// route group carries `optional_signature`, so an unsigned caller reaches here
/// with no `AuthenticatedDid` at all and a public repo still answers.
pub async fn commit_status(
State(state): State<AppState>,
Path((owner, name, sha)): Path<(String, String, String)>,
auth: Option<Extension<AuthenticatedDid>>,
) -> Result<Json<CombinedStatus>> {
let caller = auth.as_ref().map(|Extension(a)| a.0.as_str());
// The gate runs first and on the requested path, before any claim data is
// touched. Its deny is the repo's own not-found, byte-identical to a missing
// repo, so the status surface cannot answer "does this repo exist".
let (record, _rules) =
crate::api::authorize_repo_read(&state, &owner, &name, caller, "/").await?;
let commit_sha = normalize_sha(&sha)?;
let statuses = project_claims(&state, &record, &commit_sha).await?;
Ok(Json(CombinedStatus {
state: combined_state(&statuses).to_string(),
sha: commit_sha,
total_count: statuses.len(),
statuses,
reported_only: true,
}))
}
/// The projection, in one place. Both read surfaces call it, so R3's "the rollup
/// is derived by the same projection as the commit read" holds by construction
/// rather than by two implementations happening to agree.
///
/// KTD-5: current authorization, not write-time authorization. An ownership
/// transfer drops the prior owner's claims from the projection while the
/// append-only history keeps them.
async fn project_claims(
state: &AppState,
record: &crate::db::RepoRecord,
commit_sha: &str,
) -> Result<Vec<StatusEntry>> {
let claims = state
.db
.latest_status_claims(&record.id, commit_sha, &record.owner_did)
.await?;
Ok(claims
.into_iter()
.map(|c| StatusEntry {
state: c.state,
context: c.context,
target_url: c.target_url,
description: c.description,
producer_did: c.producer_did,
created_at: c.created_at,
})
.collect())
}
/// The pull request head rollup (R11). `state` stays inside the four wire values
/// whatever happened to the head (KTD-1): an unresolvable head is carried by
/// `head_resolved`, alongside the pull request's own state, so a client can tell
/// "the head could not be resolved" from "the head resolved and nothing reported"
/// without a fifth state value no client understands.
#[derive(Serialize)]
pub struct PullRequestStatus {
pub number: i64,
/// The pull request's own state: open, closed, or merged.
pub pull_request_state: String,
pub head_resolved: bool,
/// The target commit, absent exactly when `head_resolved` is false.
pub sha: Option<String>,
pub state: String,
pub total_count: usize,
pub statuses: Vec<StatusEntry>,
pub reported_only: bool,
}
/// GET /api/v1/repos/:owner/:repo/pulls/:number/status
///
/// Same optional-auth shape as the commit read, and the auth extension MUST stay
/// last in the extractor list.
pub async fn pull_request_status(
State(state): State<AppState>,
Path((owner, name, number)): Path<(String, String, i64)>,
auth: Option<Extension<AuthenticatedDid>>,
) -> Result<Json<PullRequestStatus>> {
let caller = auth.as_ref().map(|Extension(a)| a.0.as_str());
// The gate runs first, on the requested path, before the pull request row is
// loaded. Its deny is the repo's own not-found, byte-identical to a missing
// repo. Once it passes, a missing pull request number is the plain not-found:
// existence is no longer secret.
let (record, _rules) =
crate::api::authorize_repo_read(&state, &owner, &name, caller, "/").await?;
let pr = state
.db
.get_pr(&record.id, number)
.await?
.ok_or_else(|| AppError::NotFound(format!("PR #{number} not found")))?;
let head = rollup_head(&state, &record, &pr).await?;
let statuses = match &head {
Some(sha) => project_claims(&state, &record, sha).await?,
None => Vec::new(),
};
Ok(Json(PullRequestStatus {
number: pr.number,
pull_request_state: pr.status,
head_resolved: head.is_some(),
sha: head,
state: combined_state(&statuses).to_string(),
total_count: statuses.len(),
statuses,
reported_only: true,
}))
}
/// Branch-head lookups performed by the rollup fallback, counted so a test can
/// assert WORK DONE rather than only the answer returned. A fallback that
/// re-resolved on every read and then discarded the answer would be invisible in
/// the response and visible here.
#[cfg(test)]
pub(crate) static BRANCH_RESOLVES: std::sync::atomic::AtomicUsize =
std::sync::atomic::AtomicUsize::new(0);
/// The rollup's target commit (KTD-4).
///
/// The stored head wins whenever it is set. It is frozen at close or merge, so a
/// closed or merged pull request keeps the commit the decision was made against,
/// and only an OPEN pull request with no stored head falls back to the branch.
///
/// The fallback is one database read: the latest push this node recorded for the
/// pull request's source branch, from `repo_push_events`. It deliberately does
/// NOT go through `git::store` ref listing, which needs an acquired repository
/// path — on a cold node that downloads the whole repository from object storage,
/// and this read group carries no rate limiter, so an anonymous caller could
/// drive repeated downloads with a URL.
///
/// `repo_push_events` is the source rather than `branch_cids` because
/// `record_push_events` writes it for every ref update unconditionally, whereas
/// the sole writer of `branch_cids` sits behind a pin CID. A node with no object
/// pinning configured never writes that table, so a resolve keyed on it could
/// never answer there and every open pull request without a stored head reported
/// `head_resolved: false` forever. The residual limit is that only pushes taken
/// after this shipped have rows, so a branch last pushed before then still does
/// not resolve.
///
/// The resolved head is persisted, which makes an unauthenticated GET write. That
/// is only acceptable because the write is self-limiting: it fires exactly when
/// `head_commit` is absent and it fills it, and the fill is conditioned on that
/// same absence in SQL, so the PERSIST happens at most once per pull request.
/// The resolve attempt itself is not once-only — a pull request whose branch does
/// not resolve stores nothing, so there is nothing to short-circuit on and the
/// lookup runs again on every read until it succeeds.
async fn rollup_head(
state: &AppState,
record: &crate::db::RepoRecord,
pr: &crate::db::PullRequest,
) -> Result<Option<String>> {
if let Some(stored) = pr.head_commit.clone() {
return Ok(Some(stored));
}
if pr.status != "open" {
return Ok(None);
}
// The push path records the full ref while a pull request stores the bare
// branch name, the same mismatch `crate::api::repos::branch_from_ref`
// reconciles on the write side; this is that mapping run backwards. The empty
// branch is excluded for the same reason it is there: `refs/heads/` is not a
// branch and nothing can have written it.
if pr.source_branch.is_empty() {
return Ok(None);
}
let ref_name = crate::git::store::local_branch_ref(&pr.source_branch);
#[cfg(test)]
BRANCH_RESOLVES.fetch_add(1, std::sync::atomic::Ordering::SeqCst);
let resolved = state
.db
.latest_push_sha_for_ref(&record.id, &ref_name)
.await?
// A recorded head that is not a commit SHA is no target at all, and
// storing it would leave the rollup pointing at nothing forever.
.and_then(|sha| normalize_sha(&sha).ok());
let Some(sha) = resolved else {
return Ok(None);
};
// Best effort, and deliberately not `?`. The head is already resolved and the
// response does not depend on this write landing, so a transient database
// failure here must not turn a served read into a 500; the next read resolves
// again and re-attempts the fill. Same catch-and-log the sibling writes on the
// push path use (`update_open_pr_heads`, `record_push_events`).
if let Err(e) = state.db.set_pr_head_if_absent(&pr.id, &sha).await {
tracing::warn!(
err = %e,
pr_id = %pr.id,
"failed to persist a resolved pull request head; serving the resolved head anyway"
);
}
Ok(Some(sha))
}
/// KTD-1's fold: any error or failure yields failure, else any pending yields
/// pending, else success. An empty set is pending, never success — total absence
/// of a verdict is its own state and must not read as a pass (R10).
fn combined_state(statuses: &[StatusEntry]) -> &'static str {
if statuses.is_empty() {
return "pending";
}
if statuses
.iter()
.any(|s| s.state == "error" || s.state == "failure")
{
"failure"
} else if statuses.iter().any(|s| s.state == "pending") {
"pending"
} else if statuses.iter().all(|s| s.state == "success") {
"success"
} else {
// A stored state outside the vocabulary is not a pass: an
// unrecognized verdict folds toward failure, never silently to success.
"failure"
}
}
/// 40 hex characters, lowercased. The SHA is never existence-checked (KTD-7):
/// a claim describes a commit object, and the object may not have arrived yet.
fn normalize_sha(sha: &str) -> Result<String> {
if crate::api::repos::is_hex_sha(sha) {
Ok(sha.to_ascii_lowercase())
} else {
Err(AppError::BadRequest(
"commit sha must be exactly 40 hexadecimal characters".into(),
))
}
}
/// One size bound on persisted signature material, measured in bytes.
fn bound(what: &str, len: usize, max: usize) -> Result<()> {
if len > max {
return Err(AppError::BadRequest(format!(
"{what} must be at most {max} bytes"
)));
}
Ok(())
}
fn validate_state(state: &str) -> Result<String> {
if CLAIM_STATES.contains(&state) {
Ok(state.to_string())
} else {
Err(AppError::BadRequest(format!(
"state must be one of {}",
CLAIM_STATES.join(", ")
)))
}
}
/// The context is the projection key, so it is trimmed, bounded, and required to
/// be free of control characters.
fn validate_context(context: &str) -> Result<String> {
let trimmed = context.trim();
if trimmed.is_empty() || trimmed.chars().count() > MAX_CONTEXT_CHARS {
return Err(AppError::BadRequest(format!(
"context must be 1 to {MAX_CONTEXT_CHARS} characters"
)));
}
if trimmed.chars().any(char::is_control) {
return Err(AppError::BadRequest(
"context must not contain control characters".into(),
));
}
Ok(trimmed.to_string())
}
fn validate_target_url(url: Option<&str>) -> Result<Option<String>> {
let Some(url) = url else { return Ok(None) };
if url.chars().count() > MAX_TARGET_URL_CHARS {
return Err(AppError::BadRequest(format!(
"target_url must be at most {MAX_TARGET_URL_CHARS} characters"
)));
}
let lower = url.to_ascii_lowercase();
if lower.starts_with("http://") || lower.starts_with("https://") {
Ok(Some(url.to_string()))
} else {
Err(AppError::BadRequest(
"target_url must be an http or https URL".into(),
))
}
}
fn validate_description(description: Option<&str>) -> Result<Option<String>> {
match description {
Some(d) if d.chars().count() > MAX_DESCRIPTION_CHARS => Err(AppError::BadRequest(format!(
"description must be at most {MAX_DESCRIPTION_CHARS} characters"
))),
other => Ok(other.map(str::to_string)),
}
}
#[cfg(test)]
mod tests;
File diff suppressed because it is too large. Load diff
+782
View File
@@ -0,0 +1,782 @@
//! Opaque, integrity-protected continuation tokens for the task read surfaces.
//!
//! The task list pages by keyset over `(created_at, id)`, but the rows a
//! caller may *see* are a filtered subset of the rows the query has to
//! *examine*. Handing the caller a raw `(created_at, id)` cursor therefore
//! forced a choice between two broken options (#327 review): anchor the cursor
//! on the last visible row, and a window of denied tasks longer than the scan
//! budget stalls paging forever; or anchor it on the last examined row, and a
//! denied read leaks the id and timestamp of a task `GET /tasks/{id}` other-
//! wise 404s.
//!
//! A server-issued token removes the choice. The position it carries is the
//! last *examined* candidate, so paging always advances, and the payload is
//! opaque and MAC'd, so the caller learns nothing from it and cannot forge one
//! naming a row of their choosing.
//!
//! The position must be *confidential*, not merely unforgeable: a token that
//! merely signed a base64 payload would still let its holder read the id and
//! timestamp of the denied row it names, which is the disclosure the token
//! exists to prevent. The payload is therefore encrypted under a
//! synthetic-IV construction (SIV): the tag is an HMAC over the filter and the
//! plaintext, and it doubles as the IV seeding the keystream the plaintext is
//! XORed with. That needs no randomness source and no dependency beyond the
//! `hmac`/`sha2` pair already used for webhook signatures and blob recipient
//! tags, and it is decrypt-last: nothing is parsed until the tag verifies.
//!
//! Making the token the *only* accepted cursor also fixes the ordering-domain
//! bug the same review found. `agent_tasks.created_at` is TEXT and compared as
//! TEXT, so `...Z` and `...+00:00` denote one instant but sort differently. A
//! caller-typed timestamp could silently skip or repeat same-time rows. The
//! token instead carries the stored string verbatim, so the value compared is
//! always a value the server wrote.
//!
//! A token is bound to the *caller* as well as the filter. The position it
//! names is the last examined candidate, not the last visible one, so it
//! encodes how far a scan got under one caller's visibility. Resuming it as a
//! different caller would start the scan past rows that caller is entitled to
//! read, silently dropping them from the answer (#327 review). The presenting
//! caller's normalized DID (empty and flagged absent when anonymous) is
//! therefore part of the MAC input, so a token presented by anyone else fails
//! to verify rather than under-reporting.
//!
//! Wire form: `v1.<iv>.<body>`, both parts base64url unpadded: the 16-byte
//! synthetic IV (the truncated MAC, which also seeds the keystream) and the
//! padded, encrypted JSON payload.
use base64::engine::general_purpose::URL_SAFE_NO_PAD;
use base64::Engine;
use hmac::{Hmac, Mac};
use serde::{Deserialize, Serialize};
use sha2::Sha256;
use crate::error::AppError;
type HmacSha256 = Hmac<Sha256>;
const CURSOR_PREFIX: &str = "v1";
const KEY_DERIVATION_LABEL: &[u8] = b"gitlawb/tasks-cursor-key/v1";
const TAG_DOMAIN: &[u8] = b"gitlawb/tasks-cursor-tag/v1";
const STREAM_DOMAIN: &[u8] = b"gitlawb/tasks-cursor-stream/v1";
/// Truncated MAC length, and the synthetic IV width. 128 bits is far beyond
/// forgery reach for a token that carries no authority of its own (every page
/// re-runs the visibility gate against the presenting caller), and keeps the
/// token short enough to sit in a query string.
const TAG_LEN: usize = 16;
/// Padding bucket for the serialized cursor payload, in bytes. The keystream
/// XORs the plaintext byte for byte, so token length would otherwise track
/// payload length and leak the named row's field widths to a holder who
/// cannot decrypt anything. 128 comfortably covers the widest realistic
/// payload: a 35-char RFC3339 timestamp (`to_rfc3339()` renders 29 chars at
/// millisecond precision and 32 at microsecond, 35 only with all nine
/// fractional digits), a 36-char server-minted UUID, and the fixed JSON
/// envelope plus a 10-digit expiry, which together total 101 bytes. Should a future payload ever exceed the bucket, the length
/// rounds up to the next multiple instead of failing, so `encode` never
/// breaks — it just mints a longer token.
const PAYLOAD_BUCKET: usize = 128;
/// How long an issued cursor stays acceptable. Keyset positions never go stale
/// on their own — `created_at`/`id` are immutable — so this is not a
/// correctness bound. It bounds how long a token stays valid across a node
/// restart-and-rotate and keeps an abandoned page from being resumed
/// indefinitely.
const CURSOR_TTL_SECS: i64 = 24 * 60 * 60;
/// Node-keyed MAC key for continuation tokens, derived from the node keypair
/// seed so it needs no configuration and survives restarts of the same node.
/// Derived rather than used directly so a token forgery oracle could not
/// bear on the signing key itself.
#[derive(Clone)]
pub struct TaskCursorKey([u8; 32]);
impl std::fmt::Debug for TaskCursorKey {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.write_str("TaskCursorKey(<redacted>)")
}
}
impl TaskCursorKey {
pub fn derive(node_seed: &[u8; 32]) -> Self {
let mut mac = HmacSha256::new_from_slice(node_seed).expect("HMAC accepts any key length");
mac.update(KEY_DERIVATION_LABEL);
let mut key = [0u8; 32];
key.copy_from_slice(&mac.finalize().into_bytes());
Self(key)
}
}
/// The keyset position a token carries: the last candidate row the previous
/// request examined, whether or not the caller was allowed to see it.
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct TaskPosition {
pub created_at: String,
pub id: String,
}
impl TaskPosition {
pub fn new(created_at: impl Into<String>, id: impl Into<String>) -> Self {
Self {
created_at: created_at.into(),
id: id.into(),
}
}
pub fn as_pair(&self) -> (&str, &str) {
(self.created_at.as_str(), self.id.as_str())
}
}
#[derive(Serialize, Deserialize)]
struct CursorPayload<'a> {
/// `created_at` of the last examined candidate, verbatim as stored.
t: &'a str,
/// `id` of the last examined candidate.
i: &'a str,
/// Unix expiry.
e: i64,
}
/// The filter a page was issued under. A cursor is only meaningful against the
/// same `status`/`assignee_did` filter that produced it: resuming a filtered
/// scan from an unfiltered position (or the reverse) silently skips rows.
/// Bound into the MAC rather than stored in the payload, so it costs no token
/// length and cannot be edited without invalidating the tag.
#[derive(Debug, Clone, Copy)]
pub struct TaskFilter<'a> {
pub status: Option<&'a str>,
pub assignee_did: Option<&'a str>,
}
/// The visibility identity a token was issued to. Normalized through
/// `normalize_owner_key` so the two spellings of one `did:key` identity
/// (`did:key:X` and bare `X`) bind identically, matching `did_matches` on the
/// read path: a caller who authenticates with the other spelling of their own
/// DID must be able to resume their own page.
fn caller_binding(caller: Option<&str>) -> &str {
caller.map(crate::db::normalize_owner_key).unwrap_or("")
}
/// The assignee filter a token was issued under. Normalized through
/// `normalize_owner_key` so the two spellings of one `did:key` identity
/// (`did:key:X` and bare `X`) bind identically, matching `list_tasks_keyset`
/// in SQL: minting under `did:key:X` and resuming with bare `X` (or the reverse)
/// hits the same rows and must verify under the MAC.
fn assignee_binding(assignee: Option<&str>) -> &str {
assignee.map(crate::db::normalize_owner_key).unwrap_or("")
}
fn cursor_mac(
key: &TaskCursorKey,
filter: TaskFilter<'_>,
caller: Option<&str>,
plaintext: &[u8],
) -> HmacSha256 {
let mut mac = HmacSha256::new_from_slice(&key.0).expect("HMAC accepts any key length");
mac.update(TAG_DOMAIN);
// Length-prefix every field so no two distinct filter/caller/plaintext
// tuples can produce the same MAC input.
for field in [
filter.status.unwrap_or("").as_bytes(),
assignee_binding(filter.assignee_did).as_bytes(),
caller_binding(caller).as_bytes(),
plaintext,
] {
mac.update(&(field.len() as u64).to_be_bytes());
mac.update(field);
}
// Presence is distinct from emptiness for all three optional fields, so an
// anonymous token cannot collide with one issued to a caller whose
// normalized DID is the empty string.
mac.update(&[
u8::from(filter.status.is_some()),
u8::from(filter.assignee_did.is_some()),
u8::from(caller.is_some()),
]);
mac
}
/// Synthetic IV: the authentication tag over the filter and plaintext, which
/// also seeds the keystream. Deterministic by construction, so no randomness
/// source is needed and two tokens for the same page are byte-identical.
fn siv(
key: &TaskCursorKey,
filter: TaskFilter<'_>,
caller: Option<&str>,
plaintext: &[u8],
) -> [u8; TAG_LEN] {
let mut out = [0u8; TAG_LEN];
out.copy_from_slice(
&cursor_mac(key, filter, caller, plaintext)
.finalize()
.into_bytes()[..TAG_LEN],
);
out
}
/// XOR `buf` with the keystream for `iv`. Its own inverse, so encrypt and
/// decrypt are the same call.
fn apply_keystream(key: &TaskCursorKey, iv: &[u8; TAG_LEN], buf: &mut [u8]) {
for (block_index, chunk) in buf.chunks_mut(32).enumerate() {
let mut mac = HmacSha256::new_from_slice(&key.0).expect("HMAC accepts any key length");
mac.update(STREAM_DOMAIN);
mac.update(iv);
mac.update(&(block_index as u64).to_be_bytes());
let block = mac.finalize().into_bytes();
for (byte, k) in chunk.iter_mut().zip(block.iter()) {
*byte ^= k;
}
}
}
/// Mint a token resuming at `position` for `filter`, usable only by `caller`.
///
/// The serialized payload is padded with trailing whitespace to a fixed
/// bucket before the SIV tag is computed: the keystream preserves plaintext
/// length, so an unpadded payload would let the token's length leak the
/// named row's field widths (a 25-char `to_rfc3339()` rendering mints a
/// shorter token than a 35-char one). `decode` tolerates the padding because
/// `serde_json` ignores trailing whitespace, and the tag covers the padded
/// bytes, so a token minted before padding landed still verifies.
pub fn encode(
key: &TaskCursorKey,
filter: TaskFilter<'_>,
caller: Option<&str>,
position: &TaskPosition,
) -> String {
let mut payload = serde_json::to_vec(&CursorPayload {
t: &position.created_at,
i: &position.id,
e: chrono::Utc::now().timestamp() + CURSOR_TTL_SECS,
})
.expect("cursor payload is plain strings and an integer");
let padded_len = payload.len().next_multiple_of(PAYLOAD_BUCKET);
payload.resize(padded_len, b' ');
let iv = siv(key, filter, caller, &payload);
apply_keystream(key, &iv, &mut payload);
format!(
"{CURSOR_PREFIX}.{}.{}",
URL_SAFE_NO_PAD.encode(iv),
URL_SAFE_NO_PAD.encode(&payload)
)
}
/// One rejection message for every way a token can fail to verify. A caller
/// who mangled a token, replayed an expired one, or tried to move one to a
/// different filter or a different presenting identity learns only that the
/// cursor is not usable — never which of those it was, and never anything
/// about the row it named.
const INVALID_CURSOR: &str = "invalid or expired cursor";
fn reject() -> AppError {
AppError::BadRequest(INVALID_CURSOR.into())
}
/// Verify a token against `filter` and the presenting `caller`, and return the
/// position it carries.
pub fn decode(
key: &TaskCursorKey,
filter: TaskFilter<'_>,
caller: Option<&str>,
token: &str,
) -> crate::error::Result<TaskPosition> {
let mut parts = token.split('.');
let (Some(version), Some(iv_b64), Some(body_b64), None) =
(parts.next(), parts.next(), parts.next(), parts.next())
else {
return Err(reject());
};
if version != CURSOR_PREFIX {
return Err(reject());
}
let iv_bytes = URL_SAFE_NO_PAD.decode(iv_b64).map_err(|_| reject())?;
let mut plaintext = URL_SAFE_NO_PAD.decode(body_b64).map_err(|_| reject())?;
let iv: [u8; TAG_LEN] = iv_bytes.try_into().map_err(|_| reject())?;
apply_keystream(key, &iv, &mut plaintext);
// Authenticate before parsing: until the tag matches, `plaintext` is just
// attacker-chosen bytes run through a keystream. `verify_truncated_left`
// is the constant-time compare, so a forgery attempt cannot be steered by
// timing the first differing byte.
cursor_mac(key, filter, caller, &plaintext)
.verify_truncated_left(&iv)
.map_err(|_| reject())?;
let decoded: CursorPayload<'_> = serde_json::from_slice(&plaintext).map_err(|_| reject())?;
if decoded.e < chrono::Utc::now().timestamp() {
return Err(reject());
}
Ok(TaskPosition::new(decoded.t, decoded.i))
}
#[cfg(test)]
mod tests {
use super::*;
fn key() -> TaskCursorKey {
TaskCursorKey::derive(&[7u8; 32])
}
fn unfiltered() -> TaskFilter<'static> {
TaskFilter {
status: None,
assignee_did: None,
}
}
#[test]
fn round_trips_position_verbatim() {
let k = key();
// A stored timestamp the server wrote, fractional digits and all.
let pos = TaskPosition::new("2026-01-03T00:00:00.123456789+00:00", "task-a");
let token = encode(&k, unfiltered(), None, &pos);
assert_eq!(decode(&k, unfiltered(), None, &token).unwrap(), pos);
// A sub-bucket payload pads to exactly one bucket: the padding
// claim is an exact width, and every other width assertion here is
// bucket-relative, so only an absolute check catches a pad that
// skips "next multiple" or a drifted bucket constant.
let body = token.split('.').nth(2).expect("token has a body part");
let raw = URL_SAFE_NO_PAD.decode(body).unwrap();
assert_eq!(
raw.len(),
PAYLOAD_BUCKET,
"normal token must pad to exactly one bucket"
);
}
/// The whole point of the token is that the caller may hold it without
/// learning the denied row it names. A signed-but-plaintext payload would
/// pass every other test here and still fail this one.
#[test]
fn token_does_not_expose_the_row_it_names() {
let k = key();
let token = encode(
&k,
unfiltered(),
None,
&TaskPosition::new("2026-01-03T00:00:00+00:00", "denied-task-id"),
);
let body = token.split('.').nth(2).expect("token has a body part");
let raw = URL_SAFE_NO_PAD.decode(body).unwrap();
let as_text = String::from_utf8_lossy(&raw);
for secret in ["denied-task-id", "2026-01-03", "\"t\"", "\"i\""] {
assert!(
!as_text.contains(secret),
"token body must not carry {secret:?} in the clear: {as_text:?}"
);
}
// Also assert it is not merely reordered or whitespace-mangled JSON.
assert!(
serde_json::from_slice::<serde_json::Value>(&raw).is_err(),
"token body must not be parseable as JSON"
);
}
/// The keystream preserves plaintext length, so without the fixed-width
/// padding a token naming a 35-char timestamp would be measurably longer
/// than one naming a 32-char one: the holder would learn the withheld
/// row's field widths without decrypting anything. Mint the same shape
/// both ways and require identical token length.
#[test]
fn token_length_does_not_vary_with_position_width() {
let k = key();
// Ids are server-minted fixed-length UUIDs; the timestamp is the
// width that varies in production (`to_rfc3339()` renders 35 chars
// with nanoseconds, 25 without fractional seconds).
let id = "b3f7a1c2-4d5e-4f6a-8b9c-0d1e2f3a4b5c";
let narrow = encode(
&k,
unfiltered(),
None,
&TaskPosition::new("2026-01-03T00:00:00+00:00", id),
);
let wide = encode(
&k,
unfiltered(),
None,
&TaskPosition::new("2026-01-03T00:00:00.123456789+00:00", id),
);
assert_eq!(
narrow.len(),
wide.len(),
"tokens must not leak the named row's field widths: {narrow} vs {wide}"
);
// The padded token still decodes to the exact stored string.
assert_eq!(
decode(&k, unfiltered(), None, &wide).unwrap(),
TaskPosition::new("2026-01-03T00:00:00.123456789+00:00", id)
);
}
/// A token minted before the fixed-width padding landed (unpadded payload,
/// tag over the raw bytes) must still verify: `decode` MACs whatever
/// plaintext the keystream recovers and `serde_json` tolerates the
/// missing trailing whitespace. This pins the back-compat claim in the
/// `encode` docstring with a frozen v1 artifact, so a change to the v1
/// recipe pins (domains, MAC input, keystream, segment layout, field
/// names) breaks this test instead of silently re-deriving under the
/// new internals. It does not pin the padding: `decode` never consults
/// `PAYLOAD_BUCKET` and the fixture is unpadded, so the padding-width
/// claim belongs to `oversized_payload_rounds_up_to_next_bucket`.
///
/// Fixture expiry is fixed at 2100-01-01 (`4102444800`): it must stay in
/// the future whenever the test runs.
#[test]
fn unpadded_legacy_token_still_verifies() {
let k = key();
let filter = unfiltered();
let position = TaskPosition::new("2026-01-03T00:00:00+00:00", "legacy-task");
// Pre-padding recipe minted once and frozen: serialize with no
// padding, tag, and encrypt. Deterministic by construction (SIV needs
// no randomness), so this literal is stable across runs.
let token = "v1.ILJyy7JZScm_h-HM1-CggA.l4kJUoHTbqSGTVKL0DP4hORnbOS1_gHYV2fwJYXd2QtM-6TBMCKh-wvluIiaEYC9rUSUPVSteAKkp7jBCmK_bR2D";
assert_eq!(
decode(&k, filter, None, token).unwrap(),
position,
"pre-padding token must verify under the current decode"
);
// The fixture is a genuine pre-padding artifact, so its decoded
// body must stay below one bucket: a literal re-minted under the
// padded recipe would retire this back-compat pin silently.
let body = token.split('.').nth(2).expect("token has a body part");
let raw = URL_SAFE_NO_PAD.decode(body).unwrap();
assert!(
raw.len() < PAYLOAD_BUCKET,
"legacy fixture must be the unpadded shape: {} bytes",
raw.len()
);
}
/// The `PAYLOAD_BUCKET` docstring promises an oversized payload rounds up
/// to the next multiple instead of failing. Mint a position whose id
/// pushes the serialized payload past one bucket and require the token
/// to round-trip (at two buckets).
#[test]
fn oversized_payload_rounds_up_to_next_bucket() {
let k = key();
// ~140-char id: payload crosses the 128-byte bucket.
let long_id = "a".repeat(140);
let position = TaskPosition::new("2026-01-03T00:00:00+00:00", long_id.clone());
let probe = serde_json::to_vec(&CursorPayload {
t: &position.created_at,
i: &position.id,
e: 1_700_000_000,
})
.expect("cursor payload is plain strings and an integer");
assert!(
probe.len() > PAYLOAD_BUCKET,
"fixture payload must exceed one bucket: {} bytes",
probe.len()
);
assert!(
probe.len() <= 2 * PAYLOAD_BUCKET,
"fixture payload must fit in two buckets: {} bytes",
probe.len()
);
let token = encode(&k, unfiltered(), None, &position);
assert_eq!(
decode(&k, unfiltered(), None, &token).unwrap(),
position,
"oversized payload must round-trip at two buckets"
);
// The padded body must be exactly two buckets, not merely longer
// than one: the padding claim is an exact width, not a direction.
let body = token.split('.').nth(2).expect("token has a body part");
let raw = URL_SAFE_NO_PAD.decode(body).unwrap();
assert_eq!(
raw.len(),
2 * PAYLOAD_BUCKET,
"oversized payload must pad to exactly two buckets"
);
}
/// Callers paste the token straight into a query string, so it must carry
/// no character that needs percent-encoding.
#[test]
fn token_is_url_safe_verbatim() {
let token = encode(
&key(),
TaskFilter {
status: Some("pending"),
assignee_did: Some("did:key:z6MkAssignee"),
},
None,
&TaskPosition::new("2026-01-03T00:00:00.123456789+00:00", "task-a"),
);
assert!(
token
.chars()
.all(|c| c.is_ascii_alphanumeric() || matches!(c, '-' | '_' | '.')),
"token must be query-safe without encoding: {token}"
);
}
/// Flipping any byte of a token must fail the tag, not decode to a
/// different position: the keystream is malleable on its own, the tag is
/// what stops a chosen-position forgery.
#[test]
fn rejects_any_tampered_byte() {
let k = key();
let token = encode(
&k,
unfiltered(),
None,
&TaskPosition::new("2026-01-03T00:00:00+00:00", "task-a"),
);
let parts: Vec<&str> = token.split('.').collect();
for part in [1usize, 2] {
for position in [0usize, 3] {
let mut bytes = parts[part].as_bytes().to_vec();
bytes[position] = if bytes[position] == b'A' { b'B' } else { b'A' };
let mut mangled: Vec<String> = parts.iter().map(|p| p.to_string()).collect();
mangled[part] = String::from_utf8(bytes).unwrap();
assert!(
decode(&k, unfiltered(), None, &mangled.join(".")).is_err(),
"byte {position} of part {part} must not be malleable"
);
}
}
// Sanity: the untouched token still decodes, so the loop above is not
// passing because every token is rejected.
assert!(decode(&k, unfiltered(), None, &token).is_ok());
}
#[test]
fn rejects_token_from_another_node() {
let pos = TaskPosition::new("2026-01-03T00:00:00+00:00", "task-a");
let token = encode(&TaskCursorKey::derive(&[1u8; 32]), unfiltered(), None, &pos);
assert!(decode(
&TaskCursorKey::derive(&[2u8; 32]),
unfiltered(),
None,
&token
)
.is_err());
}
#[test]
fn rejects_cursor_moved_to_a_different_filter() {
let k = key();
let pos = TaskPosition::new("2026-01-03T00:00:00+00:00", "task-a");
let token = encode(
&k,
TaskFilter {
status: Some("pending"),
assignee_did: None,
},
None,
&pos,
);
assert!(decode(&k, unfiltered(), None, &token).is_err());
assert!(decode(
&k,
TaskFilter {
status: Some("claimed"),
assignee_did: None
},
None,
&token
)
.is_err());
assert!(decode(
&k,
TaskFilter {
status: Some("pending"),
assignee_did: None
},
None,
&token
)
.is_ok());
}
/// A filter of `Some("")` must not verify a token minted with `None`.
#[test]
fn distinguishes_absent_filter_from_empty_filter() {
let k = key();
let pos = TaskPosition::new("2026-01-03T00:00:00+00:00", "task-a");
let token = encode(&k, unfiltered(), None, &pos);
assert!(decode(
&k,
TaskFilter {
status: Some(""),
assignee_did: None
},
None,
&token
)
.is_err());
}
/// A cursor names how far a scan got under one caller's visibility. Moved
/// to a different presenting identity it would start that caller's scan
/// past rows they are entitled to read, so it must fail to verify rather
/// than silently under-report (#327 review).
#[test]
fn rejects_cursor_moved_to_a_different_caller() {
let k = key();
let pos = TaskPosition::new("2026-01-03T00:00:00+00:00", "task-a");
let alice = Some("did:key:z6MkAlice");
let bob = Some("did:key:z6MkBob");
let anon_token = encode(&k, unfiltered(), None, &pos);
assert!(
decode(&k, unfiltered(), alice, &anon_token).is_err(),
"an anonymously minted cursor must not resume an authenticated scan"
);
assert!(decode(&k, unfiltered(), None, &anon_token).is_ok());
let alice_token = encode(&k, unfiltered(), alice, &pos);
assert!(
decode(&k, unfiltered(), bob, &alice_token).is_err(),
"one caller's cursor must not resume another caller's scan"
);
assert!(
decode(&k, unfiltered(), None, &alice_token).is_err(),
"an authenticated cursor must not resume an anonymous scan"
);
assert!(decode(&k, unfiltered(), alice, &alice_token).is_ok());
}
/// The binding is on identity, not spelling: `did:key:X` and bare `X` are
/// one caller everywhere else on the read path (`did_matches`), so a caller
/// who presents the other form of their own DID must keep their own page.
#[test]
fn caller_binding_is_normalized_across_did_key_spellings() {
let k = key();
let pos = TaskPosition::new("2026-01-03T00:00:00+00:00", "task-a");
let token = encode(&k, unfiltered(), Some("did:key:z6MkAlice"), &pos);
assert_eq!(
decode(&k, unfiltered(), Some("z6MkAlice"), &token).unwrap(),
pos
);
// A different method sharing the base58 tail is a different principal.
assert!(decode(&k, unfiltered(), Some("did:web:z6MkAlice"), &token).is_err());
}
/// The filter binding is on identity, not spelling: `assignee_did` matches
/// normalized keys in SQL (`normalize_owner_key`), so minting under
/// `did:key:X` and resuming under bare `X` (or vice versa) must succeed,
/// while `did:web:X` remains distinct.
#[test]
fn assignee_filter_binding_is_normalized_across_did_key_spellings() {
let k = key();
let pos = TaskPosition::new("2026-01-03T00:00:00+00:00", "task-a");
let filter_did_key = TaskFilter {
status: Some("pending"),
assignee_did: Some("did:key:z6MkAssignee"),
};
let filter_bare = TaskFilter {
status: Some("pending"),
assignee_did: Some("z6MkAssignee"),
};
let filter_web = TaskFilter {
status: Some("pending"),
assignee_did: Some("did:web:z6MkAssignee"),
};
let token_from_did_key = encode(&k, filter_did_key, None, &pos);
assert_eq!(
decode(&k, filter_bare, None, &token_from_did_key).unwrap(),
pos
);
assert!(decode(&k, filter_web, None, &token_from_did_key).is_err());
let token_from_bare = encode(&k, filter_bare, None, &pos);
assert_eq!(
decode(&k, filter_did_key, None, &token_from_bare).unwrap(),
pos
);
assert!(decode(&k, filter_web, None, &token_from_bare).is_err());
}
/// Anonymous is a distinct binding from a caller whose normalized DID is
/// the empty string, the same way `Some("")` is distinct from `None` for
/// the filter fields.
#[test]
fn distinguishes_anonymous_from_empty_caller() {
let k = key();
let pos = TaskPosition::new("2026-01-03T00:00:00+00:00", "task-a");
let token = encode(&k, unfiltered(), None, &pos);
assert!(decode(&k, unfiltered(), Some(""), &token).is_err());
}
/// Length-prefixing must stop a status/assignee pair from being re-split.
#[test]
fn rejects_field_boundary_shift() {
let k = key();
let pos = TaskPosition::new("2026-01-03T00:00:00+00:00", "task-a");
let token = encode(
&k,
TaskFilter {
status: Some("pend"),
assignee_did: Some("ing"),
},
None,
&pos,
);
assert!(decode(
&k,
TaskFilter {
status: Some("pending"),
assignee_did: Some("")
},
None,
&token
)
.is_err());
}
#[test]
fn rejects_expired_token() {
let k = key();
// Minted the same way `encode` does, but already past its expiry.
let mut payload = serde_json::to_vec(&CursorPayload {
t: "2026-01-03T00:00:00+00:00",
i: "task-a",
e: chrono::Utc::now().timestamp() - 1,
})
.unwrap();
let iv = siv(&k, unfiltered(), None, &payload);
apply_keystream(&k, &iv, &mut payload);
let expired = format!(
"v1.{}.{}",
URL_SAFE_NO_PAD.encode(iv),
URL_SAFE_NO_PAD.encode(&payload)
);
let err = decode(&k, unfiltered(), None, &expired).unwrap_err();
assert!(err.to_string().contains(INVALID_CURSOR));
}
#[test]
fn rejects_malformed_shapes() {
let k = key();
for bad in [
"",
"v1",
"v1.",
"v1.abc",
"v2.abc.def",
"v1.abc.def.ghi",
"v1.!!!.def",
] {
assert!(
decode(&k, unfiltered(), None, bad).is_err(),
"must reject {bad:?}"
);
}
}
/// Every rejection path renders the same text, so a caller cannot tell a
/// forged token from an expired one from a filter mismatch.
#[test]
fn every_rejection_is_indistinguishable() {
let k = key();
for bad in ["v1.abc.def", "not-a-cursor", "v1..", "v9.a.b"] {
assert_eq!(
decode(&k, unfiltered(), None, bad).unwrap_err().to_string(),
format!("invalid request: {INVALID_CURSOR}")
);
}
}
}
File diff suppressed because it is too large. Load diff
+16 -32
View File
@@ -85,11 +85,10 @@ pub async fn set_visibility(
Path((owner, repo)): Path<(String, String)>, Path((owner, repo)): Path<(String, String)>,
Json(req): Json<SetVisibilityRequest>, Json(req): Json<SetVisibilityRequest>,
) -> Result<(StatusCode, Json<serde_json::Value>)> { ) -> Result<(StatusCode, Json<serde_json::Value>)> {
let record = state // Quarantine first (via authorize_repo_read), then owner — same posture as
.db // list_visibility so a quarantined repo cannot be mutated while reads 404.
.get_repo(&owner, &repo) let (record, _rules) =
.await? crate::api::authorize_repo_read(&state, &owner, &repo, Some(&auth.0), "/").await?;
.ok_or_else(|| AppError::RepoNotFound(format!("{owner}/{repo}")))?;
require_owner(&record, &auth.0)?; require_owner(&record, &auth.0)?;
validate_path_glob(&req.path_glob)?; validate_path_glob(&req.path_glob)?;
@@ -141,11 +140,8 @@ pub async fn remove_visibility(
Path((owner, repo)): Path<(String, String)>, Path((owner, repo)): Path<(String, String)>,
Json(req): Json<RemoveVisibilityRequest>, Json(req): Json<RemoveVisibilityRequest>,
) -> Result<Json<serde_json::Value>> { ) -> Result<Json<serde_json::Value>> {
let record = state let (record, _rules) =
.db crate::api::authorize_repo_read(&state, &owner, &repo, Some(&auth.0), "/").await?;
.get_repo(&owner, &repo)
.await?
.ok_or_else(|| AppError::RepoNotFound(format!("{owner}/{repo}")))?;
require_owner(&record, &auth.0)?; require_owner(&record, &auth.0)?;
state state
@@ -171,14 +167,12 @@ pub async fn list_visibility(
Extension(auth): Extension<AuthenticatedDid>, Extension(auth): Extension<AuthenticatedDid>,
Path((owner, repo)): Path<(String, String)>, Path((owner, repo)): Path<(String, String)>,
) -> Result<Json<serde_json::Value>> { ) -> Result<Json<serde_json::Value>> {
let record = state // Quarantine first (via authorize_repo_read), then owner — a quarantined
.db // mirror must be opaque even to a caller matching owner_did.
.get_repo(&owner, &repo) let (record, rules) =
.await? crate::api::authorize_repo_read(&state, &owner, &repo, Some(&auth.0), "/").await?;
.ok_or_else(|| AppError::RepoNotFound(format!("{owner}/{repo}")))?;
require_owner(&record, &auth.0)?; require_owner(&record, &auth.0)?;
let rules = state.db.list_visibility_rules(&record.id).await?;
let rules_json: Vec<_> = rules let rules_json: Vec<_> = rules
.into_iter() .into_iter()
.map(|r| { .map(|r| {
@@ -205,28 +199,18 @@ pub async fn list_visibility(
/// denied one (`reinclude`), so a clean-clone client can sparse-exclude the /// denied one (`reinclude`), so a clean-clone client can sparse-exclude the
/// denied subtrees while re-including the allowed nested paths. Unlike /// denied subtrees while re-including the allowed nested paths. Unlike
/// `list_visibility` this is not owner-gated and never exposes reader_dids. /// `list_visibility` this is not owner-gated and never exposes reader_dids.
///
/// Quarantined repos are opaque 404 via [`crate::api::authorize_repo_read`] —
/// same posture as encrypted-blob discovery — so admission and private-subtree
/// layout are not disclosed to anon, owner, or peers.
pub async fn withheld_paths( pub async fn withheld_paths(
State(state): State<AppState>, State(state): State<AppState>,
auth: Option<Extension<AuthenticatedDid>>, auth: Option<Extension<AuthenticatedDid>>,
Path((owner, repo)): Path<(String, String)>, Path((owner, repo)): Path<(String, String)>,
) -> Result<Json<serde_json::Value>> { ) -> Result<Json<serde_json::Value>> {
let record = state
.db
.get_repo(&owner, &repo)
.await?
.ok_or_else(|| AppError::RepoNotFound(format!("{owner}/{repo}")))?;
let rules = state.db.list_visibility_rules(&record.id).await?;
let caller = auth.as_ref().map(|e| e.0 .0.as_str()); let caller = auth.as_ref().map(|e| e.0 .0.as_str());
let (record, rules) =
// Whole-repo read gate: a caller who cannot read "/" gets repo-not-found, crate::api::authorize_repo_read(&state, &owner, &repo, caller, "/").await?;
// matching the git read endpoints, so this never discloses a private repo's
// existence or its path layout to an unauthorized caller.
if crate::visibility::visibility_check(&rules, record.is_public, &record.owner_did, caller, "/")
== crate::visibility::Decision::Deny
{
return Err(AppError::RepoNotFound(format!("{owner}/{repo}")));
}
let withheld = let withheld =
crate::visibility::withheld_globs(&rules, record.is_public, &record.owner_did, caller); crate::visibility::withheld_globs(&rules, record.is_public, &record.owner_did, caller);
+217 -27
View File
@@ -33,11 +33,8 @@ pub async fn create_webhook(
Path((owner, name)): Path<(String, String)>, Path((owner, name)): Path<(String, String)>,
Json(req): Json<CreateWebhookRequest>, Json(req): Json<CreateWebhookRequest>,
) -> Result<(StatusCode, Json<Webhook>)> { ) -> Result<(StatusCode, Json<Webhook>)> {
let record = state let (record, _rules) =
.db crate::api::authorize_repo_read(&state, &owner, &name, Some(auth.0.as_str()), "/").await?;
.get_repo(&owner, &name)
.await?
.ok_or_else(|| AppError::RepoNotFound(format!("{owner}/{name}")))?;
crate::api::require_repo_owner(&record, &auth.0)?; crate::api::require_repo_owner(&record, &auth.0)?;
// Gate the target through the same hardened public-host validator the peer // Gate the target through the same hardened public-host validator the peer
@@ -46,9 +43,10 @@ pub async fn create_webhook(
// endpoints (SSRF). Delivery runs on the shared no-redirect client // endpoints (SSRF). Delivery runs on the shared no-redirect client
// (main.rs), which closes the 3xx-to-internal bounce. // (main.rs), which closes the 3xx-to-internal bounce.
if !crate::api::peers::is_public_http_url(&req.url) { if !crate::api::peers::is_public_http_url(&req.url) {
return Err(AppError::BadRequest( return Err(AppError::BadRequest(format!(
"webhook URL must be a public http(s) URL (no loopback, private, or .internal/.local hosts)".into(), "webhook URL {}",
)); crate::api::peers::PUBLIC_HTTP_URL_REQUIREMENT
)));
} }
let events = req.events.unwrap_or_else(|| vec!["*".into()]); let events = req.events.unwrap_or_else(|| vec!["*".into()]);
@@ -58,7 +56,14 @@ pub async fn create_webhook(
id: Uuid::new_v4().to_string(), id: Uuid::new_v4().to_string(),
repo_id: record.id, repo_id: record.id,
url: req.url, url: req.url,
secret: req.secret, // Seal the secret before it touches the database (#506): the column and
// any dump/backup of it hold ciphertext, never the caller's signing key.
secret: req.secret.map(|plain| {
crate::webhook_secret::WebhookSecret::new(crate::webhook_secret::seal(
state.webhook_secret_key.as_ref(),
&plain,
))
}),
events, events,
created_by_did, created_by_did,
created_at: Utc::now().to_rfc3339(), created_at: Utc::now().to_rfc3339(),
@@ -95,19 +100,11 @@ pub async fn list_webhooks(
crate::api::require_repo_owner(&record, &caller)?; crate::api::require_repo_owner(&record, &caller)?;
let hooks = state.db.list_webhooks(&record.id).await?; let hooks = state.db.list_webhooks(&record.id).await?;
// Redact secrets in list response // `WebhookSecret` redacts itself on serialize, so a set secret is emitted as
let redacted: Vec<_> = hooks // "***" and the signing material never appears in the body (see webhook_secret).
.into_iter() let count = hooks.len();
.map(|mut h| {
if h.secret.is_some() {
h.secret = Some("***".into());
}
h
})
.collect();
Ok(Json( Ok(Json(
serde_json::json!({ "webhooks": redacted, "count": redacted.len() }), serde_json::json!({ "webhooks": hooks, "count": count }),
)) ))
} }
@@ -117,11 +114,8 @@ pub async fn delete_webhook(
Extension(auth): Extension<AuthenticatedDid>, Extension(auth): Extension<AuthenticatedDid>,
Path((owner, name, id)): Path<(String, String, String)>, Path((owner, name, id)): Path<(String, String, String)>,
) -> Result<Json<serde_json::Value>> { ) -> Result<Json<serde_json::Value>> {
let record = state let (record, _rules) =
.db crate::api::authorize_repo_read(&state, &owner, &name, Some(auth.0.as_str()), "/").await?;
.get_repo(&owner, &name)
.await?
.ok_or_else(|| AppError::RepoNotFound(format!("{owner}/{name}")))?;
crate::api::require_repo_owner(&record, &auth.0)?; crate::api::require_repo_owner(&record, &auth.0)?;
// Verify the webhook belongs to this repo // Verify the webhook belongs to this repo
@@ -141,7 +135,20 @@ pub async fn delete_webhook(
#[cfg(test)] #[cfg(test)]
mod tests { mod tests {
use crate::api::peers::is_public_http_url; use axum::extract::{Extension, Path, State};
use axum::Json;
use chrono::Utc;
use gitlawb_core::identity::Keypair;
use sqlx::PgPool;
use uuid::Uuid;
use super::{create_webhook, CreateWebhookRequest};
use crate::api::peers::{is_public_http_url, PUBLIC_HTTP_URL_REQUIREMENT};
use crate::auth::AuthenticatedDid;
use crate::db::RepoRecord;
use crate::error::AppError;
use crate::test_support::test_state;
use rand::Rng;
// create_webhook gates req.url through is_public_http_url. Pin the exact // create_webhook gates req.url through is_public_http_url. Pin the exact
// SSRF targets from issue #81 so the webhook path can never regress to the // SSRF targets from issue #81 so the webhook path can never regress to the
@@ -152,6 +159,7 @@ mod tests {
"http://127.0.0.1:5432/", "http://127.0.0.1:5432/",
"http://169.254.169.254/latest/meta-data/", "http://169.254.169.254/latest/meta-data/",
"http://localhost/", "http://localhost/",
"http://node.localhost/",
"http://10.0.0.5/", "http://10.0.0.5/",
"http://[::1]/", "http://[::1]/",
// IPv6 transition encodings smuggling loopback v4 (6to4 / NAT64). // IPv6 transition encodings smuggling loopback v4 (6to4 / NAT64).
@@ -169,4 +177,186 @@ mod tests {
assert!(is_public_http_url("https://hooks.example.com/gitlawb")); assert!(is_public_http_url("https://hooks.example.com/gitlawb"));
assert!(is_public_http_url("http://203.0.113.10:7545/")); assert!(is_public_http_url("http://203.0.113.10:7545/"));
} }
#[sqlx::test]
async fn webhook_localhost_rejection_uses_shared_public_url_contract(pool: PgPool) {
let state = test_state(pool).await;
let owner = Keypair::generate().did().to_string();
let repo_id = Uuid::new_v4().to_string();
state
.db
.create_repo(&RepoRecord {
id: repo_id.clone(),
name: "webhook-contract".to_string(),
owner_did: owner.clone(),
description: None,
is_public: true,
default_branch: "main".to_string(),
created_at: Utc::now(),
updated_at: Utc::now(),
disk_path: "/tmp/webhook-contract.git".to_string(),
forked_from: None,
machine_id: None,
})
.await
.unwrap();
let result = create_webhook(
State(state.clone()),
Extension(AuthenticatedDid(owner.clone())),
Path((owner, "webhook-contract".to_string())),
Json(CreateWebhookRequest {
url: "https://node.localhost/hook".to_string(),
secret: None,
events: None,
}),
)
.await;
match result {
Err(AppError::BadRequest(message)) => assert_eq!(
message,
format!("webhook URL {PUBLIC_HTTP_URL_REQUIREMENT}")
),
Err(error) => panic!("unexpected webhook rejection: {error}"),
Ok(_) => panic!("a .localhost webhook must be rejected"),
}
assert!(
state.db.list_webhooks(&repo_id).await.unwrap().is_empty(),
"a rejected .localhost webhook must leave no row behind"
);
}
/// The startup re-encryption pass must seal any webhook secret still resting
/// as plaintext (a row written before at-rest encryption shipped), and leave
/// a second pass with nothing to do.
#[sqlx::test]
async fn reencrypt_legacy_secrets_encrypts_existing_cleartext(pool: PgPool) {
let state = test_state(pool).await;
// Seed a webhook the way a pre-encryption node would have: plaintext
// secret written straight into the row, bypassing the handler's seal.
let hook = crate::db::Webhook {
id: Uuid::new_v4().to_string(),
repo_id: "owner/legacy-repo".to_string(),
url: "https://hooks.example.com/legacy".to_string(),
secret: Some(crate::webhook_secret::WebhookSecret::new(
"legacy-plaintext".to_string(),
)),
events: vec!["*".to_string()],
created_by_did: "did:key:zLEGACYOWNER".to_string(),
created_at: Utc::now().to_rfc3339(),
active: true,
};
state.db.create_webhook(&hook).await.unwrap();
let rewritten = state
.db
.reencrypt_legacy_webhook_secrets(state.webhook_secret_key.as_ref())
.await
.unwrap();
assert_eq!(rewritten, 1, "exactly the one plaintext row is rewritten");
let stored = state.db.get_webhook(&hook.id).await.unwrap().unwrap();
let secret = stored.secret.expect("secret retained after re-encryption");
assert!(secret.is_sealed(), "re-encrypted secret must be sealed");
assert_eq!(
crate::webhook_secret::open(state.webhook_secret_key.as_ref(), secret.as_str())
.as_deref(),
Some("legacy-plaintext"),
"the node-held key must recover the original secret"
);
// Idempotent: a second pass finds nothing left to rewrite.
let again = state
.db
.reencrypt_legacy_webhook_secrets(state.webhook_secret_key.as_ref())
.await
.unwrap();
assert_eq!(again, 0);
}
/// A legacy plaintext secret that literally begins with the seal prefix is
/// not ciphertext: prefix-matching alone would skip it forever while
/// `reveal` fails to open it, wedging the hook. Classification is by
/// `open`, so the pass seals it and the literal secret survives.
#[sqlx::test]
async fn reencrypt_seals_a_prefixed_plaintext_secret(pool: PgPool) {
let state = test_state(pool).await;
let literal = format!(
"{}not-a-real-ciphertext",
crate::webhook_secret::SEAL_PREFIX
);
let hook = crate::db::Webhook {
id: Uuid::new_v4().to_string(),
repo_id: "owner/prefixed-repo".to_string(),
url: "https://hooks.example.com/prefixed".to_string(),
secret: Some(crate::webhook_secret::WebhookSecret::new(literal.clone())),
events: vec!["*".to_string()],
created_by_did: "did:key:zLEGACYOWNER".to_string(),
created_at: Utc::now().to_rfc3339(),
active: true,
};
state.db.create_webhook(&hook).await.unwrap();
let rewritten = state
.db
.reencrypt_legacy_webhook_secrets(state.webhook_secret_key.as_ref())
.await
.unwrap();
assert_eq!(rewritten, 1, "the prefixed plaintext row is rewritten");
let stored = state.db.get_webhook(&hook.id).await.unwrap().unwrap();
let secret = stored.secret.unwrap();
assert!(secret.is_sealed());
assert_eq!(
crate::webhook_secret::reveal(state.webhook_secret_key.as_ref(), &secret).as_deref(),
Some(literal.as_str()),
"the literal plaintext, prefix included, is the recovered secret"
);
}
/// A well-formed sealed payload that does not open under this node's key
/// (foreign or rotated ciphertext) must survive the re-encrypt pass
/// untouched: re-wrapping it would make `reveal` return the outer blob as
/// a signing secret and flip a broken hook from withheld into garbage
/// signatures.
#[sqlx::test]
async fn reencrypt_leaves_foreign_ciphertext_untouched(pool: PgPool) {
let state = test_state(pool).await;
let foreign_key: [u8; 32] = chacha20poly1305::aead::OsRng.gen();
let foreign_ciphertext = crate::webhook_secret::seal(&foreign_key, "owner-real-secret");
let hook = crate::db::Webhook {
id: Uuid::new_v4().to_string(),
repo_id: "owner/foreign-repo".to_string(),
url: "https://hooks.example.com/foreign".to_string(),
secret: Some(crate::webhook_secret::WebhookSecret::new(
foreign_ciphertext.clone(),
)),
events: vec!["*".to_string()],
created_by_did: "did:key:zLEGACYOWNER".to_string(),
created_at: Utc::now().to_rfc3339(),
active: true,
};
state.db.create_webhook(&hook).await.unwrap();
let rewritten = state
.db
.reencrypt_legacy_webhook_secrets(state.webhook_secret_key.as_ref())
.await
.unwrap();
assert_eq!(rewritten, 0, "foreign ciphertext is left unchanged");
let stored = state.db.get_webhook(&hook.id).await.unwrap().unwrap();
let secret = stored.secret.unwrap();
assert_eq!(
secret.as_str(),
foreign_ciphertext,
"the stored value is still the unopenable ciphertext, not a re-wrap"
);
assert!(
crate::webhook_secret::reveal(state.webhook_secret_key.as_ref(), &secret).is_none(),
"delivery still withholds: reveal cannot open the foreign blob"
);
}
} }
+717 -14
View File
@@ -17,16 +17,121 @@ use crate::state::AppState;
#[derive(Clone, Debug)] #[derive(Clone, Debug)]
pub struct AuthenticatedDid(pub String); pub struct AuthenticatedDid(pub String);
/// The RFC 9421 material this request was actually verified against, injected
/// alongside [`AuthenticatedDid`] by `require_signature`.
///
/// A handler that records a signed claim as history has to store it: the header
/// values and the canonical signing string are gone once the request ends, and a
/// stored claim nobody can re-verify is not history. `signing_string` is the exact
/// byte sequence the Ed25519 verification succeeded over, not a rebuild — a
/// handler reconstructing it from headers would be verifying a different string
/// than the middleware did.
#[derive(Clone, Debug)]
pub struct SignatureMaterial {
/// The `Signature` header value.
pub signature: String,
/// The `Signature-Input` header value.
pub signature_input: String,
/// The canonical bytes the signature covered.
pub signing_string: String,
/// The buffered request body, on the routes that persist it. The signing
/// string covers it only through the content-digest, so without these bytes
/// a stored claim can be shown to carry *a* valid signature over *some*
/// digest and nothing more. Carried here because the body is consumed
/// downstream by the extractor and cannot be recovered afterwards.
///
/// `Option`, and populated only behind [`PersistsSignedBody`], because a
/// route that never stores the body has no use for a second handle to it.
/// The unlike-sized field is this one: the three above are header-derived
/// and bounded by the server's header limit, while a receive-pack POST
/// carries up to GITLAWB_MAX_PACK_BYTES (2 GB by default). `Bytes`, not
/// `Vec<u8>`, so the handle the marked routes do take is a refcount bump
/// over the buffer the middleware already holds rather than a copy.
pub body: Option<axum::body::Bytes>,
}
/// Marks a route group whose handler persists the signed request body.
///
/// Applied as an extension layer OUTSIDE the auth layers, so it is on the
/// request before [`require_signature`] decides whether to carry the body. A
/// marker applied inside them is never seen, and the failure is silent: the
/// handler still compiles, tests that inject material by hand still pass, and
/// production stores nothing. The status write handler refuses an absent body
/// for that reason.
///
/// A marker rather than a path check because `require_signature` must not learn
/// which URL persists claims; that knowledge belongs at the route table.
#[derive(Clone, Copy, Debug)]
pub struct PersistsSignedBody;
/// A UCAN that passed full chain validation, with the root issuer the chain
/// rests on. Inserted into request extensions by [`require_ucan_chain`] when
/// `X-Ucan` is present; absent when the header is.
///
/// `root` is carried rather than recomputed so the chain is walked once per
/// request. Holding this is not itself an authorization decision — a caller must
/// still compare `root` against an identity it independently trusts, because
/// `did:key` is self-certifying and anyone can mint a chain that verifies.
#[derive(Clone, Debug)]
pub struct VerifiedUcan {
pub ucan: Ucan,
pub root: Did,
}
/// Whether `caller` is authorized to push to `record`. /// Whether `caller` is authorized to push to `record`.
/// ///
/// Phase 1 (`GITLAWB_ENFORCE_OWNER_PUSH`): owner-only, via the canonical /// The repo owner, or a caller presenting a verified UCAN whose chain roots at
/// [`crate::api::did_matches`] owner comparison (DID-safe on both sides). This is /// that owner and which carries `git/push` for this repo.
/// intentionally a distinct, intent-named gate rather than a bare owner check so ///
/// that Phase 2 can extend it to honor a verified UCAN `git/push` capability as a /// `verified` is optional because `X-Ucan` is: a push carrying no token reaches
/// pure addition (`did_matches(..) || ucan_grants_push(..)`) without rewriting /// the same owner-only decision it always did. The owner check is unconditional
/// call sites. /// and runs first, so this can only ever turn a refusal into an acceptance,
pub fn caller_authorized_to_push(record: &crate::db::RepoRecord, caller: &str) -> bool { /// never the reverse.
pub fn caller_authorized_to_push(
record: &crate::db::RepoRecord,
caller: &str,
verified: Option<&VerifiedUcan>,
) -> bool {
crate::api::did_matches(caller, &record.owner_did) crate::api::did_matches(caller, &record.owner_did)
|| verified.is_some_and(|v| ucan_grants_push(record, v))
}
/// Whether a verified UCAN authorizes a push to `record`.
///
/// Three conditions, all required:
/// 1. The chain roots at this repo's owner. This is the trust anchor — the
/// repo record is data the node holds independently of the token, so a
/// self-minted chain cannot satisfy it.
/// 2. Every link is bounded. There is no revocation path, so an unbounded link
/// is a permanent grant.
/// 3. Every link — the leaf and every proof behind it — carries an
/// unconstrained push-class capability naming this repository. That is
/// [`gitlawb_core::ucan::Ucan::chain_grants_push_to`], the same rule
/// `gl ucan import` applies before storing a token and `git-remote-gitlawb`
/// applies before minting an invocation, so what one boundary accepts the
/// next does not refuse.
///
/// (3) walks the whole chain on purpose. Refusing a wildcard leaf alone did
/// nothing: `is_attenuated_by` accepts a concrete child under a `*` parent, and
/// that narrowing was exactly what the helper performed, so one owner-issued
/// `with: "*"` proof let a delegate mint a concrete leaf for ANY repository the
/// owner had — or created later — and both the chain check and the owner-root
/// check passed. A delegation's scope is fixed when it is issued.
pub fn ucan_grants_push(record: &crate::db::RepoRecord, verified: &VerifiedUcan) -> bool {
if !crate::api::did_matches(&verified.root.to_string(), &record.owner_did) {
return false;
}
// A write capability must lapse on its own. `exp` is optional in the format and
// there is no revocation path, so a chain with an unbounded link is a permanent
// grant: once the token leaks, the owner cannot withdraw it short of rotating
// the DID the repo is keyed on. Refusing here is what makes "the damage window
// is the token's expiry" a true statement rather than an aspiration.
if !verified.ucan.chain_lifetime_is_bounded() {
return false;
}
verified
.ucan
.chain_grants_push_to(&record.owner_did, &record.name)
} }
use gitlawb_core::http_sig::{ use gitlawb_core::http_sig::{
@@ -317,10 +422,28 @@ async fn verify_signature(limit: usize, request: Request, next: Next) -> Respons
tracing::info!(did = %sig.key_id, "✓ authenticated request"); tracing::info!(did = %sig.key_id, "✓ authenticated request");
// The body travels only where it is stored. On every other signed route the
// middleware's handle would keep the buffer alive for the rest of the layer
// chain, until the handler's own extractor consumes the request.
let body = parts
.extensions
.get::<PersistsSignedBody>()
.map(|_| body_bytes.clone());
let material = SignatureMaterial {
signature: sig_header,
signature_input: sig_input,
signing_string,
body,
};
let mut request = Request::from_parts(parts, Body::from(body_bytes)); let mut request = Request::from_parts(parts, Body::from(body_bytes));
request request
.extensions_mut() .extensions_mut()
.insert(AuthenticatedDid(sig.key_id.to_string())); .insert(AuthenticatedDid(sig.key_id.to_string()));
// Carry the verified material forward for handlers that persist a signed
// claim; none of it can be recovered once the request is gone.
request.extensions_mut().insert(material);
next.run(request).await next.run(request).await
} }
@@ -355,7 +478,7 @@ fn validate_ucan_chain(
token: &str, token: &str,
expected_aud: &Did, expected_aud: &Did,
signer_did: &Did, signer_did: &Did,
) -> Result<(), (StatusCode, Json<serde_json::Value>)> { ) -> Result<VerifiedUcan, (StatusCode, Json<serde_json::Value>)> {
let ucan = Ucan::decode(token).map_err(|e| { let ucan = Ucan::decode(token).map_err(|e| {
( (
StatusCode::UNAUTHORIZED, StatusCode::UNAUTHORIZED,
@@ -383,14 +506,14 @@ fn validate_ucan_chain(
) )
})?; })?;
ucan.verify_chain().map_err(|e| { let root = ucan.verify_chain().map_err(|e| {
( (
StatusCode::UNAUTHORIZED, StatusCode::UNAUTHORIZED,
Json(json!({ "error": "invalid_ucan", "message": e.to_string() })), Json(json!({ "error": "invalid_ucan", "message": e.to_string() })),
) )
})?; })?;
Ok(()) Ok(VerifiedUcan { ucan, root })
} }
/// Axum middleware that validates a UCAN chain when `X-Ucan` is present. /// Axum middleware that validates a UCAN chain when `X-Ucan` is present.
@@ -443,11 +566,18 @@ pub async fn require_ucan_chain(
} }
}; };
if let Err((status, body)) = validate_ucan_chain(&token, &state.node_did, &signer_did) { let verified = match validate_ucan_chain(&token, &state.node_did, &signer_did) {
return (status, body).into_response(); Ok(v) => v,
} Err((status, body)) => return (status, body).into_response(),
};
tracing::debug!(did = %signer_did, "UCAN chain validated"); tracing::debug!(did = %signer_did, root = %verified.root, "UCAN chain validated");
// Park the verified token where a handler can reach it. Validation alone
// grants nothing; the authorization decision is made downstream, by a caller
// that knows which identity it trusts for the resource being touched.
let mut request = request;
request.extensions_mut().insert(verified);
next.run(request).await next.run(request).await
} }
@@ -483,6 +613,37 @@ mod tests {
Ucan::bootstrap(node, agent_did).unwrap() Ucan::bootstrap(node, agent_did).unwrap()
} }
/// The middleware validated a token and threw the result away, so no handler
/// could ever read it and `Ucan::can` had no call site in the node. Validation
/// must hand back both the token and the root the chain rests on.
#[test]
fn validate_ucan_chain_hands_back_the_root_and_the_token() {
let owner = Keypair::generate();
let agent = Keypair::generate();
let node = Keypair::generate();
let caps_vec = vec![Capability::new("gitlawb://repos/zowner/r", caps::GIT_PUSH)];
let delegation =
Ucan::issue(&owner, agent.did(), caps_vec.clone(), None).expect("issue delegation");
let invocation = Ucan::delegate(&agent, node.did(), caps_vec, None, &delegation)
.expect("wrap invocation");
let token = invocation.encode().expect("encode");
let verified = validate_ucan_chain(&token, &node.did(), &agent.did())
.expect("a well-formed owner-rooted invocation must validate");
assert_eq!(
verified.root,
owner.did(),
"the root must be the owner, so a caller can anchor against the repo record"
);
assert_eq!(
verified.ucan.payload.iss,
agent.did(),
"the token itself must come back so a caller can read its capabilities"
);
}
fn delegation_ucan(agent: &Keypair, node_did: Did, proof: &Ucan) -> Ucan { fn delegation_ucan(agent: &Keypair, node_did: Did, proof: &Ucan) -> Ucan {
Ucan::delegate( Ucan::delegate(
agent, agent,
@@ -628,16 +789,19 @@ mod tests {
let keypair = Keypair::generate(); let keypair = Keypair::generate();
let scan_token_key = crate::state::AppState::derive_scan_token_key(&keypair); let scan_token_key = crate::state::AppState::derive_scan_token_key(&keypair);
let webhook_secret_key = crate::state::AppState::derive_webhook_secret_key(&keypair);
let (ref_tx, _) = tokio::sync::broadcast::channel(1); let (ref_tx, _) = tokio::sync::broadcast::channel(1);
let (task_tx, _) = tokio::sync::broadcast::channel(1); let (task_tx, _) = tokio::sync::broadcast::channel(1);
let pool = sqlx::postgres::PgPoolOptions::new() let pool = sqlx::postgres::PgPoolOptions::new()
.connect_lazy("postgres://localhost/gitlawb_test_placeholder") .connect_lazy("postgres://localhost/gitlawb_test_placeholder")
.expect("lazy pool creation should not fail"); .expect("lazy pool creation should not fail");
let db = Arc::new(crate::db::Db::for_testing(pool.clone())); let db = Arc::new(crate::db::Db::for_testing(pool.clone()));
let task_cursor_key = crate::api::task_cursor::TaskCursorKey::derive(&keypair.to_seed());
let schema = Arc::new(graphql::build_schema( let schema = Arc::new(graphql::build_schema(
db.clone(), db.clone(),
ref_tx.clone(), ref_tx.clone(),
task_tx.clone(), task_tx.clone(),
task_cursor_key.clone(),
)); ));
crate::state::AppState { crate::state::AppState {
config: Arc::new(Config::parse_from(["gitlawb-node"])), config: Arc::new(Config::parse_from(["gitlawb-node"])),
@@ -649,6 +813,7 @@ mod tests {
ref_update_tx: ref_tx, ref_update_tx: ref_tx,
task_event_tx: task_tx, task_event_tx: task_tx,
graphql_schema: schema, graphql_schema: schema,
task_cursor_key,
machine_id: None, machine_id: None,
repo_store: crate::git::repo_store::RepoStore::for_testing(PathBuf::from("/tmp"), pool), repo_store: crate::git::repo_store::RepoStore::for_testing(PathBuf::from("/tmp"), pool),
rate_limiter: RateLimiter::new(100, Duration::from_secs(60)), rate_limiter: RateLimiter::new(100, Duration::from_secs(60)),
@@ -664,10 +829,12 @@ mod tests {
ipfs_max_legacy_scan_rule_bytes: ipfs_max_legacy_scan_rule_bytes:
crate::api::ipfs::MAX_LEGACY_SCAN_RULE_BYTES_PER_REQUEST, crate::api::ipfs::MAX_LEGACY_SCAN_RULE_BYTES_PER_REQUEST,
ipfs_scan_token_key: Arc::new(scan_token_key), ipfs_scan_token_key: Arc::new(scan_token_key),
webhook_secret_key: Arc::new(webhook_secret_key),
ipfs_max_served_object_bytes: crate::api::ipfs::MAX_SERVED_OBJECT_BYTES, ipfs_max_served_object_bytes: crate::api::ipfs::MAX_SERVED_OBJECT_BYTES,
push_limiter_trust: crate::rate_limit::TrustedProxy::None, push_limiter_trust: crate::rate_limit::TrustedProxy::None,
sync_trigger_rate_limiter: RateLimiter::new(60, Duration::from_secs(3600)), sync_trigger_rate_limiter: RateLimiter::new(60, Duration::from_secs(3600)),
peer_write_rate_limiter: RateLimiter::new(600, Duration::from_secs(3600)), peer_write_rate_limiter: RateLimiter::new(600, Duration::from_secs(3600)),
federated_rate_limiter: RateLimiter::new_bounded(12, Duration::from_secs(60), 10_000),
shutdown_tx: tokio::sync::watch::channel(false).0, shutdown_tx: tokio::sync::watch::channel(false).0,
git_read_semaphore: Arc::new(tokio::sync::Semaphore::new(64)), git_read_semaphore: Arc::new(tokio::sync::Semaphore::new(64)),
git_write_semaphore: Arc::new(tokio::sync::Semaphore::new(64)), git_write_semaphore: Arc::new(tokio::sync::Semaphore::new(64)),
@@ -683,10 +850,114 @@ mod tests {
git_ipfs_walk_semaphore: Arc::new(tokio::sync::Semaphore::new(64)), git_ipfs_walk_semaphore: Arc::new(tokio::sync::Semaphore::new(64)),
git_ipfs_walk_per_caller: git_ipfs_walk_per_caller:
crate::rate_limit::PerCallerConcurrency::with_default_max_keys(16), crate::rate_limit::PerCallerConcurrency::with_default_max_keys(16),
task_read_rate_limiter: RateLimiter::new(1200, Duration::from_secs(3600)),
git_bin: "git".to_string(), git_bin: "git".to_string(),
} }
} }
/// Reports what the signature middleware put in `SignatureMaterial.body`,
/// so a test can tell "captured" from "deliberately not captured" apart
/// from "the middleware never ran".
async fn report_body(material: Option<axum::Extension<SignatureMaterial>>) -> String {
match material {
Some(axum::Extension(m)) => match m.body {
Some(bytes) => format!("captured:{}", bytes.len()),
None => "absent".to_string(),
},
None => "no-material".to_string(),
}
}
fn signed_post(path: &str, body: &[u8]) -> Request {
let kp = Keypair::generate();
let signed = gitlawb_core::http_sig::sign_request(&kp, "POST", path, body);
Request::builder()
.method("POST")
.uri(path)
.header("content-digest", signed.content_digest)
.header("signature-input", signed.signature_input)
.header("signature", signed.signature)
.body(axum::body::Body::from(body.to_vec()))
.unwrap()
}
/// The body handle travels only on a route that asked for it.
///
/// Both arms drive a genuinely signed request through `require_signature`;
/// the only difference is the [`PersistsSignedBody`] marker layer. A
/// middleware that captured unconditionally would make both arms read
/// `captured`, and one that never read the marker would make both read
/// `absent`, so the pair pins the decision rather than the mechanism.
#[tokio::test]
async fn the_body_is_carried_only_when_the_route_marks_itself_as_persisting_it() {
let body = br#"{"state":"success"}"#;
// The marker layer is added AFTER the middleware layer, which is what
// makes it the outer of the two and therefore the one that runs first.
// This is the same relative position `build_router` uses on the status
// write group, where the marker sits outside `add_auth_layers`.
let marked = Router::new()
.route("/", axum::routing::post(report_body))
.layer(middleware::from_fn_with_state(
crate::auth::SignatureBodyLimit(65536),
require_signature,
))
.layer(axum::Extension(PersistsSignedBody));
let resp = marked.oneshot(signed_post("/", body)).await.unwrap();
assert_eq!(resp.status(), StatusCode::OK);
let seen = axum::body::to_bytes(resp.into_body(), 2048).await.unwrap();
assert_eq!(
String::from_utf8(seen.to_vec()).unwrap(),
format!("captured:{}", body.len()),
"a route carrying the persist marker must receive the buffered body"
);
let unmarked = Router::new()
.route("/", axum::routing::post(report_body))
.layer(middleware::from_fn_with_state(
crate::auth::SignatureBodyLimit(65536),
require_signature,
));
let resp = unmarked.oneshot(signed_post("/", body)).await.unwrap();
assert_eq!(resp.status(), StatusCode::OK);
let seen = axum::body::to_bytes(resp.into_body(), 2048).await.unwrap();
assert_eq!(
String::from_utf8(seen.to_vec()).unwrap(),
"absent",
"a signed route that never persists the body must not be handed a handle to it"
);
}
/// The marker is read from the request the middleware was given, and the
/// request it hands downstream still carries it. Both halves matter: the
/// first is the capture decision, the second is that rebuilding the request
/// from its parts does not drop extensions a later layer may want.
#[tokio::test]
async fn the_marker_survives_the_request_rebuild() {
async fn report_marker(marker: Option<axum::Extension<PersistsSignedBody>>) -> String {
match marker {
Some(_) => "marked".to_string(),
None => "unmarked".to_string(),
}
}
let app = Router::new()
.route("/", axum::routing::post(report_marker))
.layer(middleware::from_fn_with_state(
crate::auth::SignatureBodyLimit(65536),
require_signature,
))
.layer(axum::Extension(PersistsSignedBody));
let resp = app.oneshot(signed_post("/", b"{}")).await.unwrap();
assert_eq!(resp.status(), StatusCode::OK);
let seen = axum::body::to_bytes(resp.into_body(), 2048).await.unwrap();
assert_eq!(
String::from_utf8(seen.to_vec()).unwrap(),
"marked",
"the marker must reach the handler, so the middleware saw it too"
);
}
#[tokio::test] #[tokio::test]
async fn require_ucan_chain_no_header_passes_through() { async fn require_ucan_chain_no_header_passes_through() {
let state = make_test_state(Keypair::generate().did()); let state = make_test_state(Keypair::generate().did());
@@ -849,3 +1120,435 @@ mod tests {
); );
} }
} }
#[cfg(test)]
mod ucan_push_tests {
use super::*;
use gitlawb_core::identity::Keypair;
use gitlawb_core::ucan::{caps, Capability, Ucan};
const OWNER_KEY: &str = "z6MkOwnerAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA";
/// `RepoRecord` does not derive `Default`, and adding the derive to a
/// production DB type purely to serve a test is the wrong direction.
fn repo(owner_did: &str, name: &str) -> crate::db::RepoRecord {
crate::db::RepoRecord {
id: "repo-id".to_string(),
name: name.to_string(),
owner_did: owner_did.to_string(),
description: None,
is_public: true,
default_branch: "main".to_string(),
created_at: chrono::Utc::now(),
updated_at: chrono::Utc::now(),
disk_path: "/unused".to_string(),
forked_from: None,
machine_id: None,
}
}
/// The token's own issuer and audience are irrelevant to this predicate: the
/// middleware has already bound `iss` to the request signer and `aud` to this
/// node. Only the capabilities and the chain's root matter here.
fn verified(root: &str, caps_vec: Vec<Capability>) -> VerifiedUcan {
verified_with_exp(
root,
caps_vec,
Some(chrono::Utc::now() + chrono::Duration::hours(1)),
)
}
fn verified_with_exp(
root: &str,
caps_vec: Vec<Capability>,
exp: Option<chrono::DateTime<chrono::Utc>>,
) -> VerifiedUcan {
let agent = Keypair::generate();
let node = Keypair::generate();
let ucan = Ucan::issue(&agent, node.did(), caps_vec, exp).expect("issue");
VerifiedUcan {
ucan,
root: root.parse().expect("root DID must parse"),
}
}
fn owner_full() -> String {
format!("did:key:{OWNER_KEY}")
}
fn push_cap_for(owner: &str, name: &str) -> Capability {
Capability::new(format!("gitlawb://repos/{owner}/{name}"), caps::GIT_PUSH)
}
#[test]
fn grants_push_when_the_chain_roots_at_the_owner_and_names_the_repo() {
let rec = repo(&owner_full(), "myrepo");
let v = verified(&owner_full(), vec![push_cap_for(&owner_full(), "myrepo")]);
assert!(ucan_grants_push(&rec, &v));
}
#[test]
fn matches_a_bare_owner_key_against_a_full_did_record() {
// Mirror rows store the bare key. A literal string compare would fail
// here, denying a delegation that is in fact valid.
let rec = repo(OWNER_KEY, "myrepo");
let v = verified(&owner_full(), vec![push_cap_for(&owner_full(), "myrepo")]);
assert!(ucan_grants_push(&rec, &v));
}
/// A perpetual grant is refused even when it is otherwise perfectly valid:
/// owner-rooted, right repo, right action. Without revocation, an unbounded
/// delegation cannot be withdrawn once it leaks.
#[test]
fn refuses_a_delegation_that_never_expires() {
let rec = repo(&owner_full(), "myrepo");
let v = verified_with_exp(
&owner_full(),
vec![push_cap_for(&owner_full(), "myrepo")],
None,
);
assert!(!ucan_grants_push(&rec, &v));
}
#[test]
fn refuses_a_self_minted_root() {
// The whole point: a token nobody delegated grants nothing, however
// permissive its capabilities look.
let stranger = Keypair::generate();
let rec = repo(&owner_full(), "myrepo");
let v = verified(&stranger.did().to_string(), vec![Capability::new("*", "*")]);
assert!(!ucan_grants_push(&rec, &v));
}
#[test]
fn refuses_a_capability_for_a_different_repo() {
let rec = repo(&owner_full(), "myrepo");
let v = verified(
&owner_full(),
vec![push_cap_for(&owner_full(), "otherrepo")],
);
assert!(!ucan_grants_push(&rec, &v));
}
/// A delegation can be issued against the remote's spelling (`myrepo.git`):
/// the matcher strips the same single `.git` the smart-HTTP routes do, so
/// the capability lands on the canonical record name.
#[test]
fn grants_push_when_the_capability_carries_the_git_suffix() {
let rec = repo(&owner_full(), "myrepo");
let v = verified(
&owner_full(),
vec![push_cap_for(&owner_full(), "myrepo.git")],
);
assert!(ucan_grants_push(&rec, &v));
}
/// Mirror-created rows can carry a literal `.git` name; reaching one takes
/// `foo.git.git`, the same spelling the remote URL uses. A single `.git`
/// capability resolves to `foo` and must not fall through to `foo.git`.
#[test]
fn a_repo_literally_named_dot_git_needs_the_doubled_suffix() {
let rec = repo(&owner_full(), "foo.git");
let single = verified(&owner_full(), vec![push_cap_for(&owner_full(), "foo.git")]);
assert!(!ucan_grants_push(&rec, &single));
let doubled = verified(
&owner_full(),
vec![push_cap_for(&owner_full(), "foo.git.git")],
);
assert!(ucan_grants_push(&rec, &doubled));
}
#[test]
fn refuses_a_capability_carrying_constraints() {
// `nb` is not interpreted yet. An owner who writes {"refs": [...]} means
// to restrict; honouring the capability while ignoring nb would grant
// strictly more than they intended, so it authorizes nothing.
let rec = repo(&owner_full(), "myrepo");
let v = verified(
&owner_full(),
vec![push_cap_for(&owner_full(), "myrepo")
.with_constraints(serde_json::json!({ "refs": ["refs/heads/feat/*"] }))],
);
assert!(!ucan_grants_push(&rec, &v));
}
#[test]
fn refuses_a_non_push_capability() {
let rec = repo(&owner_full(), "myrepo");
let v = verified(
&owner_full(),
vec![Capability::new(
format!("gitlawb://repos/{}/myrepo", owner_full()),
caps::ISSUE_CREATE,
)],
);
assert!(!ucan_grants_push(&rec, &v));
}
/// A resource wildcard must NOT authorize a push, even though attenuation
/// accepts it. The helper narrows a `*` delegation before signing, but the node
/// cannot rely on that: a delegate can sign an invocation that keeps the
/// wildcard, and it would otherwise reach every repository the owner has — or
/// will later create. This test previously asserted the opposite.
#[test]
fn refuses_a_resource_wildcard_and_honours_repo_admin() {
let rec = repo(&owner_full(), "myrepo");
let wildcard = verified(&owner_full(), vec![Capability::new("*", caps::GIT_PUSH)]);
assert!(
!ucan_grants_push(&rec, &wildcard),
"a wildcard resource must not authorize a push at the node"
);
// The ACTION wildcard is a different axis and stays: attenuation bounds it,
// and it still has to name a concrete repository.
let action_wildcard = verified(
&owner_full(),
vec![Capability::new(
format!("gitlawb://repos/{}/myrepo", owner_full()),
"*",
)],
);
assert!(ucan_grants_push(&rec, &action_wildcard));
let admin = verified(
&owner_full(),
vec![Capability::new(
format!("gitlawb://repos/{}/myrepo", owner_full()),
caps::REPO_ADMIN,
)],
);
assert!(ucan_grants_push(&rec, &admin));
}
/// The attack the wildcard refusal exists to stop: one `*` delegation reaching a
/// repository it was never issued against, including one created afterwards.
#[test]
fn a_wildcard_delegation_cannot_reach_a_second_repository() {
let other = repo(&owner_full(), "a-repo-created-later");
let wildcard = verified(&owner_full(), vec![Capability::new("*", caps::GIT_PUSH)]);
assert!(!ucan_grants_push(&other, &wildcard));
}
/// The round-8 P1, executed rather than reasoned: an owner-issued `*` PROOF with
/// a concrete leaf for a repository that did not exist at issuance. Refusing a
/// wildcard *leaf* did nothing here — `is_attenuated_by` accepts a concrete child
/// under a `*` parent, and that narrowing is exactly what `build_invocation` did,
/// so the first-party helper was the working mint path.
#[test]
fn a_wildcard_proof_cannot_reach_a_repo_created_later() {
let owner = Keypair::generate();
let agent = Keypair::generate();
let node = Keypair::generate();
let hour = chrono::Utc::now() + chrono::Duration::hours(1);
let parent = Ucan::issue(
&owner,
agent.did(),
vec![Capability::new("*", caps::GIT_PUSH)],
Some(hour),
)
.unwrap();
let later = format!("gitlawb://repos/{}/a-repo-created-later", owner.did());
let invocation = Ucan::delegate(
&agent,
node.did(),
vec![Capability::new(&later, caps::GIT_PUSH)],
Some(hour),
&parent,
)
.unwrap();
let root = invocation.verify_chain().expect("the chain still verifies");
let rec = repo(&owner.did().to_string(), "a-repo-created-later");
assert!(
!ucan_grants_push(
&rec,
&VerifiedUcan {
ucan: invocation,
root
}
),
"a wildcard proof must not authorize a repo it never named"
);
}
/// The shipping flow must keep working: a proof that names the repository
/// authorizes a push to it. Guards against fixing the wildcard by refusing
/// everything with a `prf`.
#[test]
fn a_concrete_proof_still_authorizes_the_repo_it_names() {
let owner = Keypair::generate();
let agent = Keypair::generate();
let node = Keypair::generate();
let hour = chrono::Utc::now() + chrono::Duration::hours(1);
let resource = format!("gitlawb://repos/{}/myrepo", owner.did());
let parent = Ucan::issue(
&owner,
agent.did(),
vec![Capability::new(&resource, caps::GIT_PUSH)],
Some(hour),
)
.unwrap();
let invocation = Ucan::delegate(
&agent,
node.did(),
vec![Capability::new(&resource, caps::GIT_PUSH)],
Some(hour),
&parent,
)
.unwrap();
let root = invocation.verify_chain().expect("chain verifies");
let rec = repo(&owner.did().to_string(), "myrepo");
assert!(ucan_grants_push(
&rec,
&VerifiedUcan {
ucan: invocation,
root
}
));
}
/// A `*` two links up. The immediate proof names the repository, so a walk that
/// checks only one level of `prf` stays green here — this is the case that
/// makes the recursion load-bearing rather than incidental.
#[test]
fn a_wildcard_grandparent_cannot_reach_the_repo_through_a_concrete_parent() {
let owner = Keypair::generate();
let intermediary = Keypair::generate();
let agent = Keypair::generate();
let node = Keypair::generate();
let hour = chrono::Utc::now() + chrono::Duration::hours(1);
let resource = format!("gitlawb://repos/{}/myrepo", owner.did());
let grandparent = Ucan::issue(
&owner,
intermediary.did(),
vec![Capability::new("*", caps::GIT_PUSH)],
Some(hour),
)
.unwrap();
let parent = Ucan::delegate(
&intermediary,
agent.did(),
vec![Capability::new(&resource, caps::GIT_PUSH)],
Some(hour),
&grandparent,
)
.unwrap();
let invocation = Ucan::delegate(
&agent,
node.did(),
vec![Capability::new(&resource, caps::GIT_PUSH)],
Some(hour),
&parent,
)
.unwrap();
let root = invocation.verify_chain().expect("chain verifies");
assert_eq!(root, owner.did());
let rec = repo(&owner.did().to_string(), "myrepo");
assert!(
!ucan_grants_push(
&rec,
&VerifiedUcan {
ucan: invocation,
root
}
),
"a wildcard anywhere in the chain must deny, not just in the immediate proof"
);
}
/// And the same three-link shape with every link concrete must still grant, so
/// the walk is refusing the wildcard and not the depth.
#[test]
fn a_three_link_concrete_chain_still_authorizes() {
let owner = Keypair::generate();
let intermediary = Keypair::generate();
let agent = Keypair::generate();
let node = Keypair::generate();
let hour = chrono::Utc::now() + chrono::Duration::hours(1);
let resource = format!("gitlawb://repos/{}/myrepo", owner.did());
let cap = || vec![Capability::new(&resource, caps::GIT_PUSH)];
let grandparent = Ucan::issue(&owner, intermediary.did(), cap(), Some(hour)).unwrap();
let parent =
Ucan::delegate(&intermediary, agent.did(), cap(), Some(hour), &grandparent).unwrap();
let invocation = Ucan::delegate(&agent, node.did(), cap(), Some(hour), &parent).unwrap();
let root = invocation.verify_chain().unwrap();
let rec = repo(&owner.did().to_string(), "myrepo");
assert!(ucan_grants_push(
&rec,
&VerifiedUcan {
ucan: invocation,
root
}
));
}
/// The round-ten P1, pinned where it bites. The middleware verifies the chain
/// on the `X-Ucan` header of any signed request before the scope walk runs,
/// so a depth bound that lived only in `chain_grants_push_to` protected
/// nothing: a hand-built header reached the unbounded recursion first. A
/// chain at `MAX_CHAIN_DEPTH` links validates; one link more is refused here,
/// with 401 and a message that names depth, before any authorization runs.
#[test]
fn validate_ucan_chain_stops_at_the_depth_bound() {
use gitlawb_core::ucan::MAX_CHAIN_DEPTH;
let owner = Keypair::generate();
let node = Keypair::generate();
let hour = chrono::Utc::now() + chrono::Duration::hours(1);
let resource = format!("gitlawb://repos/{}/myrepo", owner.did());
let cap = || vec![Capability::new(&resource, caps::GIT_PUSH)];
// owner -> k1 -> ... -> signer: MAX_CHAIN_DEPTH - 1 links, all concrete,
// all signed, all expiring. Only the length is in question.
let mut signer = Keypair::generate();
let mut cur = Ucan::issue(&owner, signer.did(), cap(), Some(hour)).unwrap();
for _ in 2..MAX_CHAIN_DEPTH {
let next = Keypair::generate();
cur = Ucan::delegate(&signer, next.did(), cap(), Some(hour), &cur).unwrap();
signer = next;
}
// Wrapped into an invocation for the node: exactly MAX_CHAIN_DEPTH links.
let at_bound = Ucan::delegate(&signer, node.did(), cap(), Some(hour), &cur).unwrap();
let verified = validate_ucan_chain(&at_bound.encode().unwrap(), &node.did(), &signer.did())
.expect("a chain at the bound must still validate");
assert_eq!(verified.root, owner.did());
// One more link and it is past the bound.
let next = Keypair::generate();
let deeper = Ucan::delegate(&signer, next.did(), cap(), Some(hour), &cur).unwrap();
let past_bound = Ucan::delegate(&next, node.did(), cap(), Some(hour), &deeper).unwrap();
let (status, body) =
match validate_ucan_chain(&past_bound.encode().unwrap(), &node.did(), &next.did()) {
Ok(_) => panic!("a chain past the bound must be refused, not walked"),
Err(refusal) => refusal,
};
assert_eq!(status, StatusCode::UNAUTHORIZED);
let message = body.0["message"].as_str().unwrap_or_default();
assert!(
message.contains("deeper than"),
"the refusal must name depth as the reason: {message}"
);
}
#[test]
fn refuses_a_malformed_resource_uri() {
let rec = repo(&owner_full(), "myrepo");
for bad in [
"",
"myrepo",
"https://repos/x/myrepo",
"gitlawb://repos/myrepo",
] {
let v = verified(&owner_full(), vec![Capability::new(bad, caps::GIT_PUSH)]);
assert!(!ucan_grants_push(&rec, &v), "{bad} must not grant push");
}
}
}
+535 -13
View File
@@ -11,23 +11,34 @@ use uuid::Uuid;
use crate::db::RefCertificate; use crate::db::RefCertificate;
use crate::state::AppState; use crate::state::AppState;
/// Issue a signed ref-update certificate for a successful push. /// Issue a signed ref-update certificate for a successful push. The
/// live receive-pack handler calls this on every successful push.
/// Build the v1 signing payload — the EXACT bytes the node signs and
/// gl's `verify_signature` reconstructs. This is the single source
/// of truth shared by `issue_ref_certificate` and the frozen
/// canonical-form tests below; drift here is what would break every
/// existing cert.
/// ///
/// Builds a canonical JSON payload, signs it with the node's Ed25519 key, /// Field order and key set are fixed: a default `serde_json::Value`
/// persists the certificate, and returns it. /// serializes `Map<String, Value>` with sorted keys, so any change to
pub async fn issue_ref_certificate( /// the literal is observable as a different byte sequence on the
state: &AppState, /// wire.
///
/// v1 is the unversioned legacy shape: no `version` key in the signed
/// JSON. From v2 onward the version belongs INSIDE the signed bytes
/// (see `v2_signing_payload`): leaving it as an unsigned sibling
/// would let a future v2 cert be downgraded to v1 and still verify
/// cleanly, since the bytes would be identical.
pub(crate) fn v1_signing_payload(
repo_id: &str, repo_id: &str,
ref_name: &str, ref_name: &str,
old_sha: &str, old_sha: &str,
new_sha: &str, new_sha: &str,
pusher_did: &str, pusher_did: &str,
) -> Result<RefCertificate> { node_did: &str,
let node_did = state.node_did.to_string(); issued_at: &str,
let issued_at = Utc::now().to_rfc3339(); ) -> serde_json::Value {
serde_json::json!({
// Build the canonical signing payload.
let payload = serde_json::json!({
"repo_id": repo_id, "repo_id": repo_id,
"ref": ref_name, "ref": ref_name,
"old": old_sha, "old": old_sha,
@@ -35,13 +46,180 @@ pub async fn issue_ref_certificate(
"pusher": pusher_did, "pusher": pusher_did,
"node": node_did, "node": node_did,
"ts": issued_at, "ts": issued_at,
}); })
}
/// Build the v2 signing payload — the future shape for versioned
/// certs. Identical to v1 except the wire `version` is part of the
/// signed bytes, so stripping or flipping the `version` column
/// invalidates the signature instead of verifying cleanly under the
/// other version's path.
///
/// NOT yet issued: `issue_ref_certificate` still stamps v1 because
/// the shipped `gl` verifier only supports v1. This builder exists
/// to pin the downgrade-resistant shape now, before any v2 cert is
/// ever signed.
#[allow(dead_code)]
pub(crate) fn v2_signing_payload(
repo_id: &str,
ref_name: &str,
old_sha: &str,
new_sha: &str,
pusher_did: &str,
node_did: &str,
issued_at: &str,
) -> serde_json::Value {
serde_json::json!({
"repo_id": repo_id,
"ref": ref_name,
"old": old_sha,
"new": new_sha,
"pusher": pusher_did,
"node": node_did,
"ts": issued_at,
"version": 2,
})
}
/// Issue a signed ref-update certificate for a successful push.
///
/// `cert_id` is the deterministic id derived from `(request_id,
/// ref_name)` (see [`crate::db::ref_cert_id_for`]). It is required so
/// the recovery drain and the live handler produce the same primary
/// key: a live push followed by a recovery pass collapses to a
/// single cert row, and a re-push to the same `(repo, ref)` updates
/// the existing row's `old_sha` / `new_sha` / `pusher_did` /
/// `issued_at` / `signature` to the new transition while preserving
/// the original `id` (the `insert_ref_certificate` upsert is
/// keyed on `(repo_id, ref_name)` and only updates fields when the
/// new `issued_at` is strictly greater).
///
/// #26 Split PR 1 P1-B: the live handler routes through this
/// function (the upsert). After the reviewer-1 round-2 fix, the
/// recovery drain also routes through this function (P1: refresh a
/// stale cert), so both paths use the same deterministic `cert_id`
/// and the same upsert. A re-pass is always safe:
///
/// - Live handler → live upsert: re-push updates the row, preserves
/// the original `id`. The contract pinned by
/// `insert_ref_certificate_upserts_on_repo_ref` is restored.
/// - Live handler → recovery: live's `ON CONFLICT (id) DO UPDATE`
/// preserves the original `id`; the recovery's same upsert is
/// a no-op for an equal-`issued_at` re-run and a refresh for a
/// strictly-newer one.
/// - Recovery → live handler: the recovery wrote a row with the
/// deterministic `id`; the live upsert (which preserves `id` and
/// only updates other fields when `issued_at` is strictly newer)
/// is a no-op for an equal-`issued_at` re-run and a refresh for
/// a strictly-newer one.
#[allow(dead_code)] // round-trip test in db/mod.rs pins the upsert contract; the live path and the drain use issue_ref_certificate_with_issued_at
pub async fn issue_ref_certificate(
state: &AppState,
repo_id: &str,
ref_name: &str,
old_sha: &str,
new_sha: &str,
pusher_did: &str,
cert_id: &str,
) -> Result<RefCertificate> {
issue_ref_certificate_with_issued_at(
state, repo_id, ref_name, old_sha, new_sha, pusher_did, cert_id, None,
)
.await
}
/// #26 Split PR 1 round 4 — variant that lets the caller stamp the
/// cert's `issued_at` with a transition-time timestamp instead of
/// `Utc::now()`. The recovery drain passes the persisted
/// `row.created_at` so a replay after a later live cert does not
/// outrank the live cert in the `EXCLUDED.issued_at >
/// ref_certificates.issued_at` upsert guard.
///
/// The live handler uses the default `issue_ref_certificate` (no
/// override), which keeps `Utc::now()` — the reviewer's invariant
/// is that `issued_at` reflects the transition time, and for a
/// live push the transition time and the wall-clock are the same.
///
/// `issued_at_override` is honored verbatim; passing a value not in
/// RFC 3339 form is a logic bug (the upsert will mis-order), so
/// callers must use the row's persisted `created_at`.
///
/// # Clippy allow — too many arguments
/// This is the explicit "stamp a transition-time `issued_at`"
/// variant of `issue_ref_certificate`. The drain
/// (`durable_outbox::derive_one`) is the in-crate caller; the
/// test `replay_of_stale_row_does_not_overwrite_live_cert_b` pins
/// the contract that a recovery replay's `issued_at` does NOT
/// outrank a later live cert. Adding a struct-arg would be a
/// larger refactor for two callers (live + drain) and obscure the
/// parallel to `issue_ref_certificate` (which is `#[allow]`'d for
/// the same reason historically).
#[allow(clippy::too_many_arguments)]
pub async fn issue_ref_certificate_with_issued_at(
state: &AppState,
repo_id: &str,
ref_name: &str,
old_sha: &str,
new_sha: &str,
pusher_did: &str,
cert_id: &str,
issued_at_override: Option<String>,
) -> Result<RefCertificate> {
let cert = build_ref_certificate(
state,
repo_id,
ref_name,
old_sha,
new_sha,
pusher_did,
Some(cert_id.to_string()),
issued_at_override,
)
.await?;
state.db.insert_ref_certificate(&cert).await
}
/// Shared cert construction: build the JSON payload, sign it with the
/// node key, and assemble the `RefCertificate` row. `cert_id_override`
/// lets the recovery path plug in a deterministic id; the live path
/// passes `None` and gets a fresh UUID. `issued_at_override` lets
/// the recovery path stamp the cert with the original transition
/// time so the upsert's `issued_at > issued_at` guard correctly
/// orders transitions regardless of write order.
#[allow(clippy::too_many_arguments)]
async fn build_ref_certificate(
state: &AppState,
repo_id: &str,
ref_name: &str,
old_sha: &str,
new_sha: &str,
pusher_did: &str,
cert_id_override: Option<String>,
issued_at_override: Option<String>,
) -> Result<RefCertificate> {
let node_did = state.node_did.to_string();
// P1 (reviewer-1 round 4): when the caller passes a transition-
// time `issued_at` (the recovery drain passes `row.created_at`),
// use it verbatim so the upsert's per-column guard
// `EXCLUDED.issued_at > ref_certificates.issued_at` correctly
// orders transitions regardless of write order. The live handler
// passes `None` and gets `Utc::now()` — for a live push the
// transition time and the wall-clock are the same.
let issued_at = issued_at_override.unwrap_or_else(|| Utc::now().to_rfc3339());
// Build the canonical signing payload via the shared builder so
// the frozen-vector test below and the live signer cannot drift
// apart — a regression in either side fails both.
let payload = v1_signing_payload(
repo_id, ref_name, old_sha, new_sha, pusher_did, &node_did, &issued_at,
);
let payload_bytes = serde_json::to_vec(&payload)?; let payload_bytes = serde_json::to_vec(&payload)?;
let signature = state.node_keypair.sign_b64(&payload_bytes); let signature = state.node_keypair.sign_b64(&payload_bytes);
let id = cert_id_override.unwrap_or_else(|| Uuid::new_v4().to_string());
let cert = RefCertificate { let cert = RefCertificate {
id: Uuid::new_v4().to_string(), id,
repo_id: repo_id.to_string(), repo_id: repo_id.to_string(),
ref_name: ref_name.to_string(), ref_name: ref_name.to_string(),
old_sha: old_sha.to_string(), old_sha: old_sha.to_string(),
@@ -50,9 +228,353 @@ pub async fn issue_ref_certificate(
node_did, node_did,
signature, signature,
issued_at, issued_at,
// #26 Split PR 3: the wire-format version. New certs stamp
// v1 — the pre-versioning 7-field payload with no `version`
// key in the signed JSON — because the shipped `gl`
// verifier only supports v1. Stamping v2 here would issue
// certificates the shipped client refuses (round-3
// finding: node pinned `version == 2` while gl pinned
// "2 is refused", with nothing exercising the
// composition). v2 is reserved for the future shape
// defined by `v2_signing_payload`, which carries the
// version INSIDE the signed bytes; land the v2 verify
// path before stamping v2.
version: 1,
}; };
// Persist and return the row as it exists in the database (on a // Persist and return the row as it exists in the database (on a
// conflict the existing row survives when it is newer). // conflict the existing row survives when it is newer).
state.db.insert_ref_certificate(&cert).await state.db.insert_ref_certificate(&cert).await
} }
#[cfg(test)]
mod v1_payload_tests {
//! #26 Split PR 3 — cert payload version compat.
//!
//! The v1 cert payload is the frozen canonical byte form the
//! node signs. gl's `verify_signature` reconstructs the same
//! payload and verifies. Any drift in field order, key set, or
//! whitespace breaks every existing cert. The tests here pin
//! the v1 payload shape, the round-trip, the cert id
//! determinism per the v1 version, and the v2 read forward
//! compat (an unknown version refuses to verify rather than
//! guessing).
//!
//! Reviewer 1 finding: the live signer (`issue_ref_certificate`)
//! and the frozen-vector test (`v1_payload_matches_frozen_canonical_form`)
//! previously held two separate `serde_json::json!` literals,
//! so a regression in the signer could not fail this test.
//! Both now go through the shared `v1_signing_payload` builder.
use super::{v1_signing_payload, v2_signing_payload};
use crate::db::RefCertificate;
use gitlawb_core::identity::Keypair;
/// v1 payload is the same shape the legacy code signed, byte
/// for byte. Reverting this assertion to the v2 shape (which
/// includes a `version` key) is what the versioned format
/// forbids: an old client reading a v1 cert must not see a
/// `version` key in the signed JSON.
///
/// The literal is built via the shared `v1_signing_payload`
/// helper that `issue_ref_certificate` also uses, so the live
/// signer and the frozen vector share one source of truth.
#[test]
fn v1_payload_matches_frozen_canonical_form() {
let payload = v1_signing_payload(
"repo-1",
"refs/heads/main",
&"0".repeat(40),
&"a".repeat(40),
"did:key:z6MkPusher",
"did:key:z6MkNode",
"2026-07-22T00:00:00+00:00",
);
let frozen = format!(
r#"{{"new":"{}","node":"did:key:z6MkNode","old":"{}","pusher":"did:key:z6MkPusher","ref":"refs/heads/main","repo_id":"repo-1","ts":"2026-07-22T00:00:00+00:00"}}"#,
"a".repeat(40),
"0".repeat(40),
);
assert_eq!(serde_json::to_string(&payload).unwrap(), frozen);
}
/// Round-trip: sign the v1 payload with a keypair, build a
/// RefCertificate with version: 1, and verify the structure.
/// This is the construction the live code does on every push.
///
/// Uses the shared builder for both halves so a regression in
/// the live signer (adding/removing/renaming a field) cannot
/// pass this test by accident.
#[test]
fn v1_ref_certificate_structure_is_well_formed() {
let kp = Keypair::generate();
let node_did = kp.did().as_str().to_string();
let payload = v1_signing_payload(
"repo-1",
"refs/heads/main",
&"0".repeat(40),
&"a".repeat(40),
"did:key:z6MkPusher",
&node_did,
"2026-07-22T00:00:00+00:00",
);
let payload_bytes = serde_json::to_vec(&payload).unwrap();
let sig = kp.sign_b64(&payload_bytes);
let cert = RefCertificate {
id: "cert-id".into(),
repo_id: "repo-1".into(),
ref_name: "refs/heads/main".into(),
old_sha: "0".repeat(40),
new_sha: "a".repeat(40),
pusher_did: "did:key:z6MkPusher".into(),
node_did: node_did.clone(),
signature: sig.clone(),
issued_at: "2026-07-22T00:00:00+00:00".into(),
version: 1,
};
assert_eq!(cert.version, 1, "new certs carry version: 1");
// The signed payload reconstructs identically: gl's
// verify_signature would build the same JSON, hash the
// same bytes, and verify the same signature.
let reconstructed = v1_signing_payload(
&cert.repo_id,
&cert.ref_name,
&cert.old_sha,
&cert.new_sha,
&cert.pusher_did,
&cert.node_did,
&cert.issued_at,
);
let reconstructed_bytes = serde_json::to_vec(&reconstructed).unwrap();
assert_eq!(
reconstructed_bytes, payload_bytes,
"the round-trip serialization must be byte-identical; \
this is what makes gl's verify_signature succeed"
);
// Round 3 (P3 reviewer): byte-equality alone never calls
// `identity::verify`, so a bad signature helper would pass.
// Verify the computed signature against the reconstructed
// bytes — the same check gl performs.
let vk = node_did
.parse::<gitlawb_core::did::Did>()
.unwrap()
.to_verifying_key()
.unwrap();
use base64::{engine::general_purpose::URL_SAFE_NO_PAD, Engine};
let sig_bytes: [u8; 64] = URL_SAFE_NO_PAD
.decode(&cert.signature)
.unwrap()
.try_into()
.unwrap();
gitlawb_core::identity::verify(&vk, &reconstructed_bytes, &sig_bytes)
.expect("well-formed v1 cert signature must verify");
}
/// Reviewer 1 finding: the live signer must share the
/// canonical-form literal so a regression in either side
/// fails both. This test pins the shared builder's output
/// against the literal a future v2 byte stream would have
/// to break — if someone adds a `version` key to
/// `v1_signing_payload`, every existing cert breaks verify,
/// and this test is one of the canaries that fires.
#[test]
fn v1_signing_payload_has_no_version_key() {
let payload = v1_signing_payload(
"repo-1",
"refs/heads/main",
"oldsha",
"newsha",
"did:key:z6MkPusher",
"did:key:z6MkNode",
"2026-07-22T00:00:00+00:00",
);
let obj = payload.as_object().expect("payload is a JSON object");
assert!(
!obj.contains_key("version"),
"v1 signing payload must not carry a `version` key — adding \
one changes the bytes and breaks every existing cert's \
signature. v2 certs build a different payload, not this one"
);
// Pin the exact set so a future field addition is caught.
let expected: std::collections::BTreeSet<&str> =
["new", "node", "old", "pusher", "ref", "repo_id", "ts"]
.into_iter()
.collect();
let actual: std::collections::BTreeSet<&str> = obj.keys().map(String::as_str).collect();
assert_eq!(actual, expected, "v1 payload key set is frozen");
}
/// Round 3 (P2 reviewer): from v2 onward the version belongs
/// INSIDE the signed bytes. v1 stays the unversioned legacy
/// shape; v2 carries `"version": 2` in the payload so a
/// version flip without re-signing invalidates the signature
/// instead of verifying cleanly under the other path.
#[test]
fn v2_signing_payload_binds_version_and_resists_downgrade() {
let kp = Keypair::generate();
let node_did = kp.did().as_str().to_string();
let ts = "2026-07-22T00:00:00+00:00";
let v1 = v1_signing_payload(
"repo-1",
"refs/heads/main",
"oldsha",
"newsha",
"did:key:z6MkPusher",
&node_did,
ts,
);
let v2 = v2_signing_payload(
"repo-1",
"refs/heads/main",
"oldsha",
"newsha",
"did:key:z6MkPusher",
&node_did,
ts,
);
let v2_obj = v2.as_object().expect("v2 payload is a JSON object");
assert_eq!(
v2_obj.get("version"),
Some(&serde_json::json!(2)),
"v2 payload must carry the version inside the signed bytes"
);
let v1_bytes = serde_json::to_vec(&v1).unwrap();
let v2_bytes = serde_json::to_vec(&v2).unwrap();
assert_ne!(
v1_bytes, v2_bytes,
"v1 and v2 payloads must differ, otherwise a version flip is cryptographically invisible"
);
// A signature over v1 must NOT verify as v2 and vice versa.
let vk = node_did
.parse::<gitlawb_core::did::Did>()
.unwrap()
.to_verifying_key()
.unwrap();
use base64::{engine::general_purpose::URL_SAFE_NO_PAD, Engine};
let sig_v1 = kp.sign_b64(&v1_bytes);
let sig_v1_bytes: [u8; 64] = URL_SAFE_NO_PAD.decode(&sig_v1).unwrap().try_into().unwrap();
assert!(
gitlawb_core::identity::verify(&vk, &v1_bytes, &sig_v1_bytes).is_ok(),
"v1 signature must verify under v1 bytes"
);
assert!(
gitlawb_core::identity::verify(&vk, &v2_bytes, &sig_v1_bytes).is_err(),
"a v1 signature presented as v2 (downgrade/upgrade flip) must not verify"
);
}
/// The live issuer stamps the version the shipped client can
/// verify. The gl client refuses to verify v2 certs explicitly;
/// this test pins that the version the ISSUING PATH claims
/// agrees with the payload shape it actually signs.
///
/// Round 2: the prior form built a `RefCertificate` literal with
/// `version: 1` and asserted `version == 1` — true no matter what the
/// signer does, so flipping the issuer's version (or its payload shape)
/// left it green. This one drives the actual
/// `issue_ref_certificate` so a regression in the issuer flips the test.
/// The DB row is created against `test_support::test_state`, which
/// gives us a real `AppState` with a real node keypair.
///
/// Round 3: the issuer stamps v1 because the shipped `gl`
/// verifier only supports v1 (stamping v2 shipped certs the
/// client refuses). The binding to the live issuer is kept —
/// flipping the stamp still turns this test red — without
/// changing what production issues.
#[sqlx::test]
#[allow(clippy::async_yields_async)]
async fn issuer_stamps_v1_over_v1_payload(pool: sqlx::PgPool) {
let state = crate::test_support::test_state(pool.clone()).await;
let repo_id = "repo-issuer-observe";
let ref_name = "refs/heads/main";
let old = "0".repeat(40);
let new = "a".repeat(40);
let pusher = "did:key:z6MkPusher";
let cert_id = "cert-issuer-observe";
// Drive the live issuer and read the persisted row back by
// id, so the test covers the stored `version` column and
// not just the in-process return value.
let cert = crate::cert::issue_ref_certificate(
&state, repo_id, ref_name, &old, &new, pusher, cert_id,
)
.await
.expect("issue_ref_certificate must succeed");
// The cert returned by the issuer is the source of truth
// for both the version claim and the signed bytes.
assert_eq!(cert.repo_id, repo_id);
assert_eq!(cert.ref_name, ref_name);
// Cross-check 1: the issuer's claimed version must match
// what the live function stamps. This assertion is bound
// to the live function, not a hand-built literal:
// flipping the stamp in `issue_ref_certificate` breaks it
// through the actual call path.
assert_eq!(
cert.version, 1,
"the live issuer must stamp v1 until a v2 verifier ships; \
flipping the stamp breaks this assertion through the actual call path"
);
// Cross-check 1b: the persisted row carries the same stamp.
let stored = state
.db
.get_ref_certificate(&cert.id)
.await
.expect("persisted cert must be readable")
.expect("persisted cert must exist");
assert_eq!(
stored.version, cert.version,
"the stored version column must match the issued claim"
);
// Cross-check 2: the signature on the cert must verify
// against the v1 payload builder's bytes, not against some
// other shape the issuer might have introduced. This is
// what gl's `verify_signature` does for a v1 cert: rebuild
// the payload from the cert's own fields and verify the
// signature. If the issuer ever signs a different shape
// while still claiming v1, every shipped client breaks —
// and so does this.
let rebuilt = v1_signing_payload(
&cert.repo_id,
&cert.ref_name,
&cert.old_sha,
&cert.new_sha,
&cert.pusher_did,
&cert.node_did,
&cert.issued_at,
);
let vk = cert
.node_did
.parse::<gitlawb_core::did::Did>()
.unwrap()
.to_verifying_key()
.unwrap();
use base64::{engine::general_purpose::URL_SAFE_NO_PAD, Engine};
let sig_bytes: [u8; 64] = URL_SAFE_NO_PAD
.decode(&cert.signature)
.unwrap()
.try_into()
.unwrap();
gitlawb_core::identity::verify(&vk, &serde_json::to_vec(&rebuilt).unwrap(), &sig_bytes)
.expect("a v1 cert's signature must verify against the v1 payload shape");
// Cross-check 3: the v1 signed payload must NOT contain a
// version key — v2 is the future shape that adds one inside
// the signed bytes (see `v2_signing_payload`). The v1
// payload (which the cert is signed over) is frozen at 7
// fields. A change that adds `version` to the v1 payload
// breaks every existing cert and this test.
let obj = rebuilt
.as_object()
.expect("rebuilt v1 payload is a JSON object");
assert!(
!obj.contains_key("version"),
"the v1 signing payload must not carry a version key — \
adding one changes the bytes and breaks every existing \
cert's signature"
);
}
}
+457 -10
View File
@@ -66,6 +66,12 @@ pub struct Config {
#[arg(long, env = "GITLAWB_PUBLIC_URL")] #[arg(long, env = "GITLAWB_PUBLIC_URL")]
pub public_url: Option<String>, pub public_url: Option<String>,
/// Base URL for the web view of repos on this node (e.g. https://gitlawb.com).
/// When set, `GET /` advertises it as `web_url` so the CLI can print a
/// working `View:` link. Omit for nodes that have no web front-end.
#[arg(long, env = "GITLAWB_WEB_URL")]
pub web_url: Option<String>,
/// Comma-separated list of bootstrap peer URLs to announce to on startup /// Comma-separated list of bootstrap peer URLs to announce to on startup
#[arg(long, env = "GITLAWB_BOOTSTRAP_PEERS", value_delimiter = ',')] #[arg(long, env = "GITLAWB_BOOTSTRAP_PEERS", value_delimiter = ',')]
pub bootstrap_peers: Vec<String>, pub bootstrap_peers: Vec<String>,
@@ -146,6 +152,57 @@ pub struct Config {
#[arg(long, env = "GITLAWB_AUTO_SYNC", default_value_t = false)] #[arg(long, env = "GITLAWB_AUTO_SYNC", default_value_t = false)]
pub auto_sync: bool, pub auto_sync: bool,
/// Periodically fetch configured INBOUND repositories from their external
/// HTTPS upstream. Disabled by default: enabling this also requires
/// `GITLAWB_ENFORCE_OWNER_PUSH=true`, so an upstream-authoritative mirror
/// cannot accept writes from arbitrary signed identities between fetches.
#[arg(long, env = "GITLAWB_UPSTREAM_MIRROR_ENABLED", default_value_t = false)]
pub upstream_mirror_enabled: bool,
/// Pause between complete upstream-mirror scans. A scan is sequential and
/// paginated; the pause begins only after the prior scan has finished, so a
/// slow forge cannot create overlapping worker cycles.
#[arg(
long,
env = "GITLAWB_UPSTREAM_MIRROR_INTERVAL_SECS",
default_value_t = 300,
value_parser = clap::value_parser!(u64).range(5..=86_400)
)]
pub upstream_mirror_interval_secs: u64,
/// Maximum wall-clock time for one external `git fetch`. The child runs in
/// its own process group and is fully reaped on timeout. The preflight
/// URL-rewrite and local-config probes run on their own short bounds and
/// do not consume this budget.
#[arg(
long,
env = "GITLAWB_UPSTREAM_MIRROR_FETCH_TIMEOUT_SECS",
default_value_t = 600,
value_parser = clap::value_parser!(u64).range(1..=GIT_SERVICE_TIMEOUT_SECS_MAX)
)]
pub upstream_mirror_fetch_timeout_secs: u64,
/// Number of durable INBOUND rows read per database page. Every page is
/// processed before the scan sleeps; this bounds query memory without
/// permanently starving repositories beyond the first page.
#[arg(
long,
env = "GITLAWB_UPSTREAM_MIRROR_PAGE_SIZE",
default_value_t = 100,
value_parser = clap::builder::RangedU64ValueParser::<usize>::new().range(1..=1_000)
)]
pub upstream_mirror_page_size: usize,
/// Enable the periodic reconciliation sweep that re-derives pin/seal sets
/// and fills durability gaps. Defaults to true; set to false to disable
/// the sweep even when a pin backend (IPFS/Pinata) is configured.
#[arg(
long,
env = "GITLAWB_RECONCILIATION_SWEEP",
default_value_t = true,
action = clap::ArgAction::Set
)]
pub reconciliation_sweep: bool,
/// Irys URL for Arweave permanent anchoring. /// Irys URL for Arweave permanent anchoring.
/// Leave empty to disable. Use https://devnet.irys.xyz for free devnet. /// Leave empty to disable. Use https://devnet.irys.xyz for free devnet.
#[arg(long, env = "GITLAWB_IRYS_URL", default_value = "")] #[arg(long, env = "GITLAWB_IRYS_URL", default_value = "")]
@@ -221,12 +278,35 @@ pub struct Config {
#[arg(long, env = "GITLAWB_METRICS_ADDR", default_value = "")] #[arg(long, env = "GITLAWB_METRICS_ADDR", default_value = "")]
pub metrics_addr: String, pub metrics_addr: String,
/// Maximum time to wait for in-flight requests to drain on shutdown, in /// Shutdown grace setting, in seconds. Currently not enforced: the value
/// seconds. After this elapses, the server returns 503 to anything still /// is logged at startup and axum drains in-flight requests on the shutdown
/// in flight and exits. Default: 30s. /// signal with no upper bound. Default: 30s.
#[arg(long, env = "GITLAWB_SHUTDOWN_GRACE_SECS", default_value_t = 30)] #[arg(long, env = "GITLAWB_SHUTDOWN_GRACE_SECS", default_value_t = 30)]
pub shutdown_grace_secs: u64, pub shutdown_grace_secs: u64,
/// Maximum number of creation requests (repo create, issue create, PR
/// create, register, fork) allowed per DID within the rate-limit window.
/// 0 is rejected at parse time: it would silently disable the limiter
/// (RateLimiter treats max_requests == 0 as allow-all), not block every
/// creation request.
#[arg(
long,
env = "GITLAWB_RATE_LIMIT_MAX",
default_value_t = 10,
value_parser = clap::value_parser!(u32).range(1..)
)]
pub rate_limit_max_requests: u32,
/// Rate-limit window size, in seconds, over which
/// `rate_limit_max_requests` applies. 0 is rejected at parse time: every
/// timestamp would prune immediately, silently disabling the limiter.
#[arg(
long,
env = "GITLAWB_RATE_LIMIT_WINDOW_SECS",
default_value_t = 3600,
value_parser = clap::value_parser!(u64).range(1..)
)]
pub rate_limit_window_secs: u64,
/// Maximum wall-clock time a single served git operation (upload-pack / /// Maximum wall-clock time a single served git operation (upload-pack /
/// receive-pack through `run_git_service`) may run before it is aborted and /// receive-pack through `run_git_service`) may run before it is aborted and
/// its process group torn down, in seconds. Bounds a git that neither /// its process group torn down, in seconds. Bounds a git that neither
@@ -756,6 +836,60 @@ pub struct Config {
value_parser = clap::builder::RangedU64ValueParser::<u64>::new().range(0..=86_400) value_parser = clap::builder::RangedU64ValueParser::<u64>::new().range(0..=86_400)
)] )]
pub pin_repair_sweep_delay_secs: u64, pub pin_repair_sweep_delay_secs: u64,
/// #26 Split PR 1 step 4 — receive-pack queue retention window.
/// Terminal `complete` and `rejected_at_git` rows older than
/// this are eligible for the periodic purge. `quarantined`
/// rows are never purged on a timer. The v30 partial index
/// `idx_receive_pack_requests_completed_at` keeps the scan
/// cheap regardless of the value.
#[arg(
long,
env = "GITLAWB_QUEUE_RETENTION_DAYS",
default_value_t = 7,
value_parser = clap::builder::RangedU64ValueParser::<i64>::new().range(1..=365)
)]
pub queue_retention_days: i64,
/// #26 Split PR 1 step 4 — receive-pack queue purge batch size.
/// Each periodic purge pass deletes at most this many terminal
/// rows per batch. The drain and the purge share the same
/// `DRAIN_PER_PASS_LIMIT` budget; see the spawn function in
/// `main.rs` for the wiring.
#[arg(
long,
env = "GITLAWB_QUEUE_PURGE_BATCH",
default_value_t = 1000,
value_parser = clap::builder::RangedU64ValueParser::<i64>::new().range(1..=100_000)
)]
pub queue_purge_batch: i64,
/// #26 Split PR 1 step 5 — effects-executor retry bound. The
/// drain flips a request to `quarantined` after this many
/// `EffectsOutcome::Retry` returns, closing the
/// infinite-retry DoS window. Default 8 matches the spec; set
/// lower in tests.
#[arg(
long,
env = "GITLAWB_EFFECTS_MAX_ATTEMPTS",
default_value_t = 8,
value_parser = clap::builder::RangedU64ValueParser::<i32>::new().range(1..=1000)
)]
pub effects_max_attempts: i32,
/// Per-client-IP rate limit for the anonymous task read routes
/// (`GET /api/v1/tasks`, `GET /api/v1/tasks/{id}`), in requests per hour.
/// Both are publicly reachable (`optional_signature`). `GET /api/v1/tasks`
/// runs `collect_visible_tasks` and returns a visibility-filtered page;
/// `GET /api/v1/tasks/{id}` runs `get_visible_task` and returns an opaque 404
/// when the task is hidden or missing. These reads can require task,
/// repository, and visibility-rule queries even when no task is returned,
/// so the brake bounds the cost of anonymous probes. Keyed on the resolved
/// client IP via `GITLAWB_TRUSTED_PROXY`. `0` disables. Default: 1200 (a list page
/// followed by per-task reads is a normal client pattern, so this sits above
/// the `/ipfs` budget). GraphQL `tasks` / `task` queries, including WebSocket
/// operations, share this per-IP budget with REST task reads.
#[arg(long, env = "GITLAWB_TASK_READ_RATE_LIMIT", default_value_t = 1200)]
pub task_read_rate_limit: usize,
} }
impl Config { impl Config {
@@ -803,7 +937,13 @@ impl Config {
/// Cross-field boot validation. Single-field ranges are enforced by clap; this /// Cross-field boot validation. Single-field ranges are enforced by clap; this
/// catches combinations that ship a denial-of-service under otherwise-valid /// catches combinations that ship a denial-of-service under otherwise-valid
/// values. Call once at startup and fail fast on `Err`. /// values. Call once at startup and fail fast on `Err`.
pub fn validate(&self) -> Result<(), String> { ///
/// `&mut self` is required so the validator can normalize `web_url` to
/// its canonical form after validation: the configured value is
/// advertised on public `GET /`, and any consumer reading
/// `config.web_url` later must see the same string that was actually
/// validated.
pub fn validate(&mut self) -> Result<(), String> {
// Concurrent git writes pin the dedicated lock pool for their whole // Concurrent git writes pin the dedicated lock pool for their whole
// duration (connection-affine advisory lock in acquire_write). The main // duration (connection-affine advisory lock in acquire_write). The main
// pool no longer carries that occupancy. // pool no longer carries that occupancy.
@@ -820,6 +960,13 @@ impl Config {
Self::DB_LOCK_POOL_NON_PUSH_HEADROOM Self::DB_LOCK_POOL_NON_PUSH_HEADROOM
)); ));
} }
if self.upstream_mirror_enabled && !self.enforce_owner_push {
return Err(
"GITLAWB_UPSTREAM_MIRROR_ENABLED requires GITLAWB_ENFORCE_OWNER_PUSH=true: \
an upstream-authoritative mirror must not accept writes from arbitrary signed identities"
.to_string(),
);
}
let main_floor = Self::DB_POOL_APP_HEADROOM as u64; let main_floor = Self::DB_POOL_APP_HEADROOM as u64;
if (self.db_max_connections as u64) < main_floor { if (self.db_max_connections as u64) < main_floor {
return Err(format!( return Err(format!(
@@ -920,14 +1067,150 @@ impl Config {
.map_err(|e| e.to_string())?; .map_err(|e| e.to_string())?;
} }
// GITLAWB_WEB_URL is advertised in GET / so the CLI can print a working
// View: link. Clap maps "" to Some("") which would produce a broken URL;
// reject early instead of serving a malformed link. Non-blank values must
// additionally parse as an absolute http(s) URL — the CLI appends
// `/{owner}/{repo}` to it, so scheme-less hosts ("gitlawb.com") and other
// garbage would render links no browser can follow.
if let Some(raw) = &self.web_url {
match validate_web_url(raw) {
Ok(()) => {}
Err(reason) => {
return Err(format!(
"GITLAWB_WEB_URL {reason} — \
set it to an absolute URL like https://gitlawb.com or leave it unset."
));
}
}
// Normalize to one canonical form so every consumer (the regular
// and degraded `GET /` handlers, anyone who reads
// `config.web_url` directly) sees the same string that was
// actually validated. The CLI's own defensive trim would paper
// over the divergence on the `View:` line, but that hides the
// bug from any other consumer that parses the value verbatim.
let canonical = raw
.trim()
.parse::<url::Url>()
.map(|u| u.to_string())
.map_err(|_| "is not a valid absolute URL".to_string())?;
if canonical != *raw {
self.web_url = Some(canonical);
}
}
Ok(()) Ok(())
} }
} }
/// Validate a `web_url` value for use as a browser-reachable base URL.
/// Blank values are rejected (clap maps `--web-url ""` to `Some("")`), and
/// non-blank values must parse as absolute `http`/`https` URLs with no
/// query, fragment, or userinfo — the CLI treats this as a string prefix
/// to append paths to, so anything else produces links no browser can
/// follow (`?a=1/owner/repo` puts the repo path inside the query string);
/// and userinfo is rejected outright because the configured value is
/// published on public `GET /` and reprinted on the CLI's `View:` line,
/// so an accidentally configured password would land in public metadata,
/// terminal scrollback, and CI logs.
pub(crate) fn validate_web_url(raw: &str) -> Result<(), String> {
if raw.trim().is_empty() {
return Err("must not be empty or whitespace-only".into());
}
let parsed: url::Url = raw
.trim()
.parse()
.map_err(|_| "is not a valid absolute URL".to_string())?;
match parsed.scheme() {
"http" | "https" => {}
other => return Err(format!("must use http or https, got '{other}:'")),
}
// url::Url cannot represent a URL without a host for http/https schemes, so
// reaching here guarantees an absolute browser-usable base.
if parsed.query().is_some() {
return Err(
"must not contain a query string — it is used as a path prefix for View links".into(),
);
}
if parsed.fragment().is_some() {
return Err(
"must not contain a fragment — it is used as a path prefix for View links".into(),
);
}
if !parsed.username().is_empty() || parsed.password().is_some() {
return Err(
"must not contain a username or password — it is advertised publicly and printed on the View: line"
.into(),
);
}
// Bidi/control bytes can hide behind percent-encoding: a literal scan of
// the wire form misses `.../%E2%80%AE` even though a browser decodes it
// into a reordering character on the rendered View: link. Check the
// decoded host and path before the value is advertised on GET / (INV-6).
let host = parsed.host_str().unwrap_or("");
let decoded_path = percent_encoding::percent_decode_str(parsed.path()).decode_utf8_lossy();
if host
.chars()
.chain(decoded_path.chars())
.any(|c| c.is_control() || gitlawb_core::sanitize::is_bidi_format(c))
{
return Err(
"must not contain control or bidi characters, even percent-encoded — the View: line is terminal output"
.into(),
);
}
Ok(())
}
#[cfg(test)] #[cfg(test)]
mod tests { mod tests {
use super::*; use super::*;
#[test]
fn upstream_mirror_worker_is_default_off_and_bounded() {
let mut config = Config::parse_from(["gitlawb-node"]);
assert!(!config.upstream_mirror_enabled);
assert_eq!(config.upstream_mirror_interval_secs, 300);
assert_eq!(config.upstream_mirror_fetch_timeout_secs, 600);
assert_eq!(config.upstream_mirror_page_size, 100);
config.validate().unwrap();
assert!(
Config::try_parse_from(["gitlawb-node", "--upstream-mirror-interval-secs", "4",])
.is_err()
);
assert!(
Config::try_parse_from(["gitlawb-node", "--upstream-mirror-page-size", "0",]).is_err()
);
}
#[test]
fn upstream_mirror_worker_requires_owner_push_enforcement() {
// enforce_owner_push defaults true on main, so the only unsafe shape
// is the explicit opt-out: mirror enabled plus enforcement refused.
let mut unsafe_config = Config::parse_from([
"gitlawb-node",
"--upstream-mirror-enabled",
"--enforce-owner-push",
"false",
]);
let error = unsafe_config.validate().unwrap_err();
assert!(error.contains("GITLAWB_ENFORCE_OWNER_PUSH=true"));
Config::parse_from([
"gitlawb-node",
"--upstream-mirror-enabled",
"--enforce-owner-push",
"true",
])
.validate()
.unwrap();
// The default posture alone also validates: enforcement is on.
Config::parse_from(["gitlawb-node", "--upstream-mirror-enabled"])
.validate()
.unwrap();
}
#[test] #[test]
fn lock_pool_size_defaults_to_40_and_rejects_zero() { fn lock_pool_size_defaults_to_40_and_rejects_zero() {
assert_eq!( assert_eq!(
@@ -1156,6 +1439,30 @@ mod tests {
); );
} }
#[test]
fn queue_lifecycle_knobs_default_conservatively() {
let c = Config::parse_from(["gitlawb-node"]);
// 7-day retention matches the v30 partial index comment
// and the spec at .gravirei/plans/state-model-durable-post-receive.md.
assert_eq!(c.queue_retention_days, 7);
// 1000 rows per pass matches DRAIN_PER_PASS_LIMIT in durable_outbox.
assert_eq!(c.queue_purge_batch, 1000);
assert_eq!(
Config::parse_from(["gitlawb-node", "--queue-retention-days", "30"])
.queue_retention_days,
30
);
assert!(Config::try_parse_from(["gitlawb-node", "--queue-retention-days", "0"]).is_err());
assert!(Config::try_parse_from(["gitlawb-node", "--queue-retention-days", "366"]).is_err());
assert_eq!(
Config::parse_from(["gitlawb-node", "--queue-purge-batch", "500"]).queue_purge_batch,
500
);
assert!(Config::try_parse_from(["gitlawb-node", "--queue-purge-batch", "0"]).is_err());
}
#[test] #[test]
fn ipfs_walk_per_source_defaults_and_rejects_out_of_range() { fn ipfs_walk_per_source_defaults_and_rejects_out_of_range() {
assert_eq!(Config::parse_from(["gitlawb-node"]).ipfs_walk_per_source, 4); assert_eq!(Config::parse_from(["gitlawb-node"]).ipfs_walk_per_source, 4);
@@ -1566,7 +1873,7 @@ mod tests {
.expect("default config must validate"); .expect("default config must validate");
// An under-sized lock pool relative to the push cap is rejected. // An under-sized lock pool relative to the push cap is rejected.
let under_lock = Config::parse_from([ let mut under_lock = Config::parse_from([
"gitlawb-node", "gitlawb-node",
"--db-lock-pool-max-connections", "--db-lock-pool-max-connections",
"16", "16",
@@ -1579,7 +1886,7 @@ mod tests {
); );
// Exactly at the push cap without non-push headroom is rejected. // Exactly at the push cap without non-push headroom is rejected.
let push_only = Config::parse_from([ let mut push_only = Config::parse_from([
"gitlawb-node", "gitlawb-node",
"--db-lock-pool-max-connections", "--db-lock-pool-max-connections",
"32", "32",
@@ -1592,7 +1899,7 @@ mod tests {
); );
// Main pool can be smaller than pushes + headroom when the lock pool carries writes. // Main pool can be smaller than pushes + headroom when the lock pool carries writes.
let split = Config::parse_from([ let mut split = Config::parse_from([
"gitlawb-node", "gitlawb-node",
"--db-max-connections", "--db-max-connections",
"16", "16",
@@ -1607,13 +1914,153 @@ mod tests {
); );
// Main pool still needs the app headroom floor. // Main pool still needs the app headroom floor.
let under_main = Config::parse_from(["gitlawb-node", "--db-max-connections", "4"]); let mut under_main = Config::parse_from(["gitlawb-node", "--db-max-connections", "4"]);
assert!( assert!(
under_main.validate().is_err(), under_main.validate().is_err(),
"db_max_connections below DB_POOL_APP_HEADROOM must be rejected" "db_max_connections below DB_POOL_APP_HEADROOM must be rejected"
); );
} }
#[test]
fn web_url_rejects_empty_and_whitespace() {
// Unset is fine.
Config::parse_from(["gitlawb-node"])
.validate()
.expect("no web_url must validate");
// A real URL validates.
let mut cfg = Config::parse_from(["gitlawb-node"]);
cfg.web_url = Some("https://gitlawb.com".into());
assert!(cfg.validate().is_ok());
// Empty string is rejected.
let mut cfg = Config::parse_from(["gitlawb-node"]);
cfg.web_url = Some("".into());
assert!(cfg.validate().is_err(), "empty web_url must be rejected");
// Whitespace-only is rejected.
let mut cfg = Config::parse_from(["gitlawb-node"]);
cfg.web_url = Some(" ".into());
assert!(
cfg.validate().is_err(),
"whitespace-only web_url must be rejected"
);
}
#[test]
fn web_url_rejects_non_absolute_or_non_browser_urls() {
// Scheme-less host: parses nowhere, renders a broken View: link.
for bad in [
"gitlawb.com",
"not a url",
"ftp://files.example.com",
"javascript:alert(1)",
"https://", // no host
] {
let mut cfg = Config::parse_from(["gitlawb-node"]);
cfg.web_url = Some(bad.into());
assert!(cfg.validate().is_err(), "web_url {bad:?} must be rejected");
}
// Query strings and fragments corrupt the appended repo path, since
// web_url is used as a raw string prefix (`?a=1` would swallow
// `/{owner}/{repo}` into the query).
for bad in [
"https://gitlawb.com?a=1",
"https://gitlawb.com/?utm_source=docs",
"https://gitlawb.com#section",
"https://gitlawb.com/#faq",
] {
let mut cfg = Config::parse_from(["gitlawb-node"]);
cfg.web_url = Some(bad.into());
assert!(
cfg.validate().is_err(),
"web_url {bad:?} (query or fragment) must be rejected"
);
}
// Surrounding whitespace around a valid URL is tolerated.
let mut cfg = Config::parse_from(["gitlawb-node"]);
cfg.web_url = Some(" https://gitlawb.com ".into());
assert!(
cfg.validate().is_ok(),
"padded-but-valid web_url must validate"
);
// Percent-encoded bidi must not boot: the wire form carries no Cf
// code point, but a browser decodes %E2%80%AE into a reordering
// character on the rendered View: link (INV-6). Literal bidi and
// control bytes are rejected the same way.
for bad in [
"https://example.com/%E2%80%AE",
"https://example.com/\u{202e}spoof",
"https://example.com/%1b[31m",
] {
let mut cfg = Config::parse_from(["gitlawb-node"]);
cfg.web_url = Some(bad.into());
assert!(
cfg.validate().is_err(),
"web_url {bad:?} (control or bidi, even encoded) must be rejected"
);
}
// Non-default ports and paths are fine.
let mut cfg = Config::parse_from(["gitlawb-node"]);
cfg.web_url = Some("http://localhost:8080/ui".into());
assert!(cfg.validate().is_ok(), "port+path web_url must validate");
// Userinfo must be rejected: `web_url` is published on public
// `GET /` and reprinted on the CLI's `View:` line, so an
// accidentally configured password would land in public metadata,
// terminal scrollback, and CI logs. Reject username-only and
// user+password forms.
for bad in [
"https://[email protected]",
"https://user:[email protected]",
"https://:[email protected]",
] {
let mut cfg = Config::parse_from(["gitlawb-node"]);
cfg.web_url = Some(bad.into());
assert!(
cfg.validate().is_err(),
"web_url {bad:?} (userinfo) must be rejected"
);
}
}
/// `validate()` accepts whitespace-padded input but the stored value
/// must match the canonical form actually validated, so every consumer
/// (the regular and degraded `GET /` handlers, anyone who reads
/// `config.web_url` directly) sees the same string. Without this, an
/// operator that sets `GITLAWB_WEB_URL=" https://git.example "` boots
/// a node that advertises `" https://git.example "` on public
/// metadata — different from what was validated, different from what
/// any other consumer would parse.
#[test]
fn web_url_is_normalized_to_canonical_form_after_validation() {
// Padded input is rewritten to the canonical form. `url::Url`'s
// own `to_string` is what we use for the canonicalization, and
// it always appends a path-separator slash to a bare-host URL,
// so the stored value is `https://git.example/` (which is also
// what every browser would normalize the operator-typed form to).
let mut cfg = Config::parse_from(["gitlawb-node"]);
cfg.web_url = Some(" https://git.example ".into());
cfg.validate().unwrap();
assert_eq!(
cfg.web_url.as_deref(),
Some("https://git.example/"),
"stored web_url must be the canonical form, not the operator-typed one"
);
// Already-canonical input is left alone after a no-op
// re-normalization — same form comes back out, so any consumer
// reading the stored value sees a stable string across calls.
let mut cfg = Config::parse_from(["gitlawb-node"]);
cfg.web_url = Some("https://git.example/".into());
cfg.validate().unwrap();
assert_eq!(cfg.web_url.as_deref(), Some("https://git.example/"));
}
/// The DECLARED default, read off the parser rather than out of a parse. /// The DECLARED default, read off the parser rather than out of a parse.
/// ///
/// `Config::parse_from` consults the process environment, so on a host that /// `Config::parse_from` consults the process environment, so on a host that
@@ -1878,7 +2325,7 @@ mod tests {
std::os::unix::fs::symlink(&target, &link).unwrap(); std::os::unix::fs::symlink(&target, &link).unwrap();
let key_path = link.join("p2p.key"); let key_path = link.join("p2p.key");
let config = Config::parse_from([ let mut config = Config::parse_from([
"gitlawb-node", "gitlawb-node",
"--p2p-port", "--p2p-port",
"0", "0",
@@ -1973,7 +2420,7 @@ mod tests {
let key_path = build(base.path()); let key_path = build(base.path());
let before = snapshot(base.path()); let before = snapshot(base.path());
let config = Config::parse_from([ let mut config = Config::parse_from([
"gitlawb-node", "gitlawb-node",
"--p2p-port", "--p2p-port",
"7546", "7546",
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
+387 -2
View File
@@ -6,6 +6,7 @@
use std::collections::{BTreeSet, HashMap}; use std::collections::{BTreeSet, HashMap};
use std::path::Path; use std::path::Path;
use std::str::FromStr; use std::str::FromStr;
use std::time::Duration;
use ed25519_dalek::VerifyingKey; use ed25519_dalek::VerifyingKey;
use gitlawb_core::did::Did; use gitlawb_core::did::Did;
@@ -127,17 +128,60 @@ fn plan_seal(node_seed: &[u8; 32], dids: &BTreeSet<String>, stored_tag: Option<&
/// `node_seed` keys the opaque recipients tag. Returns `(oid, cid)` for each blob /// `node_seed` keys the opaque recipients tag. Returns `(oid, cid)` for each blob
/// actually sealed and recorded this call (the per-push delta), used by Option B3 /// actually sealed and recorded this call (the per-push delta), used by Option B3
/// to anchor a manifest. Recipient identities are never stored or returned. /// to anchor a manifest. Recipient identities are never stored or returned.
///
/// Nine args (the fence joins the seal's eight) but grouping them would churn
/// both callers and the race/hung-git tests for no behavioral gain.
#[allow(clippy::too_many_arguments)]
pub async fn encrypt_and_pin( pub async fn encrypt_and_pin(
ipfs_api: &str, ipfs_api: &str,
repo_path: &Path, repo_path: &Path,
db: &Db, db: &Db,
repo_id: &str, repo_id: &str,
node_seed: &[u8; 32], node_seed: &[u8; 32],
git_bin: &str,
batch_budget: Duration,
recipients: &HashMap<String, BTreeSet<String>>, recipients: &HashMap<String, BTreeSet<String>>,
fence: Option<&crate::ipfs_pin::PolicyFence>,
) -> Vec<(String, String)> { ) -> Vec<(String, String)> {
let mut sealed = Vec::new(); let mut sealed = Vec::new();
let mut skipped_unresolvable = 0usize; let mut skipped_unresolvable = 0usize;
for (oid, dids) in recipients { // One shared read deadline for the whole batch, like `pin_new_objects`: a
// hung git child is watchdog-reaped at this bound, so the outer
// `PIN_PHASE_DEADLINE` timeout cannot be held open by a blocking read
// (R1-P2). Each read runs under `spawn_blocking` — it is synchronous child
// spawn + pipe drain + watchdog join.
let read_deadline = std::time::Instant::now() + batch_budget;
let total = recipients.len();
for (attempted, (oid, dids)) in recipients.iter().enumerate() {
// Batch budget gate (R2-P3), mirroring the public pin loops: an object
// is never started with a remainder too small to cover a bounded read's
// teardown. This is consistency (the seal is bounded by the outer
// `PIN_PHASE_DEADLINE` either way), but it keeps the three loops from
// drifting apart in how they report a truncated batch.
if crate::ipfs_pin::batch_budget_gate(
"encrypted-seal",
read_deadline,
sealed.len(),
total - attempted,
)
.is_none()
{
break;
}
// Policy fence (R1-P1): the recipients snapshot was derived before the
// long withheld-blob walk; if a visibility rule moved while that walk
// ran (a reader added or removed), stop sealing instead of pinning to a
// stale recipient set. Checked FIRST so a changed policy costs nothing.
if let Some(f) = fence {
if !f.is_current().await {
tracing::warn!(
repo = %f.repo_id(),
oid = %oid,
"visibility policy changed after the recipients snapshot; stopping the seal loop"
);
break;
}
}
// A DB read failure is not a cache miss: re-sealing here would do an // A DB read failure is not a cache miss: re-sealing here would do an
// avoidable IPFS write during a partial outage. Skip and retry next push. // avoidable IPFS write during a partial outage. Skip and retry next push.
let stored_tag = match db.encrypted_blob_recipients_tag(repo_id, oid).await { let stored_tag = match db.encrypted_blob_recipients_tag(repo_id, oid).await {
@@ -188,7 +232,9 @@ pub async fn encrypt_and_pin(
} }
SealPlan::Seal { keys, tag } => (keys, tag), SealPlan::Seal { keys, tag } => (keys, tag),
}; };
let data = match crate::git::store::read_object(repo_path, oid) { let data = match read_object_bounded_spawn_blocking(git_bin, repo_path, oid, read_deadline)
.await
{
Ok(Some((_t, bytes))) => bytes, Ok(Some((_t, bytes))) => bytes,
Ok(None) => { Ok(None) => {
tracing::warn!(oid = %oid, "git object not found; skipping encrypted pin"); tracing::warn!(oid = %oid, "git object not found; skipping encrypted pin");
@@ -206,6 +252,22 @@ pub async fn encrypt_and_pin(
continue; continue;
} }
}; };
// Dispatch fence (R1-P1): re-read the policy epoch immediately before
// the irreversible HTTP POST. The iteration-top check catches a narrow
// that landed before work began; THIS check catches a narrow that landed
// during the tag lookup, recipient resolution, git read, or seal — all
// of which can take seconds. Without this, a reader removed during
// preparation can still receive a newly published envelope.
if let Some(f) = fence {
if !f.is_current().await {
tracing::warn!(
repo = %f.repo_id(),
oid = %oid,
"visibility policy changed during encrypted seal preparation; aborting upload"
);
break;
}
}
let cid = match crate::ipfs_pin::pin_git_object(ipfs_api, oid, &envelope, None).await { let cid = match crate::ipfs_pin::pin_git_object(ipfs_api, oid, &envelope, None).await {
Ok(c) if !c.is_empty() => c, Ok(c) if !c.is_empty() => c,
Ok(_) => { Ok(_) => {
@@ -237,10 +299,33 @@ pub async fn encrypt_and_pin(
sealed sealed
} }
/// Bounded, reaped git object read for the seal loop, run off the async thread:
/// `read_object_bounded` is synchronous child spawn + pipe drain + watchdog
/// join, so blocking the runtime task on it would let a hung git hold a worker
/// thread (R1-P2). The `deadline` is the batch's shared read deadline; a child
/// still alive at it is SIGTERM/SIGKILL group-reaped by the watchdog.
async fn read_object_bounded_spawn_blocking(
git_bin: &str,
repo_path: &Path,
sha256_hex: &str,
deadline: std::time::Instant,
) -> anyhow::Result<Option<(String, Vec<u8>)>> {
let git_bin = git_bin.to_string();
let repo_path = repo_path.to_path_buf();
let sha256_hex = sha256_hex.to_string();
tokio::task::spawn_blocking(move || {
crate::git::store::read_object_bounded(&git_bin, &repo_path, &sha256_hex, deadline)
.map_err(anyhow::Error::from)
})
.await
.map_err(|e| anyhow::anyhow!("read_object spawn_blocking join failed: {e}"))?
}
#[cfg(test)] #[cfg(test)]
mod tests { mod tests {
use super::*; use super::*;
use ed25519_dalek::SigningKey; use ed25519_dalek::SigningKey;
use std::time::Duration;
/// A did:key encoding a small-order point. Well-formed, indistinguishable /// A did:key encoding a small-order point. Well-formed, indistinguishable
/// from a real one by eye, and refused at resolution since the choke-point /// from a real one by eye, and refused at resolution since the choke-point
@@ -475,4 +560,304 @@ mod tests {
other => panic!("changed recipient set must re-seal; got {other:?}"), other => panic!("changed recipient set must re-seal; got {other:?}"),
} }
} }
/// A reader removed mid-seal must stop the seal loop (R1-P1 "race test for
/// reader removal"): `encrypt_and_pin` re-checks the policy fence before
/// each blob, so a `remove_visibility_rule` landing while the first seal is
/// in flight aborts before a later blob is pinned to a stale recipient set.
#[sqlx::test]
async fn encrypt_and_pin_stops_sealing_when_reader_removed_mid_batch(pool: sqlx::PgPool) {
let db = crate::db::Db::for_testing(pool);
db.run_migrations().await.expect("migrations");
let tmp = tempfile::TempDir::new().unwrap();
let repo_path = tmp.path().join("seal-race.git");
// Three loose blobs, each withheld (path-scoped deny exists so the sweep
// would have derived recipients for them).
let oids: Vec<String> = {
crate::git::store::init_bare(&repo_path).expect("init bare repo");
(0..3)
.map(|i| {
let mut cmd = std::process::Command::new("git");
cmd.args(["hash-object", "-w", "--stdin"])
.current_dir(&repo_path)
.stdin(std::process::Stdio::piped())
.stdout(std::process::Stdio::piped());
let mut child = cmd.spawn().expect("spawn git hash-object");
{
use std::io::Write;
child
.stdin
.as_mut()
.expect("stdin")
.write_all(format!("secret blob {i}\n").as_bytes())
.expect("write stdin");
}
let out = child.wait_with_output().expect("hash-object output");
assert!(out.status.success());
String::from_utf8_lossy(&out.stdout).trim().to_string()
})
.collect()
};
// A real repos row so the fence has an epoch and a reader can be removed.
let now = chrono::Utc::now();
let repo_id = uuid::Uuid::new_v4().to_string();
db.create_repo(&crate::db::RepoRecord {
id: repo_id.clone(),
name: "seal-race-repo".into(),
owner_did: "did:key:zSealRaceOwner".into(),
description: None,
is_public: true,
default_branch: "main".into(),
created_at: now,
updated_at: now,
disk_path: repo_path.display().to_string(),
forked_from: None,
machine_id: None,
})
.await
.expect("create repo");
// A rule whose removal is the "reader removed" mutation: one reader per
// blob, all under the same path glob.
let reader = did_key(1);
db.set_visibility_rule(
&repo_id,
"**/secret/*",
crate::db::VisibilityMode::B,
std::slice::from_ref(&reader),
"did:key:zSealRaceOwner",
)
.await
.expect("set rule");
// IPFS endpoint that delays the FIRST add 2s so the removal lands while
// that seal is in flight, then answers immediately.
let endpoint = delaying_cid_endpoint(vec![Duration::from_secs(2)]).await;
let recipients: HashMap<String, BTreeSet<String>> = oids
.iter()
.cloned()
.map(|oid| {
let mut s = BTreeSet::new();
s.insert(reader.clone());
(oid, s)
})
.collect();
let fence = crate::ipfs_pin::PolicyFence::capture(&db, &repo_id)
.await
.expect("fence captures");
let sealed = tokio::time::timeout(Duration::from_secs(30), async {
let seal_db = db.clone();
let seal_repo = repo_path.clone();
let seal_endpoint = endpoint.clone();
let seal_repo_id = repo_id.clone();
let handle = tokio::spawn(async move {
encrypt_and_pin(
&seal_endpoint,
&seal_repo,
&seal_db,
&seal_repo_id,
&SEED,
"git",
Duration::from_secs(60),
&recipients,
Some(&fence),
)
.await
});
// Let the first add start (endpoint sleeps 2s), then remove the
// reader so the fence is stale before the loop checks again.
// The window has to cover blob 1's encrypt phase: if the removal
// lands before the loop reaches dispatch, the fence aborts with
// nothing sealed, so give the in-flight add room to start under
// a loaded box while staying inside the 2s endpoint delay.
tokio::time::sleep(Duration::from_millis(1200)).await;
db.remove_visibility_rule(&repo_id, "**/secret/*")
.await
.expect("remove rule");
handle.await.expect("seal task")
})
.await
.expect("wedge guard: the fence abort must not take 30s");
assert!(
sealed.len() < oids.len(),
"a reader removal landing mid-batch must abort before every blob is sealed: {}",
sealed.len()
);
assert!(
!sealed.is_empty(),
"at least the blob already in flight before the removal completes"
);
}
/// A hung git must not hold the seal loop past its read budget (R1-P2): the
/// git read runs under `spawn_blocking` against `read_object_bounded`, so
/// the watchdog reaps a wedged child at the batch deadline and the loop
/// keeps its shape instead of blocking a runtime worker indefinitely.
#[cfg(unix)]
#[sqlx::test]
async fn encrypt_and_pin_returns_by_budget_with_a_hung_git(pool: sqlx::PgPool) {
use std::os::unix::fs::PermissionsExt;
let db = crate::db::Db::for_testing(pool);
db.run_migrations().await.expect("migrations");
let tmp = tempfile::TempDir::new().unwrap();
let repo_path = tmp.path().join("seal-hung.git");
let oids: Vec<String> = {
crate::git::store::init_bare(&repo_path).expect("init bare repo");
(0..2)
.map(|i| {
let mut cmd = std::process::Command::new("git");
cmd.args(["hash-object", "-w", "--stdin"])
.current_dir(&repo_path)
.stdin(std::process::Stdio::piped())
.stdout(std::process::Stdio::piped());
let mut child = cmd.spawn().expect("spawn git hash-object");
{
use std::io::Write;
child
.stdin
.as_mut()
.expect("stdin")
.write_all(format!("secret blob {i}\n").as_bytes())
.expect("write stdin");
}
let out = child.wait_with_output().expect("hash-object output");
assert!(out.status.success());
String::from_utf8_lossy(&out.stdout).trim().to_string()
})
.collect()
};
// A git that wedges forever, ignoring SIGTERM, so only the watchdog's
// SIGKILL can reap it.
let fake = tmp.path().join("hanging-git");
std::fs::write(&fake, "#!/bin/sh\ntrap '' TERM\necho $$ > pid\nsleep 30\n").unwrap();
let mut perm = std::fs::metadata(&fake).unwrap().permissions();
perm.set_mode(0o755);
std::fs::set_permissions(&fake, perm).unwrap();
let now = chrono::Utc::now();
let repo_id = uuid::Uuid::new_v4().to_string();
db.create_repo(&crate::db::RepoRecord {
id: repo_id.clone(),
name: "seal-hung-repo".into(),
owner_did: "did:key:zSealHungOwner".into(),
description: None,
is_public: true,
default_branch: "main".into(),
created_at: now,
updated_at: now,
disk_path: repo_path.display().to_string(),
forked_from: None,
machine_id: None,
})
.await
.expect("create repo");
db.set_visibility_rule(
&repo_id,
"**/secret/*",
crate::db::VisibilityMode::B,
&[did_key(1)],
"did:key:zSealHungOwner",
)
.await
.expect("set rule");
let recipients: HashMap<String, BTreeSet<String>> = oids
.iter()
.cloned()
.map(|oid| {
let mut s = BTreeSet::new();
s.insert(did_key(1));
(oid, s)
})
.collect();
// Unreachable endpoint: even if a read somehow succeeded, the pin would
// fail; the read itself is the thing under test.
let started = std::time::Instant::now();
let sealed = tokio::time::timeout(
Duration::from_secs(60),
encrypt_and_pin(
"http://127.0.0.1:9",
&repo_path,
&db,
&repo_id,
&SEED,
fake.to_str().unwrap(),
Duration::from_secs(2),
&recipients,
None,
),
)
.await
.expect("a hung git must not hold the seal past the outer wedge guard");
let elapsed = started.elapsed();
assert!(
elapsed < Duration::from_secs(10),
"a hung git must be watchdog-reaped inside the read budget, not block the loop for ~10s+ (took {elapsed:?})"
);
assert!(
sealed.is_empty(),
"with a hung git no blob can be read, so nothing may be reported sealed"
);
}
/// Local TCP endpoint that answers `{ "Hash": "QmMock" }` after an optional
/// per-request delay, so a seal can be made to straddle a policy mutation.
async fn delaying_cid_endpoint(delays: Vec<Duration>) -> String {
use tokio::io::{AsyncReadExt, AsyncWriteExt};
let listener = tokio::net::TcpListener::bind("127.0.0.1:0").await.unwrap();
let endpoint = format!("http://{}", listener.local_addr().unwrap());
tokio::spawn(async move {
let mut seen = 0usize;
while let Ok((mut sock, _)) = listener.accept().await {
let delay = *delays
.get(seen)
.or_else(|| delays.last())
.unwrap_or(&Duration::ZERO);
seen += 1;
tokio::spawn(async move {
let mut acc = Vec::new();
let mut buf = [0u8; 4096];
loop {
let n = match sock.read(&mut buf).await {
Ok(0) | Err(_) => break,
Ok(n) => n,
};
acc.extend_from_slice(&buf[..n]);
if let Some(hdr_end) =
acc.windows(4).position(|w| w == b"\r\n\r\n").map(|p| p + 4)
{
let headers = String::from_utf8_lossy(&acc[..hdr_end]).to_lowercase();
let len: usize = headers
.lines()
.find_map(|l| l.strip_prefix("content-length:"))
.and_then(|v| v.trim().parse().ok())
.unwrap_or(0);
if acc.len() >= hdr_end + len {
break;
}
}
}
tokio::time::sleep(delay).await;
let body = br#"{"Hash":"bafkreifjjcie6lypi6ny7amxnfftagclbuxndqonfipmb64f2km2devei4"}"#;
let _ = sock
.write_all(
format!("HTTP/1.1 200 OK\r\nContent-Length: {}\r\n\r\n", body.len())
.as_bytes(),
)
.await;
let _ = sock.write_all(body).await;
let _ = sock.flush().await;
});
}
});
endpoint
}
} }
+25 -1
View File
@@ -12,6 +12,9 @@ pub enum AppError {
#[error("repo already exists: {0}")] #[error("repo already exists: {0}")]
RepoExists(String), RepoExists(String),
#[error("conflict: {0}")]
Conflict(String),
#[error("not found: {0}")] #[error("not found: {0}")]
NotFound(String), NotFound(String),
@@ -101,7 +104,7 @@ pub const DB_ERROR_MESSAGE: &str = "a database error occurred";
/// Connection-level sqlx failures that mean the database is unreachable right /// Connection-level sqlx failures that mean the database is unreachable right
/// now (retryable, 503), as opposed to server-reported query errors. /// now (retryable, 503), as opposed to server-reported query errors.
fn db_unavailable(e: &sqlx::Error) -> bool { pub(crate) fn db_unavailable(e: &sqlx::Error) -> bool {
matches!( matches!(
e, e,
sqlx::Error::PoolTimedOut sqlx::Error::PoolTimedOut
@@ -189,6 +192,7 @@ impl IntoResponse for AppError {
"repo_exists", "repo_exists",
format!("repository '{r}' already exists"), format!("repository '{r}' already exists"),
), ),
AppError::Conflict(msg) => (StatusCode::CONFLICT, "conflict", msg.clone()),
AppError::NotFound(msg) => (StatusCode::NOT_FOUND, "not_found", msg.clone()), AppError::NotFound(msg) => (StatusCode::NOT_FOUND, "not_found", msg.clone()),
AppError::Unauthorized(msg) => (StatusCode::UNAUTHORIZED, "not_an_agent", msg.clone()), AppError::Unauthorized(msg) => (StatusCode::UNAUTHORIZED, "not_an_agent", msg.clone()),
AppError::Forbidden(msg) => (StatusCode::FORBIDDEN, "forbidden", msg.clone()), AppError::Forbidden(msg) => (StatusCode::FORBIDDEN, "forbidden", msg.clone()),
@@ -321,6 +325,16 @@ pub type Result<T> = std::result::Result<T, AppError>;
mod tests { mod tests {
use super::*; use super::*;
#[test]
fn conflict_maps_to_409() {
assert_eq!(
AppError::Conflict("task not claimable".into())
.into_response()
.status(),
StatusCode::CONFLICT
);
}
#[test] #[test]
fn timeout_maps_to_504_distinct_from_git_500() { fn timeout_maps_to_504_distinct_from_git_500() {
assert_eq!( assert_eq!(
@@ -344,6 +358,16 @@ mod tests {
); );
} }
/// A 429 does not advertise a retry interval: the status-claim caps behind
/// it are per-commit and append-only, so a client honoring a 1s hint would
/// retry a request that can never succeed.
#[test]
fn too_many_requests_carries_no_retry_after() {
let resp = AppError::TooManyRequests("x".into()).into_response();
assert_eq!(resp.status(), StatusCode::TOO_MANY_REQUESTS);
assert!(resp.headers().get("retry-after").is_none());
}
/// #226: raw sqlx/DB detail must never appear in the Internal 500 body. /// #226: raw sqlx/DB detail must never appear in the Internal 500 body.
#[tokio::test] #[tokio::test]
async fn internal_error_body_is_opaque() { async fn internal_error_body_is_opaque() {
+76 -29
View File
@@ -141,19 +141,10 @@ pub fn resolve_issue_id(repo_path: &Path, id_or_prefix: &str) -> Result<Option<S
return Ok(None); return Ok(None);
} }
// Try exact match first — fast path for callers passing the full UUID. // List all refs that start with the given string; the glob also matches the exact
// ref. The id handed back is always git's own refname, never the request input.
let exact_ref = format!("refs/gitlawb/issues/{id_or_prefix}"); let exact_ref = format!("refs/gitlawb/issues/{id_or_prefix}");
let check = Command::new("git") let prefix_glob = format!("{exact_ref}*");
.args(["cat-file", "-e", &exact_ref])
.current_dir(repo_path)
.output()
.context("failed to run git cat-file -e")?;
if check.status.success() {
return Ok(Some(id_or_prefix.to_string()));
}
// Prefix search: list all refs that start with the given string.
let prefix_glob = format!("refs/gitlawb/issues/{id_or_prefix}*");
let list = Command::new("git") let list = Command::new("git")
.args(["for-each-ref", "--format=%(refname)", &prefix_glob]) .args(["for-each-ref", "--format=%(refname)", &prefix_glob])
.current_dir(repo_path) .current_dir(repo_path)
@@ -165,25 +156,44 @@ pub fn resolve_issue_id(repo_path: &Path, id_or_prefix: &str) -> Result<Option<S
} }
let output = String::from_utf8_lossy(&list.stdout); let output = String::from_utf8_lossy(&list.stdout);
let matches: Vec<&str> = output.lines().filter(|l| !l.trim().is_empty()).collect(); let matches: Vec<&str> = output
.lines()
.map(str::trim)
.filter(|l| !l.is_empty())
.collect();
match matches.len() { // An exact id wins over longer ids that share it as a prefix, but only if its
0 => Ok(None), // object exists: for-each-ref also lists a ref whose blob is missing, so a
1 => { // dangling exact ref falls through to the prefix rules below.
// Strip the "refs/gitlawb/issues/" prefix to get the bare ID. let exact = match matches.iter().find(|m| **m == exact_ref) {
let full_id = matches[0] Some(m) => {
.trim() let check = Command::new("git")
.strip_prefix("refs/gitlawb/issues/") .args(["cat-file", "-e", m])
.unwrap_or(matches[0].trim()) .current_dir(repo_path)
.to_string(); .output()
Ok(Some(full_id)) .context("failed to run git cat-file -e")?;
check.status.success().then_some(*m)
} }
_ => anyhow::bail!( None => None,
"ambiguous issue prefix '{}': matches {} issues", };
id_or_prefix, let found = match exact {
matches.len() Some(m) => m,
), None => match matches.as_slice() {
} [] => return Ok(None),
[m] => *m,
_ => anyhow::bail!(
"ambiguous issue prefix '{}': matches {} issues",
id_or_prefix,
matches.len()
),
},
};
// Strip the "refs/gitlawb/issues/" prefix to get the bare ID.
let full_id = found
.strip_prefix("refs/gitlawb/issues/")
.unwrap_or(found)
.to_string();
Ok(Some(full_id))
} }
/// Close an issue by updating its status to "closed" in the git ref. /// Close an issue by updating its status to "closed" in the git ref.
@@ -267,6 +277,43 @@ mod tests {
assert_eq!(resolved, Some(full_id.to_string())); assert_eq!(resolved, Some(full_id.to_string()));
} }
#[test]
fn test_resolve_exact_id_wins_over_longer_id() {
let dir = TempDir::new().unwrap();
init_repo(&dir);
create_issue(dir.path(), "abc12345", r#"{"status":"open"}"#).unwrap();
create_issue(dir.path(), "abc12345-0000", r#"{"status":"open"}"#).unwrap();
let resolved = resolve_issue_id(dir.path(), "abc12345").unwrap();
assert_eq!(resolved, Some("abc12345".to_string()));
}
#[test]
fn test_resolve_dangling_exact_id_does_not_win_over_longer_id() {
let dir = TempDir::new().unwrap();
init_repo(&dir);
create_issue(dir.path(), "abc12345", r#"{"status":"open","n":1}"#).unwrap();
create_issue(dir.path(), "abc12345-0000", r#"{"status":"open","n":2}"#).unwrap();
// Dangle the exact ref: the ref stays listed by for-each-ref, but its blob is gone.
let out = Command::new("git")
.args(["rev-parse", "refs/gitlawb/issues/abc12345"])
.current_dir(dir.path())
.output()
.unwrap();
let hash = String::from_utf8(out.stdout).unwrap();
let hash = hash.trim();
std::fs::remove_file(
dir.path()
.join(".git/objects")
.join(&hash[..2])
.join(&hash[2..]),
)
.unwrap();
let result = resolve_issue_id(dir.path(), "abc12345");
assert!(result.unwrap_err().to_string().contains("ambiguous"));
assert!(close_issue(dir.path(), "abc12345").is_err());
}
#[test] #[test]
fn test_resolve_prefix_matches_unique() { fn test_resolve_prefix_matches_unique() {
let dir = TempDir::new().unwrap(); let dir = TempDir::new().unwrap();
+15 -8
View File
@@ -185,10 +185,13 @@ fn rev_list_delta(
/// List every object in the repository via /// List every object in the repository via
/// `git cat-file --batch-all-objects --batch-check='%(objectname)'`. /// `git cat-file --batch-all-objects --batch-check='%(objectname)'`.
/// ///
/// This is the whole-repo enumeration the push path falls back to and the /// This is the whole-repo enumeration the push path falls back to. It
/// reconciliation sweep relies on. It returns *all* objects (including /// returns *all* objects (including unreachable/dangling ones), which is
/// unreachable/dangling ones), which is what the sweep needs to catch /// what the push path needs to catch stragglers — do not swap it for a
/// stragglers — do not swap it for a reachability walk. /// reachability walk. The reconciliation sweep no longer uses it: sweep
/// candidates come from the windowed walk (window commits plus walked
/// pairs), which are reachable by construction, so dangling objects are
/// absent rather than filtered.
pub fn list_all_objects(repo_path: &Path, git_bin: &str, deadline: Instant) -> Result<Vec<String>> { pub fn list_all_objects(repo_path: &Path, git_bin: &str, deadline: Instant) -> Result<Vec<String>> {
let out = crate::git::visibility_pack::run_bounded_git( let out = crate::git::visibility_pack::run_bounded_git(
git_bin, git_bin,
@@ -214,6 +217,7 @@ pub fn list_all_objects(repo_path: &Path, git_bin: &str, deadline: Instant) -> R
/// filter needs to tell blobs (content, withholdable) and trees (content, /// filter needs to tell blobs (content, withholdable) and trees (content,
/// withholdable since #135/#173) from commits/tags (structural, never withheld) /// withholdable since #135/#173) from commits/tags (structural, never withheld)
/// without typing the candidate list itself. /// without typing the candidate list itself.
#[allow(dead_code)] // used by tests and all_blob_oids
pub fn list_all_objects_with_type( pub fn list_all_objects_with_type(
repo_path: &Path, repo_path: &Path,
git_bin: &str, git_bin: &str,
@@ -261,6 +265,7 @@ pub fn all_blob_oids(
/// candidate blob or tree absent from the matching reachable visibility-allowed /// candidate blob or tree absent from the matching reachable visibility-allowed
/// set; a dangling or withheld object is in this universe but not the allowed /// set; a dangling or withheld object is in this universe but not the allowed
/// set, so it never replicates (#99, #172). /// set, so it never replicates (#99, #172).
#[allow(dead_code)] // production went to the typed sweep path; tests still use it
pub fn all_blob_and_tree_oids( pub fn all_blob_and_tree_oids(
repo_path: &Path, repo_path: &Path,
git_bin: &str, git_bin: &str,
@@ -303,10 +308,12 @@ pub struct PinCandidateSet {
/// Every degraded path is **logged**, not silent: a full-scan fallback, a /// Every degraded path is **logged**, not silent: a full-scan fallback, a
/// failed full scan, and a panicked blocking task each emit a warning. On a /// failed full scan, and a panicked blocking task each emit a warning. On a
/// failed full scan or a task panic the candidate set is empty (pin nothing /// failed full scan or a task panic the candidate set is empty (pin nothing
/// this push); that is a durability gap the reconciliation sweep backstops, and /// this push); that is a durability gap the reconciliation sweep backstops
/// it can never leak because the withheld/fail-closed filter still runs on /// when it is enabled and a pin backend is configured (a node running with the
/// whatever set is returned. `full_scan` rides on the returned set so the caller /// sweep disabled or with no IPFS/Pinata backend has no backstop), and it can
/// knows when the dangling-inclusive filter is required. /// never leak because the withheld/fail-closed filter still runs on whatever
/// set is returned. `full_scan` rides on the returned set so the caller knows
/// when the dangling-inclusive filter is required.
/// ///
/// `scan_sem` is the post-receive scan admission pool (`git_encrypt_semaphore`, /// `scan_sem` is the post-receive scan admission pool (`git_encrypt_semaphore`,
/// #174 F4): both git-spawning stages — the per-tip `cat-file` probe + delta /// #174 F4): both git-spawning stages — the per-tip `cat-file` probe + delta
+11 -17
View File
@@ -82,18 +82,9 @@ pub struct RepoStore {
} }
impl RepoStore { impl RepoStore {
#[cfg(test)] #[cfg(any(test, feature = "test-harness"))]
pub fn for_testing(repos_dir: PathBuf, lock_pool: PgPool) -> Self { pub fn for_testing(repos_dir: PathBuf, lock_pool: PgPool) -> Self {
Self { Self::new(repos_dir, None, lock_pool, Duration::from_secs(300))
repos_dir,
tigris: None,
lock_pool,
lock_held_transfer_timeout: Duration::from_secs(300),
lock_acquire_deadline: LOCK_ACQUIRE_DEADLINE,
migrated: Arc::new(tokio::sync::Mutex::new(std::collections::HashSet::new())),
pre_unlock_gate: None,
upload_site_reached: Arc::new(std::sync::atomic::AtomicUsize::new(0)),
}
} }
/// Same as [`RepoStore::for_testing`] but with Tigris enabled, so the paths /// Same as [`RepoStore::for_testing`] but with Tigris enabled, so the paths
@@ -118,7 +109,7 @@ impl RepoStore {
/// Test-only: every guard from this store parks in `release` right before the /// Test-only: every guard from this store parks in `release` right before the
/// `pg_advisory_unlock` await, until `gate` is notified. Dropping the future /// `pg_advisory_unlock` await, until `gate` is notified. Dropping the future
/// while it is parked reproduces a client disconnect inside `release`. /// while it is parked reproduces a client disconnect inside `release`.
#[cfg(test)] #[cfg(all(test, unix))]
pub fn with_pre_unlock_gate(mut self, gate: Arc<tokio::sync::Notify>) -> Self { pub fn with_pre_unlock_gate(mut self, gate: Arc<tokio::sync::Notify>) -> Self {
self.pre_unlock_gate = Some(gate); self.pre_unlock_gate = Some(gate);
self self
@@ -126,7 +117,7 @@ impl RepoStore {
/// Test-only: how many write guards from this store have reached the Tigris upload /// Test-only: how many write guards from this store have reached the Tigris upload
/// site. See [`RepoWriteGuard::release`]. /// site. See [`RepoWriteGuard::release`].
#[cfg(test)] #[cfg(all(test, unix))]
pub fn tigris_upload_site_reached(&self) -> usize { pub fn tigris_upload_site_reached(&self) -> usize {
self.upload_site_reached self.upload_site_reached
.load(std::sync::atomic::Ordering::SeqCst) .load(std::sync::atomic::Ordering::SeqCst)
@@ -1237,7 +1228,7 @@ async fn retry_fork_archive_delete(
/// `remove_dir_all` / `rename` can take it and static analysers can treat it as /// `remove_dir_all` / `rename` can take it and static analysers can treat it as
/// sanitised input (CodeQL `rust/path-injection`). /// sanitised input (CodeQL `rust/path-injection`).
#[derive(Debug, Clone)] #[derive(Debug, Clone)]
pub(crate) struct ValidatedRepoDiskPath(PathBuf); pub struct ValidatedRepoDiskPath(PathBuf);
impl ValidatedRepoDiskPath { impl ValidatedRepoDiskPath {
pub(crate) fn as_path(&self) -> &Path { pub(crate) fn as_path(&self) -> &Path {
@@ -1717,8 +1708,11 @@ fn validate_repo_name(repo_name: &str) -> Result<()> {
if repo_name.is_empty() { if repo_name.is_empty() {
anyhow::bail!("repo_name is empty"); anyhow::bail!("repo_name is empty");
} }
if repo_name.len() > 100 { // Byte bound shared with the API-side validator via `MAX_REPO_NAME_LEN`:
anyhow::bail!("repo_name exceeds 100 chars"); // the two validators differ on charset by design, but must not drift on
// length, or an API-admitted name fails here after the proof is spent.
if repo_name.len() > crate::db::MAX_REPO_NAME_LEN {
anyhow::bail!("repo_name exceeds {} chars", crate::db::MAX_REPO_NAME_LEN);
} }
// Repo names are `[A-Za-z0-9._-]+` minus path-traversal traps. // Repo names are `[A-Za-z0-9._-]+` minus path-traversal traps.
if repo_name.contains("..") { if repo_name.contains("..") {
@@ -2405,7 +2399,7 @@ impl RepoWriteGuard {
/// Only the UNLOCK has to wait for the reaper. This token carries everything /// Only the UNLOCK has to wait for the reaper. This token carries everything
/// the settlement itself needs, so the request future can hold it and settle /// the settlement itself needs, so the request future can hold it and settle
/// the tree at the disconnect instant while the reaper still holds the lock. /// the tree at the disconnect instant while the reaper still holds the lock.
pub(crate) struct TreeSettlement { pub struct TreeSettlement {
local_path: ValidatedRepoDiskPath, local_path: ValidatedRepoDiskPath,
repo_name: String, repo_name: String,
/// Shared with the guard, so the stage read here is the stage the publish /// Shared with the guard, so the stage read here is the stage the publish
+465
View File
@@ -220,6 +220,7 @@ pub fn response_served_pack(output: &[u8]) -> bool {
/// ///
/// Accepts a push. The caller MUST verify HTTP Signature auth before /// Accepts a push. The caller MUST verify HTTP Signature auth before
/// calling this function. /// calling this function.
#[allow(dead_code)] // used by tests; production uses receive_pack_raw
pub async fn receive_pack( pub async fn receive_pack(
git_bin: &str, git_bin: &str,
repo_path: &Path, repo_path: &Path,
@@ -244,6 +245,219 @@ pub async fn receive_pack(
.body(Body::from(output))?) .body(Body::from(output))?)
} }
/// Run `git-receive-pack` and return the raw stdout bytes together with
/// the process exit status. Unlike [`receive_pack`], this does NOT bail
/// on a non-zero exit: the caller needs the stdout (which contains the
/// report-status with per-ref ok/ng results) even when the process
/// exits non-zero. A timeout still returns `Err`.
///
/// `reflog_action` (typically `gitlawb-request:<request_id>`) is
/// exported as `GIT_REFLOG_ACTION` so the ref transaction writes
/// request-bound landing evidence into the reflog. Recovery requires
/// that evidence before auto-promoting a stranded row; without it a
/// later request recreating the same old/new tuple is
/// indistinguishable from the stranded one.
#[allow(dead_code)]
pub async fn receive_pack_raw(
git_bin: &str,
repo_path: &Path,
request_body: Bytes,
timeout: Duration,
admission: Option<AdmissionGuard>,
) -> Result<(Vec<u8>, bool)> {
receive_pack_raw_with_reflog(git_bin, repo_path, request_body, timeout, admission, None).await
}
/// [`receive_pack_raw`] with explicit reflog binding.
pub async fn receive_pack_raw_with_reflog(
git_bin: &str,
repo_path: &Path,
request_body: Bytes,
timeout: Duration,
admission: Option<AdmissionGuard>,
reflog_action: Option<&str>,
) -> Result<(Vec<u8>, bool)> {
let mut command = Command::new(git_bin);
command
.arg("receive-pack")
.arg("--stateless-rpc")
.arg(repo_path);
if let Some(action) = reflog_action {
command.env("GIT_REFLOG_ACTION", action);
}
let (out, err, status, _admission) = drive_git_child_raw(
command,
request_body,
timeout,
"git-receive-pack",
admission,
)
.await?;
// On timeout, drive_git_child_raw returns Err — we never reach here.
// On success/non-zero exit, we have the stdout and exit status.
if !status.success() {
let stderr = String::from_utf8_lossy(&err);
tracing::warn!(stderr = %stderr, "git-receive-pack exited non-zero");
}
Ok((out, status.success()))
}
/// Parse the git-receive-pack report-status output to determine per-ref
/// success/failure. Returns `(unpack_ok, per_ref_results)` where
/// `per_ref_results` is a list of `(ref_name, is_ok)`.
///
/// The report-status format (after the sideband framing) is:
/// ```text
/// unpack ok\n (or: unpack fail\n)
/// ok <refname>\n (per successful ref)
/// ng <refname> <reason>\n (per rejected ref)
/// \n (empty line terminates)
/// ```
///
/// Returns `None` if the output cannot be parsed (e.g. the client did
/// not request report-status, or the output is truncated). In that
/// case the caller should treat all refs as uncertain.
pub fn parse_report_status(output: &[u8]) -> Option<(bool, Vec<(String, bool)>)> {
let text = std::str::from_utf8(output).ok()?;
// Strip sideband framing: each line starts with a pkt-line length
// prefix and a channel byte (1 = stdout, 2 = stderr). The actual
// data starts after the first `0000` flush packet or after we
// strip sideband bytes.
//
// Framing completeness is part of authority, at whichever level the
// framing lives. A double-framed capture carries the report stream
// (itself pkt-line framed, flush-terminated) inside the sideband
// envelope — real git output has been observed with NO outer flush,
// so the outer level cannot require one; the inner frame's flush is
// authoritative there. A single-framed (or unframed) payload has no
// inner frame, so its only completion marker is the outer flush:
// output ending after a complete `unpack ok` / `ok <ref>` packet but
// before the outer `0000` is a truncated report and yields `None`
// (uncertain), not a successful report.
let (stripped, outer_flush) = strip_sideband(text, false)?;
// The report-status vocabulary (`unpack ...`, `ok ...`, `ng ...`)
// never opens with four hex digits, so a hex quartet here is inner
// pkt-line framing, not report text.
let stripped = if stripped.len() >= 4
&& stripped.as_bytes()[..4]
.iter()
.all(|b| b.is_ascii_hexdigit())
{
strip_sideband(&stripped, true)?.0
} else if !outer_flush {
return None;
} else {
stripped
};
// The report is framed TWICE when the client negotiated side-band-64k,
// which `git push` over smart HTTP does — so this is the common case, not
// an exotic one. The outer frame is the side-band envelope; band 1 carries
// the report-status stream, which is ITSELF pkt-line encoded. Real bytes
// from git 2.50.1 rejecting one ref of a two-ref push:
//
// 0057\x01000eunpack ok\n0028ng refs/heads/main non-fast-forward\n...
//
// After one pass the first line reads `000eunpack ok`, which fails the
// `unpack ok` check below and makes this return None — i.e. "no report",
// which the caller treats as inconclusive and keeps every declared ref.
// The per-ref gate would then be inert for exactly the pushes it exists to
// filter. The second pass above already removed the inner pkt-line
// framing where present (and required its flush); it is a no-op
// on the single-framed shape, because plain report text does not begin
// with four hex digits.
let lines: Vec<&str> = stripped.lines().collect();
if lines.is_empty() {
return None;
}
// First non-empty line is "unpack ok" or "unpack fail".
let unpack_line = lines.iter().find(|l| !l.is_empty())?;
let unpack_ok = if unpack_line.starts_with("unpack ok") {
true
} else if unpack_line.starts_with("unpack fail") {
false
} else {
return None;
};
let mut results = Vec::new();
for line in &lines[1..] {
let line = line.trim();
if line.is_empty() {
break;
}
if let Some(rest) = line.strip_prefix("ok ") {
results.push((rest.to_string(), true));
} else if let Some(rest) = line.strip_prefix("ng ") {
// "ng <refname> <reason>" — skip the reason
let ref_name = rest.split_whitespace().next()?.to_string();
results.push((ref_name, false));
}
}
Some((unpack_ok, results))
}
/// Strip git sideband framing from a pkt-line encoded output.
/// Sideband-encoded lines start with a 4-hex-digit length, then a
/// channel byte (0x01=stdout, 0x02=stderr), then payload. Returns the
/// decoded payload plus whether the stream saw its terminating flush
/// packet, or `None` if the framing is malformed. With `require_flush`,
/// a stream that never sees its flush is also `None` — the caller uses
/// this for the level whose completion marker is authoritative (see
/// [`parse_report_status`]).
fn strip_sideband(text: &str, require_flush: bool) -> Option<(String, bool)> {
let mut output = String::new();
let mut pos = 0;
let bytes = text.as_bytes();
let mut saw_flush = false;
loop {
if pos == bytes.len() {
break;
}
if pos + 4 > bytes.len() {
// Truncated framing mid-header: fail closed.
return None;
}
let len_str = std::str::from_utf8(&bytes[pos..pos + 4]).ok()?;
let len = usize::from_str_radix(len_str, 16).ok()?;
if len == 0 {
// Flush packet — end of sideband stream
saw_flush = true;
break;
}
if len < 4 || pos + len > bytes.len() {
// Truncated pkt-line extends past EOF: incomplete framing.
return None;
}
let pkt_data = std::str::from_utf8(&bytes[pos + 4..pos + len]).ok()?;
pos += len;
// Sideband: first byte is channel (1=stdout, 2=stderr)
if let Some(payload) = pkt_data.strip_prefix('\x01') {
output.push_str(payload);
} else if pkt_data.starts_with('\x02') {
// stderr — skip (git error messages)
} else {
// Not sideband encoded — pass through
output.push_str(pkt_data);
}
}
if require_flush && !saw_flush {
// Truncated at a packet boundary: every packet parsed, but the
// stream never completed. Indistinguishable from a report git was
// killed mid-write, so no authority.
return None;
}
if output.is_empty() {
None
} else {
Some((output, saw_flush))
}
}
/// Sends SIGTERM to a child's whole process group on drop, unless disarmed first. /// Sends SIGTERM to a child's whole process group on drop, unless disarmed first.
/// ///
/// A served `git upload-pack`/`receive-pack` forks helpers such as `pack-objects`. /// A served `git upload-pack`/`receive-pack` forks helpers such as `pack-objects`.
@@ -578,6 +792,121 @@ async fn drive_git_child(
Ok((out, admission)) Ok((out, admission))
} }
/// Like [`drive_git_child`], but returns stdout and stderr even on a
/// non-zero exit status. Used by [`receive_pack_raw`] so the caller
/// can parse the report-status output from a failed `git-receive-pack`.
async fn drive_git_child_raw(
mut command: Command,
input: Bytes,
timeout: Duration,
what: &str,
admission: Option<AdmissionGuard>,
) -> Result<(
Vec<u8>,
Vec<u8>,
std::process::ExitStatus,
Option<AdmissionGuard>,
)> {
command
.stdin(Stdio::piped())
.stdout(Stdio::piped())
.stderr(Stdio::piped());
#[cfg(unix)]
command.process_group(0);
let mut child = command.spawn()?;
let mut stdin = child.stdin.take();
let mut stdout = child.stdout.take().context("git stdout was not piped")?;
let mut stderr = child.stderr.take().context("git stderr was not piped")?;
#[cfg(unix)]
let pgid = child.id().map(|id| id as i32);
#[cfg(unix)]
let mut group_guard = KillGroupOnDrop {
child: Some(child),
pgid,
admission,
};
let mut out = Vec::new();
let mut err = Vec::new();
let interact = async {
let write = async {
match stdin.take() {
Some(mut s) => s.write_all(&input).await,
None => Ok(()),
}
};
#[cfg(unix)]
let child_ref = group_guard.child_mut();
#[cfg(not(unix))]
let child_ref = &mut child;
let (write_result, r_out, r_err, status) = tokio::join!(
write,
stdout.read_to_end(&mut out),
stderr.read_to_end(&mut err),
child_ref.wait(),
);
r_out?;
r_err?;
Ok::<_, anyhow::Error>((write_result, status?))
};
let timed = tokio::time::timeout(timeout, interact).await;
let (write_result, status, admission) = match timed {
Ok(result) => {
#[cfg(unix)]
let admission = group_guard.disarm();
let (write_result, status) = result?;
(write_result, status, admission)
}
Err(_elapsed) => {
#[cfg(unix)]
{
reap_group_on_timeout(group_guard.child_mut()).await;
drop(group_guard.disarm());
}
#[cfg(not(unix))]
{
let _ = child.start_kill();
let _ = tokio::time::timeout(Duration::from_secs(2), child.wait()).await;
drop(admission);
}
return Err(GitServiceTimeout.into());
}
};
// Prefer the child's collected answer over a stdin-write error: an early
// rejection (unpack failure, command-level deny) makes git print a complete
// report-status and exit WITHOUT draining stdin, so a body larger than the
// pipe buffer EPIPEs the write while the answer is already on stdout. The
// child is reaped (the join above waited on it), so `out`/`err`/`status`
// are complete and authoritative — returning Err here would discard them:
// `receive_pack_raw_with_reflog` would propagate the write error and the
// handler would mark every prepared row `uncertain` instead of driving
// `cancelled` per the report. That durable fate is the effect: on a
// non-zero exit the client still gets `Err(AppError::Git)` (the handler
// maps the failed run itself, not this write), while on exit 0 an error
// here would answer a push git actually landed with a server error. The
// non-raw `drive_git_child` orders the same way (exit status and git's own
// failure first, stdin-write error last — pinned by
// `run_git_service_surfaces_git_stderr_over_a_stdin_epipe`), but it must
// fail on non-zero because its callers do not parse output from a failed
// child; the raw driver exists precisely to parse that output.
if let Err(e) = &write_result {
tracing::warn!(
err = %e,
what = %what,
"git stdin write failed after child exit; surfacing collected output"
);
}
Ok((out, err, status, admission))
}
/// Map a validated service name to its git subcommand. Only the two fixed /// Map a validated service name to its git subcommand. Only the two fixed
/// subcommands can come out, so an unrecognized service can never reach git's argv. /// subcommands can come out, so an unrecognized service can never reach git's argv.
fn service_to_command(service: &str) -> Result<&'static str> { fn service_to_command(service: &str) -> Result<&'static str> {
@@ -868,6 +1197,86 @@ mod tests {
use std::process::Command; use std::process::Command;
use tempfile::TempDir; use tempfile::TempDir;
/// Byte-for-byte capture from `git receive-pack` 2.50.1 rejecting one ref of
/// a two-ref push, with side-band-64k negotiated — what `git push` over
/// smart HTTP actually sends. The report is framed twice: the outer
/// side-band envelope, then the report-status stream's own pkt-lines.
///
/// This fixture is the point of the test. A hand-written single-framed
/// string parses fine with only one strip pass, so a fixture invented to
/// match the parser hides the exact case the parser is for.
const REAL_SIDEBAND_REPORT: &[u8] =
b"0057\x01000eunpack ok\n0028ng refs/heads/main non-fast-forward\n0018ok refs/heads/third\n0000";
#[test]
fn a_real_double_framed_sideband_report_is_read_not_treated_as_absent() {
let (unpack_ok, refs) =
parse_report_status(REAL_SIDEBAND_REPORT).expect("a real git report must parse");
assert!(unpack_ok, "unpack line must be read through both frames");
assert_eq!(
refs,
vec![
("refs/heads/main".to_string(), false),
("refs/heads/third".to_string(), true),
],
"the rejected ref must be distinguished from the accepted one"
);
}
/// The single-framed shape must keep working: the second pass has to be a
/// no-op there, not a corruption. Plain report text does not begin with
/// four hex digits, which is what makes that safe.
#[test]
fn a_single_framed_report_still_parses_after_the_second_pass() {
// Framed programmatically rather than by a hand-counted hex prefix: a
// wrong length makes the parser return None, which would look exactly
// like the regression this pair of tests exists to catch.
let payload = "\x01unpack ok\nok refs/heads/x\n";
let single = format!("{:04x}{payload}0000", payload.len() + 4);
let (unpack_ok, refs) =
parse_report_status(single.as_bytes()).expect("single-framed must parse");
assert!(unpack_ok);
assert_eq!(refs, vec![("refs/heads/x".to_string(), true)]);
}
#[test]
fn truncated_mid_packet_report_is_indeterminate_not_partial() {
// Truncate a valid single-framed report mid-packet (overrun):
// must return None so the handler persists the whole request as
// indeterminate instead of committing a partial ok set.
let payload = "\x01unpack ok\nok refs/heads/x\n";
let single = format!("{:04x}{payload}0000", payload.len() + 4);
let truncated = &single.as_bytes()[..single.len() - 6];
assert!(
parse_report_status(truncated).is_none(),
"mid-packet truncation must be indeterminate"
);
}
/// Truncation exactly at a packet boundary (every packet complete, but
/// the outer flush never arrived — e.g. git killed after the last `ok`
/// line) must also be indeterminate: the payload is complete but the
/// frame is not, and framing completeness is part of authority. The
/// mid-packet sibling above cannot catch this; only the flush gate can.
/// Removing the `require_flush` enforcement returns `Some` and turns
/// this red.
#[test]
fn truncated_at_packet_boundary_report_is_indeterminate_not_partial() {
let payload = "\x01unpack ok\nok refs/heads/x\n";
let single = format!("{:04x}{payload}0000", payload.len() + 4);
let no_flush = &single.as_bytes()[..single.len() - 4];
assert!(
parse_report_status(no_flush).is_none(),
"a report missing its outer flush must be indeterminate, \
even with every packet intact"
);
// Sanity: the untruncated fixture still parses (flush present).
assert!(
parse_report_status(single.as_bytes()).is_some(),
"the same bytes with the flush must parse"
);
}
#[test] #[test]
fn service_to_command_maps_allowlist_and_rejects_passthrough() { fn service_to_command_maps_allowlist_and_rejects_passthrough() {
assert_eq!( assert_eq!(
@@ -2641,6 +3050,62 @@ mod tests {
); );
} }
// The raw driver's counterpart: a rejection-shaped early exit where git
// prints a COMPLETE report-status and exits without draining stdin, so a
// body larger than the pipe buffer (~64 KiB) EPIPEs the write while the
// answer is already on stdout. `receive_pack_raw` must surface that report
// (Ok + stdout + exit status), not discard it as a stdin-write error — an
// Err here loses parseable output: `receive_pack_raw_with_reflog`
// propagates it and the handler marks every prepared row `uncertain`
// instead of driving `cancelled` from the report (the durable fate is the
// effect; the client-visible status on a failed run is `AppError::Git`
// either way). Goes RED if the stdin-write check returns Err after the
// join. Both exit codes are pinned: receive-pack rejections and successes
// both carry reports the handler must parse.
#[cfg(unix)]
#[tokio::test]
async fn receive_pack_raw_surfaces_report_status_over_a_stdin_epipe() {
for exit_code in [0, 1] {
let tmp = tempfile::TempDir::new().unwrap();
std::fs::write(tmp.path().join("report.bin"), REAL_SIDEBAND_REPORT).unwrap();
// Never reads stdin: the pending body blocks the pipe, then EPIPEs
// once the fake exits. `cat` delivers the report first.
let git_bin = write_fake_git(
tmp.path(),
&format!("#!/bin/sh\ncat \"$(dirname \"$0\")/report.bin\"\nexit {exit_code}\n"),
);
let body = Bytes::from(vec![0u8; 256 * 1024]);
let (out, ok) = receive_pack_raw(
git_bin.to_str().unwrap(),
tmp.path(),
body,
Duration::from_secs(60),
None,
)
.await
.expect(
"a report-status printed before exit must not be discarded as a \
stdin-write error",
);
assert_eq!(
ok,
exit_code == 0,
"the exit status rides the Ok return for both report shapes"
);
let (unpack_ok, results) = parse_report_status(&out)
.expect("git's report survives the EPIPE and stays parseable");
assert!(unpack_ok, "unpack ok parses");
assert_eq!(
results,
vec![
("refs/heads/main".to_string(), false),
("refs/heads/third".to_string(), true),
],
"the per-ref report reaches the handler instead of an HTTP error"
);
}
}
// ── #192 F1: response_served_pack detector (real git output) ──────────── // ── #192 F1: response_served_pack detector (real git output) ────────────
// A completing request (want+flush+done) yields NAK + a real packfile; // A completing request (want+flush+done) yields NAK + a real packfile;
File diff suppressed because it is too large. Load diff
+4 -2
View File
@@ -627,7 +627,9 @@ impl TigrisClient {
} }
/// Compress a bare repo directory into a tar.zst byte vector. /// Compress a bare repo directory into a tar.zst byte vector.
fn compress_repo(repo_path: &Path) -> Result<Vec<u8>> { /// `pub(crate)` for sweep tests that seed a fake object store with a
/// byte-identical archive (rather than duplicating the format).
pub(crate) fn compress_repo(repo_path: &Path) -> Result<Vec<u8>> {
let buf = Vec::new(); let buf = Vec::new();
let encoder = zstd::stream::Encoder::new(buf, 3)?; // level 3 = fast + decent ratio let encoder = zstd::stream::Encoder::new(buf, 3)?; // level 3 = fast + decent ratio
let mut tar = tar::Builder::new(encoder); let mut tar = tar::Builder::new(encoder);
@@ -661,7 +663,7 @@ pub(crate) fn publish_lock(local_path: &Path) -> Arc<Mutex<()>> {
/// Owns a snapshot temp dir from extraction through handoff to [`RepoSnapshot`]. /// Owns a snapshot temp dir from extraction through handoff to [`RepoSnapshot`].
/// Dropped when the `spawn_blocking` join result is abandoned, so cancellation /// Dropped when the `spawn_blocking` join result is abandoned, so cancellation
/// before the outer future resumes still removes the directory. /// before the outer future resumes still removes the directory.
pub(crate) struct TempSnapshotDir { pub struct TempSnapshotDir {
path: PathBuf, path: PathBuf,
} }
File diff suppressed because it is too large. Load diff
+767 -4
View File
@@ -3,7 +3,7 @@ pub mod query;
pub mod subscription; pub mod subscription;
pub mod types; pub mod types;
use async_graphql::Schema; use async_graphql::{Schema, SchemaBuilder};
use std::sync::Arc; use std::sync::Arc;
use crate::db::Db; use crate::db::Db;
@@ -14,6 +14,22 @@ use subscription::SubscriptionRoot;
pub type GitlawbSchema = Schema<QueryRoot, MutationRoot, SubscriptionRoot>; pub type GitlawbSchema = Schema<QueryRoot, MutationRoot, SubscriptionRoot>;
// Keep ordinary schema discovery and application queries usable while
// bounding validation work and the number of resolver selections per request.
const GRAPHQL_MAX_COMPLEXITY: usize = 400;
// The current public schema is shallow; this leaves headroom for composed
// clients and the canonical getIntrospectionQuery without allowing
// recursively nested documents to grow unchecked.
const GRAPHQL_MAX_DEPTH: usize = 14;
fn apply_query_limits<Query, Mutation, Subscription>(
builder: SchemaBuilder<Query, Mutation, Subscription>,
) -> SchemaBuilder<Query, Mutation, Subscription> {
builder
.limit_complexity(GRAPHQL_MAX_COMPLEXITY)
.limit_depth(GRAPHQL_MAX_DEPTH)
}
/// Client-facing message for GraphQL resolver failures that wrap a real /// Client-facing message for GraphQL resolver failures that wrap a real
/// `sqlx::Error`. The real error is logged server-side; never put sqlx/Postgres /// `sqlx::Error`. The real error is logged server-side; never put sqlx/Postgres
/// detail in the GraphQL `errors` array (#250). /// detail in the GraphQL `errors` array (#250).
@@ -60,6 +76,7 @@ pub(crate) fn graphql_app_err(e: crate::error::AppError) -> async_graphql::Error
// Curated client-safe variants — `Display` is intentional API text. // Curated client-safe variants — `Display` is intentional API text.
safe @ (crate::error::AppError::RepoNotFound(_) safe @ (crate::error::AppError::RepoNotFound(_)
| crate::error::AppError::RepoExists(_) | crate::error::AppError::RepoExists(_)
| crate::error::AppError::Conflict(_)
| crate::error::AppError::NotFound(_) | crate::error::AppError::NotFound(_)
| crate::error::AppError::Unauthorized(_) | crate::error::AppError::Unauthorized(_)
| crate::error::AppError::Forbidden(_) | crate::error::AppError::Forbidden(_)
@@ -76,21 +93,91 @@ pub(crate) fn graphql_app_err(e: crate::error::AppError) -> async_graphql::Error
} }
} }
/// Classify a failed `claim_task` for GraphQL exactly as REST's claim handler
/// classifies it: a lost race is a client-safe conflict, a real sqlx fault
/// stays opaque. Lives here rather than at the three call sites so the
/// classification cannot drift between transports (#327 review), and so
/// `every_graphql_map_err_uses_opaque_helpers` can keep whitelisting by name.
pub(crate) fn graphql_claim_conflict(e: anyhow::Error) -> async_graphql::Error {
graphql_app_err(crate::api::tasks::task_write_conflict(
e,
"task not claimable: not found or already claimed",
))
}
/// The `finish_task` half of [`graphql_claim_conflict`], covering both
/// `completeTask` and `failTask`.
pub(crate) fn graphql_finish_conflict(e: anyhow::Error) -> async_graphql::Error {
graphql_app_err(crate::api::tasks::task_write_conflict(
e,
"task not found or not in claimed state",
))
}
pub struct TaskReadBrakeExtension;
impl async_graphql::extensions::ExtensionFactory for TaskReadBrakeExtension {
fn create(&self) -> Arc<dyn async_graphql::extensions::Extension> {
Arc::new(TaskReadBrakeExtensionImpl)
}
}
use async_graphql::async_trait::async_trait;
struct TaskReadBrakeExtensionImpl;
#[async_trait]
impl async_graphql::extensions::Extension for TaskReadBrakeExtensionImpl {
async fn prepare_request(
&self,
ctx: &async_graphql::extensions::ExtensionContext<'_>,
mut request: async_graphql::Request,
next: async_graphql::extensions::NextPrepareRequest<'_>,
) -> async_graphql::ServerResult<async_graphql::Request> {
if let Some(session_brake) = ctx
.session_data
.get(&std::any::TypeId::of::<crate::rate_limit::TaskReadBrake>())
.and_then(|d| d.downcast_ref::<crate::rate_limit::TaskReadBrake>())
{
request = request.data(crate::rate_limit::TaskReadBrake {
limiter: session_brake.limiter.clone(),
key: session_brake.key.clone(),
request_count: Arc::new(std::sync::atomic::AtomicUsize::new(0)),
});
}
if let Some(did) = ctx
.session_data
.get(&std::any::TypeId::of::<crate::auth::AuthenticatedDid>())
.and_then(|d| d.downcast_ref::<crate::auth::AuthenticatedDid>())
{
request = request.data(did.clone());
}
next.run(ctx, request).await
}
}
pub fn build_schema( pub fn build_schema(
db: Arc<Db>, db: Arc<Db>,
ref_update_tx: tokio::sync::broadcast::Sender<RefUpdateBroadcast>, ref_update_tx: tokio::sync::broadcast::Sender<RefUpdateBroadcast>,
task_event_tx: tokio::sync::broadcast::Sender<TaskEventBroadcast>, task_event_tx: tokio::sync::broadcast::Sender<TaskEventBroadcast>,
task_cursor_key: crate::api::task_cursor::TaskCursorKey,
) -> GitlawbSchema { ) -> GitlawbSchema {
Schema::build(QueryRoot, MutationRoot, SubscriptionRoot) apply_query_limits(Schema::build(QueryRoot, MutationRoot, SubscriptionRoot))
.data(db) .data(db)
.data(ref_update_tx) .data(ref_update_tx)
.data(task_event_tx) .data(task_event_tx)
.data(task_cursor_key)
.extension(TaskReadBrakeExtension)
.finish() .finish()
} }
#[cfg(test)] #[cfg(test)]
mod tests { mod tests {
use super::*; use super::*;
use crate::api::events::MAX_VISIBLE_REF_UPDATES;
use crate::db::{MAX_VISIBLE_REPO_PAGE_SIZE, MAX_VISIBLE_TASK_PAGE_SIZE};
use async_graphql::{EmptyMutation, EmptySubscription, Object, Value};
use std::sync::atomic::{AtomicUsize, Ordering};
#[test] #[test]
fn graphql_db_err_opaques_sqlx_chain() { fn graphql_db_err_opaques_sqlx_chain() {
@@ -160,9 +247,683 @@ mod tests {
assert!(!err.message.contains("failed to open")); assert!(!err.message.contains("failed to open"));
} }
fn production_test_schema() -> GitlawbSchema {
let pool = sqlx::postgres::PgPoolOptions::new()
.connect_lazy("postgres://localhost/unused")
.unwrap();
production_test_schema_with_db(Arc::new(Db::for_testing(pool)))
}
fn production_test_schema_with_db(db: Arc<Db>) -> GitlawbSchema {
build_schema(
db,
tokio::sync::broadcast::channel(1).0,
tokio::sync::broadcast::channel(1).0,
crate::api::task_cursor::TaskCursorKey::derive(&[42u8; 32]),
)
}
#[tokio::test]
async fn ref_update_subscription_alias_budget() {
assert_subscription_alias_budget("refUpdates { repo }").await;
}
#[tokio::test]
async fn task_event_subscription_alias_budget() {
assert_subscription_alias_budget("taskEvents { taskId }").await;
}
async fn assert_subscription_alias_budget(field: &str) {
use futures::StreamExt;
use std::time::Duration;
let pool = sqlx::postgres::PgPoolOptions::new()
.connect_lazy("postgres://localhost/unused")
.unwrap();
let (ref_tx, _) = tokio::sync::broadcast::channel(16);
let (task_tx, _) = tokio::sync::broadcast::channel(16);
let schema = build_schema(
Arc::new(Db::for_testing(pool)),
ref_tx.clone(),
task_tx.clone(),
crate::api::task_cursor::TaskCursorKey::derive(&[42u8; 32]),
);
for count in [7, 8] {
let fields = (0..count)
.map(|n| format!("r{n}: {field}"))
.collect::<Vec<_>>()
.join(" ");
let query = format!("subscription {{ {fields} }}");
let mut responses = Box::pin(schema.execute_stream(query));
if count == 8 {
let response = tokio::time::timeout(Duration::from_secs(1), responses.next())
.await
.expect("over-budget subscriptions must reject before waiting for events")
.expect("the validation error must be returned");
assert_eq!(response.errors.len(), 1);
assert_eq!(response.errors[0].message, "Query is too complex.");
assert_eq!(ref_tx.receiver_count() + task_tx.receiver_count(), 0);
continue;
}
// Poll the accepted operation far enough to register its receivers.
assert!(
tokio::time::timeout(Duration::from_millis(30), responses.next())
.await
.is_err()
);
assert_eq!(ref_tx.receiver_count() + task_tx.receiver_count(), 7);
let _ = ref_tx.send(RefUpdateBroadcast {
repo: "owner/repo".into(),
ref_name: "refs/heads/main".into(),
old_sha: "0".repeat(40),
new_sha: "1".repeat(40),
pusher_did: "did:key:test".into(),
node_did: "did:key:node".into(),
timestamp: "2026-01-01T00:00:00Z".into(),
owner_did: "did:key:test".into(),
});
let _ = task_tx.send(TaskEventBroadcast {
task_id: "task".into(),
old_status: "pending".into(),
new_status: "claimed".into(),
by_did: "did:key:test".into(),
at: "2026-01-01T00:00:00Z".into(),
});
let response = tokio::time::timeout(Duration::from_secs(1), responses.next())
.await
.unwrap()
.unwrap();
assert!(response.errors.is_empty(), "{:?}", response.errors);
assert_ne!(response.data, Value::Null);
drop(responses);
assert_eq!(ref_tx.receiver_count() + task_tx.receiver_count(), 0);
}
}
#[sqlx::test]
async fn production_mutation_budget_accepts_seven_and_rejects_eight(pool: sqlx::PgPool) {
let db = Arc::new(Db::for_testing(pool));
db.run_migrations().await.unwrap();
let caller = "did:key:reviewer";
let now = chrono::Utc::now().to_rfc3339();
for index in 0..8 {
db.create_task(&crate::db::AgentTask {
id: format!("task-{index}"),
repo_id: None,
kind: "test".into(),
status: "pending".into(),
delegator_did: caller.into(),
assignee_did: None,
capability: "test".into(),
ucan_token: None,
payload: None,
result: None,
created_at: now.clone(),
updated_at: now.clone(),
deadline: None,
})
.await
.unwrap();
}
let schema = production_test_schema_with_db(db.clone());
for count in [8, 7] {
let fields = (0..count)
.map(|n| {
format!("r{n}: claimTask(id: \"task-{n}\", assigneeDid: \"{caller}\") {{ id }}")
})
.collect::<Vec<_>>()
.join(" ");
let response = schema
.execute(
async_graphql::Request::new(format!("mutation {{ {fields} }}"))
.data(crate::auth::AuthenticatedDid(caller.into())),
)
.await;
if count == 8 {
assert_eq!(response.errors.len(), 1);
assert_eq!(response.errors[0].message, "Query is too complex.");
assert_eq!(
db.list_tasks_keyset(Some("pending"), None, 8, None)
.await
.unwrap()
.len(),
8
);
continue;
}
assert!(response.errors.is_empty(), "{:?}", response.errors);
assert_eq!(
response
.data
.into_json()
.unwrap()
.as_object()
.unwrap()
.len(),
7
);
assert_eq!(
db.list_tasks_keyset(Some("claimed"), Some(caller), 8, None)
.await
.unwrap()
.len(),
7
);
assert_eq!(
db.get_task("task-7").await.unwrap().unwrap().status,
"pending"
);
}
}
#[sqlx::test]
async fn production_list_cost_scales_with_the_same_alias_count(pool: sqlx::PgPool) {
let db = Arc::new(Db::for_testing(pool));
db.run_migrations().await.unwrap();
let schema = production_test_schema_with_db(db);
// Bounds are per-root named constants so the cases move with the
// documented maximums instead of going stale as literals.
for (field, selection, max) in [
("refUpdates", "repo", MAX_VISIBLE_REF_UPDATES),
("tasks", "items { id }", MAX_VISIBLE_TASK_PAGE_SIZE),
(
"reposPage",
"nodes { name }",
MAX_VISIBLE_REPO_PAGE_SIZE as i64,
),
] {
// Two aliases at the maximum always exceed the budget.
let fields = (0..2)
.map(|n| format!("r{n}: {field}(limit: {max}) {{ {selection} }}"))
.collect::<Vec<_>>()
.join(" ");
let response = schema.execute(format!("{{ {fields} }}")).await;
assert_eq!(response.errors.len(), 1);
assert_eq!(response.errors[0].message, "Query is too complex.");
// At limit 1 two aliases fit for the row-priced roots; reposPage
// prices the full corpus scan, so a single alias is the most that
// fits under the budget.
let aliases = if field == "reposPage" { 1 } else { 2 };
let fields = (0..aliases)
.map(|n| format!("r{n}: {field}(limit: 1) {{ {selection} }}"))
.collect::<Vec<_>>()
.join(" ");
let response = schema.execute(format!("{{ {fields} }}")).await;
assert!(response.errors.is_empty(), "{field}: {:?}", response.errors);
assert_eq!(
response
.data
.into_json()
.unwrap()
.as_object()
.unwrap()
.len(),
aliases
);
}
}
#[tokio::test]
async fn production_limits_reject_mutation_aliases_and_large_lists() {
let schema = production_test_schema();
// Limits are derived from the MAX_* constants so the boundaries move
// with the documented maximums instead of going stale as literals.
for (prefix, count, field) in [
("mutation", 8, "claimTask(id: \"missing\", assigneeDid: \"did:key:test\") { id }".to_owned()),
("mutation", 8, "createTask(delegatorDid: \"did:key:test\", input: { kind: \"test\", capability: \"test\" }) { id }".to_owned()),
("mutation", 8, "completeTask(id: \"missing\", byDid: \"did:key:test\", input: {}) { id }".to_owned()),
("mutation", 8, "failTask(id: \"missing\", byDid: \"did:key:test\", input: {}) { id }".to_owned()),
("query", 8, "task(id: \"missing\") { id }".to_owned()),
("query", 2, format!("refUpdates(limit: {MAX_VISIBLE_REF_UPDATES}) {{ repo }}")),
("query", 2, format!("tasks(limit: {MAX_VISIBLE_TASK_PAGE_SIZE}) {{ id }}")),
("query", 2, format!("reposPage(limit: {MAX_VISIBLE_REPO_PAGE_SIZE}) {{ nodes {{ name }} }}")),
("query", 8, "reposPage(limit: 1) { nodes { name } }".to_owned()),
] {
let fields = (0..count)
.map(|n| format!("r{n}: {field}"))
.collect::<Vec<_>>()
.join(" ");
let response = schema.execute(format!("{prefix} {{ {fields} }}")).await;
assert_eq!(response.errors.len(), 1, "{:?}", response.errors);
assert_eq!(response.errors[0].message, "Query is too complex.");
}
}
#[sqlx::test]
async fn production_repos_page_serves_the_documented_maximum(pool: sqlx::PgPool) {
let db = Arc::new(Db::for_testing(pool));
db.run_migrations().await.unwrap();
let schema = production_test_schema_with_db(db);
for selection in [
"nodes { name }",
"nodes { name ownerDid } hasNextPage endCursor",
] {
let response = schema
.execute(format!(
"{{ reposPage(limit: {MAX_VISIBLE_REPO_PAGE_SIZE}) {{ {selection} }} }}"
))
.await;
assert!(response.errors.is_empty(), "{:?}", response.errors);
assert_eq!(
response.data.into_json().unwrap()["reposPage"]["nodes"],
serde_json::json!([])
);
}
}
#[tokio::test]
async fn seven_root_aliases_are_accepted() {
struct Root(Arc<AtomicUsize>);
#[Object]
impl Root {
#[graphql(complexity = "50 + child_complexity")]
async fn repos(&self) -> Vec<Nested> {
self.0.fetch_add(1, Ordering::Relaxed);
vec![Nested]
}
}
let calls = Arc::new(AtomicUsize::new(0));
let schema = apply_query_limits(Schema::build(
Root(calls.clone()),
EmptyMutation,
EmptySubscription,
))
.finish();
let fields = (0..7)
.map(|n| format!("r{n}: repos {{ value }}"))
.collect::<Vec<_>>()
.join(" ");
let response = schema.execute(format!("{{ {fields} }}")).await;
assert!(response.errors.is_empty(), "{:?}", response.errors);
assert_eq!(calls.load(Ordering::Relaxed), 7);
}
#[tokio::test]
async fn expensive_root_aliases_are_rejected_before_database_access() {
// Use the production builder with a lazy pool: rejection must precede DB access.
let schema = production_test_schema();
let fields = (0..8)
.map(|n| format!("r{n}: repos {{ name }}"))
.collect::<Vec<_>>()
.join(" ");
let response = schema.execute(format!("{{ {fields} }}")).await;
assert_eq!(response.data, Value::Null);
assert_eq!(response.errors.len(), 1);
assert_eq!(response.errors[0].message, "Query is too complex.");
}
#[tokio::test]
async fn ordinary_schema_introspection_remains_available() {
let schema = production_test_schema();
let canonical_introspection = r#"
query IntrospectionQuery {
__schema {
queryType { name }
mutationType { name }
subscriptionType { name }
types {
...FullType
}
directives {
name
description
locations
args {
...InputValue
}
}
}
}
fragment FullType on __Type {
kind
name
description
fields(includeDeprecated: true) {
name
description
args {
...InputValue
}
type {
...TypeRef
}
isDeprecated
deprecationReason
}
inputFields {
...InputValue
}
interfaces {
...TypeRef
}
enumValues(includeDeprecated: true) {
name
description
isDeprecated
deprecationReason
}
possibleTypes {
...TypeRef
}
}
fragment InputValue on __InputValue {
name
description
type { ...TypeRef }
defaultValue
}
fragment TypeRef on __Type {
kind
name
ofType {
kind
name
ofType {
kind
name
ofType {
kind
name
ofType {
kind
name
ofType {
kind
name
ofType {
kind
name
ofType {
kind
name
}
}
}
}
}
}
}
}
"#;
let response = schema.execute(canonical_introspection).await;
assert!(
response.errors.is_empty(),
"canonical introspection query failed against production schema: {:?}",
response.errors
);
assert_ne!(response.data, Value::Null);
}
#[tokio::test]
async fn ref_updates_and_tasks_single_alias_max_limit_complexity_contract() {
let pool = sqlx::postgres::PgPoolOptions::new()
.connect_lazy("postgres://localhost/unused")
.unwrap();
pool.close().await;
let (ref_tx, _) = tokio::sync::broadcast::channel(16);
let (task_tx, _) = tokio::sync::broadcast::channel(16);
let schema = build_schema(
Arc::new(Db::for_testing(pool)),
ref_tx,
task_tx,
crate::api::task_cursor::TaskCursorKey::derive(&[42u8; 32]),
);
// refUpdates(limit: MAX_VISIBLE_REF_UPDATES):
// Single-field selection has child_complexity 1 -> cost 100 + 200 + 1 = 301 <= 400.
// It passes validation and reaches the resolver (which yields db error on lazy pool).
let single_ref = schema
.execute(format!(
"{{ refUpdates(limit: {MAX_VISIBLE_REF_UPDATES}) {{ repo }} }}"
))
.await;
assert!(
!single_ref
.errors
.iter()
.any(|e| e.message == "Query is too complex."),
"single-field refUpdates at max limit must pass complexity validation: {:?}",
single_ref.errors
);
assert!(
single_ref
.errors
.iter()
.any(|e| e.message == GRAPHQL_DB_ERROR_MESSAGE),
"single-field refUpdates at max limit must reach the resolver: {:?}",
single_ref.errors
);
// Two-field selection has child_complexity 2 -> cost 100 + 200 + 2 = 302 <= 400.
// The documented max-200 request stays reachable with multiple fields
// (the old multiplicative price rejected it at 450); it passes
// validation and reaches the resolver like the single-field case.
let multi_ref = schema
.execute(format!(
"{{ refUpdates(limit: {MAX_VISIBLE_REF_UPDATES}) {{ repo refName }} }}"
))
.await;
assert!(
!multi_ref
.errors
.iter()
.any(|e| e.message == "Query is too complex."),
"two-field refUpdates at max limit must pass complexity validation: {:?}",
multi_ref.errors
);
assert!(
multi_ref
.errors
.iter()
.any(|e| e.message == GRAPHQL_DB_ERROR_MESSAGE),
"two-field refUpdates at max limit must reach the resolver: {:?}",
multi_ref.errors
);
// tasks(limit: MAX_VISIBLE_TASK_PAGE_SIZE):
// Selection items { id } has child_complexity 2, so the cost is
// 50 + MAX_VISIBLE_TASK_PAGE_SIZE + 2 <= 400.
// It passes validation and reaches the resolver.
let single_task = schema
.execute(format!(
"{{ tasks(limit: {MAX_VISIBLE_TASK_PAGE_SIZE}) {{ items {{ id }} }} }}"
))
.await;
assert!(
!single_task
.errors
.iter()
.any(|e| e.message == "Query is too complex."),
"single-field tasks at the maximum page size must pass complexity validation: {:?}",
single_task.errors
);
assert!(
single_task
.errors
.iter()
.any(|e| e.message == GRAPHQL_DB_ERROR_MESSAGE),
"single-field tasks at the maximum page size must reach the resolver: {:?}",
single_task.errors
);
// Selection items { id status } has child_complexity 3, so the cost is
// 50 + MAX_VISIBLE_TASK_PAGE_SIZE + 3 <= 400.
// A request at the documented maximum stays reachable with multiple
// fields (the old multiplicative price rejected it at 450); it passes
// validation and reaches the resolver like the single-field case.
let multi_task = schema
.execute(format!(
"{{ tasks(limit: {MAX_VISIBLE_TASK_PAGE_SIZE}) {{ items {{ id status }} }} }}"
))
.await;
assert!(
!multi_task
.errors
.iter()
.any(|e| e.message == "Query is too complex."),
"two-field tasks at the maximum page size must pass complexity validation: {:?}",
multi_task.errors
);
assert!(
multi_task
.errors
.iter()
.any(|e| e.message == GRAPHQL_DB_ERROR_MESSAGE),
"two-field tasks at the maximum page size must reach the resolver: {:?}",
multi_task.errors
);
}
#[tokio::test]
async fn two_repos_aliases_exceed_the_budget() {
// `repos` is priced as the full page it scans
// (50 + MAX_VISIBLE_REPO_PAGE_SIZE + child), so two `repos { name }`
// aliases cost 2 * 251 = 502 > 400 and the document is rejected before
// any resolver runs. If the price reverts to `50 + child_complexity`
// the aliases cost 2 * 51 = 102, validation passes, and the failure
// mode changes (resolver DB errors on the lazy pool): this test goes
// red on revert.
let schema = production_test_schema();
let response = schema
.execute("{ r0: repos { name } r1: repos { name } }")
.await;
assert_eq!(response.errors.len(), 1, "{:?}", response.errors);
assert_eq!(response.errors[0].message, "Query is too complex.");
}
#[tokio::test]
async fn seven_repos_page_aliases_exceed_the_budget() {
// `reposPage` is priced as the full corpus scan
// (50 + MAX_VISIBLE_REPO_PAGE_SIZE + child_complexity), so seven
// `reposPage(limit: 1) { nodes { name } }` aliases cost
// 7 * (50 + 200 + 2) = 1764 > 400 and the document is rejected
// before any resolver runs. Under the previous row-priced formula
// the same aliases cost 7 * (50 + 1 + 2) = 371, validation passes,
// and the failure mode changes (resolver DB errors on the lazy
// pool): this test goes red on revert. Eight aliases are rejected
// under both formulas, so seven is the load-bearing boundary.
let schema = production_test_schema();
let fields = (0..7)
.map(|n| format!("r{n}: reposPage(limit: 1) {{ nodes {{ name }} }}"))
.collect::<Vec<_>>()
.join(" ");
let response = schema.execute(format!("{{ {fields} }}")).await;
assert_eq!(response.errors.len(), 1, "{:?}", response.errors);
assert_eq!(response.errors[0].message, "Query is too complex.");
}
#[tokio::test]
async fn ref_updates_fixed_base_is_charged() {
// The 100 base in the refUpdates complexity formula charges for the
// collector's fixed work (loading the deduped repo set, the
// quarantined set, and the visibility rules, then walking up to 16
// keyset pages) even at limit: 1. Four aliases at limit: 1 cost
// 4 * (100 + 1 + 1) = 408 > 400 and must be rejected. If the base
// disappears, the cost drops to 4 * (1 + 1) = 8 and this test fails.
let schema = production_test_schema();
let fields = (0..4)
.map(|n| format!("r{n}: refUpdates(limit: 1) {{ repo }}"))
.collect::<Vec<_>>()
.join(" ");
let response = schema.execute(format!("{{ {fields} }}")).await;
assert_eq!(response.errors.len(), 1, "{:?}", response.errors);
assert_eq!(response.errors[0].message, "Query is too complex.");
}
#[derive(Clone, Copy)]
struct Nested;
#[Object]
impl Nested {
async fn child(&self) -> Nested {
Nested
}
async fn value(&self) -> i32 {
1
}
}
struct CountingQuery(Arc<AtomicUsize>);
#[Object]
impl CountingQuery {
async fn nested(&self) -> Nested {
self.0.fetch_add(1, Ordering::Relaxed);
Nested
}
}
#[tokio::test]
async fn query_depth_limit_accepts_fourteen_and_rejects_fifteen() {
let calls = Arc::new(AtomicUsize::new(0));
let schema = apply_query_limits(Schema::build(
CountingQuery(Arc::clone(&calls)),
EmptyMutation,
EmptySubscription,
))
.finish();
let query_at_depth = |depth: usize| {
let selection = (2..depth).fold("value".to_string(), |selection, _| {
format!("child {{ {selection} }}")
});
format!("{{ nested {{ {selection} }} }}")
};
let accepted = schema.execute(query_at_depth(GRAPHQL_MAX_DEPTH)).await;
assert!(
accepted.errors.is_empty(),
"depth {GRAPHQL_MAX_DEPTH} should be accepted: {:?}",
accepted.errors
);
assert_eq!(calls.swap(0, Ordering::Relaxed), 1);
let rejected = schema.execute(query_at_depth(GRAPHQL_MAX_DEPTH + 1)).await;
assert_eq!(rejected.data, Value::Null);
assert_eq!(rejected.errors.len(), 1);
assert_eq!(rejected.errors[0].message, "Query is nested too deep.");
assert_eq!(calls.load(Ordering::Relaxed), 0);
}
#[tokio::test]
async fn production_schema_enforces_depth_limit() {
let schema = production_test_schema();
// Depth 14 is accepted through the production build_schema path.
let of_types_14 = (0..9).fold("name".to_string(), |acc, _| format!("ofType {{ {acc} }}"));
let query_14 =
format!("{{ __schema {{ types {{ fields {{ type {{ {of_types_14} }} }} }} }} }}");
let accepted = schema.execute(&query_14).await;
assert!(accepted.errors.is_empty(), "{:?}", accepted.errors);
// Construct an introspection query of depth 15 using __schema.
// 1: __schema
// 2: types
// 3: fields
// 4: type
// 5..14: ofType (10 times)
// 15: name
// Total depth = 15 > GRAPHQL_MAX_DEPTH (14).
let of_types = (0..10).fold("name".to_string(), |acc, _| format!("ofType {{ {acc} }}"));
let query = format!("{{ __schema {{ types {{ fields {{ type {{ {of_types} }} }} }} }} }}");
let rejected = schema.execute(&query).await;
assert_eq!(rejected.data, async_graphql::Value::Null);
assert_eq!(rejected.errors.len(), 1);
assert_eq!(rejected.errors[0].message, "Query is nested too deep.");
}
/// Every `.map_err(` in the GraphQL query/mutation resolvers must route /// Every `.map_err(` in the GraphQL query/mutation resolvers must route
/// through the opaque helpers, or discard the error (`|_|`). Same source- /// through one of the curated helpers in this module, or discard the error
/// scrape pattern as `api::authz_guard` (#255 review). /// (`|_|`). Same source-scrape pattern as `api::authz_guard` (#255
/// review). The list is deliberately a whitelist of names rather than a
/// prefix match, so adding a new mapper is a decision a reviewer sees.
#[test] #[test]
fn every_graphql_map_err_uses_opaque_helpers() { fn every_graphql_map_err_uses_opaque_helpers() {
for (file, src) in [ for (file, src) in [
@@ -177,6 +938,8 @@ mod tests {
let after = code[idx + ".map_err(".len()..].trim_start(); let after = code[idx + ".map_err(".len()..].trim_start();
let ok = after.starts_with("crate::graphql::graphql_db_err") let ok = after.starts_with("crate::graphql::graphql_db_err")
|| after.starts_with("crate::graphql::graphql_app_err") || after.starts_with("crate::graphql::graphql_app_err")
|| after.starts_with("crate::graphql::graphql_claim_conflict")
|| after.starts_with("crate::graphql::graphql_finish_conflict")
|| after.starts_with("|_|") || after.starts_with("|_|")
|| after.starts_with("|_ "); || after.starts_with("|_ ");
assert!( assert!(
+740 -44
View File
@@ -22,6 +22,7 @@ pub struct MutationRoot;
#[Object] #[Object]
impl MutationRoot { impl MutationRoot {
#[graphql(complexity = "50 + child_complexity")]
async fn create_task( async fn create_task(
&self, &self,
ctx: &Context<'_>, ctx: &Context<'_>,
@@ -36,6 +37,33 @@ impl MutationRoot {
} }
let delegator_did = caller.to_string(); let delegator_did = caller.to_string();
let db = ctx.data_unchecked::<Arc<Db>>(); let db = ctx.data_unchecked::<Arc<Db>>();
// #496: same write gate as the REST handler — a repo_id naming a
// hosted, non-quarantined repo admits tasks only from its owner, so a
// stranger cannot file under a foreign repo id. An id naming no hosted
// repo (unknown, or quarantined — hidden as if it did not exist) is
// stored verbatim as an opaque label; the gate does not bind an
// unresolved id to a repo. Task reads are caller-independent on this
// base (the tasks resolver is not repo-scoped), before and after this
// change. As in tasks.rs, the success-vs-denial split between a hosted
// repo id and an unknown one is itself a one-bit oracle — a stranger
// holding a private repo's id learns it is hosted here — but denying
// foreign repos requires drawing exactly that line.
if let Some(repo_id) = input.repo_id.as_deref() {
let record = db
.get_repo_by_id(repo_id)
.await
.map_err(crate::graphql::graphql_db_err)?;
if let Some(record) = record {
let quarantined = db
.is_repo_quarantined(&record.id)
.await
.map_err(crate::graphql::graphql_db_err)?;
if !quarantined {
crate::api::require_repo_owner(&record, caller)
.map_err(crate::graphql::graphql_app_err)?;
}
}
}
let now = Utc::now().to_rfc3339(); let now = Utc::now().to_rfc3339();
let task = AgentTask { let task = AgentTask {
id: Uuid::new_v4().to_string(), id: Uuid::new_v4().to_string(),
@@ -52,12 +80,17 @@ impl MutationRoot {
updated_at: now, updated_at: now,
deadline: input.deadline, deadline: input.deadline,
}; };
db.create_task(&task) // Return the stored row, not the pre-insert input: create_task
// normalizes the assignee (whitespace-only becomes NULL/open), and a
// response that shows the raw input lies about the reservation.
let stored = db
.create_task(&task)
.await .await
.map_err(crate::graphql::graphql_db_err)?; .map_err(crate::graphql::graphql_db_err)?;
Ok(AgentTaskType::from(task)) Ok(AgentTaskType::from(stored))
} }
#[graphql(complexity = "50 + child_complexity")]
async fn claim_task( async fn claim_task(
&self, &self,
ctx: &Context<'_>, ctx: &Context<'_>,
@@ -73,20 +106,41 @@ impl MutationRoot {
let assignee_did = caller.to_string(); let assignee_did = caller.to_string();
let db = ctx.data_unchecked::<Arc<Db>>(); let db = ctx.data_unchecked::<Arc<Db>>();
let tx = ctx.data_unchecked::<tokio::sync::broadcast::Sender<TaskEventBroadcast>>(); let tx = ctx.data_unchecked::<tokio::sync::broadcast::Sender<TaskEventBroadcast>>();
crate::api::tasks::get_claimable_task(db, &id, caller)
.await
.map_err(crate::graphql::graphql_app_err)?
.ok_or_else(|| async_graphql::Error::new("task not found"))?;
// Same classification REST's claim handler applies: a claim race is a
// client-safe conflict, a genuine sqlx failure stays opaque. Routing
// both transports through `task_write_conflict` is what stops a normal
// race from reaching a GraphQL caller as a generic database error
// (#327 review).
let task = db let task = db
.claim_task(&id, &assignee_did) .claim_task(&id, &assignee_did)
.await .await
.map_err(crate::graphql::graphql_db_err)?; .map_err(crate::graphql::graphql_claim_conflict)?;
let _ = tx.send(TaskEventBroadcast { let by_did = task
task_id: id, .assignee_did
old_status: "pending".to_string(), .as_deref()
new_status: "claimed".to_string(), .map(crate::db::trim_assignee_did)
by_did: assignee_did, .unwrap_or(assignee_did.as_str())
at: Utc::now().to_rfc3339(), .to_string();
}); crate::api::tasks::announce_task_event(
db,
tx,
TaskEventBroadcast {
task_id: id,
old_status: "pending".to_string(),
new_status: "claimed".to_string(),
by_did,
at: Utc::now().to_rfc3339(),
},
)
.await;
Ok(AgentTaskType::from(task)) Ok(AgentTaskType::from(task))
} }
#[graphql(complexity = "50 + child_complexity")]
async fn complete_task( async fn complete_task(
&self, &self,
ctx: &Context<'_>, ctx: &Context<'_>,
@@ -103,32 +157,47 @@ impl MutationRoot {
let by_did = caller.to_string(); let by_did = caller.to_string();
let db = ctx.data_unchecked::<Arc<Db>>(); let db = ctx.data_unchecked::<Arc<Db>>();
let tx = ctx.data_unchecked::<tokio::sync::broadcast::Sender<TaskEventBroadcast>>(); let tx = ctx.data_unchecked::<tokio::sync::broadcast::Sender<TaskEventBroadcast>>();
// Authorize the actor: binding by_did to the signer is necessary but not // Authorize the actor: the task must be visible to the caller (returning
// sufficient — only the task's assignee may finish it. // not found for invisible tasks so existence is not leaked), and only
let existing = db // the task's assignee may finish it.
.get_task(&id) let existing = crate::api::tasks::get_visible_task(db, &id, Some(caller))
.await .await
.map_err(crate::graphql::graphql_db_err)? .map_err(crate::graphql::graphql_app_err)?
.ok_or_else(|| async_graphql::Error::new("task not found"))?; .ok_or_else(|| async_graphql::Error::new("task not found"))?;
if !crate::api::did_matches(caller, existing.assignee_did.as_deref().unwrap_or_default()) { if !crate::api::did_matches(
caller,
crate::db::trim_assignee_did(existing.assignee_did.as_deref().unwrap_or_default()),
) {
return Err(async_graphql::Error::new( return Err(async_graphql::Error::new(
"only the task assignee can complete it", "only the task assignee can complete it",
)); ));
} }
let task = db let task = db
.finish_task(&id, "completed", input.result.as_deref()) .finish_task(&id, "completed", input.result.as_deref(), caller)
.await .await
.map_err(crate::graphql::graphql_db_err)?; .map_err(crate::graphql::graphql_finish_conflict)?;
let _ = tx.send(TaskEventBroadcast { let by_did = task
task_id: id, .assignee_did
old_status: "claimed".to_string(), .as_deref()
new_status: "completed".to_string(), .map(crate::db::trim_assignee_did)
by_did, .unwrap_or(by_did.as_str())
at: Utc::now().to_rfc3339(), .to_string();
}); crate::api::tasks::announce_task_event(
db,
tx,
TaskEventBroadcast {
task_id: id,
old_status: "claimed".to_string(),
new_status: "completed".to_string(),
by_did,
at: Utc::now().to_rfc3339(),
},
)
.await;
Ok(AgentTaskType::from(task)) Ok(AgentTaskType::from(task))
} }
#[graphql(complexity = "50 + child_complexity")]
async fn fail_task( async fn fail_task(
&self, &self,
ctx: &Context<'_>, ctx: &Context<'_>,
@@ -145,29 +214,44 @@ impl MutationRoot {
let by_did = caller.to_string(); let by_did = caller.to_string();
let db = ctx.data_unchecked::<Arc<Db>>(); let db = ctx.data_unchecked::<Arc<Db>>();
let tx = ctx.data_unchecked::<tokio::sync::broadcast::Sender<TaskEventBroadcast>>(); let tx = ctx.data_unchecked::<tokio::sync::broadcast::Sender<TaskEventBroadcast>>();
// Authorize the actor: only the task's assignee may fail it. // Authorize the actor: the task must be visible to the caller (returning
let existing = db // not found for invisible tasks so existence is not leaked), and only
.get_task(&id) // the task's assignee may fail it.
let existing = crate::api::tasks::get_visible_task(db, &id, Some(caller))
.await .await
.map_err(crate::graphql::graphql_db_err)? .map_err(crate::graphql::graphql_app_err)?
.ok_or_else(|| async_graphql::Error::new("task not found"))?; .ok_or_else(|| async_graphql::Error::new("task not found"))?;
if !crate::api::did_matches(caller, existing.assignee_did.as_deref().unwrap_or_default()) { if !crate::api::did_matches(
caller,
crate::db::trim_assignee_did(existing.assignee_did.as_deref().unwrap_or_default()),
) {
return Err(async_graphql::Error::new( return Err(async_graphql::Error::new(
"only the task assignee can fail it", "only the task assignee can fail it",
)); ));
} }
let reason = input.reason.unwrap_or_default(); let reason = input.reason.unwrap_or_default();
let task = db let task = db
.finish_task(&id, "failed", Some(&reason)) .finish_task(&id, "failed", Some(&reason), caller)
.await .await
.map_err(crate::graphql::graphql_db_err)?; .map_err(crate::graphql::graphql_finish_conflict)?;
let _ = tx.send(TaskEventBroadcast { let by_did = task
task_id: id, .assignee_did
old_status: "claimed".to_string(), .as_deref()
new_status: "failed".to_string(), .map(crate::db::trim_assignee_did)
by_did, .unwrap_or(by_did.as_str())
at: Utc::now().to_rfc3339(), .to_string();
}); crate::api::tasks::announce_task_event(
db,
tx,
TaskEventBroadcast {
task_id: id,
old_status: "claimed".to_string(),
new_status: "failed".to_string(),
by_did,
at: Utc::now().to_rfc3339(),
},
)
.await;
Ok(AgentTaskType::from(task)) Ok(AgentTaskType::from(task))
} }
} }
@@ -186,6 +270,22 @@ mod tests {
.join("; ") .join("; ")
} }
/// The `id` scalar returned by a successful `createTask` mutation.
fn created_id(resp: &Response) -> String {
assert!(resp.errors.is_empty(), "graphql errors: {:?}", resp.errors);
let async_graphql::Value::Object(obj) = &resp.data else {
panic!("data not an object: {:?}", resp.data);
};
let async_graphql::Value::Object(task) = obj.get("createTask").expect("createTask key")
else {
panic!("createTask not an object");
};
let async_graphql::Value::String(id) = task.get("id").expect("id key") else {
panic!("id not a string");
};
id.clone()
}
/// N2: every GraphQL mutation requires a verified signer and binds the acting /// N2: every GraphQL mutation requires a verified signer and binds the acting
/// DID to it. Unsigned is rejected; a signer other than the claimed actor is /// DID to it. Unsigned is rejected; a signer other than the claimed actor is
/// rejected; the matching signer passes the auth gate. /// rejected; the matching signer passes the auth gate.
@@ -249,7 +349,7 @@ mod tests {
); );
if *name == "claimTask" { if *name == "claimTask" {
assert!( assert!(
errs.contains("task not claimable"), errs.contains("task not found"),
"claim race / missing task must keep its business message: {errs}" "claim race / missing task must keep its business message: {errs}"
); );
assert!( assert!(
@@ -293,6 +393,210 @@ mod tests {
); );
} }
/// #327 review: `claimTask`/`completeTask`/`failTask` called
/// `graphql_db_err` directly on db-layer business failures, so an ordinary
/// claim race or stale finish reached GraphQL clients as a generic
/// database error while REST clients got an actionable conflict. All three
/// now route through the shared `task_write_conflict` classifier.
///
/// Deleting either `map_err` from a mutation turns this red: the message
/// becomes the raw anyhow text instead of the fixed conflict form.
#[sqlx::test]
async fn task_write_races_surface_as_conflicts_not_db_errors(pool: PgPool) {
let state = crate::test_support::test_state(pool).await;
let schema = state.graphql_schema.as_ref();
let assignee = "did:key:zGQLRACEASSIGNEEAAAAAAAAAAAAAAAAAAAAAAA";
let rival = "did:key:zGQLRACERIVALBBBBBBBBBBBBBBBBBBBBBBBBBB";
let delegator = "did:key:zGQLRACEDELEGATORCCCCCCCCCCCCCCCCCCCCC";
let now = chrono::Utc::now().to_rfc3339();
state
.db
.create_repo(&crate::db::RepoRecord {
id: "race-repo".into(),
name: "race".into(),
owner_did: delegator.into(),
description: None,
is_public: true,
default_branch: "main".into(),
created_at: chrono::Utc::now(),
updated_at: chrono::Utc::now(),
disk_path: "/tmp/race-repo".into(),
forked_from: None,
machine_id: None,
})
.await
.unwrap();
state
.db
.create_task(&crate::db::AgentTask {
id: "race-task".into(),
repo_id: Some("race-repo".into()),
kind: "build".into(),
status: "pending".into(),
delegator_did: delegator.into(),
assignee_did: None,
capability: "repo:write".into(),
ucan_token: None,
payload: None,
result: None,
created_at: now.clone(),
updated_at: now,
deadline: None,
})
.await
.unwrap();
// The assignee wins the claim.
let claim = |did: &str| {
format!(r#"mutation {{ claimTask(id: "race-task", assigneeDid: "{did}") {{ id }} }}"#)
};
let resp = schema
.execute(Request::new(claim(assignee)).data(AuthenticatedDid(assignee.into())))
.await;
assert!(resp.errors.is_empty(), "first claim: {}", errors(&resp));
// The rival loses the race: ineligible caller receives opaque not-found.
let resp = schema
.execute(Request::new(claim(rival)).data(AuthenticatedDid(rival.into())))
.await;
let errs = errors(&resp);
assert!(
errs.contains("task not found"),
"ineligible caller after task is claimed must receive opaque not-found: {errs}"
);
assert!(
!errs.contains(crate::graphql::GRAPHQL_DB_ERROR_MESSAGE),
"ineligible claim is not a database failure: {errs}"
);
// A second claim that reaches the conditional SQL write (e.g. assignee claiming
// again on an already-claimed task) exercises the conflict mapper:
let resp = schema
.execute(Request::new(claim(assignee)).data(AuthenticatedDid(assignee.into())))
.await;
let errs = errors(&resp);
assert!(
errs.contains("task not claimable: not found or already claimed"),
"a lost claim race must render the fixed conflict message: {errs}"
);
assert!(
!errs.contains(crate::graphql::GRAPHQL_DB_ERROR_MESSAGE),
"a claim race is not a database failure: {errs}"
);
// A stale finish on an already-completed task is the same class.
let complete = format!(
r#"mutation {{ completeTask(id: "race-task", byDid: "{assignee}", input: {{ result: "ok" }}) {{ id }} }}"#
);
let resp = schema
.execute(Request::new(&complete).data(AuthenticatedDid(assignee.into())))
.await;
assert!(resp.errors.is_empty(), "first complete: {}", errors(&resp));
let resp = schema
.execute(Request::new(&complete).data(AuthenticatedDid(assignee.into())))
.await;
let errs = errors(&resp);
assert!(
errs.contains("task not found or not in claimed state"),
"a stale complete must render the fixed conflict message: {errs}"
);
assert!(
!errs.contains(crate::graphql::GRAPHQL_DB_ERROR_MESSAGE),
"a stale complete is not a database failure: {errs}"
);
let fail = format!(
r#"mutation {{ failTask(id: "race-task", byDid: "{assignee}", input: {{ reason: "nope" }}) {{ id }} }}"#
);
let resp = schema
.execute(Request::new(&fail).data(AuthenticatedDid(assignee.into())))
.await;
let errs = errors(&resp);
assert!(
errs.contains("task not found or not in claimed state"),
"a stale fail must render the fixed conflict message: {errs}"
);
assert!(
!errs.contains(crate::graphql::GRAPHQL_DB_ERROR_MESSAGE),
"a stale fail is not a database failure: {errs}"
);
}
/// The other half of the same classification: a genuine SQL fault on the
/// write itself must stay opaque. Without this, routing conflicts through
/// `task_write_conflict` could be "fixed" by making every db failure a
/// client-visible conflict message.
#[sqlx::test]
async fn task_write_sql_faults_stay_opaque(pool: PgPool) {
let state = crate::test_support::test_state(pool.clone()).await;
let assignee = "did:key:zGQLOPAQUEASSIGNEEAAAAAAAAAAAAAAAAAAAAA";
let delegator = "did:key:zGQLOPAQUEDELEGATORBBBBBBBBBBBBBBBBBBB";
let now = chrono::Utc::now().to_rfc3339();
state
.db
.create_task(&crate::db::AgentTask {
id: "opaque-task".into(),
repo_id: None,
kind: "build".into(),
status: "pending".into(),
delegator_did: delegator.into(),
assignee_did: Some(assignee.into()),
capability: "repo:write".into(),
ucan_token: None,
payload: None,
result: None,
created_at: now.clone(),
updated_at: now,
deadline: None,
})
.await
.unwrap();
// Fault the write itself, not the schema. Dropping a column the
// visibility pre-check also SELECTs would fail in `get_visible_task`
// and never reach `graphql_claim_conflict`, so this test would pass
// against a claim mapper that leaks (#327 review). A BEFORE UPDATE
// trigger keeps every read valid and faults only inside
// `Db::claim_task`.
sqlx::raw_sql(
"CREATE FUNCTION gl_test_fault_task_update() RETURNS trigger
LANGUAGE plpgsql AS $fn$
BEGIN RAISE EXCEPTION 'gl_test_forced_update_fault'; END;
$fn$;
CREATE TRIGGER gl_test_fault_task_update
BEFORE UPDATE ON agent_tasks
FOR EACH ROW EXECUTE FUNCTION gl_test_fault_task_update();",
)
.execute(&pool)
.await
.unwrap();
let resp = state
.graphql_schema
.execute(
Request::new(format!(
r#"mutation {{ claimTask(id: "opaque-task", assigneeDid: "{assignee}") {{ id }} }}"#
))
.data(AuthenticatedDid(assignee.into())),
)
.await;
let errs = errors(&resp);
assert!(
errs.contains(crate::graphql::GRAPHQL_DB_ERROR_MESSAGE),
"a real sqlx fault on the write must stay opaque: {errs}"
);
assert!(
!errs.contains("task not claimable"),
"a write-time sqlx fault must not be reclassified as a claim race: {errs}"
);
assert!(
!errs.contains("gl_test_forced_update_fault") && !errs.contains("trigger"),
"database text leaked through the conflict mapper: {errs}"
);
}
/// Adversarial-review GATE-1 (GraphQL): completing a task requires being its /// Adversarial-review GATE-1 (GraphQL): completing a task requires being its
/// assignee, not merely signing as the by_did you pass. A signer who is not /// assignee, not merely signing as the by_did you pass. A signer who is not
/// the assignee is rejected even though the by_did binding passes; the /// the assignee is rejected even though the by_did binding passes; the
@@ -315,7 +619,7 @@ mod tests {
payload: None, payload: None,
result: None, result: None,
created_at: now.clone(), created_at: now.clone(),
updated_at: now, updated_at: now.clone(),
deadline: None, deadline: None,
}; };
state.db.create_task(&task).await.expect("seed task"); state.db.create_task(&task).await.expect("seed task");
@@ -332,14 +636,69 @@ mod tests {
) )
}; };
// Stranger signs as themselves and passes byDid=self (so the signer // Stranger signs as themselves on a repo-less task they cannot see:
// binding passes), but is not the assignee → rejected by authorization. // invisible task returns "task not found" so existence is not leaked.
let resp = schema let resp = schema
.execute(Request::new(q(stranger)).data(AuthenticatedDid(stranger.into()))) .execute(Request::new(q(stranger)).data(AuthenticatedDid(stranger.into())))
.await; .await;
assert!(
errors(&resp).contains("task not found"),
"an invisible task must return not found, got: {}",
errors(&resp)
);
// Seed a task on a public repo that stranger CAN see, but is not assignee of:
let pub_repo = crate::db::RepoRecord {
id: "pub-r".into(),
name: "pub-r".into(),
owner_did: "did:key:zGQLDELEGATORCCCCCCCCCCCCCCCCCCCCCCCCCC".into(),
description: None,
is_public: true,
default_branch: "main".into(),
created_at: chrono::Utc::now(),
updated_at: chrono::Utc::now(),
disk_path: "/tmp/pub-r".into(),
forked_from: None,
machine_id: None,
};
state.db.create_repo(&pub_repo).await.expect("create repo");
let pub_task = crate::db::AgentTask {
id: "task-pub".into(),
repo_id: Some("pub-r".into()),
kind: "build".into(),
status: "pending".into(),
delegator_did: "did:key:zGQLDELEGATORCCCCCCCCCCCCCCCCCCCCCCCCCC".into(),
assignee_did: None,
capability: "repo:write".into(),
ucan_token: None,
payload: None,
result: None,
created_at: now.clone(),
updated_at: now,
deadline: None,
};
state
.db
.create_task(&pub_task)
.await
.expect("seed pub task");
state
.db
.claim_task("task-pub", assignee)
.await
.expect("claim pub task");
let q_pub = |actor: &str| {
format!(
r#"mutation {{ completeTask(id: "task-pub", byDid: "{actor}", input: {{}}) {{ id status }} }}"#
)
};
let resp = schema
.execute(Request::new(q_pub(stranger)).data(AuthenticatedDid(stranger.into())))
.await;
assert!( assert!(
errors(&resp).contains("assignee"), errors(&resp).contains("assignee"),
"a non-assignee signer must be rejected: {}", "a non-assignee signer on a visible task must be rejected: {}",
errors(&resp) errors(&resp)
); );
@@ -353,4 +712,341 @@ mod tests {
errors(&resp) errors(&resp)
); );
} }
/// #496 (GraphQL): createTask applies the same repo-ownership gate as the
/// REST handler — a stranger naming a hosted repo is rejected without
/// persisting anything, while the owner files. The quarantined and
/// unknown-id arms are pinned too, including that each accepted write
/// stores the supplied id verbatim, so this copy of the gate cannot regress
/// silently.
#[sqlx::test]
async fn create_task_rejects_foreign_repo_id(pool: PgPool) {
let state = crate::test_support::test_state(pool).await;
let owner = "did:key:zGQLTASKOWNERAAAAAAAAAAAAAAAAAAAAAAAAAA";
let stranger = "did:key:zGQLTASKSTRANGERBBBBBBBBBBBBBBBBBBBBBB";
let now = chrono::Utc::now();
let repo = crate::db::RepoRecord {
id: uuid::Uuid::new_v4().to_string(),
name: "gql-task-gate-repo".to_string(),
owner_did: owner.to_string(),
description: None,
is_public: true,
default_branch: "main".to_string(),
created_at: now,
updated_at: now,
disk_path: "/tmp/gql-task-gate-repo".to_string(),
forked_from: None,
machine_id: None,
};
state.db.create_repo(&repo).await.expect("seed repo");
let schema = state.graphql_schema.as_ref();
let q = |actor: &str| {
format!(
r#"mutation {{ createTask(delegatorDid: "{actor}", input: {{ kind: "build", capability: "repo:write", repoId: "{}" }}) {{ id }} }}"#,
repo.id
)
};
// Stranger signs as themselves (so the signer binding passes) but
// names the victim's repo → rejected by the ownership gate, leaking
// nothing about the repo.
let resp = schema
.execute(Request::new(q(stranger)).data(AuthenticatedDid(stranger.into())))
.await;
let errs = errors(&resp);
assert!(
errs.contains(crate::api::REPO_OWNER_ONLY_MESSAGE),
"a non-owner must not file under a foreign repo id (shared denial text): {errs}"
);
// The denial must not persist: no task row may carry the victim's
// repo id afterwards.
let stored = state
.db
.list_tasks_keyset(None, None, 50, None)
.await
.expect("list tasks");
assert!(
stored
.iter()
.all(|t| t.repo_id.as_deref() != Some(repo.id.as_str())),
"a denied write must not leave a task under the foreign repo id"
);
// The owner files under their own repo id.
let resp = schema
.execute(Request::new(q(owner)).data(AuthenticatedDid(owner.into())))
.await;
assert!(
errors(&resp).is_empty(),
"the owner should file against their repo: {}",
errors(&resp)
);
// Unknown repo ids stay accepted as opaque labels.
let q_unknown = format!(
r#"mutation {{ createTask(delegatorDid: "{stranger}", input: {{ kind: "build", capability: "repo:write", repoId: "no-such-repo-id" }}) {{ id }} }}"#
);
let resp = schema
.execute(Request::new(q_unknown).data(AuthenticatedDid(stranger.into())))
.await;
assert!(
errors(&resp).is_empty(),
"an id naming no hosted repo stays an opaque label: {}",
errors(&resp)
);
// Read back the row: the unresolved id must be stored verbatim, not
// dropped or rewritten to a repo it never named.
let stored = state
.db
.get_task(&created_id(&resp))
.await
.expect("get task")
.expect("task must be stored");
assert_eq!(
stored.repo_id.as_deref(),
Some("no-such-repo-id"),
"an unresolved id must be stored verbatim as a label"
);
// A quarantined repo id is treated as unknown (accepted, never a 403
// that would confirm the id names a real repo). This pins the
// `!quarantined` arm on this copy of the gate.
state
.db
.set_repo_quarantine(&repo.id, true)
.await
.expect("quarantine");
let resp = schema
.execute(Request::new(q(stranger)).data(AuthenticatedDid(stranger.into())))
.await;
assert!(
errors(&resp).is_empty(),
"a quarantined repo id must not 403 (existence oracle): {}",
errors(&resp)
);
// Read back the row: the quarantined id is stored verbatim as a label
// too, so the accepted write is not silently dropped.
let stored = state
.db
.get_task(&created_id(&resp))
.await
.expect("get task")
.expect("task must be stored");
assert_eq!(
stored.repo_id.as_deref(),
Some(repo.id.as_str()),
"a quarantined id must be stored verbatim as a label"
);
}
/// #496 (GraphQL): a repo-lookup fault in createTask's gate must stay
/// opaque. Dropping a column `get_repo_by_id` selects faults that lookup;
/// the resolver must return the fixed DB message and leak no schema text
/// (#250), as `create_task_db_error_message_is_opaque` pins for the
/// agent_tasks fault.
#[sqlx::test]
async fn create_task_gate_repo_lookup_fault_is_opaque(pool: PgPool) {
let state = crate::test_support::test_state(pool.clone()).await;
let owner = "did:key:zGQLTASKFAULTOWNERAAAAAAAAAAAAAAAAAAAAAAAA";
let now = chrono::Utc::now();
let repo = crate::db::RepoRecord {
id: uuid::Uuid::new_v4().to_string(),
name: "gql-task-gate-fault-repo".to_string(),
owner_did: owner.to_string(),
description: None,
is_public: true,
default_branch: "main".to_string(),
created_at: now,
updated_at: now,
disk_path: "/tmp/gql-task-gate-fault-repo".to_string(),
forked_from: None,
machine_id: None,
};
state.db.create_repo(&repo).await.expect("seed repo");
sqlx::query("ALTER TABLE repos DROP COLUMN machine_id")
.execute(&pool)
.await
.expect("drop machine_id");
let q = format!(
r#"mutation {{ createTask(delegatorDid: "{owner}", input: {{ kind: "build", capability: "repo:write", repoId: "{}" }}) {{ id }} }}"#,
repo.id
);
let resp = state
.graphql_schema
.execute(Request::new(&q).data(AuthenticatedDid(owner.into())))
.await;
let errs = errors(&resp);
assert!(
errs.contains(crate::graphql::GRAPHQL_DB_ERROR_MESSAGE),
"gate sqlx fault must be opaque: {errs}"
);
assert!(
!errs.contains("machine_id") && !errs.contains("column"),
"schema text leaked through the gate: {errs}"
);
}
/// Pre-assigned GraphQL claimTask must reject a stranger (no UCAN leak)
/// and admit the reserved assignee.
#[sqlx::test]
async fn claim_task_honors_preassigned_assignee(pool: PgPool) {
let state = crate::test_support::test_state(pool).await;
let reserved = "did:key:zGQLRESERVEDAAAAAAAAAAAAAAAAAAAAAAAAAAAA";
let thief = "did:key:zGQLTHIEFBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB";
let now = chrono::Utc::now().to_rfc3339();
let task = crate::db::AgentTask {
id: "task-gql-reserved".into(),
repo_id: None,
kind: "build".into(),
status: "pending".into(),
delegator_did: "did:key:zGQLDELEGATORCCCCCCCCCCCCCCCCCCCCCCCCCC".into(),
assignee_did: Some(reserved.into()),
capability: "repo:write".into(),
ucan_token: Some("gql-ucan-secret".into()),
payload: None,
result: None,
created_at: now.clone(),
updated_at: now,
deadline: None,
};
state.db.create_task(&task).await.expect("seed");
let schema = state.graphql_schema.as_ref();
let q = |actor: &str| {
format!(
r#"mutation {{ claimTask(id: "task-gql-reserved", assigneeDid: "{actor}") {{ id status ucanToken }} }}"#
)
};
let resp = schema
.execute(Request::new(q(thief)).data(AuthenticatedDid(thief.into())))
.await;
let errs = errors(&resp);
// The claim gate hides existence: an ineligible claimer gets the same
// not-found as a missing task, not a reserved deny that would leak it.
assert!(
errs.contains("task not found"),
"stranger claim must read as not-found, got: {errs}"
);
assert!(
!errs.contains("gql-ucan-secret"),
"UCAN must not leak in GraphQL errors: {errs}"
);
if let Ok(data) = resp.data.into_json() {
let s = data.to_string();
assert!(
!s.contains("gql-ucan-secret"),
"UCAN must not leak in GraphQL data on failed claim: {s}"
);
}
// Parity with REST: failed steal must leave the row untouched.
let still = state
.db
.get_task("task-gql-reserved")
.await
.expect("get")
.expect("task exists");
assert_eq!(
still.status, "pending",
"status must stay pending after steal"
);
assert_eq!(
still.assignee_did.as_deref(),
Some(reserved),
"assignee_did must stay reserved after steal"
);
let resp = schema
.execute(Request::new(q(reserved)).data(AuthenticatedDid(reserved.into())))
.await;
assert!(
errors(&resp).is_empty(),
"reserved assignee must claim: {}",
errors(&resp)
);
let data = resp.data.into_json().expect("json data");
assert_eq!(data["claimTask"]["status"], "claimed");
assert_eq!(data["claimTask"]["ucanToken"], "gql-ucan-secret");
}
/// create_task normalizes a whitespace-only assignee to NULL (open task),
/// so the create response must show the stored row, not echo the input: a
/// client trusting the echoed "reservation" would believe a claimable task
/// is reserved. Before the fix the response was built from the pre-insert
/// struct and returned the padded input verbatim.
#[sqlx::test]
async fn create_task_reports_stored_assignee_not_input(pool: PgPool) {
let state = crate::test_support::test_state(pool).await;
let caller = "did:key:zGQLCREATORAAAAAAAAAAAAAAAAAAAAAAAAAAAAA";
let schema = state.graphql_schema.as_ref();
let q = format!(
r#"mutation {{ createTask(delegatorDid: "{caller}", input: {{ kind: "build", capability: "repo:write", assigneeDid: " " }}) {{ id assigneeDid }} }}"#
);
let resp = schema
.execute(Request::new(q).data(AuthenticatedDid(caller.into())))
.await;
let data = resp.data.into_json().expect("json data");
assert!(
data["createTask"]["assigneeDid"].is_null(),
"whitespace-only assigneeDid must come back null (stored shape): {data}"
);
}
/// A task seeded with a whitespace-padded assignee must still be
/// completable by that assignee over GraphQL: REST and Db::finish_task
/// trim the stored value before comparing, and the GraphQL gate must do
/// the same or padded reservations finish only through REST.
#[sqlx::test]
async fn graphql_finish_accepts_padded_stored_assignee(pool: PgPool) {
let state = crate::test_support::test_state(pool).await;
let assignee = "did:key:zGQLPADDEDAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA";
let now = chrono::Utc::now().to_rfc3339();
let task = crate::db::AgentTask {
id: "task-gql-padded".into(),
repo_id: None,
kind: "build".into(),
status: "pending".into(),
delegator_did: "did:key:zGQLDELEGATORDDDDDDDDDDDDDDDDDDDDDDDDDD".into(),
assignee_did: Some(format!("\t{assignee}\n")),
capability: "repo:write".into(),
ucan_token: None,
payload: None,
result: None,
created_at: now.clone(),
updated_at: now,
deadline: None,
};
state.db.create_task(&task).await.expect("seed");
let schema = state.graphql_schema.as_ref();
let claim = format!(
r#"mutation {{ claimTask(id: "task-gql-padded", assigneeDid: "{assignee}") {{ id status }} }}"#
);
let resp = schema
.execute(Request::new(claim).data(AuthenticatedDid(assignee.into())))
.await;
assert!(
errors(&resp).is_empty(),
"reserved assignee must claim a padded reservation: {}",
errors(&resp)
);
let finish = format!(
r#"mutation {{ completeTask(id: "task-gql-padded", byDid: "{assignee}", input: {{}}) {{ id status }} }}"#
);
let resp = schema
.execute(Request::new(finish).data(AuthenticatedDid(assignee.into())))
.await;
let errs = errors(&resp);
assert!(
errs.is_empty(),
"assignee must complete a padded reservation: {errs}"
);
let data = resp.data.into_json().expect("json data");
assert_eq!(data["completeTask"]["status"], "completed");
}
} }
File diff suppressed because it is too large. Load diff
@@ -11,15 +11,16 @@ pub struct SubscriptionRoot;
#[Subscription] #[Subscription]
impl SubscriptionRoot { impl SubscriptionRoot {
/// Live ref-update stream. `/graphql/ws` is mounted outside the /// Live ref-update stream. `/graphql/ws` is mounted under the
/// `optional_signature` layer, so this resolver has NO caller identity and /// `optional_signature` layer and carries optional caller identity, but this
/// cannot gate per-subscriber — it relays whatever enters the broadcast /// resolver does not inspect it — it relays whatever enters the broadcast
/// channel to any anonymous client. Its visibility safety therefore rests /// channel to any anonymous client. Its visibility safety therefore rests
/// entirely on the WRITE side: the push handler only sends a /// entirely on the WRITE side: the push handler only sends a
/// `RefUpdateBroadcast` for repos the anonymous public may read (inside its /// `RefUpdateBroadcast` for repos the anonymous public may read (inside its
/// `if announce` block, `api/repos.rs`). This is a single-point invariant — /// `if announce` block, `api/repos.rs`). This is a single-point invariant —
/// any new sender to `ref_update_tx` MUST be `announce`-gated, or private-repo /// any new sender to `ref_update_tx` MUST be `announce`-gated, or private-repo
/// ref metadata leaks here to unauthenticated subscribers (#112/#114 class). /// ref metadata leaks here to unauthenticated subscribers (#112/#114 class).
#[graphql(complexity = "50 + child_complexity")]
async fn ref_updates( async fn ref_updates(
&self, &self,
ctx: &Context<'_>, ctx: &Context<'_>,
@@ -46,6 +47,7 @@ impl SubscriptionRoot {
}) })
} }
#[graphql(complexity = "50 + child_complexity")]
async fn task_events( async fn task_events(
&self, &self,
ctx: &Context<'_>, ctx: &Context<'_>,
+67
View File
@@ -11,6 +11,14 @@ pub struct RepoType {
pub created_at: String, pub created_at: String,
} }
#[derive(SimpleObject)]
pub struct RepoPageType {
pub nodes: Vec<RepoType>,
pub has_next_page: bool,
/// Position after the last returned visible repository; null for an empty page.
pub end_cursor: Option<String>,
}
#[derive(SimpleObject, Clone)] #[derive(SimpleObject, Clone)]
pub struct AgentTaskType { pub struct AgentTaskType {
pub id: String, pub id: String,
@@ -48,6 +56,65 @@ impl From<AgentTask> for AgentTaskType {
} }
} }
/// Read-only projection of `AgentTask` for the `tasks`/`task` queries, as
/// opposed to `AgentTaskType`, which the task mutations (`createTask`,
/// `claimTask`, `completeTask`, `failTask` — all `require_signer`-gated)
/// return. Identical except for the missing `ucan_token` (#268): a read
/// surface never needs to echo it back, since the assignee already received it
/// at delegation/claim time via the mutation response.
#[derive(SimpleObject, Clone)]
pub struct AgentTaskReadType {
pub id: String,
pub repo_id: Option<String>,
pub kind: String,
pub status: String,
pub delegator_did: String,
pub assignee_did: Option<String>,
pub capability: String,
pub payload: Option<String>,
pub result: Option<String>,
pub created_at: String,
pub updated_at: String,
pub deadline: Option<String>,
}
/// Wraps a `tasks` page with the same completion signals the REST list route
/// exposes, so the two surfaces answer pagination identically.
///
/// `hasMore` and `incomplete` are separate because they are separate facts
/// (#327 review): `hasMore` says more candidates remain, `incomplete` says
/// this page is short *only* because the authorization scan hit its safety
/// wall. `nextCursor` is present exactly when `hasMore` is true and is an
/// opaque MAC'd token — it can name the last examined candidate, denied or
/// not, without disclosing it, which is what lets a caller page past a denied
/// window instead of stalling on it.
#[derive(SimpleObject, Clone)]
pub struct TaskPageType {
pub items: Vec<AgentTaskReadType>,
pub has_more: bool,
pub incomplete: bool,
pub next_cursor: Option<String>,
}
impl From<AgentTask> for AgentTaskReadType {
fn from(t: AgentTask) -> Self {
Self {
id: t.id,
repo_id: t.repo_id,
kind: t.kind,
status: t.status,
delegator_did: t.delegator_did,
assignee_did: t.assignee_did,
capability: t.capability,
payload: t.payload,
result: t.result,
created_at: t.created_at,
updated_at: t.updated_at,
deadline: t.deadline,
}
}
}
#[derive(SimpleObject, Clone)] #[derive(SimpleObject, Clone)]
pub struct RefUpdateType { pub struct RefUpdateType {
pub repo: String, pub repo: String,
+166 -1
View File
@@ -126,15 +126,55 @@ fn decode_key(b64url: &str) -> Option<VerifyingKey> {
VerifyingKey::from_bytes(&arr).ok() VerifyingKey::from_bytes(&arr).ok()
} }
/// Upper bound on the iCaptcha `/v1/pubkey` response body. The document is a
/// handful of keys at most; anything larger is refused rather than buffered
/// into memory (the unbounded `.json()` read this replaces, #507).
const KEY_MAX_BODY_BYTES: usize = 8 * 1024;
/// Read at most [`KEY_MAX_BODY_BYTES`] bytes of a key response. Returns `None`
/// when the body exceeds the cap, so an oversized document is treated as absent
/// exactly like a network error or a timeout (fail safe, never a partial key).
async fn read_key_body(mut resp: reqwest::Response) -> Option<Vec<u8>> {
let mut buf = Vec::with_capacity(1024);
loop {
match resp.chunk().await {
Ok(Some(chunk)) => {
let take = (KEY_MAX_BODY_BYTES - buf.len()).min(chunk.len());
buf.extend_from_slice(&chunk[..take]);
if take < chunk.len() {
return None; // exceeded the cap mid-chunk
}
}
Ok(None) => return Some(buf),
Err(_) => return None,
}
}
}
async fn fetch_key(url: &str) -> Option<VerifyingKey> { async fn fetch_key(url: &str) -> Option<VerifyingKey> {
let endpoint = format!("{}/v1/pubkey", url.trim_end_matches('/')); let endpoint = format!("{}/v1/pubkey", url.trim_end_matches('/'));
// Bounded request: a hung /v1/pubkey must never block node startup. On // Bounded request: a hung /v1/pubkey must never block node startup. On
// timeout/error we return None and the gate stays inert (fail safe). // timeout/error we return None and the gate stays inert (fail safe).
//
// No redirects and a capped body read: the key this fetch produces is the
// TRUST ANCHOR for the whole proof gate, so a 3xx must not be allowed to move
// where it comes from (anchor substitution / SSRF, matching the node's other
// outbound clients — `build_http_client` in main.rs, the sync worker), and
// the body is bounded rather than handed to `.json()`'s unbounded buffering.
let client = reqwest::Client::builder() let client = reqwest::Client::builder()
.timeout(Duration::from_secs(5)) .timeout(Duration::from_secs(5))
.redirect(reqwest::redirect::Policy::none())
.build() .build()
.ok()?; .ok()?;
let jwks: Jwks = client.get(&endpoint).send().await.ok()?.json().await.ok()?; let resp = client.get(&endpoint).send().await.ok()?;
// A non-2xx response body is not a key: with Policy::none() a 3xx is
// surfaced rather than followed, but its own body must not be parsed
// either, or the redirect would move the anchor after all.
if !resp.status().is_success() {
return None;
}
let body = read_key_body(resp).await?;
let jwks: Jwks = serde_json::from_slice(&body).ok()?;
decode_key(&jwks.keys.first()?.x) decode_key(&jwks.keys.first()?.x)
} }
@@ -647,4 +687,129 @@ mod tests {
let tampered = format!("{}.{}", payload, chars.into_iter().collect::<String>()); let tampered = format!("{}.{}", payload, chars.into_iter().collect::<String>());
assert!(verify_token(&v, &tampered, SUB, IAT, false).is_err()); assert!(verify_token(&v, &tampered, SUB, IAT, false).is_err());
} }
/// `{"keys":[{"x":"<base64url ed25519 point>"}]}` for the live `PUBKEY_X`.
fn jwks_body() -> Vec<u8> {
format!(r#"{{"keys":[{{"x":"{PUBKEY_X}"}}]}}"#).into_bytes()
}
/// The no-redirect / bounded-body client still parses a valid JWKS, so the
/// hardening did not break the happy path the gate depends on at startup.
#[tokio::test]
async fn fetch_key_parses_a_valid_jwks() {
let mut server = mockito::Server::new_async().await;
let _m = server
.mock("GET", "/v1/pubkey")
.with_status(200)
.with_header("Content-Type", "application/json")
.with_body(jwks_body())
.create_async()
.await;
let key = fetch_key(&server.url()).await.expect("valid JWKS parses");
assert_eq!(key, decode_key(PUBKEY_X).unwrap());
}
/// A 302 must not move the trust anchor. The redirect target serves a REAL
/// key, so following the redirect would make `fetch_key` adopt it; instead
/// `Policy::none()` leaves the 302's (non-JWKS) body as the answer → None.
#[tokio::test]
async fn fetch_key_does_not_follow_a_redirect() {
let mut target = mockito::Server::new_async().await;
let _t = target
.mock("GET", "/v1/pubkey")
.expect(0)
.with_status(200)
.with_header("Content-Type", "application/json")
.with_body(jwks_body())
.create_async()
.await;
let mut redirector = mockito::Server::new_async().await;
// Location points at the target's /v1/pubkey, so a client that DID
// follow the redirect would fetch the real key and return Some: the
// assertion below only holds while Policy::none() is in place.
let _r = redirector
.mock("GET", "/v1/pubkey")
.with_status(302)
.with_header("Location", &format!("{}/v1/pubkey", target.url()))
.create_async()
.await;
assert!(
fetch_key(&redirector.url()).await.is_none(),
"a redirect must not be followed to a host serving a valid key"
);
}
/// A body past the cap is refused rather than buffered, so a broken
/// endpoint cannot make the node materialize an arbitrarily large document.
/// The body is a VALID JWKS (`jwks_body()`) padded with JSON whitespace past
/// the cap, so an unbounded read would still parse it; this test only passes
/// because the byte cap refuses it.
#[tokio::test]
async fn fetch_key_rejects_an_oversized_body() {
let mut server = mockito::Server::new_async().await;
let mut oversized = jwks_body();
oversized.resize(KEY_MAX_BODY_BYTES + 1, b' ');
let _m = server
.mock("GET", "/v1/pubkey")
.with_status(200)
.with_header("Content-Type", "application/json")
.with_body(oversized)
.create_async()
.await;
assert!(
fetch_key(&server.url()).await.is_none(),
"an oversized body must be refused even when it is a valid JWKS"
);
}
/// A body of exactly the cap still parses: the contract refuses bodies that
/// EXCEED it, not bodies that reach it (the padded JWKS is exactly
/// KEY_MAX_BODY_BYTES bytes of otherwise-valid JSON).
#[tokio::test]
async fn fetch_key_accepts_a_body_at_the_cap() {
let mut server = mockito::Server::new_async().await;
let mut at_cap = jwks_body();
at_cap.resize(KEY_MAX_BODY_BYTES, b' ');
let _m = server
.mock("GET", "/v1/pubkey")
.with_status(200)
.with_header("Content-Type", "application/json")
.with_body(at_cap)
.create_async()
.await;
let key = fetch_key(&server.url()).await.expect("at-cap JWKS parses");
assert_eq!(key, decode_key(PUBKEY_X).unwrap());
}
/// A non-2xx response never supplies the anchor, even when its body is a
/// perfectly valid JWKS: the redirect pin only stops the hop, and the
/// status gate is what stops the 3xx's own body from being adopted.
#[tokio::test]
async fn fetch_key_rejects_a_non_success_status() {
let mut server = mockito::Server::new_async().await;
let _m = server
.mock("GET", "/v1/pubkey")
.with_status(500)
.with_header("Content-Type", "application/json")
.with_body(jwks_body())
.create_async()
.await;
assert!(
fetch_key(&server.url()).await.is_none(),
"a non-2xx body must not be adopted as the anchor even when it parses"
);
}
/// An unreachable endpoint is the same fail-safe None as every other fetch
/// error: the gate stays inert rather than the node failing to start.
#[tokio::test]
async fn fetch_key_returns_none_when_the_endpoint_is_unreachable() {
assert!(fetch_key("http://127.0.0.1:1").await.is_none());
}
} }
+497 -58
View File
@@ -922,6 +922,12 @@ async fn sweep_pass(
// Advance FIRST: every path below this line may skip the row, and none of them // Advance FIRST: every path below this line may skip the row, and none of them
// may wedge the walk (scenario 7). // may wedge the walk (scenario 7).
last = sha.clone(); last = sha.clone();
// Round-3 P1: skip Pinata-only rows whose local cid is NULL.
// The legacy repair walk re-keys `cid` from a provider CID to
// the raw resolver CID; a row with no local cid has no string
// to re-key, and skipping is the natural behavior. The cursor
// still advances so the walk does not loop on this row.
let Some(stored) = stored else { continue };
// Same cost gate as the skip-path repair: a canonical raw CIDv1 key is already // Same cost gate as the skip-path repair: a canonical raw CIDv1 key is already
// the resolver key, so it reads no bytes and resolves no repo. // the resolver key, so it reads no bytes and resolves no repo.
if gitlawb_core::cid::is_raw_cidv1(&stored) { if gitlawb_core::cid::is_raw_cidv1(&stored) {
@@ -1457,6 +1463,97 @@ fn note_legacy_repair_read() {
/// have to be documented, validated, and kept meaningful. /// have to be documented, validated, and kept meaningful.
pub const PIN_BATCH_BUDGET: Duration = Duration::from_secs(120); pub const PIN_BATCH_BUDGET: Duration = Duration::from_secs(120);
/// A captured per-repo visibility-policy epoch that fences a pin batch.
///
/// The reconciliation sweep reads the epoch immediately before dispatching a
/// pin loop and passes a fence in; the loop re-reads the epoch before every
/// upload and aborts the batch the moment it moves. A visibility narrow that
/// lands mid-batch (a rule made private, a repo quarantined) must not let the
/// remaining pre-authorized objects still go to a public content-addressed
/// backend — the narrow is a policy change, and dispatching against the stale
/// snapshot is the exact irreversible-publication class this fence exists for
/// (R1-P1). `None` (the push path) means "no fence": the push derives its own
/// object list at admission and holds a write lease, so no sweep-style batch
/// snapshot crosses the dispatch boundary.
///
/// The fence deliberately does NOT hold any lock across the batch: a
/// visibility narrow (rule insert/remove, quarantine set) must commit
/// immediately, and the batch aborts on its next `is_current` check. Holding
/// a per-repo mutex from capture through drop would invert this — the narrow
/// would block behind the background sweep and every object in the batch
/// would still be sealed/posted to the reader being removed. The accepted
/// residual is a single in-flight object: a narrow that commits between the
/// pre-POST `is_current` check and the HTTP POST cannot be recalled, but the
/// next iteration aborts and the fenced DB record (row-locked against the
/// narrow's epoch bump) refuses to land the raced row as durable.
/// Multi-process / multi-node narrows are ordered by the same epoch column;
/// no in-process registry is involved.
///
/// Product decision (revocation model): prompt revocation wins over
/// publication atomicity. The suite intentionally allows one provider
/// upload already in flight when a rule narrows and requires every
/// LATER object to stop (`encrypt_and_pin_stops_sealing_when_reader_
/// removed_mid_batch`). A stronger contract — revocation commit
/// linearizing against every concurrent publication, or compensation
/// (unpin/delete) for an envelope that finishes after removal — is
/// explicitly OUT of scope until decided and documented here. Do not
/// reintroduce batch-spanning mutual exclusion to close the
/// single-in-flight window without that decision: it trades one
/// possibly-raced object for sealing the whole batch to a removed
/// reader.
#[derive(Clone)]
pub struct PolicyFence {
db: crate::db::Db,
repo_id: String,
epoch: i64,
}
impl PolicyFence {
/// Capture the current policy epoch for `repo_id`. A read failure is a
/// skip, not a retry-with-zero: the caller must not dispatch a batch it
/// cannot fence (fail closed on a stale allow).
pub async fn capture(db: &crate::db::Db, repo_id: &str) -> Option<Self> {
match db.repo_policy_epoch(repo_id).await {
Ok(epoch) => Some(PolicyFence {
db: db.clone(),
repo_id: repo_id.to_string(),
epoch,
}),
Err(e) => {
tracing::warn!(repo = %repo_id, err = %e, "policy-epoch read failed; not fencing pin batch");
None
}
}
}
/// Whether the repo's policy epoch is unchanged since capture. A read
/// failure is treated as "changed": never dispatch on a policy we cannot
/// prove current.
pub async fn is_current(&self) -> bool {
match self.db.repo_policy_epoch(&self.repo_id).await {
Ok(epoch) => epoch == self.epoch,
Err(_) => false,
}
}
/// The epoch value captured at `capture` time. Exposed so the
/// pinner can pass it to
/// `Db::record_pinned_cid_with_source_fenced` — the third
/// fence in the same transaction as the row insert.
/// Returning the field directly (rather than a `Option`)
/// matches the contract: a `PolicyFence` always has a
/// captured epoch; `is_current()` reports whether it still
/// matches.
pub fn captured_epoch(&self) -> i64 {
self.epoch
}
/// The repo this fence guards, for log correlation.
pub fn repo_id(&self) -> &str {
&self.repo_id
}
}
/// The smallest remainder worth starting a bounded git read (or an add) with. /// The smallest remainder worth starting a bounded git read (or an add) with.
/// ///
/// A 1ms remainder otherwise buys a child spawned already past its deadline, which /// A 1ms remainder otherwise buys a child spawned already past its deadline, which
@@ -1508,9 +1605,7 @@ pub async fn pin_git_object(
return Ok(String::new()); return Ok(String::new());
} }
// Compute the expected CIDv1 from the content bytes
let expected_cid = Cid::from_git_object_bytes(data).to_string(); let expected_cid = Cid::from_git_object_bytes(data).to_string();
let url = format!( let url = format!(
"{}/api/v0/add?cid-version=1&raw-leaves=true&pin=true", "{}/api/v0/add?cid-version=1&raw-leaves=true&pin=true",
ipfs_api.trim_end_matches('/') ipfs_api.trim_end_matches('/')
@@ -1553,10 +1648,23 @@ pub async fn pin_git_object(
// Kubo returns newline-delimited JSON; we only care about the last object // Kubo returns newline-delimited JSON; we only care about the last object
// (there's typically just one for a single-file add). // (there's typically just one for a single-file add).
//
// The response MUST carry a real `Hash`: a misconfigured `GITLAWB_IPFS_API`
// (proxy returning HTML, health check on the wrong port, truncated gateway)
// can otherwise answer 2xx with no JSON, and falling back to the locally
// computed `expected_cid` would record a row for bytes the backend never
// stored. The reconciliation sweep trusts `pinned_cids` rows as durability
// evidence, so a silent false positive at pin time becomes a permanent blind
// spot for the backstop. A missing `Hash` fails the pin rather than recording
// a phantom row (mirrors Pinata's `data.cid` check).
let body = resp let body = resp
.text() .text()
.await .await
.map_err(|e| anyhow::anyhow!("IPFS add response body read failed: {e}"))?; .map_err(|e| anyhow::anyhow!("IPFS add response body read failed: {e}"))?;
// A 200 without a parseable Hash is not a successful pin. Fabricating the
// caller-computed CID would record an address the backend never confirmed;
// above its chunk threshold that raw CID is not even a stored block, so
// every later cat fails permanently.
let cid = body let cid = body
.lines() .lines()
.filter(|l| !l.trim().is_empty()) .filter(|l| !l.trim().is_empty())
@@ -1564,8 +1672,31 @@ pub async fn pin_git_object(
let v: serde_json::Value = serde_json::from_str(line).ok()?; let v: serde_json::Value = serde_json::from_str(line).ok()?;
v["Hash"].as_str().map(|s| s.to_string()) v["Hash"].as_str().map(|s| s.to_string())
}) })
.next_back() .next_back();
.unwrap_or(expected_cid.clone()); let cid = match cid {
Some(cid) => {
if cid != expected_cid {
tracing::warn!(
sha256 = %sha256_hex,
returned = %cid,
expected = %expected_cid,
"IPFS returned a different CID than computed locally (Kubo chunking may differ); recording the backend's answer"
);
}
cid
}
None => {
return Err(anyhow::anyhow!(
"IPFS /api/v0/add returned 2xx without a Hash field; refusing to record \
a CID the backend never acknowledged (misconfigured GITLAWB_IPFS_API?)"
));
}
};
// The string still has to be a CID. A garbage Hash would be recorded and
// later cat-ed the same way a missing one was, so reject it here.
Cid::from_str(&cid).map_err(|e| {
anyhow::anyhow!("IPFS /api/v0/add returned a Hash that is not a CID ({cid:.200}): {e}")
})?;
tracing::debug!(sha256 = %sha256_hex, %cid, "pinned git object to IPFS"); tracing::debug!(sha256 = %sha256_hex, %cid, "pinned git object to IPFS");
Ok(cid) Ok(cid)
@@ -1643,8 +1774,8 @@ pub(crate) fn batch_budget_gate(
/// than [`PIN_READ_FLOOR`] left. It is a gate, not a hard ceiling, since a started /// than [`PIN_READ_FLOOR`] left. It is a gate, not a hard ceiling, since a started
/// iteration still runs to completion; /// iteration still runs to completion;
/// - the git read: `store::read_object_bounded` runs under `spawn_blocking` against the /// - the git read: `store::read_object_bounded` runs under `spawn_blocking` against the
/// ABSOLUTE batch deadline (not the loop-top remainder, which the `is_pinned` round-trip /// ABSOLUTE batch deadline (not the loop-top remainder, which the `has_ipfs_cid`
/// sitting between the two would push past it), with SIGTERM-then-SIGKILL /// round-trip sitting between the two would push past it), with SIGTERM-then-SIGKILL
/// process-group teardown, so a hung `git cat-file` costs this batch its remaining /// process-group teardown, so a hung `git cat-file` costs this batch its remaining
/// budget plus one watchdog teardown instead of holding the permit for the child's /// budget plus one watchdog teardown instead of holding the permit for the child's
/// whole lifetime and blocking a runtime worker while it does; /// whole lifetime and blocking a runtime worker while it does;
@@ -1688,10 +1819,11 @@ pub(crate) fn batch_budget_gate(
/// # Truncation semantics /// # Truncation semantics
/// ///
/// A batch stopped at the deadline leaves its remaining objects unpinned, and /// A batch stopped at the deadline leaves its remaining objects unpinned, and
/// nothing sweeps them up afterwards. There is no reconciliation pass over /// nothing sweeps them up afterwards on the push path; recovery is opportunistic
/// `pinned_cids`; recovery is opportunistic, happening only if some later push /// (a later full-scan push re-offers the skipped OIDs). The reconciliation
/// on the repo takes the full-scan fallback (`push_delta::list_all_objects`) and /// sweep is the systematic backstop: when it is enabled and a pin backend is
/// re-derives the whole object set, which then re-offers the skipped OIDs. /// configured, it re-derives the public object set each pass and fills any
/// remaining gap.
/// ///
/// The twin in `pinata.rs` is back at parity on everything that bounds or repairs an /// The twin in `pinata.rs` is back at parity on everything that bounds or repairs an
/// object: it runs the same shared budget gate at the top of every iteration, the same /// object: it runs the same shared budget gate at the top of every iteration, the same
@@ -1708,6 +1840,29 @@ pub(crate) fn batch_budget_gate(
/// ///
/// Returns a list of `(sha256_hex, cid)` pairs pinned AND durably recorded this /// Returns a list of `(sha256_hex, cid)` pairs pinned AND durably recorded this
/// call. /// call.
/// What one `pin_new_objects` call (IPFS or Pinata twin) did, with the
/// three backend states kept apart instead of inferred from each other:
///
/// - `confirmed`: `(sha, provider_cid)` pairs whose DB record durably
/// landed. ONLY these may advance `gaps_filled`, branch/gossip CID
/// state, or any "repaired" bookkeeping. A provider upload whose
/// record timed out, failed, or was refused by the fence is absent
/// here even though the bytes may sit on the provider.
/// - `last_attempted`: the last OID whose loop body was entered — i.e.
/// the backend did real work for it (reads, an upload attempt, or a
/// skip-branch decision), whether that work succeeded, failed, or hit
/// an unknown outcome. `None` when nothing was entered (empty input,
/// immediate fence abort, or immediate budget gate). Durable
/// continuation cursors advance to this, never to the tail of the
/// planned vector: the tail may never have been visited, and
/// promoting it would rotate an untouched suffix behind the backlog
/// forever.
#[derive(Debug)]
pub struct PinBatchOutcome {
pub confirmed: Vec<(String, String)>,
pub last_attempted: Option<String>,
}
// Eight because #173's git seam (`git_bin`, `git_timeout`) and pin provenance // Eight because #173's git seam (`git_bin`, `git_timeout`) and pin provenance
// (`repo_id`) sit alongside #174's batch budget. All four callers pass every one, and // (`repo_id`) sit alongside #174's batch budget. All four callers pass every one, and
// grouping them into a context struct would add a type whose only job is to be // grouping them into a context struct would add a type whose only job is to be
@@ -1722,16 +1877,34 @@ pub async fn pin_new_objects(
db: &crate::db::Db, db: &crate::db::Db,
repo_id: &str, repo_id: &str,
batch_budget: Duration, batch_budget: Duration,
) -> Vec<(String, String)> { fence: Option<&PolicyFence>,
) -> PinBatchOutcome {
if ipfs_api.is_empty() { if ipfs_api.is_empty() {
return vec![]; return PinBatchOutcome {
confirmed: Vec::new(),
last_attempted: None,
};
} }
let deadline = Instant::now() + batch_budget; let deadline = Instant::now() + batch_budget;
let total = object_list.len(); let total = object_list.len();
let mut pinned = Vec::new(); let mut pinned = Vec::new();
let mut last_attempted: Option<String> = None;
for (attempted, sha) in object_list.into_iter().enumerate() { for (attempted, sha) in object_list.into_iter().enumerate() {
// Policy fence (R1-P1): a visibility narrow that lands after the caller
// built this batch must abort it before the next irreversible upload.
// Checked FIRST so a changed policy costs nothing beyond the read.
if let Some(f) = fence {
if !f.is_current().await {
tracing::warn!(
repo = %f.repo_id,
unattempted = total - attempted,
"visibility policy changed mid-batch; stopping the pin loop"
);
break;
}
}
// Top of the iteration, before any of this object's work: an object is // Top of the iteration, before any of this object's work: an object is
// never started with a remainder too small to cover a bounded read's // never started with a remainder too small to cover a bounded read's
// teardown. Consumed as a guard only: the read below runs against the // teardown. Consumed as a guard only: the read below runs against the
@@ -1740,19 +1913,40 @@ pub async fn pin_new_objects(
if batch_budget_gate("IPFS", deadline, pinned.len(), total - attempted).is_none() { if batch_budget_gate("IPFS", deadline, pinned.len(), total - attempted).is_none() {
break; break;
} }
// Skip if already pinned, but first backfill provenance if the existing // Attempt progress (not a durability claim): from here the loop does
// pin has none. A legacy pin (recorded before repo_id existed, #173, jatmn) // real work for this OID — skip-branch reads, airgapped git reads,
// is skipped here before record_pinned_cid ever runs, so its NULL provenance // an upload attempt — whatever the outcome. The caller persists
// would never resolve to one repo and known CIDs keep hitting the scan. The // this as the continuation, never the planned vector's tail.
// backfill only sets repo_id (AND repo_id IS NULL guard preserves last_attempted = Some(sha.clone());
// first-pinner-owns) and never re-pins the bytes: the object is already on IPFS. // Skip if the object is ALREADY a real local IPFS pin, but first
// Every DB call from here to the end of the iteration is bounded by the // backfill provenance if the existing pin has none. A legacy pin
// ABSOLUTE batch deadline (F3, #173): the loop runs under a global pin permit // (recorded before repo_id existed, #173, jatmn) is skipped here
// and a bare await parked it for the whole stall. The elapsed arm is mapped per // before record_pinned_cid ever runs, so its NULL provenance would
// site below, never as a blanket "existing error arm": a timeout cancels the // never resolve to one repo and known CIDs keep hitting the scan.
// client future but not the statement Postgres is running, so it reports an // The backfill only sets repo_id (AND repo_id IS NULL guard
// UNKNOWN outcome, not a failed write. // preserves first-pinner-owns) and never re-pins the bytes: the
match db_bounded(deadline, db.is_pinned(&sha)).await { // object is already on IPFS.
//
// #218 review P1a: this check keys on `has_ipfs_cid` (writer-owned
// `local_ipfs_provenance = TRUE`), NOT on row existence
// (`is_pinned`). A Pinata-only row is `is_pinned = true` but
// `has_ipfs_cid = false`: the bytes never reached the local IPFS
// daemon, only Pinata, and we MUST fall through to the local
// writer path so a real local pin lands. Using `is_pinned` here
// was the gap that made the Pinata-only → local-IPFS repair
// path inert: every sweep pass re-entered this arm, recorded
// the source, and continued without ever calling
// `pin_git_object` or `record_pinned_cid_with_source`. The flag
// stayed FALSE forever.
//
// Every DB call from here to the end of the iteration is bounded
// by the ABSOLUTE batch deadline (F3, #173): the loop runs under
// a global pin permit and a bare await parked it for the whole
// stall. The elapsed arm is mapped per site below, never as a
// blanket "existing error arm": a timeout cancels the client
// future but not the statement Postgres is running, so it
// reports an UNKNOWN outcome, not a failed write.
match db_bounded(deadline, db.has_ipfs_cid(&sha)).await {
Ok(true) => { Ok(true) => {
// Elapsed here is free to skip: these are reads, so a late server-side // Elapsed here is free to skip: these are reads, so a late server-side
// completion costs nothing, and the backfill's own `AND repo_id IS NULL` // completion costs nothing, and the backfill's own `AND repo_id IS NULL`
@@ -1862,7 +2056,7 @@ pub async fn pin_new_objects(
} }
Ok(false) => {} Ok(false) => {}
Err(e) => { Err(e) => {
tracing::warn!(sha = %sha, err = %e, "DB error checking pinned status"); tracing::warn!(sha = %sha, err = %e, "DB error checking IPFS pinned status");
continue; continue;
} }
} }
@@ -1876,7 +2070,7 @@ pub async fn pin_new_objects(
// own deadline regardless. // own deadline regardless.
// //
// The read runs against the ABSOLUTE batch deadline, not against the remainder // The read runs against the ABSOLUTE batch deadline, not against the remainder
// measured at the top of the iteration: the `is_pinned` round-trip above sits // measured at the top of the iteration: the `has_ipfs_cid` round-trip above sits
// between the two, so `Instant::now() + budget_left` would land past `deadline` // between the two, so `Instant::now() + budget_left` would land past `deadline`
// by however long the DB took, and under a saturated pool that is the dominant // by however long the DB took, and under a saturated pool that is the dominant
// term. A slow DB check must not push the read's own bound out. // term. A slow DB check must not push the read's own bound out.
@@ -1961,6 +2155,24 @@ pub async fn pin_new_objects(
break; break;
}; };
// Dispatch fence (R1-P1): re-read the policy epoch immediately before
// the irreversible HTTP POST. The iteration-top check catches a narrow
// that landed before work began; THIS check catches a narrow that landed
// during the has_ipfs_cid round-trip or the bounded Git read — both of
// which can take seconds and during which a quarantine or rule change may
// have committed. Without this, stale plaintext can start uploading
// under authorization that is no longer current.
if let Some(f) = fence {
if !f.is_current().await {
tracing::warn!(
repo = %f.repo_id,
unattempted = total - attempted,
"visibility policy changed during preparation; aborting IPFS upload"
);
break;
}
}
// Pin to IPFS // Pin to IPFS
match pin_git_object(ipfs_api, &sha, &data, Some(add_timeout)).await { match pin_git_object(ipfs_api, &sha, &data, Some(add_timeout)).await {
Ok(cid) if !cid.is_empty() => { Ok(cid) if !cid.is_empty() => {
@@ -2010,7 +2222,31 @@ pub async fn pin_new_objects(
// per-object failure. // per-object failure.
match db_bounded( match db_bounded(
db_record_deadline(deadline), db_record_deadline(deadline),
retry_db_record(|| db.record_pinned_cid_with_source(&sha, &raw_cid, repo_id)), retry_db_record(|| {
// #218 review round 9 (guidance #3 —
// linearization): always go through the
// fenced form. The fence is either
// captured (sweep / public-pin path: the
// third fence is the linearization point
// that closes the rule-write /
// record-write race) or absent
// (push-side admission where the
// decision is made at request time —
// we pass `i64::MAX` as a sentinel that
// the fenced form treats as "no fence
// check"). The 3-arg
// `record_pinned_cid_with_source` is
// still available for tests that don't
// own a fence, but the production
// pinner routes through here.
let fence_epoch = fence.map(|f| f.captured_epoch()).unwrap_or(i64::MAX);
db.record_pinned_cid_with_source_fenced(
&sha,
&raw_cid,
repo_id,
fence_epoch,
)
}),
) )
.await .await
{ {
@@ -2027,7 +2263,10 @@ pub async fn pin_new_objects(
} }
} }
pinned PinBatchOutcome {
confirmed: pinned,
last_attempted,
}
} }
#[cfg(test)] #[cfg(test)]
@@ -2223,16 +2462,16 @@ mod tests {
endpoint endpoint
} }
/// A sleeping-but-live endpoint. Answers `200` with an empty body after /// A sleeping-but-live endpoint. Answers `200` with a conformant NDJSON add
/// `delays[i]` for the i-th request it accepts (the last entry repeats), so /// response after `delays[i]` for the i-th request it accepts (the last
/// a test can make one add slow and the next fast. Drains the full request, /// entry repeats), so a test can make one add slow and the next fast.
/// headers plus the declared `Content-Length` body, before sleeping: exactly /// Drains the full request, headers plus the declared `Content-Length`
/// as in `rejecting_endpoint`, answering early and closing would surface as /// body, before sleeping: exactly as in `rejecting_endpoint`, answering
/// a write failure on the client and turn a slow-but-healthy add into a /// early and closing would surface as a write failure on the client and
/// different failure shape. /// turn a slow-but-healthy add into a different failure shape.
/// ///
/// An empty body is a successful pin: `pin_git_object` falls back to the CID /// The Hash is a fixed, real CID: the timing tests assert on pin counts,
/// it computed from the bytes when the response carries no `Hash`. /// not on the recorded address.
async fn delaying_endpoint(delays: Vec<Duration>) -> String { async fn delaying_endpoint(delays: Vec<Duration>) -> String {
use tokio::io::{AsyncReadExt, AsyncWriteExt}; use tokio::io::{AsyncReadExt, AsyncWriteExt};
let listener = tokio::net::TcpListener::bind("127.0.0.1:0").await.unwrap(); let listener = tokio::net::TcpListener::bind("127.0.0.1:0").await.unwrap();
@@ -2269,9 +2508,14 @@ mod tests {
} }
} }
tokio::time::sleep(delay).await; tokio::time::sleep(delay).await;
let _ = sock let body = br#"{"Hash":"bafkreifjjcie6lypi6ny7amxnfftagclbuxndqonfipmb64f2km2devei4","Size":"12"}
.write_all(b"HTTP/1.1 200 OK\r\nContent-Length: 0\r\n\r\n") "#;
.await; let resp = format!(
"HTTP/1.1 200 OK\r\nContent-Type: application/x-ndjson\r\nContent-Length: {}\r\n\r\n",
body.len()
);
let _ = sock.write_all(resp.as_bytes()).await;
let _ = sock.write_all(body).await;
let _ = sock.flush().await; let _ = sock.flush().await;
}); });
} }
@@ -2358,6 +2602,74 @@ mod tests {
); );
} }
/// The misconfigured-`GITLAWB_IPFS_API` false positive (P3): a 2xx response
/// that carries no `Hash` field (proxy returning HTML, health check on the
/// wrong port, truncated gateway) must FAIL the pin, not fall back to the
/// locally computed `expected_cid`. Falling back records a `pinned_cids`
/// row for bytes the backend never stored, and the reconciliation sweep
/// trusts rows as durability evidence — so the false positive becomes a
/// permanent blind spot for the backstop. A missing `Hash` must surface as
/// an explicit error, never a successful pin.
#[tokio::test]
async fn pin_git_object_rejects_a_2xx_without_a_hash_field() {
let endpoint = empty_ok_endpoint().await;
let inner = tokio::time::timeout(
Duration::from_secs(30),
pin_git_object(&endpoint, "deadbeef", b"some object bytes\n", None),
)
.await
.expect("wedge guard: an immediate empty 200 cannot take 30s");
let err = inner.expect_err(
"a 2xx without a Hash field must not surface as a successful pin \
(would record a phantom pinned_cids row the sweep then trusts)",
);
assert!(
err.to_string().contains("without a Hash field"),
"the error must name the missing Hash so operators diagnose the endpoint: {err:#}"
);
}
/// A 200 that answers with an empty body and no `Hash` — the exact shape of
/// a proxy or health-check endpoint mistaken for a Kubo API.
async fn empty_ok_endpoint() -> String {
use tokio::io::{AsyncReadExt, AsyncWriteExt};
let listener = tokio::net::TcpListener::bind("127.0.0.1:0").await.unwrap();
let endpoint = format!("http://{}", listener.local_addr().unwrap());
tokio::spawn(async move {
while let Ok((mut sock, _)) = listener.accept().await {
tokio::spawn(async move {
let mut acc = Vec::new();
let mut buf = [0u8; 4096];
loop {
let n = match sock.read(&mut buf).await {
Ok(0) | Err(_) => break,
Ok(n) => n,
};
acc.extend_from_slice(&buf[..n]);
if let Some(hdr_end) =
acc.windows(4).position(|w| w == b"\r\n\r\n").map(|p| p + 4)
{
let headers = String::from_utf8_lossy(&acc[..hdr_end]).to_lowercase();
let len: usize = headers
.lines()
.find_map(|l| l.strip_prefix("content-length:"))
.and_then(|v| v.trim().parse().ok())
.unwrap_or(0);
if acc.len() >= hdr_end + len {
break;
}
}
}
let _ = sock
.write_all(b"HTTP/1.1 200 OK\r\nContent-Length: 0\r\n\r\n")
.await;
let _ = sock.flush().await;
});
}
});
endpoint
}
/// The second unhardened sink, reached from `sync.rs`. Same shape as above. /// The second unhardened sink, reached from `sync.rs`. Same shape as above.
#[tokio::test] #[tokio::test]
async fn cat_against_silent_endpoint_errors_within_its_own_timeout() { async fn cat_against_silent_endpoint_errors_within_its_own_timeout() {
@@ -2373,6 +2685,71 @@ mod tests {
); );
} }
/// A 200 add response with no parseable Hash must surface as an add
/// failure. Falling back to the caller-computed CID records an
/// unverifiable address: above the backend's chunk threshold that raw
/// CID is not a stored block, so every later cat 500s.
#[tokio::test]
async fn pin_git_object_rejects_a_200_without_a_hash() {
let mut server = mockito::Server::new_async().await;
let _m = server
.mock("POST", mockito::Matcher::Regex(r"^/api/v0/add".to_string()))
.with_status(200)
.with_body(r#"{"Name":"object","Size":"19"}"#)
.create_async()
.await;
let result = pin_git_object(&server.url(), "deadbeef", b"some object bytes\n", None).await;
assert!(
result.is_err(),
"a 200 without a Hash must be an add failure, not a fabricated CID: {result:?}"
);
}
/// A Hash that is a string but not a CID must also fail. It would be
/// recorded into `encrypted_blobs.cid` and every later cat would miss.
#[tokio::test]
async fn pin_git_object_rejects_a_malformed_hash() {
let mut server = mockito::Server::new_async().await;
let _m = server
.mock("POST", mockito::Matcher::Regex(r"^/api/v0/add".to_string()))
.with_status(200)
.with_body(r#"{"Hash":"not-a-cid","Size":"19"}"#)
.create_async()
.await;
let result = pin_git_object(&server.url(), "deadbeef", b"some object bytes\n", None).await;
assert!(
result.is_err(),
"a Hash that is not a CID must be an add failure: {result:?}"
);
}
/// The Hash text goes into the error and `encrypted_pin` logs that error,
/// so a backend answering with a huge invalid Hash must not produce an
/// unbounded log entry.
#[tokio::test]
async fn pin_git_object_bounds_an_oversized_invalid_hash_in_the_error() {
let long = "x".repeat(5000);
let mut server = mockito::Server::new_async().await;
let _m = server
.mock("POST", mockito::Matcher::Regex(r"^/api/v0/add".to_string()))
.with_status(200)
.with_body(format!(r#"{{"Hash":"{long}","Size":"19"}}"#))
.create_async()
.await;
let result = pin_git_object(&server.url(), "deadbeef", b"some object bytes\n", None).await;
let err = result.expect_err("an invalid Hash must be an add failure");
let msg = err.to_string();
assert!(
msg.len() < 1000,
"the error must bound the echoed Hash, got {} bytes",
msg.len()
);
assert!(
!msg.contains(&long),
"the error must not carry the full invalid Hash"
);
}
/// The permit-hold bound. `pin_new_objects` runs under a deferring /// The permit-hold bound. `pin_new_objects` runs under a deferring
/// `pin_semaphore`, so without a batch deadline the hold is O(N) with N /// `pin_semaphore`, so without a batch deadline the hold is O(N) with N
/// chosen by the pusher. Five objects against an endpoint that takes 2s /// chosen by the pusher. Five objects against an endpoint that takes 2s
@@ -2405,15 +2782,16 @@ mod tests {
&db, &db,
"repo-batch-budget", "repo-batch-budget",
Duration::from_millis(5500), Duration::from_millis(5500),
None,
), ),
) )
.await .await
.expect("wedge guard: a 5.5s budget cannot take 30s"); .expect("wedge guard: a 5.5s budget cannot take 30s");
assert!( assert!(
(1..=3).contains(&pinned.len()), (1..=3).contains(&pinned.confirmed.len()),
"the batch must stop partway, not pin all five and not stall on the first: pinned {}", "the batch must stop partway, not pin all five and not stall on the first: pinned {}",
pinned.len() pinned.confirmed.len()
); );
let text = logs.text(); let text = logs.text();
let warns: Vec<&str> = text let warns: Vec<&str> = text
@@ -2435,9 +2813,52 @@ mod tests {
}) })
.unwrap_or_else(|| panic!("the deadline warn must name the unattempted count: {text}")); .unwrap_or_else(|| panic!("the deadline warn must name the unattempted count: {text}"));
assert!( assert!(
unattempted >= 1 && unattempted + pinned.len() <= 5, unattempted >= 1 && unattempted + pinned.confirmed.len() <= 5,
"unattempted={unattempted} with {} pinned is not a partial batch of five", "unattempted={unattempted} with {} pinned is not a partial batch of five",
pinned.len() pinned.confirmed.len()
);
}
/// The outcome reports the last object actually ENTERED, not the planned
/// tail: the first upload hangs 6s against a ~2s per-request timeout, so
/// it fails, and the loop-top gate breaks the batch before the second
/// object starts. `last_attempted` must be the first OID — the only one
/// the loop body entered — even though nothing was confirmed and two
/// OIDs were never visited. A cursor persisted from `to_pin.last()`
/// would rotate the untouched suffix behind the backlog forever.
#[sqlx::test]
async fn pin_new_objects_reports_last_attempted_not_planned_tail(pool: sqlx::PgPool) {
let db = crate::db::Db::for_testing(pool);
db.run_migrations().await.expect("migrations");
let tmp = tempfile::TempDir::new().unwrap();
let repo_path = tmp.path().join("attempted.git");
let oids = seed_loose_blobs(&repo_path, 3);
let endpoint = delaying_endpoint(vec![Duration::from_secs(6)]).await;
let outcome = tokio::time::timeout(
Duration::from_secs(30),
pin_new_objects(
&endpoint,
&repo_path,
"git",
Duration::from_secs(30),
oids.clone(),
&db,
"repo-attempted",
Duration::from_secs(2),
None,
),
)
.await
.expect("a 2s budget cannot take 30s");
assert_eq!(
outcome.last_attempted,
Some(oids[0].clone()),
"only the first object was entered; the planned tail was never visited"
);
assert!(
outcome.confirmed.is_empty(),
"the hung upload timed out, so nothing was confirmed"
); );
} }
@@ -2472,12 +2893,13 @@ mod tests {
&db, &db,
"repo-batch-continues", "repo-batch-continues",
Duration::from_secs(90), Duration::from_secs(90),
None,
), ),
) )
.await .await
.expect("wedge guard: a 13s add plus an immediate one cannot take 60s"); .expect("wedge guard: a 13s add plus an immediate one cannot take 60s");
assert_eq!( assert_eq!(
pinned.len(), pinned.confirmed.len(),
2, 2,
"a slow but progressing endpoint must pin both objects: an upload past the client's \ "a slow but progressing endpoint must pin both objects: an upload past the client's \
10s default is not a dead endpoint" 10s default is not a dead endpoint"
@@ -2510,11 +2932,12 @@ mod tests {
&db, &db,
"repo-batch-rejects", "repo-batch-rejects",
Duration::from_secs(60), Duration::from_secs(60),
None,
), ),
) )
.await .await
.expect("a rejecting endpoint answers immediately, so this cannot take 30s"); .expect("a rejecting endpoint answers immediately, so this cannot take 30s");
assert!(pinned.is_empty(), "every add was rejected"); assert!(pinned.confirmed.is_empty(), "every add was rejected");
assert_eq!( assert_eq!(
requests.load(std::sync::atomic::Ordering::SeqCst), requests.load(std::sync::atomic::Ordering::SeqCst),
4, 4,
@@ -2557,6 +2980,7 @@ mod tests {
/// type probe carries its oid on stdin rather than in argv, so an oid appears in /// type probe carries its oid on stdin rather than in argv, so an oid appears in
/// the log only once an object has already got past its probe, and a healthy /// the log only once an object has already got past its probe, and a healthy
/// object costs two invocations to a faulting one's one. /// object costs two invocations to a faulting one's one.
#[cfg(unix)]
fn objects_attempted(log: &std::path::Path) -> usize { fn objects_attempted(log: &std::path::Path) -> usize {
std::fs::read_to_string(log) std::fs::read_to_string(log)
.unwrap_or_default() .unwrap_or_default()
@@ -2607,6 +3031,7 @@ mod tests {
&db, &db,
"repo-merge-test", "repo-merge-test",
Duration::from_secs(2), Duration::from_secs(2),
None,
), ),
) )
.await .await
@@ -2617,7 +3042,7 @@ mod tests {
let elapsed = started.elapsed(); let elapsed = started.elapsed();
assert!( assert!(
pinned.is_empty(), pinned.confirmed.is_empty(),
"a git that never answers cannot produce a pinned object: {pinned:?}" "a git that never answers cannot produce a pinned object: {pinned:?}"
); );
assert!( assert!(
@@ -2694,6 +3119,7 @@ mod tests {
&db, &db,
"repo-merge-test", "repo-merge-test",
Duration::from_millis(1500), Duration::from_millis(1500),
None,
), ),
) )
.await .await
@@ -2708,7 +3134,7 @@ mod tests {
that can only be spawned and reaped" that can only be spawned and reaped"
); );
assert_eq!( assert_eq!(
pinned.len(), pinned.confirmed.len(),
1, 1,
"the first object is inside the budget and must pin: {pinned:?}" "the first object is inside the budget and must pin: {pinned:?}"
); );
@@ -2767,6 +3193,7 @@ mod tests {
&db, &db,
"repo-merge-test", "repo-merge-test",
Duration::from_secs(60), Duration::from_secs(60),
None,
), ),
) )
.await .await
@@ -2776,7 +3203,7 @@ mod tests {
if genuinely_unreadable { if genuinely_unreadable {
assert!( assert!(
pinned.is_empty(), pinned.confirmed.is_empty(),
"nothing can be pinned through a store that cannot be read: {pinned:?}" "nothing can be pinned through a store that cannot be read: {pinned:?}"
); );
assert_eq!( assert_eq!(
@@ -2841,6 +3268,7 @@ mod tests {
&db, &db,
"repo-merge-test", "repo-merge-test",
Duration::from_secs(60), Duration::from_secs(60),
None,
), ),
) )
.await .await
@@ -2856,7 +3284,7 @@ mod tests {
must still be attempted, got {attempted} of {}", must still be attempted, got {attempted} of {}",
oids.len() oids.len()
); );
let pinned_shas: Vec<&String> = pinned.iter().map(|(sha, _)| sha).collect(); let pinned_shas: Vec<&String> = pinned.confirmed.iter().map(|(sha, _)| sha).collect();
let expected: Vec<&String> = oids.iter().filter(|o| !tainted.contains(o)).collect(); let expected: Vec<&String> = oids.iter().filter(|o| !tainted.contains(o)).collect();
assert_eq!( assert_eq!(
pinned_shas, expected, pinned_shas, expected,
@@ -2909,6 +3337,7 @@ mod tests {
&db, &db,
"repo-merge-test", "repo-merge-test",
Duration::from_secs(60), Duration::from_secs(60),
None,
), ),
) )
.await .await
@@ -2920,7 +3349,7 @@ mod tests {
"an object-scoped fault must not stop the batch: every object must be read" "an object-scoped fault must not stop the batch: every object must be read"
); );
assert_eq!( assert_eq!(
pinned.len(), pinned.confirmed.len(),
4, 4,
"one corrupt object must cost only itself: the other four must still pin" "one corrupt object must cost only itself: the other four must still pin"
); );
@@ -3143,6 +3572,7 @@ mod tests {
&db, &db,
"repo-stalled-db", "repo-stalled-db",
Duration::from_millis(1500), Duration::from_millis(1500),
None,
), ),
) )
.await .await
@@ -3153,7 +3583,7 @@ mod tests {
let elapsed = started.elapsed(); let elapsed = started.elapsed();
assert!( assert!(
pinned.is_empty(), pinned.confirmed.is_empty(),
"a stalled pinned-status check cannot produce a pinned object: {pinned:?}" "a stalled pinned-status check cannot produce a pinned object: {pinned:?}"
); );
assert!( assert!(
@@ -3213,6 +3643,7 @@ mod tests {
&db, &db,
"repo-skip-stalled", "repo-skip-stalled",
Duration::from_millis(1500), Duration::from_millis(1500),
None,
), ),
) )
.await .await
@@ -3223,7 +3654,7 @@ mod tests {
let elapsed = started.elapsed(); let elapsed = started.elapsed();
assert!( assert!(
pinned.is_empty(), pinned.confirmed.is_empty(),
"an already-pinned object is skipped, never re-pinned: {pinned:?}" "an already-pinned object is skipped, never re-pinned: {pinned:?}"
); );
assert!( assert!(
@@ -3279,13 +3710,17 @@ mod tests {
&db, &db,
"repo-multi-stalled", "repo-multi-stalled",
Duration::from_millis(1500), Duration::from_millis(1500),
None,
), ),
) )
.await .await
.expect("three stalled objects must still cost one budget, not three"); .expect("three stalled objects must still cost one budget, not three");
let elapsed = started.elapsed(); let elapsed = started.elapsed();
assert!(pinned.is_empty(), "nothing can pin against a stalled DB"); assert!(
pinned.confirmed.is_empty(),
"nothing can pin against a stalled DB"
);
assert!( assert!(
elapsed < Duration::from_secs(3), elapsed < Duration::from_secs(3),
"three stalled objects must charge ONE budget (1.5s), not one each; got {elapsed:?}" "three stalled objects must charge ONE budget (1.5s), not one each; got {elapsed:?}"
@@ -3308,7 +3743,7 @@ mod tests {
/// ///
/// The lock time is a MARGIN, not a boundary: taking it at 100ms left `is_pinned` /// The lock time is a MARGIN, not a boundary: taking it at 100ms left `is_pinned`
/// racing it on a loaded box, and losing that race makes the read block, time out, /// racing it on a loaded box, and losing that race makes the read block, time out,
/// and break the batch, which fails on `pinned.len() == 1` for a reason that has /// and break the batch, which fails on `pinned.confirmed.len() == 1` for a reason that has
/// nothing to do with the floor. Any time between the `is_pinned` round trip and /// nothing to do with the floor. Any time between the `is_pinned` round trip and
/// the add's 1.7s return proves the same thing. /// the add's 1.7s return proves the same thing.
#[sqlx::test] #[sqlx::test]
@@ -3346,6 +3781,7 @@ mod tests {
&db, &db,
"repo-spent-budget", "repo-spent-budget",
Duration::from_millis(2000), Duration::from_millis(2000),
None,
), ),
); );
let (pinned, ()) = tokio::join!(pin, locker); let (pinned, ()) = tokio::join!(pin, locker);
@@ -3358,7 +3794,7 @@ mod tests {
nothing can resolve the CID" nothing can resolve the CID"
); );
assert_eq!( assert_eq!(
pinned.len(), pinned.confirmed.len(),
1, 1,
"the durably recorded pin must be returned: {pinned:?}" "the durably recorded pin must be returned: {pinned:?}"
); );
@@ -3419,6 +3855,7 @@ mod tests {
&db, &db,
"repo-definite-error", "repo-definite-error",
Duration::from_millis(1200), Duration::from_millis(1200),
None,
), ),
); );
let (pinned, ()) = tokio::join!(pin, commit); let (pinned, ()) = tokio::join!(pin, commit);
@@ -3500,6 +3937,7 @@ mod tests {
&db, &db,
"repo-marker-floor", "repo-marker-floor",
Duration::from_millis(1500), Duration::from_millis(1500),
None,
), ),
); );
let (pinned, ()) = tokio::join!(pin, controller); let (pinned, ()) = tokio::join!(pin, controller);
@@ -3510,7 +3948,7 @@ mod tests {
drop(sources_lock); drop(sources_lock);
assert!( assert!(
pinned.is_empty(), pinned.confirmed.is_empty(),
"an already-pinned object is skipped, never re-pinned: {pinned:?}" "an already-pinned object is skipped, never re-pinned: {pinned:?}"
); );
assert!( assert!(
@@ -3604,6 +4042,7 @@ mod tests {
&db, &db,
"repo-repair-stalled", "repo-repair-stalled",
Duration::from_millis(2200), Duration::from_millis(2200),
None,
), ),
); );
let (pinned, mut cids_lock) = tokio::join!(pin, controller); let (pinned, mut cids_lock) = tokio::join!(pin, controller);
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
+165 -4
View File
@@ -5,6 +5,8 @@
//! //!
//! * is push traffic flowing? — `gitlawb_pushes_total` //! * is push traffic flowing? — `gitlawb_pushes_total`
//! * is fetch traffic flowing? — `gitlawb_fetches_total` //! * is fetch traffic flowing? — `gitlawb_fetches_total`
//! * how long do successful push and fetch operations take? —
//! `gitlawb_push_duration_seconds` / `gitlawb_fetch_duration_seconds`
//! * are signature checks passing or failing? — //! * are signature checks passing or failing? —
//! `gitlawb_auth_successes_total` / `gitlawb_auth_failures_total` //! `gitlawb_auth_successes_total` / `gitlawb_auth_failures_total`
//! * is the sync worker making progress? — //! * is the sync worker making progress? —
@@ -18,8 +20,13 @@
//! `gitlawb_gossip_replay_guard_saturated_total` //! `gitlawb_gossip_replay_guard_saturated_total`
//! * how big are the packs we're sending and receiving? — //! * how big are the packs we're sending and receiving? —
//! `gitlawb_pack_size_bytes` //! `gitlawb_pack_size_bytes`
//! * how long after an origin lands a push is it visible on this mirror? —
//! `gitlawb_sync_lag_seconds`
//! * a single `gitlawb_info{version, did}` gauge = 1, for joins/dashboards //! * a single `gitlawb_info{version, did}` gauge = 1, for joins/dashboards
//! * currently-connected peer count — `gitlawb_peers_connected` //! * currently-connected peer count — `gitlawb_peers_connected`
//! * reconciliation sweep gaps found and filled —
//! `gitlawb_reconciliation_gaps_found_total` /
//! `gitlawb_reconciliation_gaps_filled_total`
//! //!
//! All metrics live in a single process-wide registry initialized by //! All metrics live in a single process-wide registry initialized by
//! [`init`]. Increment helpers (`record_push`, `record_auth_failure`, ...) //! [`init`]. Increment helpers (`record_push`, `record_auth_failure`, ...)
@@ -31,11 +38,10 @@
//! free. //! free.
//! //!
//! Follow-ups (not in this module): //! Follow-ups (not in this module):
//! * per-route latency histograms (TraceLayer already gives us spans)
//! * per-peer p2p counters //! * per-peer p2p counters
//! * ipfs / pinata counters //! * ipfs / pinata counters
use std::sync::OnceLock; use std::{sync::OnceLock, time::Duration};
use prometheus::{ use prometheus::{
Encoder, Histogram, HistogramOpts, IntCounter, IntCounterVec, IntGauge, IntGaugeVec, Opts, Encoder, Histogram, HistogramOpts, IntCounter, IntCounterVec, IntGauge, IntGaugeVec, Opts,
@@ -57,7 +63,12 @@ static WEBHOOK_DELIVERIES: OnceLock<IntCounterVec> = OnceLock::new();
static GOSSIP_INGEST: OnceLock<IntCounterVec> = OnceLock::new(); static GOSSIP_INGEST: OnceLock<IntCounterVec> = OnceLock::new();
static GOSSIP_REPLAY_GUARD_SATURATED: OnceLock<IntCounter> = OnceLock::new(); static GOSSIP_REPLAY_GUARD_SATURATED: OnceLock<IntCounter> = OnceLock::new();
static PACK_SIZE: OnceLock<Histogram> = OnceLock::new(); static PACK_SIZE: OnceLock<Histogram> = OnceLock::new();
static PUSH_DURATION: OnceLock<Histogram> = OnceLock::new();
static FETCH_DURATION: OnceLock<Histogram> = OnceLock::new();
static SYNC_LAG: OnceLock<Histogram> = OnceLock::new();
static PEERS_CONNECTED: OnceLock<IntGauge> = OnceLock::new(); static PEERS_CONNECTED: OnceLock<IntGauge> = OnceLock::new();
static RECONCILIATION_GAPS_FOUND: OnceLock<IntCounter> = OnceLock::new();
static RECONCILIATION_GAPS_FILLED: OnceLock<IntCounter> = OnceLock::new();
/// 1 when the node started without its p2p identity because the key storage /// 1 when the node started without its p2p identity because the key storage
/// could not be used. The node keeps serving HTTP in that state and /health /// could not be used. The node keeps serving HTTP in that state and /health
/// stays green, so without this the outage is visible only to whoever reads /// stays green, so without this the outage is visible only to whoever reads
@@ -120,6 +131,51 @@ fn init_inner(version: &str, node_did: &str) {
.expect("register gitlawb_fetches_total"); .expect("register gitlawb_fetches_total");
FETCHES.set(fetches).expect("set FETCHES once"); FETCHES.set(fetches).expect("set FETCHES once");
let push_duration = Histogram::with_opts(
HistogramOpts::new(
"gitlawb_push_duration_seconds",
"Wall time of a successful git push (receive-pack) request in seconds, from handler entry to its response, including claim-loser returns where another worker owns the effects",
)
.buckets(operation_duration_buckets()),
)
.expect("gitlawb_push_duration_seconds definition");
registry
.register(Box::new(push_duration.clone()))
.expect("register gitlawb_push_duration_seconds");
PUSH_DURATION
.set(push_duration)
.expect("set PUSH_DURATION once");
let fetch_duration = Histogram::with_opts(
HistogramOpts::new(
"gitlawb_fetch_duration_seconds",
"Duration of successful git fetch (upload-pack) completion requests in seconds",
)
.buckets(operation_duration_buckets()),
)
.expect("gitlawb_fetch_duration_seconds definition");
registry
.register(Box::new(fetch_duration.clone()))
.expect("register gitlawb_fetch_duration_seconds");
FETCH_DURATION
.set(fetch_duration)
.expect("set FETCH_DURATION once");
let sync_lag = Histogram::with_opts(
HistogramOpts::new(
"gitlawb_sync_lag_seconds",
"Lag from the origin push-land timestamp (lag_origin) to successful local mirror visibility in seconds",
)
.buckets(vec![
1.0, 5.0, 10.0, 30.0, 60.0, 120.0, 300.0, 600.0, 1_800.0, 3_600.0,
]),
)
.expect("gitlawb_sync_lag_seconds definition");
registry
.register(Box::new(sync_lag.clone()))
.expect("register gitlawb_sync_lag_seconds");
SYNC_LAG.set(sync_lag).expect("set SYNC_LAG once");
let auth_successes = IntCounterVec::new( let auth_successes = IntCounterVec::new(
Opts::new( Opts::new(
"gitlawb_auth_successes_total", "gitlawb_auth_successes_total",
@@ -241,6 +297,29 @@ fn init_inner(version: &str, node_did: &str) {
.set(peers_connected) .set(peers_connected)
.expect("set PEERS_CONNECTED once"); .expect("set PEERS_CONNECTED once");
let gaps_found = IntCounter::with_opts(Opts::new(
"gitlawb_reconciliation_gaps_found_total",
"Total reconciliation sweep gaps detected (objects that should be pinned but are not)",
))
.expect("gitlawb_reconciliation_gaps_found_total definition");
registry
.register(Box::new(gaps_found.clone()))
.expect("register gitlawb_reconciliation_gaps_found_total");
RECONCILIATION_GAPS_FOUND
.set(gaps_found)
.expect("set RECONCILIATION_GAPS_FOUND once");
let gaps_filled = IntCounter::with_opts(Opts::new(
"gitlawb_reconciliation_gaps_filled_total",
"Total reconciliation sweep gaps successfully filled (objects pinned by the sweep)",
))
.expect("gitlawb_reconciliation_gaps_filled_total definition");
registry
.register(Box::new(gaps_filled.clone()))
.expect("register gitlawb_reconciliation_gaps_filled_total");
RECONCILIATION_GAPS_FILLED
.set(gaps_filled)
.expect("set RECONCILIATION_GAPS_FILLED once");
let p2p_key_load_failed = IntGauge::with_opts(Opts::new( let p2p_key_load_failed = IntGauge::with_opts(Opts::new(
"gitlawb_p2p_identity_key_load_failed", "gitlawb_p2p_identity_key_load_failed",
"1 if the node is running without p2p because its identity key could not be loaded", "1 if the node is running without p2p because its identity key could not be loaded",
@@ -274,6 +353,35 @@ pub fn record_fetch(repo: &str) {
} }
} }
/// Observe the elapsed time of a successful git push handler.
pub fn observe_push_duration(duration: Duration) {
observe_duration(&PUSH_DURATION, duration);
}
/// Observe the elapsed time of a successful git fetch completion request.
pub fn observe_fetch_duration(duration: Duration) {
observe_duration(&FETCH_DURATION, duration);
}
/// Observe end-to-end lag until an origin ref update is visible on this mirror.
pub fn observe_sync_lag(duration: Duration) {
observe_duration(&SYNC_LAG, duration);
}
fn observe_duration(histogram: &OnceLock<Histogram>, duration: Duration) {
if let Some(h) = histogram.get() {
h.observe(duration.as_secs_f64());
}
}
/// Test-only: current `gitlawb_sync_lag_seconds` sample count (0 if the
/// registry is not initialized). Lets the sync lag-gate tests assert that a
/// skipped observation really never reached the histogram.
#[cfg(test)]
pub fn sync_lag_sample_count_for_test() -> u64 {
SYNC_LAG.get().map(|h| h.get_sample_count()).unwrap_or(0)
}
/// Test-only: current `gitlawb_fetches_total` value for `repo` (0 if the registry /// Test-only: current `gitlawb_fetches_total` value for `repo` (0 if the registry
/// is not initialized). Lets api-layer tests assert the completed-fetch count with /// is not initialized). Lets api-layer tests assert the completed-fetch count with
/// a unique label instead of scraping the encoded text. /// a unique label instead of scraping the encoded text.
@@ -314,7 +422,11 @@ pub fn record_sync_processed(status: &str) {
} }
/// Record one webhook delivery attempt outcome. `result` ∈ /// Record one webhook delivery attempt outcome. `result` ∈
/// {ok, http_error, network_error, skipped}. /// {ok, http_error, network_error, skipped, invalid_url, secret_unopenable}.
/// `invalid_url` is a policy skip: the stored hook URL failed the public-URL
/// check at delivery time and the POST was never sent. `secret_unopenable` is
/// a withheld delivery: the hook carries a secret the node could not open, so
/// the POST was never sent rather than going out unsigned.
pub fn record_webhook_delivery(result: &str) { pub fn record_webhook_delivery(result: &str) {
if let Some(c) = WEBHOOK_DELIVERIES.get() { if let Some(c) = WEBHOOK_DELIVERIES.get() {
c.with_label_values(&[result]).inc(); c.with_label_values(&[result]).inc();
@@ -388,6 +500,12 @@ pub fn observe_pack_size(bytes: f64) {
} }
} }
fn operation_duration_buckets() -> Vec<f64> {
vec![
0.005, 0.01, 0.025, 0.05, 0.1, 0.25, 0.5, 1.0, 2.5, 5.0, 10.0, 30.0, 60.0, 120.0, 300.0,
]
}
/// Update the currently-connected peer count gauge. /// Update the currently-connected peer count gauge.
pub fn set_peers_connected(count: i64) { pub fn set_peers_connected(count: i64) {
if let Some(g) = PEERS_CONNECTED.get() { if let Some(g) = PEERS_CONNECTED.get() {
@@ -395,6 +513,20 @@ pub fn set_peers_connected(count: i64) {
} }
} }
/// Record reconciliation sweep gaps found (objects that should be pinned but are not).
pub fn record_reconciliation_gaps_found(count: u64) {
if let Some(c) = RECONCILIATION_GAPS_FOUND.get() {
c.inc_by(count);
}
}
/// Record reconciliation sweep gaps filled (objects successfully pinned by the sweep).
pub fn record_reconciliation_gaps_filled(count: u64) {
if let Some(c) = RECONCILIATION_GAPS_FILLED.get() {
c.inc_by(count);
}
}
/// Record whether the node is running without its p2p identity. /// Record whether the node is running without its p2p identity.
pub fn set_p2p_key_load_failed(failed: bool) { pub fn set_p2p_key_load_failed(failed: bool) {
if let Some(g) = P2P_KEY_LOAD_FAILED.get() { if let Some(g) = P2P_KEY_LOAD_FAILED.get() {
@@ -439,6 +571,13 @@ mod tests {
.expect("PUSHES set after init") .expect("PUSHES set after init")
.with_label_values(&["alice/repo"]) .with_label_values(&["alice/repo"])
.inc(); .inc();
observe_push_duration(Duration::from_millis(42));
observe_fetch_duration(Duration::from_millis(24));
// `observe_sync_lag` is deliberately NOT called. The sync lag-gate
// test asserts a before/after delta on the process-wide histogram, so
// a sample from this test running concurrently would flake it.
record_reconciliation_gaps_found(7);
record_reconciliation_gaps_filled(3);
let body = encode().expect("encode should succeed after init"); let body = encode().expect("encode should succeed after init");
assert!( assert!(
@@ -453,6 +592,24 @@ mod tests {
body.contains("gitlawb_pushes_total{repo=\"alice/repo\"} 1"), body.contains("gitlawb_pushes_total{repo=\"alice/repo\"} 1"),
"expected the incremented counter to be visible in: {body}" "expected the incremented counter to be visible in: {body}"
); );
for metric in [
"gitlawb_push_duration_seconds",
"gitlawb_fetch_duration_seconds",
"gitlawb_sync_lag_seconds",
] {
assert!(
body.contains(&format!("# TYPE {metric} histogram")),
"expected {metric} histogram in: {body}"
);
}
assert!(
body.contains("gitlawb_reconciliation_gaps_found_total 7"),
"expected the reconciliation gaps-found counter to be visible in: {body}"
);
assert!(
body.contains("gitlawb_reconciliation_gaps_filled_total 3"),
"expected the reconciliation gaps-filled counter to be visible in: {body}"
);
} }
/// #192 F4: `init` is idempotent and safe to call repeatedly. The panic that /// #192 F4: `init` is idempotent and safe to call repeatedly. The panic that
@@ -487,9 +644,13 @@ mod tests {
// here. It is a label-less process-wide counter and the seen-set test // here. It is a label-less process-wide counter and the seen-set test
// that asserts it asserts a before/after delta, so an increment from a // that asserts it asserts a before/after delta, so an increment from a
// second test running concurrently in this binary would be a flake with // second test running concurrently in this binary would be a flake with
// no diagnostic value. // no diagnostic value. `observe_sync_lag` is withheld for the same
// reason: the sync lag-gate test asserts a sample-count delta on the
// process-wide histogram.
record_gossip_ingest("accepted"); record_gossip_ingest("accepted");
observe_pack_size(1024.0); observe_pack_size(1024.0);
observe_push_duration(Duration::from_secs(1));
observe_fetch_duration(Duration::from_secs(1));
set_peers_connected(0); set_peers_connected(0);
} }
+779 -25
View File
@@ -7,11 +7,11 @@
//! The node's PeerId comes from an Ed25519 keypair loaded from a persistent //! The node's PeerId comes from an Ed25519 keypair loaded from a persistent
//! key file, so the PeerId is stable across restarts. //! key file, so the PeerId is stable across restarts.
use std::collections::{hash_map::DefaultHasher, HashMap}; use std::collections::{hash_map::DefaultHasher, HashMap, HashSet, VecDeque};
use std::hash::{Hash, Hasher}; use std::hash::{Hash, Hasher};
use std::path::{Component, Path, PathBuf}; use std::path::{Component, Path, PathBuf};
use std::sync::Arc; use std::sync::Arc;
use std::time::Duration; use std::time::{Duration, Instant};
use anyhow::{Context, Result}; use anyhow::{Context, Result};
use chrono::{DateTime, Utc}; use chrono::{DateTime, Utc};
@@ -306,8 +306,9 @@ impl IngestLimiters {
/// ///
/// 0 is the versionless form: the field set this struct shipped with, before /// 0 is the versionless form: the field set this struct shipped with, before
/// `v` existed. It is not a placeholder for "unset", it is a real version whose /// `v` existed. It is not a placeholder for "unset", it is a real version whose
/// wire encoding happens to omit the key. /// wire encoding happens to omit the key. 1 adds `lag_origin`, the first
pub(crate) const CURRENT_REF_UPDATE_VERSION: u32 = 0; /// version-gated field: it is emitted and verified only under v1.
pub(crate) const CURRENT_REF_UPDATE_VERSION: u32 = 1;
/// True for the version whose key is omitted from the wire form. Free function /// True for the version whose key is omitted from the wire form. Free function
/// because `skip_serializing_if` takes a path, not a closure. /// because `skip_serializing_if` takes a path, not a closure.
@@ -315,6 +316,17 @@ fn is_zero(v: &u32) -> bool {
*v == 0 *v == 0
} }
// Identify is an untrusted address source. Keep enough entries for normal
// multi-homed peers while bounding both one peer and the whole routing table.
const IDENTIFY_ADDRESS_LIMIT: usize = 8;
// Bound canonicalization and sorting too, including rejected/duplicate entries.
const IDENTIFY_REPORT_ADDRESS_LIMIT: usize = 64;
const IDENTIFY_NEW_ADDRESS_LIMIT: usize = 8;
const IDENTIFY_GLOBAL_ADDRESS_LIMIT: usize = 1024;
const IDENTIFY_ADDRESS_WINDOW: Duration = Duration::from_secs(60);
const IDENTIFY_ADDRESS_TTL: Duration = Duration::from_secs(30 * 60);
const IDENTIFY_ADDRESS_CLEANUP_INTERVAL: Duration = Duration::from_secs(60);
/// A ref-update event published to Gossipsub when a push lands. /// A ref-update event published to Gossipsub when a push lands.
/// ///
/// The signing bytes are this struct serialized with `sig` set to None (see /// The signing bytes are this struct serialized with `sig` set to None (see
@@ -365,8 +377,15 @@ pub struct RefUpdateEvent {
pub old_sha: String, pub old_sha: String,
/// SHA after the push /// SHA after the push
pub new_sha: String, pub new_sha: String,
/// RFC-3339 timestamp /// RFC-3339 timestamp of the publish, freshness-checked at ingest
pub timestamp: String, pub timestamp: String,
/// RFC-3339 instant the origin push landed, for sync-lag measurement only.
/// Kept out of `timestamp` so `check_freshness` still sees the publish
/// instant: a push whose durability wait ran long would otherwise sign an
/// already-stale event. Optional for older peers; ingest falls back to
/// `timestamp` when it is absent.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub lag_origin: Option<String>,
/// Certificate ID (from the ref certificate, if issued) /// Certificate ID (from the ref certificate, if issued)
pub cert_id: Option<String>, pub cert_id: Option<String>,
/// IPFS CID of the latest commit object (set after pinning completes) /// IPFS CID of the latest commit object (set after pinning completes)
@@ -1181,6 +1200,14 @@ pub(crate) async fn ingest_ref_update(
event.v, CURRENT_REF_UPDATE_VERSION event.v, CURRENT_REF_UPDATE_VERSION
)); ));
} }
// `lag_origin` is a v1 field: a v0 event carrying it is a protocol
// violation, not a newer peer's wire form, and refusing it keeps the field
// set inside each version's signing bytes exactly what that version means.
if event.v < 1 && event.lag_origin.is_some() {
return IngestOutcome::Rejected(
"v0 ref-update event must not carry the v1 field lag_origin".to_string(),
);
}
// The cheapest denial under enforcement, ahead of every gate that reads a // The cheapest denial under enforcement, ahead of every gate that reads a
// field: an unsigned event is refused before its `node_did` is resolved or // field: an unsigned event is refused before its `node_did` is resolved or
@@ -1487,6 +1514,12 @@ pub(crate) async fn ingest_ref_update(
&event.ref_name, &event.ref_name,
&event.new_sha, &event.new_sha,
event.cid.as_deref(), event.cid.as_deref(),
Some(
event
.lag_origin
.as_deref()
.unwrap_or(event.timestamp.as_str()),
),
) )
.await .await
{ {
@@ -1631,6 +1664,200 @@ fn did_to_kad_key(did: &str) -> kad::RecordKey {
kad::RecordKey::new(&format!("/gitlawb/did/{did}").as_bytes()) kad::RecordKey::new(&format!("/gitlawb/did/{did}").as_bytes())
} }
/// Returns true when `address` was explicitly configured for `peer` through
/// AddKnownPeer. Explicit addresses are retained in Kademlia even when an
/// overlapping Identify lease expires or is evicted.
fn is_explicit_address(
explicit_addresses: &HashMap<PeerId, HashSet<Multiaddr>>,
peer: &PeerId,
address: &Multiaddr,
) -> bool {
explicit_addresses
.get(peer)
.is_some_and(|addresses| addresses.contains(address))
}
#[derive(Debug)]
struct IdentifyAddress {
address: Multiaddr,
expires_at: Instant,
}
#[derive(Debug)]
struct IdentifyPeerAddresses {
addresses: VecDeque<IdentifyAddress>,
window_started: Instant,
new_addresses: usize,
}
#[derive(Debug, Default, PartialEq, Eq)]
struct IdentifyAddressChanges {
added: Vec<Multiaddr>,
refreshed: Vec<Multiaddr>,
removed: Vec<(PeerId, Multiaddr)>,
}
#[derive(Debug, Default)]
struct IdentifyAddressBook {
peers: HashMap<PeerId, IdentifyPeerAddresses>,
address_count: usize,
}
impl IdentifyAddressBook {
fn update(
&mut self,
peer_id: PeerId,
now: Instant,
addresses: &[Multiaddr],
) -> IdentifyAddressChanges {
let mut changes = IdentifyAddressChanges::default();
if self.peers.contains_key(&peer_id) {
changes.removed.extend(self.expire_peer(peer_id, now));
}
let reported = addresses
.iter()
.take(IDENTIFY_REPORT_ADDRESS_LIMIT)
.filter_map(|address| address.clone().with_p2p(peer_id).ok())
.collect::<HashSet<_>>();
if reported.is_empty() && !self.peers.contains_key(&peer_id) {
return changes;
}
let state = self
.peers
.entry(peer_id)
.or_insert_with(|| IdentifyPeerAddresses {
addresses: VecDeque::new(),
window_started: now,
new_addresses: 0,
});
if now.saturating_duration_since(state.window_started) >= IDENTIFY_ADDRESS_WINDOW {
state.window_started = now;
state.new_addresses = 0;
}
// Refresh every retained address in the bounded report before admission.
// Refreshed addresses are reported so the Identify handler can re-admit
// them to Kademlia. Kademlia may have dropped a retained address
// after a failed dial or never kept it for a full k-bucket, and a
// TTL refresh alone would never re-add it.
for existing in &mut state.addresses {
if reported.contains(&existing.address) {
existing.expires_at = now + IDENTIFY_ADDRESS_TTL;
changes.refreshed.push(existing.address.clone());
}
}
let mut new_addresses: Vec<Multiaddr> = reported
.iter()
.filter(|address| {
!state
.addresses
.iter()
.any(|existing| &existing.address == *address)
})
.cloned()
.collect();
// HashSet iteration is deliberately unordered. Sort new candidates so
// an oversized Identify report has deterministic admission behavior.
new_addresses.sort();
for address in new_addresses {
if state.new_addresses >= IDENTIFY_NEW_ADDRESS_LIMIT {
break;
}
let evicted = if state.addresses.len() < IDENTIFY_ADDRESS_LIMIT {
None
} else {
state
.addresses
.iter()
.position(|existing| !reported.contains(&existing.address))
.and_then(|index| state.addresses.remove(index))
};
if let Some(evicted) = evicted {
self.address_count -= 1;
changes.removed.push((peer_id, evicted.address));
} else if state.addresses.len() >= IDENTIFY_ADDRESS_LIMIT {
// Every retained address was refreshed in this report. Keep
// that stable subset and drop surplus new candidates.
break;
}
if self.address_count >= IDENTIFY_GLOBAL_ADDRESS_LIMIT {
// Reject the admission instead of evicting another peer's
// entry: cross-peer eviction lets a flood of new identities
// push honest peers' addresses out of the book and, when the
// evicted address is their last one, out of the Kademlia
// routing table. Per-peer eviction above still applies to the
// reporting peer's own addresses.
break;
}
state.new_addresses += 1;
state.addresses.push_back(IdentifyAddress {
address: address.clone(),
expires_at: now + IDENTIFY_ADDRESS_TTL,
});
self.address_count += 1;
changes.added.push(address);
}
// Do not keep a row when every candidate was rejected: a zero-address
// row is invisible to address_count and would otherwise linger until
// the expiry tick. This matches expire_peer, which drops empty rows.
if state.addresses.is_empty() {
self.peers.remove(&peer_id);
}
changes
}
fn expire(&mut self, now: Instant) -> Vec<(PeerId, Multiaddr)> {
let mut removed = Vec::new();
let peers = self.peers.keys().copied().collect::<Vec<_>>();
for peer_id in peers {
removed.extend(self.expire_peer(peer_id, now));
}
removed
}
fn expire_peer(&mut self, peer_id: PeerId, now: Instant) -> Vec<(PeerId, Multiaddr)> {
let expired = self
.peers
.get_mut(&peer_id)
.map(|state| Self::expire_state(state, now))
.unwrap_or_default();
self.address_count -= expired.len();
if self
.peers
.get(&peer_id)
.is_some_and(|state| state.addresses.is_empty())
{
self.peers.remove(&peer_id);
}
expired
.into_iter()
.map(|address| (peer_id, address))
.collect()
}
fn expire_state(state: &mut IdentifyPeerAddresses, now: Instant) -> Vec<Multiaddr> {
let mut expired = Vec::new();
let mut retained = VecDeque::with_capacity(state.addresses.len());
while let Some(address) = state.addresses.pop_front() {
if address.expires_at <= now {
expired.push(address.address);
} else {
retained.push_back(address);
}
}
state.addresses = retained;
expired
}
}
/// Combined libp2p behaviour. /// Combined libp2p behaviour.
#[derive(NetworkBehaviour)] #[derive(NetworkBehaviour)]
#[behaviour(prelude = "libp2p_swarm::derive_prelude")] #[behaviour(prelude = "libp2p_swarm::derive_prelude")]
@@ -3960,6 +4187,14 @@ pub async fn start(
// Track in-flight GetRecord queries → reply channels // Track in-flight GetRecord queries → reply channels
let mut pending_get_did: HashMap<kad::QueryId, oneshot::Sender<Option<DidRecord>>> = let mut pending_get_did: HashMap<kad::QueryId, oneshot::Sender<Option<DidRecord>>> =
HashMap::new(); HashMap::new();
// Keep Identify-derived entries across disconnects so a bootstrap address
// learned through Identify remains usable for redial. The TTL cleanup below
// removes entries that are no longer refreshed without touching explicit
// AddKnownPeer addresses.
let mut identify_addresses = IdentifyAddressBook::default();
let mut explicit_addresses: HashMap<PeerId, HashSet<Multiaddr>> = HashMap::new();
let mut identify_cleanup = tokio::time::interval(IDENTIFY_ADDRESS_CLEANUP_INTERVAL);
identify_cleanup.set_missed_tick_behavior(tokio::time::MissedTickBehavior::Skip);
// Start the event loop as a background task // Start the event loop as a background task
tokio::spawn(async move { tokio::spawn(async move {
@@ -3979,6 +4214,13 @@ pub async fn start(
_ = ingest_sweep.tick() => { _ = ingest_sweep.tick() => {
run_ingest_sweep(&ingest_limiters, &replay_guard, ingest_now()).await; run_ingest_sweep(&ingest_limiters, &replay_guard, ingest_now()).await;
} }
_ = identify_cleanup.tick() => {
for (peer_id, address) in identify_addresses.expire(Instant::now()) {
if !is_explicit_address(&explicit_addresses, &peer_id, &address) {
swarm.behaviour_mut().kademlia.remove_address(&peer_id, &address);
}
}
}
// Graceful shutdown: exit the swarm loop when the // Graceful shutdown: exit the swarm loop when the
// process-wide signal flips. This drops the Swarm // process-wide signal flips. This drops the Swarm
// which closes all libp2p connections cleanly. // which closes all libp2p connections cleanly.
@@ -4113,7 +4355,21 @@ pub async fn start(
identify::Event::Received { peer_id, info, .. } identify::Event::Received { peer_id, info, .. }
)) => { )) => {
debug!(peer = %peer_id, "identify received"); debug!(peer = %peer_id, "identify received");
for addr in info.listen_addrs { let changes = identify_addresses.update(
peer_id,
Instant::now(),
&info.listen_addrs,
);
for (removed_peer, addr) in changes.removed {
if !is_explicit_address(&explicit_addresses, &removed_peer, &addr)
{
swarm
.behaviour_mut()
.kademlia
.remove_address(&removed_peer, &addr);
}
}
for addr in changes.refreshed.into_iter().chain(changes.added) {
swarm.behaviour_mut().kademlia.add_address(&peer_id, addr); swarm.behaviour_mut().kademlia.add_address(&peer_id, addr);
} }
} }
@@ -4150,6 +4406,14 @@ pub async fn start(
} }
} }
P2pCommand::AddKnownPeer { peer_id, addr } => { P2pCommand::AddKnownPeer { peer_id, addr } => {
let Ok(addr) = addr.clone().with_p2p(peer_id) else {
warn!(%peer_id, %addr, "dropping known-peer address with a foreign peer suffix");
continue;
};
explicit_addresses
.entry(peer_id)
.or_default()
.insert(addr.clone());
swarm.behaviour_mut().kademlia.add_address(&peer_id, addr); swarm.behaviour_mut().kademlia.add_address(&peer_id, addr);
} }
P2pCommand::Dial(addr) => { P2pCommand::Dial(addr) => {
@@ -4201,6 +4465,453 @@ pub async fn start(
mod tests { mod tests {
use super::*; use super::*;
fn identify_address(peer_id: PeerId, octet: u8, port: u16) -> Multiaddr {
format!("/ip4/192.0.2.{octet}/udp/{port}/quic-v1")
.parse::<Multiaddr>()
.unwrap()
.with_p2p(peer_id)
.unwrap()
}
fn identify_address_without_peer(octet: u8, port: u16) -> Multiaddr {
format!("/ip4/192.0.2.{octet}/udp/{port}/quic-v1")
.parse::<Multiaddr>()
.unwrap()
}
#[test]
fn identify_empty_reports_do_not_retain_peer_entries() {
let now = Instant::now();
let mut book = IdentifyAddressBook::default();
let foreign = identify_address(PeerId::random(), 1, 10_000);
for _ in 0..IDENTIFY_GLOBAL_ADDRESS_LIMIT + 1 {
let peer = PeerId::random();
assert_eq!(
book.update(peer, now, &[]),
IdentifyAddressChanges::default()
);
assert_eq!(
book.update(peer, now, std::slice::from_ref(&foreign)),
IdentifyAddressChanges::default()
);
}
assert!(book.peers.is_empty());
assert_eq!(book.address_count, 0);
}
#[test]
fn identify_fully_rejected_peer_leaves_no_row_at_full_budget() {
let now = Instant::now();
let mut book = IdentifyAddressBook::default();
// Fill the global budget with one-address peers.
for index in 0..IDENTIFY_GLOBAL_ADDRESS_LIMIT {
let peer = PeerId::random();
let address = identify_address(peer, 1, 10_000 + index as u16);
book.update(peer, now, std::slice::from_ref(&address));
}
assert_eq!(book.address_count, IDENTIFY_GLOBAL_ADDRESS_LIMIT);
// A fresh peer reporting at saturation has every candidate rejected
// by the global-budget check. The row created by or_insert_with must
// be dropped: a zero-address row is invisible to address_count and
// would otherwise linger until the expiry tick.
let fresh = PeerId::random();
let fresh_address = identify_address(fresh, 2, 30_000);
let changes = book.update(fresh, now, std::slice::from_ref(&fresh_address));
assert!(changes.added.is_empty());
assert!(!book.peers.contains_key(&fresh));
assert_eq!(book.address_count, IDENTIFY_GLOBAL_ADDRESS_LIMIT);
}
#[test]
fn identify_update_reports_expired_addresses_as_removed() {
let peer = PeerId::random();
let now = Instant::now();
let address = identify_address(peer, 1, 10_000);
let mut book = IdentifyAddressBook::default();
book.update(peer, now, std::slice::from_ref(&address));
// An update past the TTL expires the stale address through
// expire_peer and reports it in changes.removed so the Identify
// handler can drop it from Kademlia.
let changes = book.update(
peer,
now + IDENTIFY_ADDRESS_TTL + Duration::from_secs(1),
std::slice::from_ref(&address),
);
assert_eq!(changes.removed, vec![(peer, address.clone())]);
assert_eq!(changes.added, vec![address]);
}
#[test]
fn identify_expire_drops_rows_emptied_by_expiry() {
let peer = PeerId::random();
let now = Instant::now();
let address = identify_address(peer, 1, 10_000);
let mut book = IdentifyAddressBook::default();
book.update(peer, now, std::slice::from_ref(&address));
assert!(book.peers.contains_key(&peer));
// Once every address in the row has expired, expire() must drop the
// row itself, matching the empty-row drop in update().
let removed = book.expire(now + IDENTIFY_ADDRESS_TTL + Duration::from_secs(1));
assert_eq!(removed, vec![(peer, address)]);
assert!(!book.peers.contains_key(&peer));
assert_eq!(book.address_count, 0);
}
#[test]
fn identify_report_processing_stops_at_the_input_limit() {
let peer = PeerId::random();
let now = Instant::now();
let foreign = identify_address(PeerId::random(), 1, 10_000);
let accepted = identify_address(peer, 2, 10_001);
let ignored = identify_address(peer, 3, 10_002);
let mut report = vec![foreign; IDENTIFY_REPORT_ADDRESS_LIMIT - 1];
report.push(accepted.clone());
report.push(ignored);
let mut book = IdentifyAddressBook::default();
let changes = book.update(peer, now, &report);
assert_eq!(changes.added, vec![accepted]);
assert!(changes.removed.is_empty());
assert_eq!(book.address_count, 1);
}
#[test]
fn identify_addresses_grow_without_eviction_below_the_peer_cap() {
let peer = PeerId::random();
let now = Instant::now();
let first = identify_address(peer, 1, 10_000);
let second = identify_address(peer, 2, 10_001);
let mut book = IdentifyAddressBook::default();
book.update(peer, now, std::slice::from_ref(&first));
let changes = book.update(
peer,
now + IDENTIFY_ADDRESS_WINDOW,
std::slice::from_ref(&second),
);
assert_eq!(changes.added, vec![second]);
assert!(changes.removed.is_empty());
assert_eq!(book.peers[&peer].addresses.len(), 2);
assert_eq!(book.address_count, 2);
}
#[test]
fn identify_global_budget_rejects_new_admissions() {
let now = Instant::now();
let mut book = IdentifyAddressBook::default();
let peers = (0..IDENTIFY_GLOBAL_ADDRESS_LIMIT)
.map(|index| {
let peer = PeerId::random();
let address = identify_address(peer, 1, 10_000 + index as u16);
book.update(peer, now, std::slice::from_ref(&address));
(peer, address)
})
.collect::<Vec<_>>();
let (peer, refreshed) = &peers[0];
let new_address = identify_address(*peer, 2, 30_000);
let changes = book.update(
*peer,
now + IDENTIFY_ADDRESS_WINDOW,
&[new_address.clone(), refreshed.clone()],
);
// The budget is full: the new address is rejected rather than
// evicting another peer's entry, and the refreshed address keeps its
// slot and is reported for re-admission to Kademlia.
assert!(changes.added.is_empty());
assert!(changes.removed.is_empty());
assert_eq!(changes.refreshed, vec![(*refreshed).clone()]);
assert_eq!(book.peers.len(), IDENTIFY_GLOBAL_ADDRESS_LIMIT);
assert_eq!(book.address_count, IDENTIFY_GLOBAL_ADDRESS_LIMIT);
assert_eq!(
book.peers[peer]
.addresses
.iter()
.map(|a| &a.address)
.collect::<Vec<_>>(),
vec![refreshed]
);
}
#[test]
fn identify_per_peer_eviction_applies_at_full_global_budget() {
let now = Instant::now();
let mut book = IdentifyAddressBook::default();
// One filler peer goes in before the victim so the evicted address
// is not also the globally-oldest insertion. If saturation evicted
// the globally-oldest entry instead of the victim's own oldest,
// the sentinel's address would be the one removed.
let sentinel = PeerId::random();
let sentinel_address = identify_address(sentinel, 200, 19_999);
book.update(sentinel, now, std::slice::from_ref(&sentinel_address));
// The victim peer holds a full per-peer row while the rest of the
// global budget is filled by one-address peers.
let victim = PeerId::random();
let victim_addresses: Vec<Multiaddr> = (0..IDENTIFY_ADDRESS_LIMIT)
.map(|index| identify_address(victim, index as u8 + 1, 10_000 + index as u16))
.collect();
book.update(victim, now, &victim_addresses);
for index in 0..IDENTIFY_GLOBAL_ADDRESS_LIMIT - IDENTIFY_ADDRESS_LIMIT - 1 {
let peer = PeerId::random();
let address = identify_address(peer, 200, 20_000 + index as u16);
book.update(peer, now, &[address]);
}
assert_eq!(book.address_count, IDENTIFY_GLOBAL_ADDRESS_LIMIT);
// The victim reports a ninth address after the rate window rolls over.
// Per-peer eviction must still apply at global saturation: the oldest
// non-reported address is evicted to make room, and the global count
// is unchanged. This ordering only holds because eviction runs before
// the global-budget check; hoisting that check above the eviction
// block makes this test fail.
let ninth = identify_address(victim, 100, 30_000);
let changes = book.update(
victim,
now + IDENTIFY_ADDRESS_WINDOW,
std::slice::from_ref(&ninth),
);
assert_eq!(changes.added, vec![ninth]);
assert_eq!(changes.removed, vec![(victim, victim_addresses[0].clone())]);
assert_eq!(book.address_count, IDENTIFY_GLOBAL_ADDRESS_LIMIT);
assert_eq!(book.peers[&victim].addresses.len(), IDENTIFY_ADDRESS_LIMIT);
// The sentinel was inserted first; per-peer eviction must leave it
// alone. A global-oldest eviction policy would have removed it.
assert_eq!(
book.peers[&sentinel]
.addresses
.iter()
.map(|a| &a.address)
.collect::<Vec<_>>(),
vec![&sentinel_address]
);
}
#[test]
fn explicit_address_helper_detects_retained_addresses() {
let peer = PeerId::random();
let other = PeerId::random();
let address = identify_address(peer, 1, 10_000);
let mut explicit: HashMap<PeerId, HashSet<Multiaddr>> = HashMap::new();
// Unknown peer: nothing is explicit.
assert!(!is_explicit_address(&explicit, &peer, &address));
explicit.entry(peer).or_default().insert(address.clone());
assert!(is_explicit_address(&explicit, &peer, &address));
// The same address under a different peer is not retained.
assert!(!is_explicit_address(&explicit, &other, &address));
// A different address for the same peer is not retained.
assert!(!is_explicit_address(
&explicit,
&peer,
&identify_address(peer, 2, 10_001)
));
}
#[test]
fn identify_addresses_are_capped_with_fifo_eviction() {
let peer_id = PeerId::random();
let now = Instant::now();
let initial = (0..IDENTIFY_ADDRESS_LIMIT)
.map(|index| identify_address(peer_id, index as u8 + 1, 10_000 + index as u16))
.collect::<Vec<_>>();
let replacement = identify_address(peer_id, 200, 20_000);
let mut book = IdentifyAddressBook::default();
let first = book.update(peer_id, now, &initial);
assert_eq!(first.added, initial);
assert!(first.removed.is_empty());
let second = book.update(
peer_id,
now + IDENTIFY_ADDRESS_WINDOW,
std::slice::from_ref(&replacement),
);
assert_eq!(second.added, vec![replacement]);
assert_eq!(second.removed, vec![(peer_id, initial[0].clone())]);
}
#[test]
fn identify_oversized_reports_keep_the_same_refreshed_subset() {
let peer_id = PeerId::random();
let now = Instant::now();
let mut report = (0..IDENTIFY_ADDRESS_LIMIT + 2)
.map(|index| identify_address(peer_id, index as u8 + 1, 10_000 + index as u16))
.collect::<Vec<_>>();
let mut book = IdentifyAddressBook::default();
let first = book.update(peer_id, now, &report);
assert_eq!(first.added.len(), IDENTIFY_ADDRESS_LIMIT);
assert_eq!(first.removed.len(), 0);
let retained = first.added.clone();
report.reverse();
let second = book.update(peer_id, now + IDENTIFY_ADDRESS_WINDOW, &report);
assert!(second.added.is_empty());
assert!(second.removed.is_empty());
let current = book
.peers
.get(&peer_id)
.unwrap()
.addresses
.iter()
.map(|entry| entry.address.clone())
.collect::<Vec<_>>();
assert_eq!(current, retained);
}
#[test]
fn identify_addresses_rate_limit_new_entries_across_updates() {
let peer_id = PeerId::random();
let now = Instant::now();
let initial = (0..IDENTIFY_NEW_ADDRESS_LIMIT)
.map(|index| identify_address(peer_id, index as u8 + 1, 10_000 + index as u16))
.collect::<Vec<_>>();
let replacement = identify_address(peer_id, 200, 20_000);
let mut book = IdentifyAddressBook::default();
assert_eq!(book.update(peer_id, now, &initial).added, initial);
assert!(book
.update(peer_id, now, std::slice::from_ref(&replacement))
.added
.is_empty());
assert_eq!(
book.update(
peer_id,
now + IDENTIFY_ADDRESS_WINDOW,
std::slice::from_ref(&replacement),
)
.added,
vec![replacement]
);
}
#[test]
fn identify_addresses_have_a_global_budget_across_peers() {
let now = Instant::now();
let mut book = IdentifyAddressBook::default();
let mut first = None;
for index in 0..IDENTIFY_GLOBAL_ADDRESS_LIMIT {
let peer_id = PeerId::random();
let address = identify_address(peer_id, (index % 254) as u8 + 1, 10_000 + index as u16);
if first.is_none() {
first = Some((peer_id, address.clone()));
}
assert!(book.update(peer_id, now, &[address]).removed.is_empty());
}
let (peer_id, first_address) = first.unwrap();
let address = identify_address(peer_id, 250, 30_000);
let changes = book.update(
peer_id,
now + IDENTIFY_ADDRESS_WINDOW,
std::slice::from_ref(&address),
);
// The global budget is full, so the admission is rejected and the
// peer's existing entry is left alone.
assert!(changes.added.is_empty());
assert!(changes.removed.is_empty());
assert_eq!(
book.peers[&peer_id]
.addresses
.iter()
.map(|a| &a.address)
.collect::<Vec<_>>(),
vec![&first_address]
);
assert_eq!(book.address_count, IDENTIFY_GLOBAL_ADDRESS_LIMIT);
}
#[test]
fn identify_addresses_refresh_and_expire() {
let peer_id = PeerId::random();
let now = Instant::now();
let address = identify_address(peer_id, 1, 10_000);
let mut book = IdentifyAddressBook::default();
assert_eq!(
book.update(peer_id, now, std::slice::from_ref(&address))
.added,
vec![address.clone()]
);
let refreshed = now + IDENTIFY_ADDRESS_TTL - Duration::from_secs(1);
let refresh = book.update(peer_id, refreshed, std::slice::from_ref(&address));
assert!(refresh.added.is_empty());
assert!(refresh.removed.is_empty());
// Refreshed addresses are reported so the Identify handler re-adds
// them to Kademlia even though they were not newly admitted.
assert_eq!(refresh.refreshed, vec![address.clone()]);
assert!(book
.expire(now + IDENTIFY_ADDRESS_TTL + Duration::from_secs(1))
.is_empty());
assert_eq!(
book.expire(refreshed + IDENTIFY_ADDRESS_TTL + Duration::from_secs(1)),
vec![(peer_id, address)]
);
}
#[test]
fn identify_refresh_reports_only_addresses_in_the_report() {
let peer_id = PeerId::random();
let now = Instant::now();
let reported = identify_address(peer_id, 1, 10_000);
let unreported = identify_address(peer_id, 2, 20_000);
let mut book = IdentifyAddressBook::default();
// Admit both addresses up front so the peer holds two.
assert_eq!(
book.update(peer_id, now, &[reported.clone(), unreported.clone()])
.added
.len(),
2
);
// A later report carrying only one of the two addresses must refresh
// just that one: pins the negative direction of the refresh
// predicate, so an address the peer stopped advertising keeps its
// original TTL instead of being refreshed back onto the Kademlia
// re-admit path on every report.
let report_at = now + IDENTIFY_ADDRESS_TTL - Duration::from_secs(1);
let changes = book.update(peer_id, report_at, std::slice::from_ref(&reported));
assert!(changes.added.is_empty());
assert!(changes.removed.is_empty());
assert_eq!(changes.refreshed, vec![reported.clone()]);
// The unreported address still expires at its original TTL, while
// the refreshed address survives until its refreshed TTL.
assert_eq!(
book.expire(now + IDENTIFY_ADDRESS_TTL + Duration::from_secs(1)),
vec![(peer_id, unreported)]
);
assert!(book.expire(report_at + Duration::from_secs(1)).is_empty());
assert_eq!(
book.expire(report_at + IDENTIFY_ADDRESS_TTL + Duration::from_secs(1)),
vec![(peer_id, reported)]
);
}
#[test]
fn identify_addresses_reject_foreign_peer_suffixes() {
let peer_id = PeerId::random();
let other_peer_id = PeerId::random();
let address = identify_address(other_peer_id, 1, 10_000);
let mut book = IdentifyAddressBook::default();
assert!(book
.update(peer_id, Instant::now(), &[address])
.added
.is_empty());
}
#[test]
fn identify_addresses_canonicalize_the_peer_suffix_once() {
let peer_id = PeerId::random();
let raw = identify_address_without_peer(1, 10_000);
let canonical = identify_address(peer_id, 1, 10_000);
let mut book = IdentifyAddressBook::default();
assert_eq!(
book.update(peer_id, Instant::now(), &[raw]).added,
vec![canonical]
);
}
#[test] #[test]
fn dir_mode_replace_authority_is_the_write_bits() { fn dir_mode_replace_authority_is_the_write_bits() {
assert!(!dir_mode_allows_untrusted_replace(0o755)); assert!(!dir_mode_allows_untrusted_replace(0o755));
@@ -6504,7 +7215,7 @@ mod tests {
#[test] #[test]
fn ref_update_event_round_trip_with_owner_did() { fn ref_update_event_round_trip_with_owner_did() {
let event = RefUpdateEvent { let event = RefUpdateEvent {
v: 0, v: CURRENT_REF_UPDATE_VERSION,
node_did: "did:key:zNode".into(), node_did: "did:key:zNode".into(),
pusher_did: "did:key:zPusher".into(), pusher_did: "did:key:zPusher".into(),
repo: "zOwner/myrepo".into(), repo: "zOwner/myrepo".into(),
@@ -6513,6 +7224,7 @@ mod tests {
old_sha: "0000000000000000000000000000000000000000".into(), old_sha: "0000000000000000000000000000000000000000".into(),
new_sha: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa".into(), new_sha: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa".into(),
timestamp: "2026-07-02T12:00:00Z".into(), timestamp: "2026-07-02T12:00:00Z".into(),
lag_origin: Some("2026-07-02T11:59:30Z".into()),
cert_id: None, cert_id: None,
cid: None, cid: None,
sig: None, sig: None,
@@ -6524,6 +7236,7 @@ mod tests {
let deserialized: RefUpdateEvent = serde_json::from_value(json).unwrap(); let deserialized: RefUpdateEvent = serde_json::from_value(json).unwrap();
assert_eq!(deserialized.owner_did, Some("did:key:zOwner".into())); assert_eq!(deserialized.owner_did, Some("did:key:zOwner".into()));
assert_eq!(deserialized.lag_origin, Some("2026-07-02T11:59:30Z".into()));
} }
#[test] #[test]
@@ -6566,7 +7279,7 @@ mod tests {
/// field is Some so the serialized form exercises the widest field set. /// field is Some so the serialized form exercises the widest field set.
fn populated_event() -> RefUpdateEvent { fn populated_event() -> RefUpdateEvent {
RefUpdateEvent { RefUpdateEvent {
v: 0, v: CURRENT_REF_UPDATE_VERSION,
node_did: "did:key:zNode".into(), node_did: "did:key:zNode".into(),
pusher_did: "did:key:zPusher".into(), pusher_did: "did:key:zPusher".into(),
repo: "zOwner/myrepo".into(), repo: "zOwner/myrepo".into(),
@@ -6575,6 +7288,7 @@ mod tests {
old_sha: "0000000000000000000000000000000000000000".into(), old_sha: "0000000000000000000000000000000000000000".into(),
new_sha: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa".into(), new_sha: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa".into(),
timestamp: "2026-07-02T12:00:00Z".into(), timestamp: "2026-07-02T12:00:00Z".into(),
lag_origin: Some("2026-07-02T11:59:30Z".into()),
cert_id: Some("cert-1".into()), cert_id: Some("cert-1".into()),
cid: Some("bafycid".into()), cid: Some("bafycid".into()),
sig: None, sig: None,
@@ -6629,30 +7343,53 @@ mod tests {
cid: Option<String>, cid: Option<String>,
} }
let mut event = populated_event(); // The fleet rule this version split is built around:
event.sig = Some("c2lnbmF0dXJl".into()); //
let bytes = serde_json::to_vec(&event).unwrap(); // * A v0 event is the pre-change wire form: no `v` key, no `lag_origin`,
// and it must still parse under the OLD struct verbatim.
// * A v1 event is for nodes that bumped with us. An un-upgraded peer
// rejects it on the `v` key itself, which is the LOUD version drop the
// ingest ceiling was designed to name, not a silent signature
// mismatch against bytes it could never compute.
let mut v0_event = populated_event();
v0_event.v = 0;
v0_event.lag_origin = None;
v0_event.sig = Some("c2lnbmF0dXJl".into());
let bytes = serde_json::to_vec(&v0_event).unwrap();
let legacy: LegacyRefUpdateEvent = serde_json::from_slice(&bytes) let legacy: LegacyRefUpdateEvent = serde_json::from_slice(&bytes)
.expect("new-code bytes must deserialize under the old field set"); .expect("v0 bytes must deserialize under the old field set");
assert_eq!(legacy.repo, "zOwner/myrepo"); assert_eq!(legacy.repo, "zOwner/myrepo");
assert_eq!(legacy.owner_did, Some("did:key:zOwner".into())); assert_eq!(legacy.owner_did, Some("did:key:zOwner".into()));
// An UNSIGNED event carries no `sig` key at all, so it is byte-identical // An UNSIGNED v0 event carries no `sig` key at all, so it is byte-identical
// in shape to the pre-change wire form and parses even under the strict // in shape to the pre-change wire form and parses even under the strict
// reader. This is what `skip_serializing_if` buys; drop it and a // reader. This is what `skip_serializing_if` buys; drop it and a
// `"sig": null` key appears here and this goes red. // `"sig": null` key appears here and this goes red.
let unsigned_bytes = serde_json::to_vec(&populated_event()).unwrap(); v0_event.sig = None;
let unsigned_bytes = serde_json::to_vec(&v0_event).unwrap();
let strict: StrictLegacyRefUpdateEvent = serde_json::from_slice(&unsigned_bytes) let strict: StrictLegacyRefUpdateEvent = serde_json::from_slice(&unsigned_bytes)
.expect("an unsigned event must carry no field the pre-change reader did not know"); .expect("an unsigned v0 event must carry no field the pre-change reader did not know");
assert_eq!(strict.repo, "zOwner/myrepo"); assert_eq!(strict.repo, "zOwner/myrepo");
// A SIGNED event does carry the new key, and that is intentional, not a // A SIGNED v0 event does carry the added `sig` key, and that is
// compatibility bug: the permissive reader above is what makes it // intentional, not a compatibility bug: the permissive reader above is
// harmless. Pinning the refusal here documents `sig` as the one added // what makes it harmless. Pinning the refusal here documents `sig` as
// field, so a SECOND addition cannot slip in unnoticed. // an added field, so a SECOND unsigned-field addition cannot slip in
serde_json::from_slice::<StrictLegacyRefUpdateEvent>(&bytes) // unnoticed.
.expect_err("a signed event must be visibly carrying the added `sig` key"); v0_event.sig = Some("c2lnbmF0dXJl".into());
serde_json::from_slice::<StrictLegacyRefUpdateEvent>(
&serde_json::to_vec(&v0_event).unwrap(),
)
.expect_err("a signed v0 event must be visibly carrying the added `sig` key");
// A v1 event is refused by the pre-change reader on the `v` key:
// refusing to parse IS the version drop old nodes perform, the same
// failure the ingest ceiling produces in the other direction.
serde_json::from_slice::<StrictLegacyRefUpdateEvent>(
&serde_json::to_vec(&populated_event()).unwrap(),
)
.expect_err("a v1 event must fail loudly on the `v` key for a pre-change reader");
} }
#[test] #[test]
@@ -6709,12 +7446,13 @@ mod tests {
/// this failure, and the emit-to-ingest round trip is structurally blind to /// this failure, and the emit-to-ingest round trip is structurally blind to
/// it because both sides re-serialize the same new declaration order. /// it because both sides re-serialize the same new declaration order.
const GOLDEN_SIGNING_BYTES: &str = concat!( const GOLDEN_SIGNING_BYTES: &str = concat!(
r#"{"node_did":"did:key:zNode","pusher_did":"did:key:zPusher","#, r#"{"v":1,"node_did":"did:key:zNode","pusher_did":"did:key:zPusher","#,
r#""repo":"zOwner/myrepo","owner_did":"did:key:zOwner","#, r#""repo":"zOwner/myrepo","owner_did":"did:key:zOwner","#,
r#""ref_name":"refs/heads/main","#, r#""ref_name":"refs/heads/main","#,
r#""old_sha":"0000000000000000000000000000000000000000","#, r#""old_sha":"0000000000000000000000000000000000000000","#,
r#""new_sha":"aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa","#, r#""new_sha":"aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa","#,
r#""timestamp":"2026-07-02T12:00:00Z","cert_id":"cert-1","cid":"bafycid"}"#, r#""timestamp":"2026-07-02T12:00:00Z","lag_origin":"2026-07-02T11:59:30Z","#,
r#""cert_id":"cert-1","cid":"bafycid"}"#,
); );
#[test] #[test]
@@ -6759,9 +7497,15 @@ mod tests {
/// The all-`None` optional shape: what the production publish site emits. /// The all-`None` optional shape: what the production publish site emits.
fn all_none_optionals_event() -> RefUpdateEvent { fn all_none_optionals_event() -> RefUpdateEvent {
RefUpdateEvent { RefUpdateEvent {
// v0 pins the versionless form: the key stays off the wire and the
// signing bytes are byte-identical to what a pre-version build emits.
v: 0,
owner_did: None, owner_did: None,
cert_id: None, cert_id: None,
cid: None, cid: None,
// None skips the key entirely, so the production-shape golden stays
// byte-identical even though lag_origin exists.
lag_origin: None,
..populated_event() ..populated_event()
} }
} }
@@ -6986,8 +7730,11 @@ mod tests {
let keypair = Keypair::generate(); let keypair = Keypair::generate();
// Signed at v0, where the key is absent from the signing bytes. // Signed at v0, where the key is absent from the signing bytes.
// Raising it on the received copy makes the key appear. // Raising it on the received copy makes the key appear. The v0 shape
// drops lag_origin too, since v0 may not carry the v1 field.
let mut at_zero = event_for(&keypair); let mut at_zero = event_for(&keypair);
at_zero.v = 0;
at_zero.lag_origin = None;
sign_ref_update(&keypair, &mut at_zero).unwrap(); sign_ref_update(&keypair, &mut at_zero).unwrap();
verify_ref_update(&at_zero).expect("baseline must verify before tampering"); verify_ref_update(&at_zero).expect("baseline must verify before tampering");
let mut raised = at_zero.clone(); let mut raised = at_zero.clone();
@@ -7292,7 +8039,14 @@ mod tests {
&bytes_of(&event), &bytes_of(&event),
"v1 event", "v1 event",
Some(( Some((
"unsupported ref-update event version 1; this build understands version 0", Box::leak(
format!(
"unsupported ref-update event version {}; this build understands version {}",
CURRENT_REF_UPDATE_VERSION + 1,
CURRENT_REF_UPDATE_VERSION
)
.into_boxed_str(),
),
"an unknown version must be named as one, never reported as a signature failure", "an unknown version must be named as one, never reported as a signature failure",
)), )),
) )
+490 -41
View File
@@ -153,6 +153,26 @@ pub async fn pin_object(
/// `upsert_branch_cid` and the p2p `publish_ref_update` gossip CID. The twin's return is /// `upsert_branch_cid` and the p2p `publish_ref_update` gossip CID. The twin's return is
/// log-only, so it omits a record-failed pin rather than logging a pin the resolver /// log-only, so it omits a record-failed pin rather than logging a pin the resolver
/// cannot serve. Moving this side to match would need that consumer moved first. /// cannot serve. Moving this side to match would need that consumer moved first.
/// Whether the committed `pinned_cids` row already names `repo_id` as
/// first pinner, making a failed redundant `record_pin_source` insert
/// irrelevant to resolvability (`pin_sources_for_oid` unions the
/// primary row). Bounded read; any failure (including its own
/// timeout) answers false — an unproven row never confirms.
/// Used only after the primary `record_pinata_cid` for this call
/// succeeded or verified: callers must not promote a row this call
/// did not establish.
async fn primary_covers_repo(
db: &crate::db::Db,
sha: &str,
repo_id: &str,
deadline: std::time::Instant,
) -> bool {
matches!(
crate::ipfs_pin::db_bounded(deadline, db.provenance_for_oid(sha)).await,
Ok(Some(owner)) if owner.as_str() == repo_id
)
}
// Ten arguments, over clippy's threshold: the three the budget and the git seam add // Ten arguments, over clippy's threshold: the three the budget and the git seam add
// (`git_bin`, `git_timeout`, `batch_budget`) plus #173's `repo_id` are what put the read // (`git_bin`, `git_timeout`, `batch_budget`) plus #173's `repo_id` are what put the read
// under test injection and under a deadline, and grouping them into a struct would only // under test injection and under a deadline, and grouping them into a struct would only
@@ -170,16 +190,33 @@ pub async fn pin_new_objects(
db: &crate::db::Db, db: &crate::db::Db,
repo_id: &str, repo_id: &str,
batch_budget: Duration, batch_budget: Duration,
) -> Vec<(String, String)> { fence: Option<&crate::ipfs_pin::PolicyFence>,
) -> crate::ipfs_pin::PinBatchOutcome {
if jwt.is_empty() { if jwt.is_empty() {
return vec![]; return crate::ipfs_pin::PinBatchOutcome {
confirmed: Vec::new(),
last_attempted: None,
};
} }
let deadline = Instant::now() + batch_budget; let deadline = Instant::now() + batch_budget;
let total = object_list.len(); let total = object_list.len();
let mut pinned = Vec::new(); let mut pinned = Vec::new();
let mut last_attempted: Option<String> = None;
for (attempted, sha) in object_list.into_iter().enumerate() { for (attempted, sha) in object_list.into_iter().enumerate() {
// Policy fence (R1-P1): a visibility narrow that lands after the caller
// built this batch must abort it before the next irreversible upload.
if let Some(f) = fence {
if !f.is_current().await {
tracing::warn!(
repo = %f.repo_id(),
unattempted = total - attempted,
"visibility policy changed mid-batch; stopping the Pinata pin loop"
);
break;
}
}
// Top of the iteration, before any of this object's work: an object is never // Top of the iteration, before any of this object's work: an object is never
// started with a remainder too small to cover a bounded read's teardown. The // started with a remainder too small to cover a bounded read's teardown. The
// gate is shared with the IPFS loop so the two cannot drift apart in how they // gate is shared with the IPFS loop so the two cannot drift apart in how they
@@ -191,6 +228,10 @@ pub async fn pin_new_objects(
{ {
break; break;
} }
// Attempt progress (not a durability claim): same contract as the
// IPFS twin — the continuation persists this, never the planned
// vector's tail.
last_attempted = Some(sha.clone());
// Every DB call from here to the end of the iteration is bounded by the // Every DB call from here to the end of the iteration is bounded by the
// ABSOLUTE batch deadline (F3, #173), through the same `db_bounded` helper the // ABSOLUTE batch deadline (F3, #173), through the same `db_bounded` helper the
@@ -395,6 +436,21 @@ pub async fn pin_new_objects(
} }
}; };
// Dispatch fence (R1-P1): re-read the policy epoch immediately before
// the irreversible HTTP POST. The iteration-top check catches a narrow
// that landed before work began; THIS check catches a narrow that landed
// during the has_pinata_cid round-trip or the bounded Git read.
if let Some(f) = fence {
if !f.is_current().await {
tracing::warn!(
repo = %f.repo_id(),
unattempted = total - attempted,
"visibility policy changed during preparation; aborting Pinata upload"
);
break;
}
}
match pin_object(client, upload_url, jwt, &sha, &data).await { match pin_object(client, upload_url, jwt, &sha, &data).await {
Ok(cid) if !cid.is_empty() => { Ok(cid) if !cid.is_empty() => {
// The resolver key (`pinned_cids.cid`) must be the locally-computed // The resolver key (`pinned_cids.cid`) must be the locally-computed
@@ -410,30 +466,109 @@ pub async fn pin_new_objects(
// runs under the shared client's own ceiling, so a successful one can // runs under the shared client's own ceiling, so a successful one can
// return with ~0 of the batch budget left, and an unfloored bound would // return with ~0 of the batch budget left, and an unfloored bound would
// fail a write that today completes in milliseconds. That costs more on // fail a write that today completes in milliseconds. That costs more on
// this side than on the twin: `pinned.push` below is UNCONDITIONAL, so // this side than on the twin: `pinned.push` below feeds
// `api/repos.rs` builds its `cid_map` from the pair either way and drives // `api/repos.rs`, which builds its `cid_map` from the pair and drives
// `upsert_branch_cid` plus the p2p `publish_ref_update` gossip from it. A // `upsert_branch_cid` plus the p2p `publish_ref_update` gossip from it. A
// dropped record therefore makes the node ADVERTISE a CID whose `/ipfs` // dropped record therefore makes the node ADVERTISE a CID whose `/ipfs`
// read 404s. If the floored bound still fires, THIS site's outcome really // read 404s. Only durably recorded pairs are pushed, so the map never
// is unknown, and unlike the source record below that is a property of // advertises an unconfirmed row.
// the operation: `record_pinata_cid` is a single autocommit upsert, so //
// the statement Postgres already started can still land after the client // Round 10 P2: a closed/failed DB write means the (sha, cid)
// future is cancelled. The warn names the arm through the error's own // pair is not durable; we suppress the `pinned.push` for this
// Display and the site keeps its existing behavior: the pair is still // sha so the reconcile cannot count a Pinata gap as filled
// returned, and the row may or may not exist. // when no row exists. The source-record failure arms are also
if let Err(e) = crate::ipfs_pin::db_bounded( // hard failures (multi-statement transaction never committed)
// and suppress the push in the same way.
//
// `record_pinata_cid` is an explicit multi-statement
// transaction (row lock + insert + commit), NOT a single
// autocommit upsert, so a timed-out future proves nothing:
// cancellation before COMMIT wrote nothing, during COMMIT is
// unknown. The Elapsed arm therefore verifies by content
// (`verify_pinata_record`: exact row + unchanged fence epoch)
// under its own bound and pushes only on proof; anything
// else stays suppressed and the gap is re-offered.
let fence_epoch = fence.map(|f| f.captured_epoch()).unwrap_or(i64::MAX);
let mut db_record_durable = false;
match crate::ipfs_pin::db_bounded(
crate::ipfs_pin::db_record_deadline(deadline), crate::ipfs_pin::db_record_deadline(deadline),
crate::ipfs_pin::retry_db_record(|| { crate::ipfs_pin::retry_db_record(|| {
db.record_pinata_cid(&sha, &raw_cid, &cid, Some(repo_id)) // P2 (reviewer round 9): Pinata's POST is
// irreversible exactly like IPFS's, so
// route the record through the fenced
// variant when a fence is in scope. The
// `i64::MAX` sentinel tells the helper
// to skip the lock + comparison (the
// unfenced caller path).
db.record_pinata_cid(&sha, &raw_cid, &cid, Some(repo_id), fence_epoch)
}), }),
) )
.await .await
{ {
tracing::warn!(sha = %sha, err = %e, "failed to record pinata_cid in DB"); Ok(()) => db_record_durable = true,
Err(crate::ipfs_pin::BoundedDbError::Elapsed) => {
tracing::warn!(
sha = %sha,
"record_pinata_cid deadline elapsed on a multi-statement transaction; \
verifying the exact row before treating it as durable"
);
match crate::ipfs_pin::db_bounded(
crate::ipfs_pin::db_record_deadline(deadline),
db.verify_pinata_record(&sha, &raw_cid, &cid, repo_id, fence_epoch),
)
.await
{
Ok(true) => {
tracing::warn!(
sha = %sha,
"timed-out pinata record verified present with unchanged epoch; \
treating as durable"
);
db_record_durable = true;
}
Ok(false) => {
tracing::warn!(
sha = %sha,
"timed-out pinata record has no matching row or the epoch moved; \
suppressing the push so the gap is re-offered"
);
}
Err(e) => {
tracing::warn!(
sha = %sha,
err = %e,
"pinata record verification did not complete; \
suppressing the push so the gap is re-offered"
);
}
}
}
Err(e) => {
tracing::warn!(
sha = %sha,
err = %e,
"failed to record pinata_cid in DB; suppressing the (sha, cid) push \
so the reconcile cannot count this gap as filled"
);
// db_record_durable stays false → push suppressed
}
} }
// F1 (#173 round 8): also record the first pinner in pin_repo_sources. // F1 (#173 round 8): also record the first pinner in pin_repo_sources.
// U3: an exhausted retry marks the set incomplete so the resolver keeps // U3: an exhausted retry marks the set incomplete so the resolver keeps
// the scan fallback rather than 404ing a copy it could serve. // the scan fallback rather than 404ing a copy it could serve.
//
// Whether the pair stays confirmed depends on what the
// PRIMARY `record_pinata_cid` above established: that row
// already names this repo as first pinner
// (`pin_sources_for_oid` unions it), so the resolver
// reaches this copy with or without the redundant
// `pin_repo_sources` insert. A source-write failure then
// keeps the pair confirmed (but still marks the set
// incomplete, preserving the U3 compensation). If the
// primary row names ANOTHER repo — or is absent — the
// source write was load-bearing and the push stays
// suppressed. (The IPFS twin needs no equivalent: its
// pin and source land in ONE transaction.)
match crate::ipfs_pin::db_bounded( match crate::ipfs_pin::db_bounded(
crate::ipfs_pin::db_record_deadline(deadline), crate::ipfs_pin::db_record_deadline(deadline),
crate::ipfs_pin::retry_db_record(|| db.record_pin_source(&sha, repo_id)), crate::ipfs_pin::retry_db_record(|| db.record_pin_source(&sha, repo_id)),
@@ -445,9 +580,7 @@ pub async fn pin_new_objects(
// wraps `record_pin_source`, an explicit transaction, so a timed-out // wraps `record_pin_source`, an explicit transaction, so a timed-out
// call definitely never committed and the source is definitely // call definitely never committed and the source is definitely
// missing. Mark the set incomplete rather than leaving it incomplete // missing. Mark the set incomplete rather than leaving it incomplete
// and unmarked. Note the contrast with `record_pinata_cid` a few // and unmarked.
// lines up: that one is a single autocommit statement, so its
// timeout genuinely is an unknown outcome and it is warn-only.
Err(e @ crate::ipfs_pin::BoundedDbError::Elapsed) => { Err(e @ crate::ipfs_pin::BoundedDbError::Elapsed) => {
tracing::warn!( tracing::warn!(
sha = %sha, sha = %sha,
@@ -464,6 +597,16 @@ pub async fn pin_new_objects(
{ {
tracing::warn!(sha = %sha, err = %e, "failed to mark pin sources incomplete"); tracing::warn!(sha = %sha, err = %e, "failed to mark pin sources incomplete");
} }
if !primary_covers_repo(
db,
&sha,
repo_id,
crate::ipfs_pin::db_record_deadline(deadline),
)
.await
{
db_record_durable = false;
}
} }
Err(e) => { Err(e) => {
tracing::warn!(sha = %sha, err = %e, "failed to record pin source"); tracing::warn!(sha = %sha, err = %e, "failed to record pin source");
@@ -475,9 +618,21 @@ pub async fn pin_new_objects(
{ {
tracing::warn!(sha = %sha, err = %e, "failed to mark pin sources incomplete"); tracing::warn!(sha = %sha, err = %e, "failed to mark pin sources incomplete");
} }
if !primary_covers_repo(
db,
&sha,
repo_id,
crate::ipfs_pin::db_record_deadline(deadline),
)
.await
{
db_record_durable = false;
}
} }
} }
pinned.push((sha, cid)); if db_record_durable {
pinned.push((sha, cid));
}
} }
Ok(_) => {} Ok(_) => {}
Err(e) => { Err(e) => {
@@ -486,7 +641,10 @@ pub async fn pin_new_objects(
} }
} }
pinned crate::ipfs_pin::PinBatchOutcome {
confirmed: pinned,
last_attempted,
}
} }
// ── Tests ───────────────────────────────────────────────────────────────────── // ── Tests ─────────────────────────────────────────────────────────────────────
@@ -688,19 +846,20 @@ mod tests {
&db, &db,
"repo-merge-test", "repo-merge-test",
Duration::from_millis(5500), Duration::from_millis(5500),
None,
), ),
) )
.await .await
.expect("wedge guard: a 5.5s budget cannot take 30s"); .expect("wedge guard: a 5.5s budget cannot take 30s");
assert!( assert!(
(1..=3).contains(&pinned.len()), (1..=3).contains(&pinned.confirmed.len()),
"the batch must stop partway, not pin all five and not stall on the first: pinned {}", "the batch must stop partway, not pin all five and not stall on the first: pinned {}",
pinned.len() pinned.confirmed.len()
); );
assert_eq!( assert_eq!(
requests.load(std::sync::atomic::Ordering::SeqCst), requests.load(std::sync::atomic::Ordering::SeqCst),
pinned.len(), pinned.confirmed.len(),
"no upload may be issued for an object the budget stopped short of" "no upload may be issued for an object the budget stopped short of"
); );
let text = logs.text(); let text = logs.text();
@@ -728,9 +887,9 @@ mod tests {
}) })
.unwrap_or_else(|| panic!("the deadline warn must name the unattempted count: {text}")); .unwrap_or_else(|| panic!("the deadline warn must name the unattempted count: {text}"));
assert!( assert!(
unattempted >= 1 && unattempted + pinned.len() <= 5, unattempted >= 1 && unattempted + pinned.confirmed.len() <= 5,
"unattempted={unattempted} with {} pinned is not a partial batch of five", "unattempted={unattempted} with {} pinned is not a partial batch of five",
pinned.len() pinned.confirmed.len()
); );
} }
@@ -779,6 +938,7 @@ mod tests {
&db, &db,
"repo-merge-test", "repo-merge-test",
Duration::from_secs(2), Duration::from_secs(2),
None,
), ),
) )
.await .await
@@ -789,7 +949,7 @@ mod tests {
let elapsed = started.elapsed(); let elapsed = started.elapsed();
assert!( assert!(
pinned.is_empty(), pinned.confirmed.is_empty(),
"a git that never answers cannot produce a pinned object: {pinned:?}" "a git that never answers cannot produce a pinned object: {pinned:?}"
); );
assert!( assert!(
@@ -868,6 +1028,7 @@ mod tests {
"repo-git-timeout", "repo-git-timeout",
// Generous, so a call that ends on time ended on `git_timeout`. // Generous, so a call that ends on time ended on `git_timeout`.
Duration::from_secs(60), Duration::from_secs(60),
None,
), ),
) )
.await .await
@@ -878,7 +1039,7 @@ mod tests {
let elapsed = started.elapsed(); let elapsed = started.elapsed();
assert!( assert!(
pinned.is_empty(), pinned.confirmed.is_empty(),
"a git that never answers cannot produce a pinned object: {pinned:?}" "a git that never answers cannot produce a pinned object: {pinned:?}"
); );
assert!( assert!(
@@ -912,6 +1073,7 @@ mod tests {
/// probe carries its oid on stdin rather than in argv, so an oid appears in the log only /// probe carries its oid on stdin rather than in argv, so an oid appears in the log only
/// once an object has already got past its probe, and a healthy object costs two /// once an object has already got past its probe, and a healthy object costs two
/// invocations to a faulting one's one. /// invocations to a faulting one's one.
#[cfg(unix)]
fn objects_attempted(log: &std::path::Path) -> usize { fn objects_attempted(log: &std::path::Path) -> usize {
std::fs::read_to_string(log) std::fs::read_to_string(log)
.unwrap_or_default() .unwrap_or_default()
@@ -969,6 +1131,7 @@ mod tests {
&db, &db,
"repo-merge-test", "repo-merge-test",
Duration::from_secs(60), Duration::from_secs(60),
None,
), ),
) )
.await .await
@@ -978,7 +1141,7 @@ mod tests {
if genuinely_unreadable { if genuinely_unreadable {
assert!( assert!(
pinned.is_empty(), pinned.confirmed.is_empty(),
"nothing can be pinned through a store that cannot be read: {pinned:?}" "nothing can be pinned through a store that cannot be read: {pinned:?}"
); );
assert_eq!( assert_eq!(
@@ -1043,6 +1206,7 @@ mod tests {
&db, &db,
"repo-merge-test", "repo-merge-test",
Duration::from_secs(60), Duration::from_secs(60),
None,
), ),
) )
.await .await
@@ -1054,7 +1218,7 @@ mod tests {
"an object-scoped fault must not stop the batch: every object must be read" "an object-scoped fault must not stop the batch: every object must be read"
); );
assert_eq!( assert_eq!(
pinned.len(), pinned.confirmed.len(),
4, 4,
"one corrupt object must cost only itself: the other four must still pin" "one corrupt object must cost only itself: the other four must still pin"
); );
@@ -1094,13 +1258,18 @@ mod tests {
&db, &db,
"repo-merge-test", "repo-merge-test",
Duration::from_secs(60), Duration::from_secs(60),
None,
), ),
) )
.await .await
.expect("an immediate endpoint and three healthy objects cannot take 30s"); .expect("an immediate endpoint and three healthy objects cannot take 30s");
assert_eq!(pinned.len(), 3, "every healthy object must pin: {pinned:?}"); assert_eq!(
for (i, (sha, cid)) in pinned.iter().enumerate() { pinned.confirmed.len(),
3,
"every healthy object must pin: {pinned:?}"
);
for (i, (sha, cid)) in pinned.confirmed.iter().enumerate() {
assert_eq!(sha, &oids[i], "the pairs must carry the objects' own oids"); assert_eq!(sha, &oids[i], "the pairs must carry the objects' own oids");
assert_eq!(cid, "QmPinataBatchTestCid"); assert_eq!(cid, "QmPinataBatchTestCid");
assert!( assert!(
@@ -1123,11 +1292,15 @@ mod tests {
&db, &db,
"repo-merge-test", "repo-merge-test",
Duration::from_secs(60), Duration::from_secs(60),
None,
), ),
) )
.await .await
.expect("a fully deduped batch cannot take 30s"); .expect("a fully deduped batch cannot take 30s");
assert!(again.is_empty(), "already-recorded objects must be skipped"); assert!(
again.confirmed.is_empty(),
"already-recorded objects must be skipped"
);
assert_eq!( assert_eq!(
requests.load(std::sync::atomic::Ordering::SeqCst), requests.load(std::sync::atomic::Ordering::SeqCst),
3, 3,
@@ -1135,6 +1308,63 @@ mod tests {
); );
} }
/// The outcome reports the last object actually ENTERED, not the planned
/// tail (Pinata twin of the IPFS test above): the first upload takes 6s
/// against a 2s batch budget, so it still completes — the Pinata loop
/// has no per-request timeout — but the loop-top gate then breaks the
/// batch before the second object starts. `last_attempted` must be the
/// first OID even though two OIDs were never visited; the 6s server
/// sleep makes the break deterministic on any box.
#[sqlx::test]
async fn pin_new_objects_reports_last_attempted_not_planned_tail(pool: sqlx::PgPool) {
let db = crate::db::Db::for_testing(pool);
db.run_migrations().await.expect("migrations");
let tmp = tempfile::TempDir::new().unwrap();
let repo_path = tmp.path().join("attempted.git");
let oids = seed_loose_blobs(&repo_path, 3);
let requests = std::sync::Arc::new(std::sync::atomic::AtomicUsize::new(0));
let endpoint = delaying_pinata_endpoint(
vec![Duration::from_secs(6)],
std::sync::Arc::clone(&requests),
)
.await;
let client = reqwest::Client::new();
let outcome = tokio::time::timeout(
Duration::from_secs(60),
pin_new_objects(
&client,
&endpoint,
"test-jwt",
&repo_path,
"git",
Duration::from_secs(60),
oids.clone(),
&db,
"repo-attempted",
Duration::from_secs(2),
None,
),
)
.await
.expect("one 6s upload plus bounded gates cannot take 60s");
assert_eq!(
outcome.last_attempted,
Some(oids[0].clone()),
"only the first object was entered; the planned tail was never visited"
);
assert_eq!(
outcome.confirmed.len(),
1,
"the slow first upload still completed and recorded"
);
assert_eq!(
requests.load(std::sync::atomic::Ordering::SeqCst),
1,
"exactly one upload was attempted"
);
}
/// The no-op configuration still short-circuits with the budgeted signature: an /// The no-op configuration still short-circuits with the budgeted signature: an
/// empty JWT must return before any git child is spawned and before any request /// empty JWT must return before any git child is spawned and before any request
/// is issued. The `git_bin` here records every invocation, so "git was never /// is issued. The `git_bin` here records every invocation, so "git was never
@@ -1176,12 +1406,13 @@ mod tests {
&db, &db,
"repo-merge-test", "repo-merge-test",
Duration::from_secs(60), Duration::from_secs(60),
None,
), ),
) )
.await .await
.expect("an unconfigured sink returns immediately"); .expect("an unconfigured sink returns immediately");
assert!(pinned.is_empty(), "an empty JWT pins nothing"); assert!(pinned.confirmed.is_empty(), "an empty JWT pins nothing");
assert!( assert!(
!log.exists(), !log.exists(),
"no git child may be spawned when the sink is not configured" "no git child may be spawned when the sink is not configured"
@@ -1281,6 +1512,7 @@ mod tests {
&db, &db,
"repo-pinata-stalled", "repo-pinata-stalled",
Duration::from_millis(1500), Duration::from_millis(1500),
None,
), ),
) )
.await .await
@@ -1291,7 +1523,7 @@ mod tests {
let elapsed = started.elapsed(); let elapsed = started.elapsed();
assert!( assert!(
pinned.is_empty(), pinned.confirmed.is_empty(),
"a stalled pinata-status check cannot produce a pinned object: {pinned:?}" "a stalled pinata-status check cannot produce a pinned object: {pinned:?}"
); );
assert!( assert!(
@@ -1337,9 +1569,15 @@ mod tests {
// bytes. // bytes.
let raw_cid = let raw_cid =
gitlawb_core::cid::Cid::from_git_object_bytes(b"pinata skip seed").to_string(); gitlawb_core::cid::Cid::from_git_object_bytes(b"pinata skip seed").to_string();
db.record_pinata_cid(&sha, &raw_cid, "QmSeedProviderCid", Some("repo-seed")) db.record_pinata_cid(
.await &sha,
.unwrap(); &raw_cid,
"QmSeedProviderCid",
Some("repo-seed"),
i64::MAX,
)
.await
.unwrap();
db.record_pin_source(&sha, "repo-seed").await.unwrap(); db.record_pin_source(&sha, "repo-seed").await.unwrap();
let requests = std::sync::Arc::new(std::sync::atomic::AtomicUsize::new(0)); let requests = std::sync::Arc::new(std::sync::atomic::AtomicUsize::new(0));
let endpoint = delaying_pinata_endpoint( let endpoint = delaying_pinata_endpoint(
@@ -1371,6 +1609,7 @@ mod tests {
&db, &db,
"repo-pinata-skip-stalled", "repo-pinata-skip-stalled",
Duration::from_millis(1500), Duration::from_millis(1500),
None,
), ),
) )
.await .await
@@ -1381,7 +1620,7 @@ mod tests {
let elapsed = started.elapsed(); let elapsed = started.elapsed();
assert!( assert!(
pinned.is_empty(), pinned.confirmed.is_empty(),
"an already-pinned object is skipped, never re-uploaded: {pinned:?}" "an already-pinned object is skipped, never re-uploaded: {pinned:?}"
); );
assert_eq!( assert_eq!(
@@ -1425,7 +1664,7 @@ mod tests {
/// ///
/// The lock time is a MARGIN, not a boundary: taking it at 100ms left /// The lock time is a MARGIN, not a boundary: taking it at 100ms left
/// `has_pinata_cid` racing it on a loaded box, and losing that race makes the read /// `has_pinata_cid` racing it on a loaded box, and losing that race makes the read
/// block, time out, and break the batch, which fails on `pinned.len() == 1` for a /// block, time out, and break the batch, which fails on `pinned.confirmed.len() == 1` for a
/// reason that has nothing to do with the floor. Any time between the /// reason that has nothing to do with the floor. Any time between the
/// `has_pinata_cid` round trip and the upload's 1.7s return proves the same thing. /// `has_pinata_cid` round trip and the upload's 1.7s return proves the same thing.
#[sqlx::test] #[sqlx::test]
@@ -1471,6 +1710,7 @@ mod tests {
&db, &db,
"repo-pinata-spent-budget", "repo-pinata-spent-budget",
Duration::from_millis(2000), Duration::from_millis(2000),
None,
), ),
); );
let (pinned, ()) = tokio::join!(pin, locker); let (pinned, ()) = tokio::join!(pin, locker);
@@ -1488,7 +1728,7 @@ mod tests {
record makes api/repos.rs advertise a CID the resolver cannot serve" record makes api/repos.rs advertise a CID the resolver cannot serve"
); );
assert_eq!( assert_eq!(
pinned.len(), pinned.confirmed.len(),
1, 1,
"the uploaded pin must still be returned: {pinned:?}" "the uploaded pin must still be returned: {pinned:?}"
); );
@@ -1556,6 +1796,7 @@ mod tests {
&db, &db,
"repo-pinata-post-upload", "repo-pinata-post-upload",
Duration::from_millis(1500), Duration::from_millis(1500),
None,
), ),
) )
.await .await
@@ -1569,10 +1810,21 @@ mod tests {
drop(lock); drop(lock);
upload.assert_async().await; upload.assert_async().await;
// The primary `record_pinata_cid` committed above naming this
// repo as first pinner, so `pin_sources_for_oid` resolves the
// copy with or without the redundant `pin_repo_sources`
// insert: the pair stays confirmed even though the source
// record timed out. The incomplete marker below is still set
// (compensation preserved); only the suppression is gone.
// Prior to the round 10 fix, the push fired regardless of
// every post-upload outcome; the correction since is that
// confirmation follows the durable primary row, not the
// redundant source write.
assert_eq!( assert_eq!(
pinned.len(), pinned.confirmed.len(),
1, 1,
"the upload succeeded, so this lane still returns the pair: {pinned:?}" "record_pin_source timed out (pin_repo_sources locked) but the primary row \
covers this repo, so the pair stays confirmed: {pinned:?}"
); );
assert!( assert!(
elapsed < Duration::from_secs(8), elapsed < Duration::from_secs(8),
@@ -1589,6 +1841,203 @@ mod tests {
); );
} }
/// The other half of the source-write contract: when the committed
/// primary row names ANOTHER repo as first pinner, the failed
/// `record_pin_source` was load-bearing (not redundant) and the
/// push stays suppressed. Fixture: a local-only row owned by
/// repo-first (no Pinata CID yet), then a Pinata loop as
/// repo-second with `pin_repo_sources` locked. The upload lands,
/// the primary upsert keeps repo-first, the source insert stalls,
/// and coverage fails — so no pair, but the incomplete marker
/// still lands and the provider CID is still recorded.
#[sqlx::test]
async fn pinata_post_upload_stalled_source_without_coverage_suppresses_push(
pool: sqlx::PgPool,
) {
let db = crate::db::Db::for_testing(pool.clone());
db.run_migrations().await.expect("migrations");
let tmp = tempfile::TempDir::new().unwrap();
let repo_path = tmp.path().join("pinata_second_source.git");
let oids = seed_loose_blobs(&repo_path, 1);
let sha = oids[0].clone();
// A local-only row owned by repo-first: the writer path below
// (repo-second, Pinata-only so far) must not claim it.
let raw =
gitlawb_core::cid::Cid::from_git_object_bytes(b"pinata loop object 0\n").to_string();
db.record_pinned_cid_with_source(&sha, &raw, "repo-first")
.await
.unwrap();
let mut server = mockito::Server::new_async().await;
let upload = server
.mock("POST", mockito::Matcher::Any)
.with_status(200)
.with_body(r#"{"data":{"cid":"QmSecondSourceProviderCid"}}"#)
.expect(1)
.create_async()
.await;
let (_logs, _log_guard) = capture_logs();
let mut lock = lock_table(&pool, "pin_repo_sources").await;
let client = reqwest::Client::new();
let pinned = tokio::time::timeout(
Duration::from_secs(20),
pin_new_objects(
&client,
&server.url(),
"test-jwt",
&repo_path,
"git",
Duration::from_secs(30),
oids,
&db,
"repo-second",
Duration::from_millis(1500),
None,
),
)
.await
.expect("record + coverage ladders are each floored");
rollback(&mut lock).await;
drop(lock);
upload.assert_async().await;
assert!(
pinned.confirmed.is_empty(),
"repo-second is not covered by the repo-first primary row, so the \
failed source write stays suppressing: {pinned:?}"
);
assert!(
db.pin_sources_incomplete(&sha).await.unwrap(),
"the compensation marker still lands"
);
// The primary upsert itself landed (provider CID recorded under
// the first-pinner row); only the pair was suppressed.
assert!(
db.has_pinata_cid(&sha).await.unwrap(),
"the committed provider row survives the suppressed push"
);
}
/// Hold a row-level lock on one repos row in an open transaction: plain
/// `SELECT`s (fence reads, skip checks) proceed, but any `SELECT ...
/// FOR UPDATE` on the row — exactly what the fenced record takes —
/// blocks until rollback. A table-level lock would also stall the
/// fence's own epoch reads and deadlock the loop under test, proving
/// nothing about the record arm.
async fn hold_repo_row_lock(
pool: &sqlx::PgPool,
repo_id: &str,
) -> sqlx::pool::PoolConnection<sqlx::Postgres> {
let mut conn = pool.acquire().await.unwrap();
sqlx::raw_sql("BEGIN").execute(&mut *conn).await.unwrap();
sqlx::query("SELECT policy_epoch FROM repos WHERE id = $1 FOR UPDATE")
.bind(repo_id)
.fetch_one(&mut *conn)
.await
.unwrap();
conn
}
/// A timed-out `record_pinata_cid` with no verifiable row suppresses the
/// push: `record_pinata_cid` is an explicit transaction, so Elapsed proves
/// nothing, and the verify read finds no row, so nothing is proven that
/// way either. The upload mock still expects the POST — Pinata accepted
/// the bytes — but without a durable row the pair must not be returned,
/// or the reconcile counts a fill that never happened and the push path
/// advertises an unresolvable CID.
#[sqlx::test]
async fn pinata_post_upload_elapsed_record_without_row_suppresses_push(pool: sqlx::PgPool) {
let db = crate::db::Db::for_testing(pool.clone());
db.run_migrations().await.expect("migrations");
let tmp = tempfile::TempDir::new().unwrap();
let repo_path = tmp.path().join("pinata_elapsed_verify.git");
let oids = seed_loose_blobs(&repo_path, 1);
let sha = oids[0].clone();
// A real repo row: the fenced record's `FOR UPDATE` must have a row
// to block on. Against a missing row the predicate takes no lock and
// the record would sail through instead of stalling.
let repo_id = "repo-pinata-elapsed-verify";
let now = chrono::Utc::now();
db.create_repo(&crate::db::RepoRecord {
id: repo_id.to_string(),
name: "elapsed-verify".into(),
owner_did: "did:key:zElapsedVerifyOwner".into(),
description: None,
is_public: true,
default_branch: "main".into(),
created_at: now,
updated_at: now,
disk_path: repo_path.display().to_string(),
forked_from: None,
machine_id: None,
})
.await
.unwrap();
// Capture the fence BEFORE locking: the capture itself is a plain
// epoch read and must succeed for the fenced record path to run.
let fence = crate::ipfs_pin::PolicyFence::capture(&db, repo_id)
.await
.expect("fence captures");
let mut server = mockito::Server::new_async().await;
let upload = server
.mock("POST", mockito::Matcher::Any)
.with_status(200)
.with_body(r#"{"data":{"cid":"QmElapsedVerifyProviderCid"}}"#)
.expect(1)
.create_async()
.await;
let (_logs, _log_guard) = capture_logs();
// Stall only the fenced record's `FOR UPDATE`: fence reads and the
// skip check are plain SELECTs and proceed, the upload lands, then
// the record elapses. The verify read proceeds too and finds no row.
let mut lock = hold_repo_row_lock(&pool, repo_id).await;
let client = reqwest::Client::new();
let pinned = tokio::time::timeout(
Duration::from_secs(30),
pin_new_objects(
&client,
&server.url(),
"test-jwt",
&repo_path,
"git",
Duration::from_secs(30),
oids,
&db,
repo_id,
Duration::from_millis(1500),
Some(&fence),
),
)
.await
.expect(
"record + verify ladders are each floored, so the call returns in seconds, \
never at the lock's lifetime",
);
rollback(&mut lock).await;
drop(lock);
upload.assert_async().await;
assert!(
pinned.confirmed.is_empty(),
"an unverified timed-out record must not return the pair: {pinned:?}"
);
assert!(
!db.has_pinata_cid(&sha).await.unwrap(),
"the cancelled record transaction must not have landed a row"
);
}
#[tokio::test] #[tokio::test]
async fn test_pin_skipped_when_jwt_empty() { async fn test_pin_skipped_when_jwt_empty() {
let client = reqwest::Client::new(); let client = reqwest::Client::new();
+67 -10
View File
@@ -70,6 +70,14 @@ impl RateLimiter {
self.window.min(Duration::from_secs(1)) self.window.min(Duration::from_secs(1))
} }
/// The configured window, exposed so a 429 can carry an honest
/// `retry-after`: a hardcoded value lies when the operator retunes the
/// window, and a client that honors the header would retry into another
/// 429 until the real window elapses.
pub fn window(&self) -> Duration {
self.window
}
pub(crate) async fn check(&self, key: &str) -> bool { pub(crate) async fn check(&self, key: &str) -> bool {
// max_requests == 0 means the limiter is disabled, not "block all". // max_requests == 0 means the limiter is disabled, not "block all".
if self.max_requests == 0 { if self.max_requests == 0 {
@@ -293,12 +301,7 @@ pub async fn rate_limit_by_did(request: Request, next: Next) -> Response {
if let (Some(limiter), Some(did)) = (limiter, did) { if let (Some(limiter), Some(did)) = (limiter, did) {
if !limiter.check(&did).await { if !limiter.check(&did).await {
return ( return too_many_requests(limiter.window().as_secs());
StatusCode::TOO_MANY_REQUESTS,
[("retry-after", "60")],
"rate limit exceeded — try again later",
)
.into_response();
} }
} }
@@ -401,18 +404,72 @@ impl<S: Send + Sync> axum::extract::FromRequestParts<S> for PeerAddr {
} }
} }
/// The message every per-IP brake answers with, shared so the GraphQL surface
/// (which cannot return a 429 status inside a 200 GraphQL envelope) says the
/// same thing as the REST routes.
pub const RATE_LIMIT_MESSAGE: &str = "rate limit exceeded — try again later";
/// The shared 429 response for the per-IP flood brakes. Route-agnostic: this /// The shared 429 response for the per-IP flood brakes. Route-agnostic: this
/// middleware now serves the push path AND the peer-sync routes, so the message /// middleware now serves the push path AND the peer-sync routes, so the message
/// stays generic (the offending path is recorded in the warn log below). /// stays generic (the offending path is recorded in the warn log below).
pub fn too_many_requests() -> Response { pub fn too_many_requests(retry_after_secs: u64) -> Response {
( (
StatusCode::TOO_MANY_REQUESTS, StatusCode::TOO_MANY_REQUESTS,
[("retry-after", "60")], [("retry-after", retry_after_secs.to_string())],
"rate limit exceeded — try again later", RATE_LIMIT_MESSAGE,
) )
.into_response() .into_response()
} }
/// The per-IP task-read brake carried as GraphQL request data: the limiter plus
/// the client key the transport already resolved.
///
/// `/graphql` is a single POST endpoint, so this brake cannot sit in middleware
/// the way `task_read_routes` mounts `rate_limit_by_ip`: that would charge every
/// unrelated query and every mutation against the task-read bucket. The two task
/// resolvers debit it instead, which also prices an aliased query honestly — ten
/// aliased `tasks` fields run the visibility gate ten times and pay ten slots,
/// where one request-scoped debit would pay one (#327 review).
/// Default cap on the number of task read fields (across all aliases) permitted
/// in a single GraphQL request before rejecting further field executions (#327 review).
pub const MAX_GRAPHQL_TASK_READS_PER_REQUEST: usize = 5;
#[derive(Clone)]
pub struct TaskReadBrake {
pub limiter: RateLimiter,
/// `None` when no key could be resolved at all (no trusted header and no
/// `ConnectInfo`, e.g. a synthetic test request). Never braked, matching
/// [`rate_limit_by_ip`].
pub key: Option<String>,
pub request_count: Arc<std::sync::atomic::AtomicUsize>,
}
impl TaskReadBrake {
/// Debit one slot for one task-read field. `true` = proceed.
pub async fn check(&self, field: &str) -> bool {
let Some(key) = self.key.as_deref() else {
return true;
};
let count = self
.request_count
.fetch_add(1, std::sync::atomic::Ordering::Relaxed);
if count >= MAX_GRAPHQL_TASK_READS_PER_REQUEST {
tracing::warn!(
key = %key,
field = %field,
count = count + 1,
"per-request GraphQL task field limit exceeded"
);
return false;
}
if self.limiter.check(key).await {
return true;
}
tracing::warn!(key = %key, field = %field, "per-IP rate limit exceeded");
false
}
}
/// Throttle the git push path by resolved client IP. The socket peer address is /// Throttle the git push path by resolved client IP. The socket peer address is
/// read from `ConnectInfo` (see `into_make_service_with_connect_info` in /// read from `ConnectInfo` (see `into_make_service_with_connect_info` in
/// `main`). Only skips the limiter when no key can be resolved at all. /// `main`). Only skips the limiter when no key can be resolved at all.
@@ -427,7 +484,7 @@ pub async fn rate_limit_by_ip(request: Request, next: Next) -> Response {
if let Some(key) = client_key(request.headers(), peer, limiter.trust) { if let Some(key) = client_key(request.headers(), peer, limiter.trust) {
if !limiter.limiter.check(&key).await { if !limiter.limiter.check(&key).await {
tracing::warn!(key = %key, path = %request.uri().path(), "per-IP rate limit exceeded"); tracing::warn!(key = %key, path = %request.uri().path(), "per-IP rate limit exceeded");
return too_many_requests(); return too_many_requests(limiter.limiter.window().as_secs());
} }
} }
} }
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
+73 -8
View File
@@ -21,6 +21,19 @@ const SCAN_TOKEN_KEY_SALT: &[u8] = b"gitlawb/hkdf-salt/ipfs-scan-token";
/// front) and needs no migration, no config, and no change to this function. /// front) and needs no migration, no config, and no change to this function.
const SCAN_TOKEN_KEY_INFO: &[u8] = b"gitlawb/ipfs-scan-token/v1"; const SCAN_TOKEN_KEY_INFO: &[u8] = b"gitlawb/ipfs-scan-token/v1";
/// HKDF salt for [`AppState::derive_webhook_secret_key`]. A constant, not a
/// secret: the confidentiality of the derived key rests on the node's private
/// seed being the input keying material, exactly as for the scan-token key.
const WEBHOOK_SECRET_KEY_SALT: &[u8] = b"gitlawb/hkdf-salt/webhook-secret";
/// HKDF `info` for [`AppState::derive_webhook_secret_key`]: the domain
/// separation from the scan-token key that shares the same input keying
/// material. Unlike the scan-token key, bumping the version is NOT a safe
/// rotation: sealed secrets persist, `open` has no old-key fallback, and a new
/// version renders every stored ciphertext permanently unopenable until a
/// re-seal path exists.
const WEBHOOK_SECRET_KEY_INFO: &[u8] = b"gitlawb/webhook-secret/v1";
#[derive(Clone, Debug)] #[derive(Clone, Debug)]
pub struct RefUpdateBroadcast { pub struct RefUpdateBroadcast {
pub repo: String, pub repo: String,
@@ -59,6 +72,10 @@ pub struct AppState {
pub task_event_tx: tokio::sync::broadcast::Sender<TaskEventBroadcast>, pub task_event_tx: tokio::sync::broadcast::Sender<TaskEventBroadcast>,
/// GraphQL schema (queries + mutations + subscriptions) /// GraphQL schema (queries + mutations + subscriptions)
pub graphql_schema: Arc<crate::graphql::GitlawbSchema>, pub graphql_schema: Arc<crate::graphql::GitlawbSchema>,
/// MAC key for task-list continuation tokens, derived from the node
/// keypair seed. Held here (and handed to the GraphQL schema) so REST and
/// GraphQL mint and accept the same tokens.
pub task_cursor_key: crate::api::task_cursor::TaskCursorKey,
/// Fly.io machine ID — used for fly-replay routing in multi-machine deployments /// Fly.io machine ID — used for fly-replay routing in multi-machine deployments
pub machine_id: Option<String>, pub machine_id: Option<String>,
/// Centralized repo storage: local disk cache + optional Tigris backend /// Centralized repo storage: local disk cache + optional Tigris backend
@@ -169,6 +186,11 @@ pub struct AppState {
/// already has on disk, so derivation is also the smaller mechanism. Rotation is a /// already has on disk, so derivation is also the smaller mechanism. Rotation is a
/// bump of the version in [`SCAN_TOKEN_KEY_INFO`]. /// bump of the version in [`SCAN_TOKEN_KEY_INFO`].
pub ipfs_scan_token_key: Arc<[u8; 32]>, pub ipfs_scan_token_key: Arc<[u8; 32]>,
/// Key sealing webhook HMAC secrets at rest (#506), derived from the node's
/// persistent identity by [`AppState::derive_webhook_secret_key`]. The
/// webhook delivery path opens a secret with it only at signing time; the
/// `webhooks.secret` column stores the ciphertext (see `webhook_secret`).
pub webhook_secret_key: Arc<[u8; 32]>,
/// Hard ceiling on the byte size of an object `GET /ipfs/{cid}` will buffer and /// Hard ceiling on the byte size of an object `GET /ipfs/{cid}` will buffer and
/// serve (default `api::ipfs::MAX_SERVED_OBJECT_BYTES`). The serve reads via a /// serve (default `api::ipfs::MAX_SERVED_OBJECT_BYTES`). The serve reads via a
/// blocking `git cat-file` and buffers the whole object; without a bound a large /// blocking `git cat-file` and buffers the whole object; without a bound a large
@@ -190,6 +212,9 @@ pub struct AppState {
/// sink as trigger and accepts unsigned requests from known peers, so it is /// sink as trigger and accepts unsigned requests from known peers, so it is
/// braked too; each peer's distinct IP gets its own bucket. /// braked too; each peer's distinct IP gets its own bucket.
pub peer_write_rate_limiter: RateLimiter, pub peer_write_rate_limiter: RateLimiter,
/// Per-client-IP federation budget: 12 requests per minute, with at most
/// 10,000 tracked source keys. Shared by router instances and swept below.
pub federated_rate_limiter: RateLimiter,
/// Process-wide graceful-shutdown signal. Sending `true` causes every /// Process-wide graceful-shutdown signal. Sending `true` causes every
/// task that holds a `watch::Receiver` to exit at its next await point. /// task that holds a `watch::Receiver` to exit at its next await point.
/// Used by: /// Used by:
@@ -323,6 +348,16 @@ pub struct AppState {
/// (`with_default_max_keys`, reject-before-insert) so a source-key farm cannot grow /// (`with_default_max_keys`, reject-before-insert) so a source-key farm cannot grow
/// it (INV-15). /// it (INV-15).
pub git_ipfs_walk_per_caller: crate::rate_limit::PerCallerConcurrency, pub git_ipfs_walk_per_caller: crate::rate_limit::PerCallerConcurrency,
/// Per-client-IP rate limiter for the task read routes. `GET /api/v1/tasks`
/// and `GET /api/v1/tasks/{id}` are anonymously reachable and run the #268
/// visibility gate (a task lookup plus deduped-repo and visibility-rule
/// queries) before returning the opaque 404, so an unauthenticated prober
/// costs the node real work per request whether or not anything is visible.
/// Keyed on the resolved client IP via `push_limiter_trust`. Layered on
/// `task_read_routes` via `rate_limit_by_ip`.
/// The same per-IP hourly budget applies to GraphQL `tasks` / `task`
/// queries and WebSocket task queries via `TaskReadBrake`.
pub task_read_rate_limiter: RateLimiter,
/// The `git` executable the served-git withheld-blob walk spawns. Production is /// The `git` executable the served-git withheld-blob walk spawns. Production is
/// `"git"` (resolved via PATH); injectable so a fake `git` can drive the walk's /// `"git"` (resolved via PATH); injectable so a fake `git` can drive the walk's
/// process-group teardown in handler tests without mutating the process-global /// process-group teardown in handler tests without mutating the process-global
@@ -351,6 +386,8 @@ impl AppState {
self.ipfs_work_rate_limiter.cleanup().await; self.ipfs_work_rate_limiter.cleanup().await;
self.sync_trigger_rate_limiter.cleanup().await; self.sync_trigger_rate_limiter.cleanup().await;
self.peer_write_rate_limiter.cleanup().await; self.peer_write_rate_limiter.cleanup().await;
self.task_read_rate_limiter.cleanup().await;
self.federated_rate_limiter.cleanup().await;
} }
/// Trigger graceful shutdown. Idempotent — calling more than once /// Trigger graceful shutdown. Idempotent — calling more than once
@@ -412,20 +449,48 @@ impl AppState {
/// identity, the token format, or any caller, so rotation is a constant change /// identity, the token format, or any caller, so rotation is a constant change
/// rather than a fork of this derivation. /// rather than a fork of this derivation.
pub(crate) fn derive_scan_token_key(keypair: &Keypair) -> [u8; 32] { pub(crate) fn derive_scan_token_key(keypair: &Keypair) -> [u8; 32] {
Self::hkdf_sha256(
SCAN_TOKEN_KEY_SALT,
SCAN_TOKEN_KEY_INFO,
keypair.to_seed().as_slice(),
)
}
/// Derive the at-rest webhook-secret sealing key from the node identity.
///
/// Same HKDF construction as [`AppState::derive_scan_token_key`] under its
/// own [`WEBHOOK_SECRET_KEY_INFO`]/[`WEBHOOK_SECRET_KEY_SALT`] pair, so it
/// is derived (a restart re-derives the same key, and old secrets remain
/// readable) yet domain-separated from the signing key and the scan-token
/// key. The doc on [`SCAN_TOKEN_KEY_INFO`] argues why derivation (not a
/// boot-random key) is the right trade: the secret's confidentiality still
/// rests on the node's private seed. Rotation is NOT a version bump here,
/// unlike for the scan-token key: [`WEBHOOK_SECRET_KEY_INFO`] explains that
/// bumping renders every stored ciphertext unopenable until a re-seal path
/// exists.
pub(crate) fn derive_webhook_secret_key(keypair: &Keypair) -> [u8; 32] {
Self::hkdf_sha256(
WEBHOOK_SECRET_KEY_SALT,
WEBHOOK_SECRET_KEY_INFO,
keypair.to_seed().as_slice(),
)
}
/// HKDF-Extract-then-Expand (RFC 5869) over the node seed: PRK =
/// HMAC(salt, ikm), then one expand block T(1) = HMAC(PRK, info || 0x01).
/// One 32-byte output needs exactly one block of SHA-256, so there is no
/// counter loop to get wrong. Salt and info are the caller's domain
/// separation; shared by every identity-derived key in this module.
fn hkdf_sha256(salt: &[u8], info: &[u8], ikm: &[u8]) -> [u8; 32] {
use hmac::{Hmac, Mac}; use hmac::{Hmac, Mac};
type HmacSha256 = Hmac<sha2::Sha256>; type HmacSha256 = Hmac<sha2::Sha256>;
let seed = keypair.to_seed(); let mut extract = HmacSha256::new_from_slice(salt).expect("HMAC takes a key of any size");
// HKDF-Extract: PRK = HMAC(salt, ikm). extract.update(ikm);
let mut extract =
HmacSha256::new_from_slice(SCAN_TOKEN_KEY_SALT).expect("HMAC takes a key of any size");
extract.update(seed.as_slice());
let prk = extract.finalize().into_bytes(); let prk = extract.finalize().into_bytes();
// HKDF-Expand, one block: T(1) = HMAC(PRK, info || 0x01). One 32-byte output
// needs exactly one block of SHA-256, so there is no counter loop to get wrong.
let mut expand = let mut expand =
HmacSha256::new_from_slice(prk.as_slice()).expect("HMAC takes a key of any size"); HmacSha256::new_from_slice(prk.as_slice()).expect("HMAC takes a key of any size");
expand.update(SCAN_TOKEN_KEY_INFO); expand.update(info);
expand.update(&[0x01]); expand.update(&[0x01]);
expand.finalize().into_bytes().into() expand.finalize().into_bytes().into()
} }
File diff suppressed because it is too large. Load diff
+78
View File
@@ -0,0 +1,78 @@
//! Native Git fixtures shared by timing tests on Unix and Windows.
use std::ops::Deref;
use std::path::{Path, PathBuf};
use std::sync::OnceLock;
/// The fixture delays or hangs in its own process, so the production watchdog
/// can terminate it without depending on a shell or descendant process support.
#[allow(dead_code)]
pub(super) enum Behavior<'a> {
Delay(u64),
Hang,
HangOid(&'a str),
HangRepo(&'a str),
LogTypes(&'a Path),
}
#[allow(dead_code)]
pub(super) struct GitShim {
_directory: tempfile::TempDir,
executable: PathBuf,
}
impl Deref for GitShim {
type Target = Path;
fn deref(&self) -> &Path {
&self.executable
}
}
#[allow(dead_code)]
pub(super) fn create(name: &str, behavior: Behavior<'_>) -> GitShim {
static COMPILED: OnceLock<tempfile::TempDir> = OnceLock::new();
let compiled = COMPILED.get_or_init(|| {
let directory = tempfile::tempdir().expect("native Git fixture directory");
let source = directory.path().join("git_shim.rs");
std::fs::write(&source, include_str!("../tests/fixtures/git_shim.rs"))
.expect("write native Git fixture source");
let executable = directory
.path()
.join(format!("git-shim{}", std::env::consts::EXE_SUFFIX));
let output =
std::process::Command::new(std::env::var_os("RUSTC").unwrap_or_else(|| "rustc".into()))
.arg("--edition=2021")
.arg("-Dwarnings")
.arg(&source)
.arg("-o")
.arg(executable)
.output()
.expect("compile native Git fixture with the installed Rust toolchain");
assert!(
output.status.success(),
"{}",
String::from_utf8_lossy(&output.stderr)
);
directory
});
let directory = tempfile::Builder::new()
.prefix(name)
.tempdir()
.expect("Git fixture instance");
let file_name = format!("git-shim{}", std::env::consts::EXE_SUFFIX);
let executable = directory.path().join(&file_name);
std::fs::copy(compiled.path().join(file_name), &executable).expect("copy native Git fixture");
let config = match behavior {
Behavior::Delay(milliseconds) => format!("delay\n{milliseconds}"),
Behavior::Hang => "hang\n".to_owned(),
Behavior::HangOid(oid) => format!("hang-oid\n{oid}"),
Behavior::HangRepo(repo) => format!("hang-repo\n{repo}"),
Behavior::LogTypes(path) => format!("log-types\n{}", path.display()),
};
std::fs::write(directory.path().join("config"), config).expect("configure native Git fixture");
GitShim {
_directory: directory,
executable,
}
}
+602
View File
@@ -0,0 +1,602 @@
//! Test-only spawn surface for the real-node deny harness (see
//! `tests/deny_harness.rs`). Feature-gated behind `test-harness` so it never
//! compiles into the production binary. It exposes a single boot constructor so
//! the out-of-crate integration test can bring up a real node over a bound TCP
//! socket without the integration crate needing access to `graphql`,
//! `rate_limit`, `repo_store`, or the other internals `AppState` is built from.
//!
//! The node is built the same way `test_support::build_state` builds it (p2p
//! disabled, real migrated pool), but bound on `127.0.0.1:0` and served through
//! the real `axum::serve` stack with connect-info, so the middleware order,
//! body limits, and per-IP rate limiters all run exactly as in production.
use std::net::SocketAddr;
use std::path::PathBuf;
use std::sync::atomic::{AtomicU64, Ordering};
use std::sync::Arc;
use std::time::Duration;
use chrono::Utc;
use clap::Parser;
use sqlx::PgPool;
use tokio::net::TcpListener;
use tokio::sync::watch;
use uuid::Uuid;
use gitlawb_core::identity::Keypair;
use crate::config::Config;
use crate::db::{Db, PullRequest, RepoRecord, VisibilityMode};
use crate::rate_limit::{RateLimiter, TrustedProxy};
use crate::state::AppState;
/// A running node bound to an ephemeral port. End every test with
/// [`TestNode::shutdown`]: it joins the serve task and closes the pool, so
/// `#[sqlx::test]`'s `DROP DATABASE` cleanup (which runs the moment the test
/// future returns) never races the server's still-open connections. Dropping
/// without it signals shutdown, aborts any remaining serve task, and removes
/// the temp repository directory, a fallback for tests that return or panic
/// before reaching teardown.
pub struct TestNode {
/// Base URL of the running node, e.g. `http://127.0.0.1:54321`.
pub base_url: String,
/// The node's own DID (for building requests that reference it).
pub node_did: String,
shutdown_tx: watch::Sender<bool>,
repos_dir: PathBuf,
/// Seeding handle over the same pool the node serves from. Kept private so
/// the integration crate seeds through the methods below rather than
/// naming `Db`/`RepoRecord` directly.
db: Arc<Db>,
/// The detached `axum::serve` task, joined by [`Self::shutdown`]. `None`
/// once taken, so the `Drop` fallback never overlaps a completed teardown.
/// Carries `serve`'s own `io::Result<()>` so `shutdown()` can observe it
/// rather than discard it (see the coverage note there).
server: Option<tokio::task::JoinHandle<std::io::Result<()>>>,
/// A clone of the served pool, kept so [`Self::shutdown`] can close it
/// (sqlx pool clones share one inner, so closing this closes the clones
/// inside `AppState`/`Db` too).
pool: PgPool,
/// Test-only: when set, `Drop` skips the graceful watch signal so ONLY
/// `handle.abort()` tears the serve task down. Defaults false, so every
/// other test keeps both redundant teardown mechanisms. Set via
/// [`Self::force_abort_only_teardown`] to fence that the abort alone
/// releases the database when the graceful chain is broken.
suppress_graceful_shutdown_on_drop: bool,
}
impl Drop for TestNode {
fn drop(&mut self) {
// Fallback for tests that return or panic before reaching
// [`TestNode::shutdown`]: flip the shared shutdown signal so the serve
// task exits gracefully, abort whatever is left of it, and remove the
// temp repos dir. Drop cannot await, so it cannot join the server or
// close the pool; a test that already failed should not panic again in
// teardown. All three actions are idempotent or no-ops after
// `shutdown()` (which takes the handle), so running afterwards is
// harmless.
//
// The abort matters when the graceful chain does not run to completion
// before sqlx's `DROP DATABASE` cleanup: `JoinHandle::abort()` is a
// synchronous cancellation request callable from non-async Drop, and
// the aborted task's future (with the `PgPool` clones inside its
// router/state) is dropped on a subsequent scheduler tick, which sqlx
// cleanup's own awaits provide before the drop statement runs. The
// race is closed by those scheduling points, not by the abort alone.
//
// `suppress_graceful_shutdown_on_drop` (test-only) forces the abort to
// carry teardown by itself, so a test can fence that the abort line is
// load-bearing; it is false everywhere else.
if !self.suppress_graceful_shutdown_on_drop {
let _ = self.shutdown_tx.send(true);
}
if let Some(handle) = self.server.take() {
handle.abort();
}
let _ = std::fs::remove_dir_all(&self.repos_dir);
}
}
/// Outcome of joining the serve task during teardown, split so the wedged-task
/// path is observable to a test (which injects a short timeout) without waiting
/// the production teardown window.
enum ServeTeardown {
/// The task returned (carrying `serve`'s `io::Result<()>`) — either within
/// the timeout, or in the race between the timeout firing and the abort
/// landing. A task that actually finished is never a wedge.
Graceful(std::io::Result<()>),
/// The task did not finish within the timeout and was aborted (cancelled)
/// and reaped.
TimedOutAborted,
/// The serve task panicked. The payload is carried out so the caller can
/// resume unwinding only AFTER the pool is closed and the temp dir removed.
Panicked(Box<dyn std::any::Any + Send>),
}
/// Join `handle` within `timeout`; on expiry `abort()` it and await the aborted
/// handle so the serve task (and the `PgPool` clones inside its router/state) is
/// reaped BEFORE the caller returns and `#[sqlx::test]` runs `DROP DATABASE`.
/// Takes `&mut` (a `JoinHandle` is `Unpin`) so ownership is retained across the
/// timeout and the handle can be aborted on expiry — the previous code moved the
/// handle into `timeout` and, on elapse, dropped it, which DETACHES the still
/// running task rather than aborting it, leaking the test database (#194 F2).
async fn join_or_abort(
handle: &mut tokio::task::JoinHandle<std::io::Result<()>>,
timeout: Duration,
) -> ServeTeardown {
match tokio::time::timeout(timeout, &mut *handle).await {
// A panic in the serve task is carried, not unwound here: the caller
// must close the pool and remove the repos dir before resuming it.
Ok(join_result) => match join_result {
Ok(result) => ServeTeardown::Graceful(result),
Err(e) => ServeTeardown::Panicked(e.into_panic()),
},
Err(_elapsed) => {
handle.abort();
match handle.await {
// The task finished in the race between the timeout firing and
// the abort landing — honor its result, not a false wedge.
Ok(result) => ServeTeardown::Graceful(result),
Err(e) if e.is_cancelled() => ServeTeardown::TimedOutAborted,
Err(e) => ServeTeardown::Panicked(e.into_panic()),
}
}
}
}
/// Allocate a process-unique temp directory for a spawned node's repositories
/// without pulling in the `tempfile` dev-dependency (which is unavailable to
/// the library crate under `--features test-harness`).
fn unique_repos_dir() -> PathBuf {
static COUNTER: AtomicU64 = AtomicU64::new(0);
let n = COUNTER.fetch_add(1, Ordering::Relaxed);
std::env::temp_dir().join(format!("gitlawb-deny-harness-{}-{}", std::process::id(), n))
}
/// Build an [`AppState`] over the given migrated pool, mirroring
/// `test_support::build_state` but with a real on-disk repository directory so
/// git smart-HTTP routes can serve. P2P is always disabled.
fn build_state(db: Arc<crate::db::Db>, pool: PgPool, repos_dir: PathBuf) -> AppState {
let keypair = Keypair::generate();
let scan_token_key = crate::state::AppState::derive_scan_token_key(&keypair);
let webhook_secret_key = crate::state::AppState::derive_webhook_secret_key(&keypair);
let task_cursor_key = crate::api::task_cursor::TaskCursorKey::derive(&keypair.to_seed());
let node_did = keypair.did();
let (ref_tx, _) = tokio::sync::broadcast::channel(1);
let (task_tx, _) = tokio::sync::broadcast::channel(1);
let schema = Arc::new(crate::graphql::build_schema(
db.clone(),
ref_tx.clone(),
task_tx.clone(),
task_cursor_key.clone(),
));
AppState {
config: Arc::new(Config::parse_from(["gitlawb-node"])),
db,
node_did,
node_keypair: Arc::new(keypair),
p2p: None,
http_client: Arc::new(reqwest::Client::new()),
ref_update_tx: ref_tx,
task_event_tx: task_tx,
graphql_schema: schema,
task_cursor_key,
machine_id: None,
repo_store: crate::git::repo_store::RepoStore::for_testing(repos_dir, pool),
rate_limiter: RateLimiter::new(100, Duration::from_secs(60)),
create_ip_rate_limiter: RateLimiter::new(1000, Duration::from_secs(3600)),
push_rate_limiter: RateLimiter::new(600, Duration::from_secs(3600)),
close_issue_rate_limiter: RateLimiter::new(600, Duration::from_secs(3600)),
federated_rate_limiter: RateLimiter::new(12, Duration::from_secs(60)),
task_read_rate_limiter: RateLimiter::new(1200, Duration::from_secs(3600)),
ipfs_rate_limiter: RateLimiter::new(600, Duration::from_secs(3600)),
ipfs_work_rate_limiter: RateLimiter::new(600, Duration::from_secs(3600)),
ipfs_max_history_walks: crate::api::ipfs::MAX_HISTORY_WALKS_PER_REQUEST,
ipfs_max_legacy_probes: crate::api::ipfs::MAX_LEGACY_PROBES_PER_REQUEST,
ipfs_legacy_scan_page_rows: crate::api::ipfs::LEGACY_SCAN_PAGE_ROWS,
ipfs_max_legacy_scan_rows: crate::api::ipfs::MAX_LEGACY_SCAN_ROWS_PER_REQUEST,
ipfs_max_legacy_scan_rule_bytes: crate::api::ipfs::MAX_LEGACY_SCAN_RULE_BYTES_PER_REQUEST,
ipfs_scan_token_key: Arc::new(scan_token_key),
webhook_secret_key: Arc::new(webhook_secret_key),
ipfs_max_served_object_bytes: crate::api::ipfs::MAX_SERVED_OBJECT_BYTES,
push_limiter_trust: TrustedProxy::None,
sync_trigger_rate_limiter: RateLimiter::new(60, Duration::from_secs(3600)),
peer_write_rate_limiter: RateLimiter::new(600, Duration::from_secs(3600)),
shutdown_tx: watch::channel(false).0,
git_read_semaphore: Arc::new(tokio::sync::Semaphore::new(64)),
git_write_semaphore: Arc::new(tokio::sync::Semaphore::new(64)),
git_push_advert_semaphore: Arc::new(tokio::sync::Semaphore::new(64)),
git_encrypt_semaphore: Arc::new(tokio::sync::Semaphore::new(64)),
pin_semaphore: Arc::new(tokio::sync::Semaphore::new(64)),
encrypt_inflight: crate::state::EncryptInflight::new(),
repo_write_leases: crate::state::RepoWriteLeases::new(8),
git_read_per_caller: crate::rate_limit::PerCallerConcurrency::with_default_max_keys(16),
git_push_advert_per_caller: crate::rate_limit::PerCallerConcurrency::with_default_max_keys(
8,
),
git_write_per_caller: crate::rate_limit::PerCallerConcurrency::with_default_max_keys(8),
git_ipfs_walk_semaphore: Arc::new(tokio::sync::Semaphore::new(64)),
git_ipfs_walk_per_caller: crate::rate_limit::PerCallerConcurrency::with_default_max_keys(
16,
),
git_bin: "git".to_string(),
}
}
/// Spawn a real node bound to `127.0.0.1:0` over the given (already-created,
/// empty) test pool. Runs the schema migrations, builds the router, and serves
/// it on a background task through the production `axum::serve` stack with
/// connect-info so per-IP layers key on the real peer. Returns once the socket
/// is bound and accepting connections.
pub async fn spawn_node(pool: PgPool) -> TestNode {
let db = Arc::new(crate::db::Db::for_testing(pool.clone()));
db.run_migrations()
.await
.expect("test schema migrations should apply");
let repos_dir = unique_repos_dir();
std::fs::create_dir_all(&repos_dir).expect("create temp repos dir");
let state = build_state(db.clone(), pool.clone(), repos_dir.clone());
let node_did = state.node_did.to_string();
let shutdown_tx = state.shutdown_tx.clone();
let mut shutdown_rx = shutdown_tx.subscribe();
let router = crate::server::build_router(state);
let listener = TcpListener::bind(("127.0.0.1", 0))
.await
.expect("bind ephemeral port");
let addr = listener.local_addr().expect("read bound addr");
let server = tokio::spawn(async move {
axum::serve(
listener,
router.into_make_service_with_connect_info::<SocketAddr>(),
)
.with_graceful_shutdown(async move {
let _ = shutdown_rx.changed().await;
})
.await
});
TestNode {
base_url: format!("http://{addr}"),
node_did,
shutdown_tx,
repos_dir,
db,
server: Some(server),
pool,
suppress_graceful_shutdown_on_drop: false,
}
}
impl TestNode {
/// Test-only: make `Drop` tear the serve task down through `handle.abort()`
/// alone by suppressing the graceful watch signal. Used to prove the abort
/// is load-bearing (that the database is released even when the graceful
/// chain is broken); no production or normal-teardown path calls this.
pub fn force_abort_only_teardown(&mut self) {
self.suppress_graceful_shutdown_on_drop = true;
}
/// Graceful async teardown: signal shutdown, join the serve task, close
/// the pool, remove the temp repos dir. Call this at the end of every
/// test. `#[sqlx::test]` issues `DROP DATABASE` as soon as the test
/// future returns, so the server (which owns pool clones inside
/// `AppState`/`Db`) must be gone and the pool closed before then, or the
/// cleanup fails and leaks per-test databases. A serve task that does not
/// exit within 10s panics loudly: a wedged graceful shutdown must fail
/// the test, not hang CI.
///
/// When `self` drops on return, `Drop` re-sends the signal and re-removes
/// the dir; both are idempotent, and the serve handle was already taken,
/// so the overlap is harmless.
pub async fn shutdown(self) {
self.shutdown_with_timeout(Duration::from_secs(10)).await;
}
/// Body of [`Self::shutdown`] with an injectable teardown timeout so a test
/// can drive the wedged-shutdown path without waiting the production 10s.
/// Production always calls this with 10s.
async fn shutdown_with_timeout(mut self, timeout: Duration) {
let _ = self.shutdown_tx.send(true);
let outcome = match self.server.take() {
Some(mut handle) => Some(join_or_abort(&mut handle, timeout).await),
None => None,
};
// Release the pool and temp dir on EVERY path before returning OR
// panicking. A wedged graceful shutdown is exactly when in-flight
// per-connection tasks still hold `PgPool` clones; aborting the serve
// future alone does not reliably drop them, so the explicit
// `pool.close()` (which closes the shared inner) is what frees them
// before `#[sqlx::test]` runs `DROP DATABASE`. Panicking before this
// (the previous behavior) skipped the close on the one path that needs
// it most (#194 F2).
self.pool.close().await;
let _ = std::fs::remove_dir_all(&self.repos_dir);
match outcome {
// Honest coverage contract: on axum 0.8.8,
// `WithGracefulShutdown::into_future` always resolves `Ok(())` (the
// shutdown signal is treated as success and `accept()` errors are
// retried forever rather than returned), so this `Err` arm is
// unreachable today - every existing harness test exercises only the
// `Ok` path here. Forward-compat insurance for a future axum where
// `serve` can fail: that should surface as a clear panic, not be
// silently discarded.
Some(ServeTeardown::Graceful(result)) => {
result.expect("axum::serve exited with an error");
}
Some(ServeTeardown::TimedOutAborted) => {
panic!(
"serve task did not exit within {timeout:?} of the shutdown \
signal (wedged graceful shutdown)"
);
}
Some(ServeTeardown::Panicked(payload)) => std::panic::resume_unwind(payload),
None => {}
}
}
/// Insert a repo owned by `owner_did` and return its repo id. Mirrors
/// `test_support::seed_repo` (the `disk_path` field is unused by the
/// integration path — `RepoStore` computes the on-disk path from
/// `repos_dir`/owner/name, see [`Self::seed_bare_repo`]).
pub async fn seed_repo(&self, owner_did: &str, name: &str, is_public: bool) -> String {
let now = Utc::now();
let record = RepoRecord {
id: Uuid::new_v4().to_string(),
name: name.to_string(),
owner_did: owner_did.to_string(),
description: None,
is_public,
default_branch: "main".to_string(),
created_at: now,
updated_at: now,
disk_path: format!("/tmp/{name}"),
forked_from: None,
machine_id: None,
};
self.db.create_repo(&record).await.expect("seed repo");
record.id
}
/// Mark `branch` as protected on `repo_id`, so a non-owner push to it is
/// rejected by the branch-protection gate (used by the deny-prober's
/// protected-push probe — #195, F3).
pub async fn seed_protected_branch(&self, repo_id: &str, branch: &str, created_by: &str) {
self.db
.protect_branch(repo_id, branch, created_by)
.await
.expect("seed protected branch");
}
/// Seed an open PR `number` in `repo_id`, authored by `author_did`. The
/// author-or-owner close gate (`close_pr`) loads the PR *before* it runs, so a
/// deny-prober row for `.../pulls/{number}/close` needs the PR to exist or a
/// stranger gets a 404 (absent entity) instead of the 403 the gate emits.
pub async fn seed_pr(&self, repo_id: &str, number: i64, author_did: &str) {
let now = Utc::now().to_rfc3339();
let pr = PullRequest {
id: Uuid::new_v4().to_string(),
repo_id: repo_id.to_string(),
number,
title: "seed pr".to_string(),
body: None,
author_did: author_did.to_string(),
source_branch: "feature".to_string(),
target_branch: "main".to_string(),
status: "open".to_string(),
merged_by_did: None,
merged_at: None,
head_commit: None,
created_at: now.clone(),
updated_at: now,
};
self.db.create_pr(&pr).await.expect("seed pr");
}
/// Seed issue `issue_id` (a git-ref JSON blob authored by `author_did`) in the
/// repo's bare on-disk store. Like [`Self::seed_pr`], the `close_issue`
/// owner-or-author gate loads the issue before it runs, so the deny-prober row
/// for `.../issues/{id}/close` needs the issue present to reach the 403.
pub fn seed_issue(&self, owner_did: &str, name: &str, issue_id: &str, author_did: &str) {
let slug = owner_did.replace([':', '/'], "_");
let bare = self.repos_dir.join(&slug).join(format!("{name}.git"));
let json = format!(r#"{{"author":"{author_did}"}}"#);
crate::git::issues::create_issue(&bare, issue_id, &json).expect("seed issue");
}
/// Add a path-scoped visibility rule restricting `path_glob` to
/// `reader_dids` (empty = only the owner). Mode B keeps object SHAs intact
/// and withholds blob content, which is what the read/replication gates
/// enforce.
pub async fn withhold_path(
&self,
repo_id: &str,
path_glob: &str,
reader_dids: &[String],
created_by: &str,
) {
self.db
.set_visibility_rule(
repo_id,
path_glob,
VisibilityMode::B,
reader_dids,
created_by,
)
.await
.expect("set visibility rule");
}
/// Record `pinned_cids` rows for every blob OID returned by
/// [`Self::seed_bare_repo`] so `/ipfs/{cid}` can resolve via provenance on
/// current main (the table is the primary lookup path). CIDs are computed
/// from raw object bytes exactly as the production pin path does
/// (`Cid::from_git_object_bytes`), not from the git oid hash.
/// Returns each seeded path's canonical CID for use in `GET /ipfs/{cid}`.
pub async fn seed_pinned_cids_for_repo(
&self,
owner_did: &str,
repo_name: &str,
repo_id: &str,
oids: &std::collections::HashMap<String, String>,
) -> std::collections::HashMap<String, String> {
let slug = owner_did.replace([':', '/'], "_");
let bare = self.repos_dir.join(&slug).join(format!("{repo_name}.git"));
let mut cids = std::collections::HashMap::new();
for (path, oid_hex) in oids {
if path == "HEAD" {
continue;
}
let (_ty, raw) = crate::git::store::read_object(&bare, oid_hex)
.expect("read seeded object")
.expect("seeded object exists in bare repo");
let preliminary = gitlawb_core::cid::Cid::from_git_object_bytes(&raw).to_string();
let cid = preliminary
.parse::<cid::CidGeneric<64>>()
.expect("parse production pin cid")
.to_string();
self.db
.record_pinned_cid_with_source(oid_hex, &cid, repo_id)
.await
.expect("seed pinned_cids row");
let roundtrip = self
.db
.oids_for_cid(&cid)
.await
.expect("oids_for_cid after seed");
assert!(
!roundtrip.is_empty(),
"pinned_cids seed for {path} did not round-trip cid {cid}"
);
cids.insert(path.clone(), cid);
}
cids
}
/// Create a real bare git repository on disk at the exact path
/// `RepoStore::acquire` reads (`<repos_dir>/<owner-slug>/<name>.git`), with
/// one commit on `main` containing `files` (`(path, contents)`). Returns the
/// blob OID of each path so callers can assert a withheld OID never appears
/// in served bytes (U8). Shells out to `git`, mirroring
/// `test_support`'s served-content seam.
/// `object_format` is `"sha1"` for the git smart-HTTP surfaces and
/// `"sha256"` for the content-addressed `/ipfs/{cid}` surface (whose CIDs
/// are the sha2-256 object ids).
pub fn seed_bare_repo(
&self,
owner_did: &str,
name: &str,
files: &[(&str, &str)],
object_format: &str,
) -> std::collections::HashMap<String, String> {
let run = |args: &[&str], cwd: &std::path::Path| {
// Test-harness-only fixture seeding, feature-gated out of the production
// binary; runs local git against a tempdir inside a test's own lifetime.
// allow-unbounded-git: test-only seeding, never on a request path holding a permit
let out = std::process::Command::new("git")
.args(args)
.current_dir(cwd)
.output() // allow-unbounded-git: test-harness fixture seeding only
.expect("git runs");
assert!(
out.status.success(),
"git {args:?} failed: {}",
String::from_utf8_lossy(&out.stderr)
);
String::from_utf8_lossy(&out.stdout).trim().to_string()
};
// Build a source work tree, then bare-clone it into the served path.
let src = self.repos_dir.join(format!("src-{name}"));
std::fs::create_dir_all(&src).expect("create src dir");
let fmt_arg = format!("--object-format={object_format}");
run(&["init", "-q", "-b", "main", &fmt_arg], &src);
run(&["config", "user.email", "t@t"], &src);
run(&["config", "user.name", "t"], &src);
for (path, contents) in files {
let full = src.join(path);
if let Some(parent) = full.parent() {
std::fs::create_dir_all(parent).expect("create file parent dir");
}
std::fs::write(&full, contents).expect("write seed file");
run(&["add", path], &src);
}
run(&["commit", "-q", "-m", "seed"], &src);
let mut oids = std::collections::HashMap::new();
for (path, _) in files {
let oid = run(&["rev-parse", &format!("HEAD:{path}")], &src);
oids.insert((*path).to_string(), oid);
}
// The HEAD commit oid, used as the `want` when driving upload-pack.
oids.insert("HEAD".to_string(), run(&["rev-parse", "HEAD"], &src));
let slug = owner_did.replace([':', '/'], "_");
let bare = self.repos_dir.join(&slug).join(format!("{name}.git"));
std::fs::create_dir_all(bare.parent().unwrap()).expect("create bare parent");
run(
&[
"clone",
"--bare",
"-q",
src.to_str().unwrap(),
bare.to_str().unwrap(),
],
&self.repos_dir,
);
oids
}
}
#[cfg(test)]
mod tests {
use super::{join_or_abort, ServeTeardown};
use std::time::Duration;
/// A wedged serve task (never returns within the window) must be ABORTED and
/// reaped on timeout, not detached. The previous teardown moved the
/// `JoinHandle` into `timeout` and dropped it on elapse, which detaches the
/// still-running task and leaks the test DB across `DROP DATABASE` (#194 F2).
/// Load-bearing: neutralize the abort in `join_or_abort` (return without
/// abort+await) and this flips RED — `handle.is_finished()` stays false
/// because the task was left running.
#[tokio::test]
async fn join_or_abort_aborts_a_wedged_task_on_timeout() {
let mut handle: tokio::task::JoinHandle<std::io::Result<()>> = tokio::spawn(async {
loop {
tokio::time::sleep(Duration::from_secs(3600)).await;
}
});
assert!(
matches!(
join_or_abort(&mut handle, Duration::from_millis(50)).await,
ServeTeardown::TimedOutAborted
),
"a task that never returns must be reported timed-out, not graceful"
);
assert!(
handle.is_finished(),
"the wedged serve task must be aborted and reaped, not detached"
);
}
/// A task that returns within the window is reported graceful, carrying its
/// `io::Result` — the normal teardown path.
#[tokio::test]
async fn join_or_abort_reports_graceful_when_task_exits_in_time() {
let mut handle: tokio::task::JoinHandle<std::io::Result<()>> =
tokio::spawn(async { Ok(()) });
assert!(
matches!(
join_or_abort(&mut handle, Duration::from_secs(5)).await,
ServeTeardown::Graceful(Ok(()))
),
"a prompt clean exit must be reported graceful"
);
}
}
File diff suppressed because it is too large. Load diff
+968
View File
@@ -0,0 +1,968 @@
//! Scheduled external-forge fetches for canonical INBOUND mirrors.
//!
//! This worker is deliberately separate from `sync`, which mirrors repositories
//! between Gitlawb peers. External upstream URLs are owner-controlled data that
//! eventually reach `git fetch`, so each cycle resolves DNS itself, validates
//! every address, and pins the approved answers into Git/libcurl with
//! `http.curloptResolve`. Redirects, proxies, credential helpers, submodule
//! recursion, and non-HTTPS protocols are disabled at the command boundary.
use anyhow::{Context, Result};
use std::collections::BTreeSet;
use std::net::IpAddr;
use std::path::Path;
use std::sync::Arc;
use std::time::{Duration, Instant};
use tracing::{info, warn};
use crate::config::Config;
use crate::db::{Db, InboundMirrorTarget, MirrorStatus, MirrorTransitionPhase};
use crate::git::repo_store::RepoStore;
use crate::state::{repo_identity_key, RepoWriteLeases};
const DNS_TIMEOUT: Duration = Duration::from_secs(10);
const MAX_PINNED_ADDRESSES: usize = 32;
const MIN_CURL_OPT_RESOLVE_GIT: (u64, u64) = (2, 37);
#[derive(Debug, Clone, Copy)]
struct EgressPolicy;
#[derive(Debug, Clone, PartialEq, Eq)]
struct PinnedUpstream {
url: String,
curlopt_resolve: Option<String>,
}
impl EgressPolicy {
async fn resolve(&self, url: reqwest::Url) -> Result<PinnedUpstream> {
let raw_host = url.host_str().context("mirror upstream URL has no host")?;
let connect_host = raw_host
.trim_start_matches('[')
.trim_end_matches(']')
.to_ascii_lowercase();
let policy_host = normalize_policy_host(&connect_host);
reject_always_local_hostname(&policy_host)?;
let port = url
.port_or_known_default()
.context("mirror upstream URL has no known port")?;
if let Ok(ip) = connect_host.parse::<IpAddr>() {
self.validate_addresses(&policy_host, &[ip])?;
return Ok(PinnedUpstream {
url: url.to_string(),
// An IP literal performs no DNS lookup, so there is no rebinding
// window to pin. TLS still verifies the literal from the URL.
curlopt_resolve: None,
});
}
let resolved = tokio::time::timeout(
DNS_TIMEOUT,
tokio::net::lookup_host((connect_host.as_str(), port)),
)
.await
.context("mirror upstream DNS lookup timed out")?
.context("resolving mirror upstream host")?;
let addresses: BTreeSet<IpAddr> = resolved.map(|addr| addr.ip()).collect();
if addresses.is_empty() {
anyhow::bail!("mirror upstream host resolved to no addresses");
}
let addresses: Vec<IpAddr> = addresses.into_iter().collect();
self.validate_addresses(&policy_host, &addresses)?;
let pinned = addresses
.iter()
.map(|ip| match ip {
IpAddr::V4(v4) => v4.to_string(),
IpAddr::V6(v6) => format!("[{v6}]"),
})
.collect::<Vec<_>>()
.join(",");
Ok(PinnedUpstream {
url: url.to_string(),
curlopt_resolve: Some(format!("+{connect_host}:{port}:{pinned}")),
})
}
fn validate_addresses(&self, host: &str, addresses: &[IpAddr]) -> Result<()> {
if addresses.is_empty() || addresses.len() > MAX_PINNED_ADDRESSES {
anyhow::bail!(
"mirror upstream host must resolve to 1..={MAX_PINNED_ADDRESSES} addresses"
);
}
for &ip in addresses {
if !address_is_permitted(ip) {
anyhow::bail!("mirror upstream host {host:?} resolved to disallowed address {ip}");
}
}
Ok(())
}
}
fn normalize_policy_host(host: &str) -> String {
host.trim_end_matches('.').to_ascii_lowercase()
}
fn reject_always_local_hostname(host: &str) -> Result<()> {
if host.is_empty()
|| host == "localhost"
|| host.ends_with(".localhost")
|| host.ends_with(".local")
|| host.ends_with(".internal")
{
anyhow::bail!("mirror upstream host is local-only and cannot be fetched");
}
Ok(())
}
fn address_is_permitted(ip: IpAddr) -> bool {
crate::api::peers::mirror_upstream_ip_permitted(ip)
}
fn parse_git_version(output: &str) -> Option<(u64, u64)> {
let version = output.split_whitespace().find(|part| {
part.as_bytes()
.first()
.is_some_and(|byte| byte.is_ascii_digit())
})?;
let mut parts = version.split('.');
Some((parts.next()?.parse().ok()?, parts.next()?.parse().ok()?))
}
fn ensure_safe_git_runtime(git_bin: &str) -> Result<()> {
// Bounded runner like every other git spawn on this path: --version does
// no I/O of its own, but the child still takes a permit and the deadline.
let (status, stdout, stderr) = crate::git::visibility_pack::run_bounded_git_raw_isolated_https(
git_bin,
&["--version"],
&std::env::temp_dir(),
b"",
Instant::now() + Duration::from_secs(10),
)?;
if !status.success() {
anyhow::bail!(
"{git_bin} --version failed: {}",
String::from_utf8_lossy(&stderr)
.chars()
.take(256)
.collect::<String>()
);
}
let stdout = String::from_utf8_lossy(&stdout);
let version = parse_git_version(&stdout).context("parsing git version")?;
if version < MIN_CURL_OPT_RESOLVE_GIT {
anyhow::bail!(
"upstream mirroring requires Git 2.37+ for DNS pinning; found {}.{}",
version.0,
version.1
);
}
Ok(())
}
#[derive(Debug, Clone, PartialEq, Eq)]
struct FetchCommand {
config: Vec<(String, String)>,
args: Vec<String>,
}
fn fetch_command(upstream: &PinnedUpstream) -> FetchCommand {
let http_scope = |name: &str| format!("http.{}.{name}", upstream.url);
let credential_scope = |name: &str| format!("credential.{}.{name}", upstream.url);
let mut config = vec![
("http.proxy".to_string(), "".to_string()),
("http.followRedirects".to_string(), "false".to_string()),
// Reset any inherited multi-valued resolver entries before adding the
// address set validated for this exact fetch.
("http.curloptResolve".to_string(), "".to_string()),
// Never forward operator or repository-scoped HTTP credentials to an
// owner-selected forge. Empty values reset Git's multi-valued headers
// and disable cookie state for this invocation.
("http.extraHeader".to_string(), "".to_string()),
("http.cookieFile".to_string(), "".to_string()),
("http.saveCookies".to_string(), "false".to_string()),
("http.sslVerify".to_string(), "true".to_string()),
// URL-specific configuration outranks generic `http.*` values even
// when the generic value is command-scoped. Repeat every security
// control at the exact validated URL so a repository-local scoped
// setting cannot re-enable redirects/proxies/headers/cookies or disable
// TLS. Key/value environment entries avoid `git -c` splitting a valid
// URL path that contains `=`.
(http_scope("proxy"), "".to_string()),
(http_scope("followRedirects"), "false".to_string()),
(http_scope("curloptResolve"), "".to_string()),
(http_scope("extraHeader"), "".to_string()),
(http_scope("cookieFile"), "".to_string()),
(http_scope("saveCookies"), "false".to_string()),
(http_scope("sslVerify"), "true".to_string()),
];
if let Some(resolve) = &upstream.curlopt_resolve {
config.extend([
("http.curloptResolve".to_string(), resolve.clone()),
(http_scope("curloptResolve"), resolve.clone()),
]);
}
config.extend([
("credential.helper".to_string(), "".to_string()),
(credential_scope("helper"), "".to_string()),
("credential.interactive".to_string(), "false".to_string()),
("core.askPass".to_string(), "false".to_string()),
("fetch.recurseSubmodules".to_string(), "false".to_string()),
// Do not let a smart server or repository-local config introduce a
// second, unvalidated pack/bundle URL outside the pinned upstream.
("fetch.uriprotocols".to_string(), "".to_string()),
("fetch.bundleURI".to_string(), "".to_string()),
("protocol.allow".to_string(), "never".to_string()),
("protocol.https.allow".to_string(), "always".to_string()),
]);
let args = vec![
"fetch".to_string(),
"--atomic".to_string(),
"--force".to_string(),
"--prune".to_string(),
"--prune-tags".to_string(),
"--no-auto-maintenance".to_string(),
"--no-recurse-submodules".to_string(),
"--no-write-fetch-head".to_string(),
upstream.url.clone(),
"+refs/heads/*:refs/heads/*".to_string(),
"+refs/tags/*:refs/tags/*".to_string(),
];
FetchCommand { config, args }
}
fn reject_local_url_rewrite(
git_bin: &str,
repo_path: &Path,
upstream_url: &str,
deadline: Instant,
) -> Result<()> {
let args = ["ls-remote", "--get-url", upstream_url];
let (status, stdout, stderr) = crate::git::visibility_pack::run_bounded_git_raw_isolated_https(
git_bin, &args, repo_path, b"", deadline,
)?;
if !status.success() {
let stderr = String::from_utf8_lossy(&stderr);
anyhow::bail!(
"checking mirror upstream URL rewrite failed: {}",
stderr.chars().take(4_096).collect::<String>()
);
}
let expanded = std::str::from_utf8(&stdout)
.context("Git returned a non-UTF-8 expanded mirror upstream URL")?
.trim_end_matches(['\r', '\n']);
if expanded != upstream_url {
anyhow::bail!("repository-local Git config attempted to rewrite the mirror upstream URL");
}
Ok(())
}
fn reject_sensitive_local_config(git_bin: &str, repo_path: &Path, deadline: Instant) -> Result<()> {
let args = [
"config",
"--local",
"--includes",
"--get-regexp",
r"^(http|credential|url)\.",
];
let (status, stdout, stderr) = crate::git::visibility_pack::run_bounded_git_raw_isolated_https(
git_bin, &args, repo_path, b"", deadline,
)?;
match status.code() {
Some(1) if stdout.is_empty() => Ok(()),
Some(0) => anyhow::bail!(
"repository-local network or credential Git config is not allowed for upstream mirrors"
),
_ => {
let stderr = String::from_utf8_lossy(&stderr);
anyhow::bail!(
"checking repository-local Git config failed: {}",
stderr.chars().take(4_096).collect::<String>()
)
}
}
}
/// Budget for each preflight probe inside [`run_fetch`] (the URL-rewrite
/// check and the sensitive-local-config scan). Both run local git children
/// against the on-disk repo, so a short bound is generous; keeping them on
/// their own deadlines means `upstream_mirror_fetch_timeout_secs` covers
/// only the external pack transfer it documents.
const PREFLIGHT_PROBE_TIMEOUT: Duration = Duration::from_secs(30);
fn run_fetch(
git_bin: &str,
repo_path: &Path,
upstream: &PinnedUpstream,
timeout: Duration,
) -> Result<()> {
let command = fetch_command(upstream);
let args: Vec<&str> = command.args.iter().map(String::as_str).collect();
let config: Vec<(&str, &str)> = command
.config
.iter()
.map(|(key, value)| (key.as_str(), value.as_str()))
.collect();
let rewrite_deadline = Instant::now()
.checked_add(PREFLIGHT_PROBE_TIMEOUT)
.context("mirror URL-rewrite probe timeout is not representable")?;
reject_local_url_rewrite(git_bin, repo_path, &upstream.url, rewrite_deadline)?;
let config_deadline = Instant::now()
.checked_add(PREFLIGHT_PROBE_TIMEOUT)
.context("mirror local-config probe timeout is not representable")?;
reject_sensitive_local_config(git_bin, repo_path, config_deadline)?;
let deadline = Instant::now()
.checked_add(timeout)
.context("upstream mirror fetch timeout is not representable")?;
let (status, _stdout, stderr) =
crate::git::visibility_pack::run_bounded_git_raw_isolated_https_with_config(
git_bin, &args, repo_path, b"", deadline, &config,
)?;
if !status.success() {
let stderr = String::from_utf8_lossy(&stderr);
let stderr = stderr.chars().take(4_096).collect::<String>();
anyhow::bail!("git upstream fetch failed: {stderr}");
}
Ok(())
}
#[derive(Clone)]
struct Worker {
db: Arc<Db>,
config: Arc<Config>,
repo_store: RepoStore,
repo_write_leases: RepoWriteLeases,
git_write_semaphore: Arc<tokio::sync::Semaphore>,
policy: EgressPolicy,
git_bin: String,
}
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
enum FetchOutcome {
Updated,
SkippedStale,
}
impl Worker {
async fn fetch_target(&self, target: InboundMirrorTarget) -> Result<FetchOutcome> {
let url = crate::db::validate_mirror_upstream_url(&target.upstream_url)?;
let upstream = self.policy.resolve(url).await?;
let repo_identity = repo_identity_key(&target.owner_did, &target.name);
let steal_after = Duration::from_secs(
self.config
.upstream_mirror_fetch_timeout_secs
.saturating_mul(2)
.saturating_add(60),
);
let _lease = self
.repo_write_leases
.acquire(&repo_identity, steal_after)
.await
.context("upstream mirror write-lease waiter cap reached")?;
let _write_permit = Arc::clone(&self.git_write_semaphore)
.acquire_owned()
.await
.context("upstream mirror write semaphore closed")?;
let acquire_timeout = Duration::from_secs(self.config.git_acquire_timeout_secs);
let guard = tokio::time::timeout(
acquire_timeout,
self.repo_store
.acquire_write(&target.owner_did, &target.name),
)
.await
.context("upstream mirror write-lock acquisition timed out")??;
let repo_path = guard.path().to_path_buf();
// A row can leave INBOUND state after selection. Recheck only after
// both the process-local lease and the cluster-wide advisory write lock
// are held. Every future transition path must take these same locks
// before changing authority, so no other node can commit a transition
// between this check and the fetch.
let current = match self.db.get_repo_mirror_state(&target.repo_id).await {
Ok(Some(current)) => current,
Ok(None) => {
let _ = guard.release(false).await;
anyhow::bail!("inbound mirror state disappeared before fetch");
}
Err(error) => {
let _ = guard.release(false).await;
return Err(error);
}
};
// Both stable shapes the CHECK admits: never transitioned, or a
// completed transition whose metadata is retained as audit. Only an
// in-flight transition makes the target stale.
let stable = (current.transition_id.is_none() && current.transition_phase.is_none())
|| (current.transition_id.is_some()
&& current.transition_phase == Some(MirrorTransitionPhase::Completed));
if current.status != MirrorStatus::Inbound
|| !stable
|| current.upstream_url != target.upstream_url
{
let _ = guard.release(false).await;
info!(repo_id = %target.repo_id, "upstream mirror state changed before fetch; skipping stale target");
return Ok(FetchOutcome::SkippedStale);
}
let git_bin = self.git_bin.clone();
let fetch_timeout = Duration::from_secs(self.config.upstream_mirror_fetch_timeout_secs);
let fetch = tokio::task::spawn_blocking(move || {
run_fetch(&git_bin, &repo_path, &upstream, fetch_timeout)
})
.await
.context("upstream mirror fetch task panicked")?;
let success = fetch.is_ok();
let outcome = guard.release(success).await;
// A refused publish (fenced or unknowable upload) must not report
// Updated; surface it as the failure it is before mapping the fetch.
outcome.into_result()?;
fetch.map(|()| FetchOutcome::Updated)
}
async fn scan_once(&self) {
let mut cursor: Option<String> = None;
loop {
let targets = match self
.db
.list_inbound_mirror_targets(
cursor.as_deref(),
self.config.upstream_mirror_page_size as i64,
)
.await
{
Ok(targets) => targets,
Err(error) => {
warn!(err = %error, "failed to list inbound mirror targets");
return;
}
};
if targets.is_empty() {
return;
}
let page_len = targets.len();
cursor = targets.last().map(|target| target.repo_id.clone());
for target in targets {
let repo_id = target.repo_id.clone();
let upstream_url = target.upstream_url.clone();
match self.fetch_target(target).await {
Ok(FetchOutcome::Updated) => {
info!(repo_id, upstream = %upstream_url, "upstream mirror fetch completed")
}
Ok(FetchOutcome::SkippedStale) => {}
Err(error) => {
warn!(repo_id, upstream = %upstream_url, err = %error, "upstream mirror fetch failed")
}
}
}
if page_len < self.config.upstream_mirror_page_size {
return;
}
}
}
}
/// Validate the opt-in runtime and spawn the scheduled worker. The worker runs
/// one scan immediately, sleeps only after a complete scan, and exits between
/// scans when graceful shutdown is signalled. An in-flight Git child is bounded
/// and reaped before the worker observes shutdown.
pub fn start(
db: Arc<Db>,
config: Arc<Config>,
repo_store: RepoStore,
repo_write_leases: RepoWriteLeases,
git_write_semaphore: Arc<tokio::sync::Semaphore>,
mut shutdown_rx: tokio::sync::watch::Receiver<bool>,
) -> Result<()> {
ensure_safe_git_runtime("git")?;
let policy = EgressPolicy;
let worker = Worker {
db,
config: Arc::clone(&config),
repo_store,
repo_write_leases,
git_write_semaphore,
policy,
git_bin: "git".to_string(),
};
tokio::spawn(async move {
info!(
interval_secs = config.upstream_mirror_interval_secs,
"upstream mirror worker started"
);
loop {
worker.scan_once().await;
tokio::select! {
_ = tokio::time::sleep(Duration::from_secs(config.upstream_mirror_interval_secs)) => {}
changed = shutdown_rx.changed() => {
if changed.is_err() || *shutdown_rx.borrow() {
info!("upstream mirror worker stopped");
return;
}
}
}
}
});
Ok(())
}
#[cfg(test)]
mod tests {
use super::*;
use clap::Parser;
use std::path::PathBuf;
async fn test_worker(
pool: sqlx::PgPool,
git_bin: PathBuf,
page_size: usize,
) -> (Worker, Arc<Db>, tempfile::TempDir) {
let db = Arc::new(Db::for_testing(pool.clone()));
db.run_migrations().await.unwrap();
let repos = tempfile::tempdir().unwrap();
let mut config = Config::parse_from([
"gitlawb-node".to_string(),
"--upstream-mirror-enabled".to_string(),
"--enforce-owner-push".to_string(),
"--upstream-mirror-fetch-timeout-secs".to_string(),
"5".to_string(),
"--upstream-mirror-page-size".to_string(),
page_size.to_string(),
]);
config.validate().unwrap();
let config = Arc::new(config);
let worker = Worker {
db: Arc::clone(&db),
config,
repo_store: RepoStore::for_testing(repos.path().to_path_buf(), pool),
repo_write_leases: RepoWriteLeases::new(8),
git_write_semaphore: Arc::new(tokio::sync::Semaphore::new(4)),
policy: EgressPolicy,
git_bin: git_bin.to_string_lossy().into_owned(),
};
(worker, db, repos)
}
async fn add_inbound_mirror(
worker: &Worker,
db: &Db,
id: &str,
name: &str,
upstream_url: &str,
) -> InboundMirrorTarget {
let now = chrono::Utc::now();
let record = crate::db::RepoRecord {
id: id.to_string(),
name: name.to_string(),
owner_did: "did:key:z6MkMirrorWorker".to_string(),
description: None,
is_public: true,
default_branch: "main".to_string(),
created_at: now,
updated_at: now,
disk_path: format!("/test/{id}.git"),
forked_from: None,
machine_id: None,
};
db.create_repo(&record).await.unwrap();
worker
.repo_store
.init(&record.owner_did, &record.name)
.await
.unwrap();
let state = db
.configure_inbound_mirror(&record.id, upstream_url)
.await
.unwrap();
InboundMirrorTarget {
repo_id: record.id,
owner_did: record.owner_did,
name: record.name,
upstream_url: state.upstream_url,
}
}
#[cfg(unix)]
fn fake_git(repos: &tempfile::TempDir) -> (PathBuf, PathBuf) {
use std::os::unix::fs::PermissionsExt;
let git = repos.path().join("fake-git");
let calls = repos.path().join("fetch-calls");
let script = format!(
r#"#!/bin/sh
if [ "$1" = "ls-remote" ]; then
printf '%s\n' "$3"
exit 0
fi
if [ "$1" = "config" ]; then
# Match real Git when no sensitive repository-local keys are configured.
exit 1
fi
case "$*" in
*fail.git*) printf '%s\n' fail >> '{}'; exit 7 ;;
*) printf '%s\n' ok >> '{}'; exit 0 ;;
esac
"#,
calls.display(),
calls.display()
);
std::fs::write(&git, script).unwrap();
let mut permissions = std::fs::metadata(&git).unwrap().permissions();
permissions.set_mode(0o700);
std::fs::set_permissions(&git, permissions).unwrap();
(git, calls)
}
#[test]
fn git_version_gate_requires_curlopt_resolve_support() {
assert_eq!(parse_git_version("git version 2.37.0"), Some((2, 37)));
assert_eq!(
parse_git_version("git version 2.39.5 (Apple Git-154)"),
Some((2, 39))
);
assert_eq!(parse_git_version("not git"), None);
assert!((2, 36) < MIN_CURL_OPT_RESOLVE_GIT);
}
#[test]
fn every_resolved_address_must_be_public() {
let policy = EgressPolicy;
let public_v4: IpAddr = "8.8.8.8".parse().unwrap();
let private_v4: IpAddr = "10.2.3.4".parse().unwrap();
let private_v6: IpAddr = "fd00::20".parse().unwrap();
assert!(policy
.validate_addresses("public.example", &[public_v4])
.is_ok());
assert!(policy
.validate_addresses("private.example", &[private_v4, private_v6])
.is_err());
assert!(policy
.validate_addresses("mixed.example", &[public_v4, private_v4])
.is_err());
let too_many = (1..=MAX_PINNED_ADDRESSES + 1)
.map(|last| IpAddr::V4(std::net::Ipv4Addr::new(8, 8, 0, last as u8)))
.collect::<Vec<_>>();
assert!(policy
.validate_addresses("oversized.example", &too_many)
.is_err());
for never in [
"127.0.0.1",
"169.254.169.254",
"192.0.2.1",
"198.18.0.1",
"198.51.100.1",
"203.0.113.10",
"::1",
"::ffff:198.18.0.1",
"2002:c612:1::",
"64:ff9b::c612:1",
"100::1",
"100:0:0:1::1",
"2001::1",
"2001:2::1",
"2001:db8::1",
"fec0::1",
"fe80::1",
"3fff::1",
"5f00::1",
] {
let ip = never.parse().unwrap();
assert!(!address_is_permitted(ip), "{never} must stay denied");
}
}
#[test]
fn local_only_hostnames_are_always_rejected() {
for invalid in [
"",
"localhost",
"forge.localhost",
"forge.local",
"ghe.internal",
] {
assert!(
reject_always_local_hostname(invalid).is_err(),
"{invalid:?}"
);
}
assert!(reject_always_local_hostname("github.example").is_ok());
}
#[test]
fn fetch_command_pins_dns_and_only_updates_branches_and_tags() {
let upstream = PinnedUpstream {
url: "https://github.example/org/repo.git".to_string(),
curlopt_resolve: Some("+github.example:443:8.8.8.8,[2606:4700:4700::1111]".to_string()),
};
let command = fetch_command(&upstream);
let has_config = |key: &str, value: &str| {
command
.config
.contains(&(key.to_string(), value.to_string()))
};
assert!(has_config("http.followRedirects", "false"));
assert!(has_config("http.proxy", ""));
assert!(has_config("http.extraHeader", ""));
assert!(has_config("http.cookieFile", ""));
assert!(has_config("http.saveCookies", "false"));
assert!(has_config("http.sslVerify", "true"));
assert!(has_config(
"http.curloptResolve",
"+github.example:443:8.8.8.8,[2606:4700:4700::1111]"
));
assert!(has_config(
"http.https://github.example/org/repo.git.followRedirects",
"false"
));
assert!(has_config(
"http.https://github.example/org/repo.git.proxy",
""
));
assert!(has_config(
"http.https://github.example/org/repo.git.extraHeader",
""
));
assert!(has_config(
"http.https://github.example/org/repo.git.curloptResolve",
"+github.example:443:8.8.8.8,[2606:4700:4700::1111]"
));
assert!(has_config(
"credential.https://github.example/org/repo.git.helper",
""
));
assert!(has_config("protocol.allow", "never"));
assert!(has_config("protocol.https.allow", "always"));
assert!(has_config("fetch.uriprotocols", ""));
assert!(has_config("fetch.bundleURI", ""));
assert!(!command
.config
.iter()
.any(|(key, _)| key.ends_with("sslCert") || key.ends_with("sslKey")));
assert!(command.args.contains(&"--atomic".to_string()));
assert!(command
.args
.contains(&"+refs/heads/*:refs/heads/*".to_string()));
assert!(command
.args
.contains(&"+refs/tags/*:refs/tags/*".to_string()));
assert!(!command.args.iter().any(|arg| arg == "+refs/*:refs/*"));
}
/// Spawn `git` inside a fixture repo with inherited repository-location
/// and config environment scrubbed, so a test run inside an outer Git
/// checkout or custom config can never leak into the fixture.
fn fixture_git(dir: &std::path::Path) -> std::process::Command {
let mut cmd = std::process::Command::new("git");
cmd.current_dir(dir);
for var in [
"GIT_DIR",
"GIT_WORK_TREE",
"GIT_CONFIG",
"GIT_CONFIG_GLOBAL",
"GIT_CONFIG_SYSTEM",
"GIT_CONFIG_NOSYSTEM",
"GIT_CONFIG_COUNT",
"GIT_EXEC_PATH",
"GIT_SSH",
"GIT_SSH_COMMAND",
"GIT_ASKPASS",
"SSH_ASKPASS",
] {
cmd.env_remove(var);
}
cmd
}
#[test]
fn repository_local_instead_of_rewrite_is_rejected() {
let repo = tempfile::tempdir().unwrap();
let init = fixture_git(repo.path())
.args(["init", "--bare"])
.output()
.unwrap();
assert!(init.status.success());
let config = fixture_git(repo.path())
.args([
"config",
"url.https://evil.example/.insteadOf",
"https://github.example/",
])
.output()
.unwrap();
assert!(config.status.success());
let error = reject_local_url_rewrite(
"git",
repo.path(),
"https://github.example/org/repo.git",
Instant::now() + Duration::from_secs(5),
)
.unwrap_err();
assert!(error.to_string().contains("attempted to rewrite"));
}
#[test]
fn repository_local_network_and_credential_config_is_rejected() {
let repo = tempfile::tempdir().unwrap();
let init = fixture_git(repo.path())
.args(["init", "--bare"])
.output()
.unwrap();
assert!(init.status.success());
reject_sensitive_local_config("git", repo.path(), Instant::now() + Duration::from_secs(5))
.unwrap();
let config = fixture_git(repo.path())
.args(["config", "http.sslCert", "/tmp/untrusted-client.pem"])
.output()
.unwrap();
assert!(config.status.success());
let error = reject_sensitive_local_config(
"git",
repo.path(),
Instant::now() + Duration::from_secs(5),
)
.unwrap_err();
assert!(error.to_string().contains("network or credential"));
}
#[cfg(unix)]
#[test]
fn hardened_real_git_reaches_https_transport_for_equals_path() {
let repo = tempfile::tempdir().unwrap();
let init = fixture_git(repo.path())
.args(["init", "--bare"])
.output()
.unwrap();
assert!(init.status.success());
// This is intentionally a loopback test seam around `run_fetch`, below
// the egress-policy layer. Reaching the listener proves the generated
// Git config neither dies on an empty client certificate nor splits the
// valid `=` path at the command-config boundary. The listener closes
// immediately, so the expected final result is a TLS transport error.
let listener = std::net::TcpListener::bind("127.0.0.1:0").unwrap();
let port = listener.local_addr().unwrap().port();
listener.set_nonblocking(true).unwrap();
let accepted = std::thread::spawn(move || {
let deadline = Instant::now() + Duration::from_secs(3);
while Instant::now() < deadline {
match listener.accept() {
Ok((_stream, _address)) => return true,
Err(error) if error.kind() == std::io::ErrorKind::WouldBlock => {
std::thread::sleep(Duration::from_millis(10));
}
Err(error) => panic!("accepting real-Git transport probe: {error}"),
}
}
false
});
let upstream = PinnedUpstream {
url: format!("https://127.0.0.1:{port}/org/repo=mirror.git"),
curlopt_resolve: None,
};
let error = run_fetch("git", repo.path(), &upstream, Duration::from_secs(5)).unwrap_err();
assert!(
accepted.join().unwrap(),
"real Git must reach the HTTPS transport; got {error:#}"
);
let error = error.to_string();
assert!(!error.contains("credential missing host field"));
assert!(!error.contains("invalid key"));
}
#[tokio::test]
async fn literal_addresses_are_checked_without_dns() {
let policy = EgressPolicy;
let allowed = policy
.resolve(reqwest::Url::parse("https://8.8.8.8/org/repo.git").unwrap())
.await
.unwrap();
assert_eq!(allowed.curlopt_resolve, None);
for denied in [
"127.0.0.1",
"10.2.3.4",
"169.254.169.254",
"192.0.2.1",
"198.18.0.1",
"203.0.113.10",
"[::1]",
"[100::1]",
"[100:0:0:1::1]",
"[2001::1]",
"[2001:db8::1]",
"[fec0::1]",
"[fd00::20]",
"[3fff::1]",
"[5f00::1]",
] {
let url = reqwest::Url::parse(&format!("https://{denied}/org/repo.git")).unwrap();
assert!(
policy.resolve(url).await.is_err(),
"{denied} must stay denied"
);
}
}
#[sqlx::test]
async fn stale_target_is_rechecked_under_the_write_locks(pool: sqlx::PgPool) {
let missing_git = PathBuf::from("/definitely/missing/git");
let (worker, db, _repos) = test_worker(pool, missing_git, 1).await;
let target = add_inbound_mirror(
&worker,
&db,
"00000000-0000-0000-0000-000000000001",
"stale",
"https://8.8.8.8/org/stale.git",
)
.await;
sqlx::query(
"UPDATE repos
SET mirror_status = 'outbound'
WHERE id = $1",
)
.bind(&target.repo_id)
.execute(db.pool())
.await
.unwrap();
let outcome = worker.fetch_target(target).await.unwrap();
assert_eq!(outcome, FetchOutcome::SkippedStale);
assert!(worker.repo_write_leases.is_empty());
}
#[cfg(unix)]
#[sqlx::test]
async fn scan_pages_past_one_failure_and_fetches_the_next_repo(pool: sqlx::PgPool) {
let bootstrap = tempfile::tempdir().unwrap();
let (git, calls) = fake_git(&bootstrap);
let (worker, db, _repos) = test_worker(pool, git, 1).await;
add_inbound_mirror(
&worker,
&db,
"00000000-0000-0000-0000-000000000001",
"fails",
"https://8.8.8.8/org/fail.git",
)
.await;
add_inbound_mirror(
&worker,
&db,
"00000000-0000-0000-0000-000000000002",
"succeeds",
"https://8.8.8.8/org/ok.git",
)
.await;
worker.scan_once().await;
let calls = std::fs::read_to_string(calls).unwrap();
assert_eq!(calls.lines().collect::<Vec<_>>(), ["fail", "ok"]);
assert!(worker.repo_write_leases.is_empty());
}
}
+221
View File
@@ -0,0 +1,221 @@
//! At-rest protection for webhook HMAC secrets (#506).
//!
//! Webhook secrets are signing keys: whoever holds one can forge delivery
//! payloads to the receiver the hook points at. They are persisted in the
//! `webhooks.secret` column and used to compute the `X-Gitlawb-Signature-256`
//! header on each delivery. That secret needs two properties:
//!
//! * It must never reach a log line or an API response. [`WebhookSecret`]
//! redacts both `Debug` and `Serialize`, so no derived `{:?}` and no JSON
//! body ever renders the value, whatever it contains.
//! * It must not rest as plaintext where a database dump or backup can read
//! it. The node seals the secret with a key derived from its persistent
//! identity (see `AppState::derive_webhook_secret_key`), so a dump alone
//! reveals nothing.
//!
//! Storage format: `enc:v1:<base64url(nonce || ciphertext)>` — a 24-byte
//! XChaCha20-Poly1305 nonce followed by the ciphertext (which carries the tag).
//! Anything without the `enc:v1:` prefix is a legacy plaintext value, written
//! before this protection shipped; the delivery path signs with those unchanged
//! and they are re-encrypted once at startup by
//! `Db::reencrypt_legacy_webhook_secrets`.
use std::fmt;
use base64::{engine::general_purpose::URL_SAFE_NO_PAD as B64URL, Engine};
use chacha20poly1305::{
aead::{Aead, AeadCore, KeyInit, OsRng},
XChaCha20Poly1305, XNonce,
};
use serde::{Deserialize, Deserializer, Serialize, Serializer};
/// Prefix marking a sealed-at-rest secret. Legacy plaintext values lack it.
pub const SEAL_PREFIX: &str = "enc:v1:";
const NONCE_LEN: usize = 24;
/// An HMAC signing secret that never renders its contents in debug or JSON
/// output. Used as `Webhook.secret`, so that field is safe to `Debug`-print and
/// `Serialize` by construction rather than by discipline at each call site.
#[derive(Clone, PartialEq, Eq)]
pub struct WebhookSecret(String);
impl WebhookSecret {
pub fn new(inner: String) -> Self {
Self(inner)
}
pub fn as_str(&self) -> &str {
&self.0
}
/// Whether the value is a sealed ciphertext (as opposed to legacy plaintext).
pub fn is_sealed(&self) -> bool {
self.0.starts_with(SEAL_PREFIX)
}
}
impl fmt::Debug for WebhookSecret {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
f.write_str("\"***\"")
}
}
impl Serialize for WebhookSecret {
/// Emit a fixed marker, never the contents. On the wire a set secret is
/// `"secret": "***"` and an unset one is `"secret": null` (`Option`), which
/// the CLI already reads as "is a secret set".
fn serialize<S: Serializer>(&self, serializer: S) -> Result<S::Ok, S::Error> {
serializer.serialize_str("***")
}
}
impl<'de> Deserialize<'de> for WebhookSecret {
fn deserialize<D: Deserializer<'de>>(deserializer: D) -> Result<Self, D::Error> {
Ok(WebhookSecret(String::deserialize(deserializer)?))
}
}
/// Seal a plaintext secret under a node-held key. A fresh `OsRng` nonce on every
/// seal guarantees repeated seals of the same secret never collide.
pub fn seal(key: &[u8; 32], plaintext: &str) -> String {
let cipher = XChaCha20Poly1305::new_from_slice(key).expect("webhook secret key is 32 bytes");
let nonce = XChaCha20Poly1305::generate_nonce(&mut OsRng);
let ciphertext = cipher
.encrypt(&nonce, plaintext.as_bytes())
.expect("XChaCha20-Poly1305 encryption is infallible for in-memory inputs");
let mut out = Vec::with_capacity(NONCE_LEN + ciphertext.len());
out.extend_from_slice(&nonce);
out.extend_from_slice(&ciphertext);
format!("{SEAL_PREFIX}{}", B64URL.encode(out))
}
/// Open a sealed secret. Returns `None` for anything that is not a well-formed
/// sealed value — legacy plaintext, a foreign key, tampered or truncated bytes —
/// so a caller that needs the legacy path checks [`WebhookSecret::is_sealed`]
/// first (see [`reveal`]).
pub fn open(key: &[u8; 32], stored: &str) -> Option<String> {
let raw = B64URL.decode(stored.strip_prefix(SEAL_PREFIX)?).ok()?;
if raw.len() <= NONCE_LEN {
return None;
}
let (nonce, ciphertext) = raw.split_at(NONCE_LEN);
let cipher = XChaCha20Poly1305::new_from_slice(key).ok()?;
let plaintext = cipher.decrypt(XNonce::from_slice(nonce), ciphertext).ok()?;
String::from_utf8(plaintext).ok()
}
/// Whether a stored value has the structural shape of a sealed payload:
/// [`SEAL_PREFIX`] plus a base64url blob long enough to hold a nonce and
/// ciphertext. This is a shape check, not an authentication check: a
/// well-formed blob sealed under a different key satisfies it while [`open`]
/// still fails. The re-encryption pass uses the distinction to leave
/// well-formed-but-foreign ciphertext alone instead of re-wrapping it as
/// plaintext, which would let `reveal` return the outer blob as a signing key.
pub fn has_seal_shape(stored: &str) -> bool {
stored
.strip_prefix(SEAL_PREFIX)
.and_then(|rest| B64URL.decode(rest).ok())
.is_some_and(|raw| raw.len() > NONCE_LEN)
}
/// Resolve a stored secret to its plaintext for signing. Sealed values are opened
/// with the node key; legacy plaintext is returned unchanged so a hook created
/// before this fix keeps signing until the startup re-encryption pass runs.
pub fn reveal(key: &[u8; 32], stored: &WebhookSecret) -> Option<String> {
if stored.is_sealed() {
open(key, stored.as_str())
} else {
Some(stored.as_str().to_string())
}
}
#[cfg(test)]
mod tests {
use super::*;
use rand::Rng;
/// Test keys need no secrecy, but they still come from the OS CSPRNG so no
/// hard-coded key material sits in the tree. A wrong key fails XChaCha20-Poly1305
/// authentication with overwhelming probability, so the "wrong key" case stays
/// deterministic in practice.
fn new_key() -> [u8; 32] {
OsRng.gen()
}
fn other_key() -> [u8; 32] {
OsRng.gen()
}
#[test]
fn seal_open_round_trip() {
let k = new_key();
let sealed = seal(&k, "s3cr3t-value");
assert!(sealed.starts_with(SEAL_PREFIX));
assert_eq!(open(&k, &sealed).as_deref(), Some("s3cr3t-value"));
}
#[test]
fn sealed_value_is_never_the_plaintext() {
let k = new_key();
let sealed = seal(&k, "s3cr3t-value");
assert!(!sealed.contains("s3cr3t-value"));
}
#[test]
fn wrong_key_does_not_open() {
let sealed = seal(&new_key(), "s3cr3t-value");
assert_eq!(open(&other_key(), &sealed), None);
}
#[test]
fn tampered_or_truncated_value_does_not_open() {
let k = new_key();
let sealed = seal(&k, "s3cr3t-value");
// Flip a byte inside the base64 payload.
let mut chars: Vec<char> = sealed.chars().collect();
let idx = SEAL_PREFIX.len();
chars[idx] = if chars[idx] == 'A' { 'B' } else { 'A' };
let tampered: String = chars.into_iter().collect();
assert_eq!(open(&k, &tampered), None);
// A value that is just the prefix (no nonce/ciphertext) is malformed.
assert_eq!(open(&k, SEAL_PREFIX), None);
}
#[test]
fn reveal_opens_a_sealed_value() {
let k = new_key();
let sealed = WebhookSecret::new(seal(&k, "s3cr3t-value"));
assert_eq!(reveal(&k, &sealed).as_deref(), Some("s3cr3t-value"));
}
#[test]
fn legacy_plaintext_is_revealed_unchanged() {
let k = new_key();
let legacy = WebhookSecret::new("plaintext-secret".into());
assert!(!legacy.is_sealed());
assert_eq!(reveal(&k, &legacy).as_deref(), Some("plaintext-secret"));
}
#[test]
fn debug_is_redacted() {
let s = WebhookSecret::new("s3cr3t-value".into());
let rendered = format!("{s:?}");
assert!(!rendered.contains("s3cr3t-value"));
assert_eq!(rendered, "\"***\"");
}
#[test]
fn serialize_is_redacted() {
let s = WebhookSecret::new("s3cr3t-value".into());
let json = serde_json::to_string(&s).unwrap();
assert!(!json.contains("s3cr3t-value"));
assert_eq!(json, "\"***\"");
}
#[test]
fn deserialize_reads_the_underlying_string() {
let s: WebhookSecret = serde_json::from_str("\"s3cr3t-value\"").unwrap();
assert_eq!(s.as_str(), "s3cr3t-value");
}
}
+293 -9
View File
@@ -30,34 +30,200 @@ fn sign_payload(secret: &str, payload: &[u8]) -> String {
format!("sha256={}", hex::encode(mac.finalize().into_bytes())) format!("sha256={}", hex::encode(mac.finalize().into_bytes()))
} }
/// What the signing step decided for one hook. Keeping this a decision (not a
/// bare `Option`) lets the delivery path tell "hook has no secret" (send
/// unsigned) apart from "hook has a secret that could not be opened" (withhold:
/// an unsigned POST would present a broken hook as working while the ledger
/// marks the event delivered).
enum SigningOutcome {
NoSecret,
Signed(String),
Withhold,
}
fn signing_decision(
secret_key: &[u8; 32],
secret: Option<&crate::webhook_secret::WebhookSecret>,
bytes: &[u8],
) -> SigningOutcome {
match secret {
None => SigningOutcome::NoSecret,
Some(s) => match crate::webhook_secret::reveal(secret_key, s) {
Some(plain) => SigningOutcome::Signed(sign_payload(&plain, bytes)),
None => SigningOutcome::Withhold,
},
}
}
/// Fire webhooks for `event` on `repo_id`. Spawns background tasks — never blocks. /// Fire webhooks for `event` on `repo_id`. Spawns background tasks — never blocks.
pub fn fire_event( pub fn fire_event(
db: Arc<Db>, db: Arc<Db>,
http_client: Arc<reqwest::Client>, http_client: Arc<reqwest::Client>,
secret_key: Arc<[u8; 32]>,
repo_id: &str, repo_id: &str,
event: &str, event: &str,
payload: serde_json::Value, payload: serde_json::Value,
) {
fire_event_occurrence(
db,
http_client,
secret_key,
repo_id,
event,
payload,
None,
None,
);
}
/// Occurrence-keyed variant: at most one delivery per
/// `(request_id, ref_name, hook)` is spawned; concurrent executors
/// collapse via the `webhook_deliveries` ledger. Best-effort on crash
/// (ledger is claimed before spawn), which the PR scope allows.
///
/// For durable outbox usage, call `claim_webhook_delivery_before_spawn` first
/// to claim deliveries synchronously, then call
/// `fire_event_occurrence_with_claimed_hooks` with the claimed hook IDs to
/// spawn HTTP POST only for claimed hooks.
#[allow(clippy::too_many_arguments)]
pub fn fire_event_occurrence(
db: Arc<Db>,
http_client: Arc<reqwest::Client>,
secret_key: Arc<[u8; 32]>,
repo_id: &str,
event: &str,
payload: serde_json::Value,
request_id: Option<&str>,
ref_name: Option<&str>,
) { ) {
let repo_id = repo_id.to_string(); let repo_id = repo_id.to_string();
let event = event.to_string(); let event = event.to_string();
let request_id = request_id.map(|s| s.to_string());
let ref_name = ref_name.map(|s| s.to_string());
tokio::spawn(async move { tokio::spawn(async move {
fire_event_async(db, http_client, &repo_id, &event, payload).await; fire_event_async_occurrence(
db,
http_client,
secret_key,
&repo_id,
&event,
payload,
request_id.as_deref(),
ref_name.as_deref(),
None,
)
.await;
}); });
} }
async fn fire_event_async( /// Variant for durable outbox: only spawn HTTP POST for previously claimed hooks.
#[allow(clippy::too_many_arguments)]
pub fn fire_event_occurrence_with_claimed_hooks(
db: Arc<Db>, db: Arc<Db>,
http_client: Arc<reqwest::Client>, http_client: Arc<reqwest::Client>,
secret_key: Arc<[u8; 32]>,
repo_id: &str, repo_id: &str,
event: &str, event: &str,
payload: serde_json::Value, payload: serde_json::Value,
request_id: Option<&str>,
ref_name: Option<&str>,
claimed_hooks: Vec<crate::db::Webhook>,
) { ) {
let hooks = match db.list_webhooks_for_event(repo_id, event).await { let repo_id = repo_id.to_string();
Ok(h) => h, let event = event.to_string();
Err(e) => { let request_id = request_id.map(|s| s.to_string());
tracing::warn!(err = %e, "failed to list webhooks for event {event}"); let ref_name = ref_name.map(|s| s.to_string());
return; tokio::spawn(async move {
fire_event_async_occurrence(
db,
http_client,
secret_key,
&repo_id,
&event,
payload,
request_id.as_deref(),
ref_name.as_deref(),
Some(claimed_hooks),
)
.await;
});
}
/// Claim webhook delivery before spawning HTTP POST.
/// Returns `Some(hook ids)` successfully claimed, or `None` when the hook
/// list itself could not be read (transient DB error, unknown membership).
/// A `None` caller must fall back to best-effort send rather than dropping
/// the delivery: children are still deleted downstream, so an empty claim
/// set on DB error would lose the webhook permanently.
/// This synchronous claim phase must be awaited before child deletion to prevent
/// permanent webhook loss on crash between deletion and claim.
pub async fn claim_webhook_delivery_before_spawn(
db: Arc<Db>,
repo_id: &str,
event: &str,
request_id: Option<&str>,
ref_name: Option<&str>,
) -> Option<Vec<crate::db::Webhook>> {
let mut claimed_hooks = Vec::new();
// Only claim when we have occurrence context (request_id + ref_name)
if let (Some(req), Some(r)) = (request_id, ref_name) {
let hooks = match db.list_webhooks_for_event(repo_id, event).await {
Ok(h) => h,
Err(e) => {
tracing::warn!(err = %e, "failed to list webhooks for event {event}");
return None;
}
};
for hook in hooks {
let delivery_id = crate::db::deterministic_id(&["webhook", req, r, &hook.id]);
match db
.claim_webhook_delivery(&delivery_id, req, repo_id, event)
.await
{
Ok(true) => claimed_hooks.push(hook),
Ok(false) => continue, // Already claimed
// Send fallback (legacy best-effort): a transient claim
// error must not drop the delivery while children are
// still deleted. The in-task claimer this replaced sent
// anyway on claim error; keep that here.
Err(e) => {
tracing::warn!(err = %e, hook_id = %hook.id, "webhook claim failed; sending anyway");
claimed_hooks.push(hook);
}
}
} }
}
Some(claimed_hooks)
}
#[allow(clippy::too_many_arguments)]
async fn fire_event_async_occurrence(
db: Arc<Db>,
http_client: Arc<reqwest::Client>,
secret_key: Arc<[u8; 32]>,
repo_id: &str,
event: &str,
payload: serde_json::Value,
request_id: Option<&str>,
ref_name: Option<&str>,
claimed_hooks: Option<Vec<crate::db::Webhook>>,
) {
// A caller that already claimed deliveries hands the claimed hook rows
// through; the send path must not re-list, because a list failure here
// would silently drop deliveries already claimed in the ledger.
let already_claimed = claimed_hooks.is_some();
let hooks = match claimed_hooks {
Some(h) => h,
None => match db.list_webhooks_for_event(repo_id, event).await {
Ok(h) => h,
Err(e) => {
tracing::warn!(err = %e, "failed to list webhooks for event {event}");
return;
}
},
}; };
if hooks.is_empty() { if hooks.is_empty() {
@@ -76,10 +242,67 @@ async fn fire_event_async(
let client = Arc::clone(&http_client); let client = Arc::clone(&http_client);
let event_name = event.to_string(); let event_name = event.to_string();
let bytes = payload_bytes.clone(); let bytes = payload_bytes.clone();
let delivery_id = uuid::Uuid::new_v4().to_string(); // Stable occurrence delivery key when request context exists;
// random UUID otherwise (legacy callers without occurrence).
let delivery_id = match (request_id, ref_name) {
(Some(req), Some(r)) => crate::db::deterministic_id(&["webhook", req, r, &hook.id]),
_ => uuid::Uuid::new_v4().to_string(),
};
// Claim before spawn so concurrent executors collapse to one
// delivery per occurrence. Failures to record fall back to
// sending (legacy best-effort) rather than dropping.
// Skip claim if already done by caller (durable_outbox path).
if !already_claimed && request_id.is_some() {
match db
.clone()
.claim_webhook_delivery(&delivery_id, request_id.unwrap_or(""), repo_id, event)
.await
{
Ok(true) => {}
Ok(false) => continue,
Err(_) => {}
}
}
let db_for_mark = db.clone();
let secret_key = Arc::clone(&secret_key);
tokio::spawn(async move { tokio::spawn(async move {
let signature = hook.secret.as_deref().map(|s| sign_payload(s, &bytes)); // Policy skip, not a delivery failure: the stored URL predates the
// create-time gate. The "invalid_url" label is documented on
// record_webhook_delivery in metrics.rs.
if !crate::api::peers::is_public_http_url(&hook.url) {
crate::metrics::record_webhook_delivery("invalid_url");
tracing::warn!(
url = %hook.url,
event = %event_name,
"webhook delivery skipped: stored URL is not a public http(s) target"
);
return;
}
let delivery_for_mark = delivery_id.clone();
// Open the secret only at signing time; at rest it stays sealed.
let signature =
match signing_decision(secret_key.as_ref(), hook.secret.as_ref(), &bytes) {
SigningOutcome::NoSecret => None,
SigningOutcome::Signed(sig) => Some(sig),
// A hook that HAS a secret must sign with it or not deliver at
// all: `reveal` returning None means the stored value could
// not be opened (a foreign or rotated key, a truncated
// ciphertext), and sending unsigned would present a broken
// hook as a working one while the ledger marks the event
// delivered. The row stays pending, like a network error, so
// a fixed key can still land it.
SigningOutcome::Withhold => {
crate::metrics::record_webhook_delivery("secret_unopenable");
tracing::warn!(
hook_id = %hook.id,
delivery_id = %delivery_for_mark,
"webhook secret could not be opened; delivery withheld"
);
return;
}
};
let mut req = client let mut req = client
.post(&hook.url) .post(&hook.url)
@@ -100,6 +323,10 @@ async fn fire_event_async(
"http_error" "http_error"
}; };
crate::metrics::record_webhook_delivery(result_label); crate::metrics::record_webhook_delivery(result_label);
// Any HTTP response proves the delivery fired; mark
// sent so the ledger never claims an unsent delivery.
// Network errors stay pending for stale-reclaim.
let _ = db_for_mark.mark_webhook_sent(&delivery_for_mark).await;
tracing::info!( tracing::info!(
url = %hook.url, url = %hook.url,
event = %event_name, event = %event_name,
@@ -120,3 +347,60 @@ async fn fire_event_async(
}); });
} }
} }
#[cfg(test)]
mod tests {
use super::*;
use rand::Rng;
fn key() -> [u8; 32] {
chacha20poly1305::aead::OsRng.gen()
}
/// A hook with no secret delivers unsigned, as before.
#[test]
fn signing_decision_no_secret_is_unsigned() {
assert!(matches!(
signing_decision(&key(), None, b"{}"),
SigningOutcome::NoSecret
));
}
/// A sealed secret that opens signs with the revealed plaintext.
#[test]
fn signing_decision_sealed_secret_signs() {
let k = key();
let sealed =
crate::webhook_secret::WebhookSecret::new(crate::webhook_secret::seal(&k, "topsecret"));
match signing_decision(&k, Some(&sealed), b"{}") {
SigningOutcome::Signed(sig) => {
assert_eq!(sig, sign_payload("topsecret", b"{}"))
}
_ => panic!("an openable sealed secret must produce a signature"),
}
}
/// A stored value that carries the sealed prefix but cannot be opened
/// (foreign key, truncated ciphertext) must withhold, never send unsigned.
#[test]
fn signing_decision_unopenable_secret_withholds() {
let garbage =
crate::webhook_secret::WebhookSecret::new("enc:v1:bm90LWNoaXBoZXJ0ZXh0".to_string());
assert!(matches!(
signing_decision(&key(), Some(&garbage), b"{}"),
SigningOutcome::Withhold
));
}
/// A legacy plaintext value signs with itself unchanged.
#[test]
fn signing_decision_legacy_plaintext_signs() {
let legacy = crate::webhook_secret::WebhookSecret::new("plainsecret".to_string());
match signing_decision(&key(), Some(&legacy), b"{}") {
SigningOutcome::Signed(sig) => {
assert_eq!(sig, sign_payload("plainsecret", b"{}"))
}
_ => panic!("legacy plaintext must keep signing"),
}
}
}
File diff suppressed because it is too large. Load diff
+59
View File
@@ -0,0 +1,59 @@
use std::io::{Read, Write};
use std::process::{Command, Stdio};
use std::time::Duration;
fn main() -> std::io::Result<()> {
let executable = std::env::current_exe()?;
let config = std::fs::read_to_string(executable.parent().unwrap().join("config"))?;
let (mode, parameter) = config.split_once('\n').unwrap_or((&config, ""));
let args: Vec<_> = std::env::args_os().skip(1).collect();
if mode == "delay" {
std::thread::sleep(Duration::from_millis(parameter.parse().unwrap()));
}
let batch = args.get(1).is_some_and(|arg| arg == "--batch-check");
let input = if batch {
let mut bytes = Vec::new();
std::io::stdin().read_to_end(&mut bytes)?;
Some(bytes)
} else {
None
};
let cwd = std::env::current_dir()?;
let hang = mode == "hang"
|| (mode == "hang-repo" && cwd.file_name().is_some_and(|name| name == parameter))
|| (mode == "hang-oid"
&& (args.iter().any(|arg| arg == parameter)
|| input
.as_ref()
.is_some_and(|bytes| String::from_utf8_lossy(bytes).trim() == parameter)));
if hang {
std::thread::sleep(Duration::from_secs(30));
std::process::exit(1);
}
if mode == "log-types" && batch {
let mut log = std::fs::OpenOptions::new()
.create(true)
.append(true)
.open(parameter)?;
writeln!(
log,
"{} {}",
String::from_utf8_lossy(input.as_ref().unwrap()).trim(),
cwd.file_name().unwrap().to_string_lossy()
)?;
}
let mut child = Command::new("git")
.args(&args)
.stdin(if input.is_some() {
Stdio::piped()
} else {
Stdio::inherit()
})
.stdout(Stdio::inherit())
.stderr(Stdio::inherit())
.spawn()?;
if let Some(bytes) = input {
child.stdin.take().unwrap().write_all(&bytes)?;
}
std::process::exit(child.wait()?.code().unwrap_or(1));
}
+595 -30
View File
@@ -416,8 +416,8 @@ fn f3_second_writer_leased_until_reap() {
NOT RepoWriteGuard (which drops at the disconnect instant, reopening F3).", NOT RepoWriteGuard (which drops at the disconnect instant, reopening F3).",
); );
let receive = repos_production let receive = repos_production
.find("smart_http::receive_pack(") .find("smart_http::receive_pack_raw")
.expect("F3 gate stale: git_receive_pack no longer calls smart_http::receive_pack"); .expect("F3 gate stale: git_receive_pack no longer calls smart_http::receive_pack_raw");
assert!( assert!(
lease_acquire < with_lease && with_lease < receive, lease_acquire < with_lease && with_lease < receive,
"F3 gate bypassed: the write lease must be acquired, then carried by the \ "F3 gate bypassed: the write lease must be acquired, then carried by the \
@@ -509,17 +509,19 @@ fn inv22_ipfs_walk_admission_reaches_every_blocking_site() {
/// too, so an ungated spawn would fire for a push git rejected, pinning and announcing /// too, so an ungated spawn would fire for a push git rejected, pinning and announcing
/// a half-applied repo. Above `release` the `?` on `receive_result` can no longer be /// a half-applied repo. Above `release` the `?` on `receive_result` can no longer be
/// what gates it, so the success check is explicit and this gate binds it: the spawn /// what gates it, so the success check is explicit and this gate binds it: the spawn
/// must sit inside `if push_succeeded`, and `release` must consume the same flag so /// must sit inside `if disposition.spawn_tail`, and `release` must consume the same
/// the two cannot drift apart. /// disposition (`disposition.release_ok`) so the two cannot drift apart.
/// ///
/// This is an ordering check rather than a cancellation-race test on purpose: it is /// This is an ordering check rather than a cancellation-race test on purpose: it is
/// the companion to `receive_pack_tail_survives_a_disconnect_during_release`, which /// the companion to `receive_pack_tail_survives_a_disconnect_during_release`, which
/// drives the actual disconnect through a parked `release`. Same instrument the F3 /// drives the actual disconnect through a parked `release`. Same instrument the F3
/// gate above uses. /// gate above uses.
/// ///
/// MUTATION (RED): move the `tokio::spawn(post_receive_replication_tail` call below /// MUTATION (RED): move the `post_receive_replication_tail(` call below
/// `guard.release(` and the ordering assertion fails; take it out of the /// `guard.release(` and the ordering assertion fails; take it out of the
/// `if push_succeeded` block and the failed-push assertion fails. /// `if disposition.spawn_tail` block and the failed-push assertion fails; gate
/// `spawn_tail`/`release_ok` on the outcome commit and
/// `post_git_disposition_replication_carve_out` fails with it.
#[test] #[test]
fn inv22_replication_tail_spawns_at_the_durability_boundary() { fn inv22_replication_tail_spawns_at_the_durability_boundary() {
let repos = src("api/repos.rs"); let repos = src("api/repos.rs");
@@ -534,41 +536,604 @@ fn inv22_replication_tail_spawns_at_the_durability_boundary() {
.expect("split always yields a first chunk"); .expect("split always yields a first chunk");
let success_flag = production let success_flag = production
.find("let push_succeeded = receive_result.is_ok();") .find("post_git_disposition(")
.expect( .expect("U5 gate missing: the tail's disposition must be bound before the gate");
"U5 gate missing: the tail's success gate must be bound from receive_result.is_ok()", // One tail per push, on the proven-landed ref set. Either spawn
// position (carve-out early, or durability-bound at release) must sit
// inside a `disposition.spawn_tail` gate and feed `landed_refs` —
// never the unfiltered `ref_updates`.
// Match the tail CALL, not the `tokio::spawn(...)` wrapper: production
// currently wraps it in `tokio::spawn(async move { ... })`, and keying on
// the wrapper's exact shape is what let this gate go blind on the refactor.
// The `pub(crate) async fn` definition is the only other text occurrence,
// filtered by the `fn ` that must precede it.
let spawn_positions: Vec<usize> = production
.match_indices("post_receive_replication_tail(")
.map(|(p, _)| p)
.filter(|&p| !production[..p].trim_end().ends_with("fn"))
.collect();
assert!(
!spawn_positions.is_empty(),
"U5 gate missing: the replication tail must be spawned by git_receive_pack"
);
for (i, pos) in spawn_positions.iter().copied().enumerate() {
let gate = production[..pos].rfind("if disposition.spawn_tail").expect(
"U5 gate missing: every tail spawn must be gated on `if disposition.spawn_tail`",
); );
let gate_open = production // The spawn must sit inside the gate's block: brace depth at the
.find("if push_succeeded {") // spawn position must still be positive measured from the `if`.
.expect("U5 gate missing: the tail spawn must be gated on the push having succeeded"); // An ungated spawn after the block closes fails this, as does a
let spawn = production // stray `disposition.spawn_tail` mention in a comment.
.find("tokio::spawn(post_receive_replication_tail(") let mut depth = 0i32;
for ch in production[gate..pos].chars() {
match ch {
'{' => depth += 1,
'}' => depth -= 1,
_ => {}
}
}
assert!(
depth >= 1,
"U5 gate bypassed: tail #{i} sits outside the `if disposition.spawn_tail` block"
);
assert!(
!production[gate..pos].contains("post_receive_replication_tail"),
"U5 gate bypassed: a second tail spawn sits between the gate and its spawn"
);
let call_end = production[pos..].find(");").unwrap() + pos;
let call = &production[pos..call_end];
// Match the ref-set ARGUMENT (third positional), not a substring of the
// call: `landed` is the in-scope alias for `landed_refs` in the
// carve-out spawn and either name is the proven-landed set, but a bare
// `call.contains("landed")` is also satisfied by the `push_landed_at`
// timestamp argument, which would let a call feeding unfiltered
// `ref_updates` pass.
let refset_arg = call
.split(',')
.nth(2)
.map(str::trim)
.expect("U5 gate stale: tail call no longer takes a third argument");
assert!(
refset_arg == "landed" || refset_arg == "landed_refs",
"U5 gate bypassed: tail #{i} must carry the proven-landed ref set, not unfiltered ref_updates"
);
}
let spawn = *spawn_positions
.first()
.expect("U5 gate missing: the replication tail must be spawned by git_receive_pack"); .expect("U5 gate missing: the replication tail must be spawned by git_receive_pack");
let release = production let release = production
.find(".release(push_succeeded)") .find(".release(disposition.release_ok)")
.expect("U5 gate stale: release must consume the same success flag as the tail gate"); .expect("U5 gate stale: release must consume the same disposition as the tail gate");
let touch = production let touch = production
.find("state.db.touch_repo(") .find("state.db.touch_repo(")
.expect("U5 gate stale: git_receive_pack no longer calls touch_repo"); .expect("U5 gate stale: git_receive_pack no longer calls touch_repo");
let webhook = production // #26 Split PR 1 step 3 — the webhook fan-out moved into
.find("webhooks::fire_event(") // `durable_outbox::apply_request_effects`, which the live
.expect("U5 gate stale: git_receive_pack no longer fires push webhooks"); // handler calls inline (the recovery drain calls the same
// function on the next startup). The gate now pins that the
// handler is wired to the executor, not to a per-ref inline
// webhook call.
let effects_executor = production
.find("apply_request_effects(&state, &request_id)")
.expect("U5 gate stale: git_receive_pack no longer calls apply_request_effects inline");
assert!( assert!(
success_flag < gate_open && gate_open < spawn, success_flag < spawn,
"U5 gate bypassed: the tail must be spawned inside `if push_succeeded`, or a \ "U5 gate bypassed: the disposition must be bound before any tail spawn"
rejected push spawns a tail that pins and announces a half-applied repo"
);
// Still inside that block: no `}` may close it between the gate and the spawn.
assert!(
!production[gate_open + "if push_succeeded {".len()..spawn].contains('}'),
"U5 gate bypassed: the tail spawn left the `if push_succeeded` block, so a \
rejected push now spawns a tail"
); );
assert!( assert!(
spawn < release && spawn < touch && spawn < webhook, spawn < release && spawn < touch && spawn < effects_executor,
"U5 gate bypassed: the tail must be spawned BEFORE guard.release, touch_repo \ "U5 gate bypassed: the tail must be spawned BEFORE guard.release, touch_repo \
and the webhook fan-out, so a disconnect in any of those windows cannot drop \ and the effect executor, so a disconnect in any of those windows cannot drop \
this push's pins, recovery copy, and announcements" this push's pins, recovery copy, and announcements"
); );
} }
/// #26 Split PR 1 step 3 — drain + handler share a single effect
/// executor. The live handler (api/repos.rs) and the recovery
/// drain (`durable_outbox::drain_receive_pack_requests_with`) both
/// call `apply_request_effects`, so the per-ref effects fan-out
/// lives in exactly one place. The v29 per-ref walk
/// (`derive_one`, `drain_pending_ref_transitions_all`,
/// `lookup_accepted_ordinal`) is dead code: any caller reintroduced
/// would be the per-ref walk the step-3 PR removed. This gate
/// fails if a call site slips back in or the new seam is bypassed.
#[test]
fn inv26_step3_live_and_drain_share_apply_request_effects() {
let repos = src("api/repos.rs");
let outbox = src("durable_outbox.rs");
// The live handler calls `apply_request_effects`. Split the
// file at the test attribute so test code can't satisfy the
// gate by itself.
let production_repos = repos
.split("\nmod tests {")
.next()
.expect("split always yields a first chunk");
assert!(
production_repos.contains("apply_request_effects(&state, &request_id)"),
"live handler must call `apply_request_effects(&state, &request_id)`; \
reverting to a per-ref inline fan-out splits live and recovery"
);
// The durable intent insert closes the pre-outbox crash window this
// PR exists to fix. Wrapping it in `if false` must turn this gate
// red; drain-only tests cannot prove the handler creates the rows.
assert!(
production_repos.contains("insert_receive_pack_request_with_children("),
"live handler must persist durable intent via \
`insert_receive_pack_request_with_children`; disabling it must fail this gate"
);
// The drain's per-request seam calls the same executor. Test
// code lives below `mod drain_tests`, so split there too.
let production_outbox = outbox
.split("\nmod drain_tests {")
.next()
.expect("split always yields a first chunk");
assert!(
production_outbox.contains("drain_receive_pack_requests_with"),
"drain seam `drain_receive_pack_requests_with` must exist; \
removing it forces a per-ref walk back into the drain"
);
assert!(
production_outbox.contains("apply_request_effects"),
"durable_outbox production code must define `apply_request_effects`; \
removing it splits the executor between live and recovery"
);
// The drain's per-request walker is wired to the executor.
// The closure body of `drain_receive_pack_requests_with` is the
// only call site — if a future change calls `derive_one`
// instead, this assertion fires.
let drain_seam_open = production_outbox
.find("pub async fn drain_receive_pack_requests_with<F, Fut>")
.expect("drain seam must be defined in the production half");
let drain_seam_close = production_outbox[drain_seam_open..]
.find("\n}\n")
.expect("drain seam body must close");
let drain_seam_body = &production_outbox[drain_seam_open..drain_seam_open + drain_seam_close];
assert!(
drain_seam_body.contains("apply_request_effects"),
"drain_receive_pack_requests_with must call apply_request_effects; \
wiring it to a per-ref helper reintroduces the v29 walk"
);
// The deleted per-ref drain must have zero call sites in the
// production half. A regression that re-adds a caller would
// bring back the per-ref fan-out.
assert!(
!production_outbox.contains("drain_pending_ref_transitions_all("),
"deleted `drain_pending_ref_transitions_all` must have zero production call sites; \
a per-ref walk is reintroduced"
);
assert!(
!production_outbox.contains("derive_one("),
"deleted `derive_one` must have zero production call sites; \
the per-ref fan-out is reintroduced"
);
assert!(
!production_outbox.contains("lookup_accepted_ordinal("),
"deleted `lookup_accepted_ordinal` must have zero production call sites; \
the per-ref ordinal lookup is reintroduced"
);
assert!(
!production_repos.contains("derive_one("),
"deleted `derive_one` must have zero live-handler call sites"
);
assert!(
!production_repos.contains("drain_pending_ref_transitions"),
"deleted per-ref drain functions must have zero live-handler call sites"
);
}
/// #26 Split PR 1 step 4 — the periodic queue-lifecycle purge is
/// wired in `lib.rs` and the contract is enforced by the `idx_receive_pack_requests_completed_at`
/// partial index from v41.
///
/// Assertions:
/// 1. `lib.rs` calls `purge_request_queue` once at boot (well, on
/// the spawn-task interval) and the spawn function exists.
/// 2. The DB helpers `purge_completed_receive_pack_requests` and
/// `purge_completed_pending_ref_transitions` exist in `db/mod.rs`.
/// 3. The config knobs `queue_retention_days` and `queue_purge_batch` exist
/// on `Config`.
/// 4. `quarantined` is NOT in the purge WHERE clause (step 4 does not
/// introduce the state, but the invariant holds for the future).
#[test]
fn inv26_step4_queue_lifecycle_purge_is_wired() {
let boot_src = src("lib.rs");
assert!(
boot_src.contains("spawn_queue_lifecycle_sweep"),
"lib.rs must spawn the periodic queue-lifecycle purge"
);
assert!(
boot_src.contains("purge_request_queue"),
"lib.rs must call purge_request_queue on the periodic sweep"
);
let db_src = src("db/mod.rs");
assert!(
db_src.contains("purge_completed_receive_pack_requests"),
"db/mod.rs must expose purge_completed_receive_pack_requests"
);
assert!(
db_src.contains("purge_completed_pending_ref_transitions"),
"db/mod.rs must expose purge_completed_pending_ref_transitions"
);
let config_src = src("config.rs");
assert!(
config_src.contains("queue_retention_days"),
"Config must expose queue_retention_days"
);
assert!(
config_src.contains("queue_purge_batch"),
"Config must expose queue_purge_batch"
);
}
/// #26 Split PR 1 step 5 — the marker gate and the retry-bound
/// quarantine are wired end-to-end. A missing or hash-mismatched
/// marker on disk quarantines the request; a Retry over the bound
/// does too. This gate pins every load-bearing seam, against the
/// production half of each file (the test modules name the same
/// identifiers in their own harnesses).
///
/// Assertions:
/// 1. `reconcile_prepared_page` calls `mark_request_quarantined`
/// and `mark_children_rejected_for_quarantined_parent` on
/// marker-gate failure.
/// 2. The drain's `EffectsOutcome::Retry` arm checks
/// `effects_max_attempts`.
/// 3. `git::store::read_ref` exists in `git/store.rs`.
/// 4. `db::mark_request_quarantined`,
/// `db::mark_children_rejected_for_quarantined_parent`,
/// `db::get_receive_pack_requests_by_ids`, and
/// `db::request_state::QUARANTINED` all exist.
/// 5. The handler writes the marker ref BEFORE calling
/// `smart_http::receive_pack_raw` (the durability window).
#[test]
fn inv26_step5_marker_quarantine_and_bound_are_wired() {
let outbox = src("durable_outbox.rs");
let store = src("git/store.rs");
let db = src("db/mod.rs");
let repos = src("api/repos.rs");
// Split at the TEST MODULE for the production-only assertions.
let production_outbox = outbox
.split("\nmod drain_tests {")
.next()
.expect("split always yields a first chunk");
let production_repos = repos
.split("\nmod tests {")
.next()
.expect("split always yields a first chunk");
// (1) The reconcile's marker gate quarantines via the DB helpers.
assert!(
production_outbox.contains("mark_request_quarantined"),
"reconcile_prepared_page must call mark_request_quarantined on marker-gate failure"
);
assert!(
production_outbox.contains("mark_children_rejected_for_quarantined_parent"),
"reconcile_prepared_page must call mark_children_rejected_for_quarantined_parent \
so quarantined parents cancel their children"
);
// The gate reads the marker ref and compares against the parent's
// `request_bytes_hash` via `git::store::read_ref` /
// `git::store::marker_value_for`. Reverting either reintroduces
// the DoS window the marker gate exists to close.
assert!(
production_outbox.contains("git::store::read_ref"),
"reconcile_prepared_page must read the marker ref via git::store::read_ref"
);
assert!(
production_outbox.contains("marker_value_for"),
"reconcile_prepared_page must compute the expected marker value via \
git::store::marker_value_for"
);
// (2) The drain's `EffectsOutcome::Retry` arm checks the bound.
assert!(
production_outbox.contains("effects_max_attempts"),
"drain_receive_pack_requests_with must consult effects_max_attempts on Retry"
);
// (3) `git::store::read_ref` is the read seam the gate depends on.
assert!(
store.contains("pub fn read_ref("),
"git::store::read_ref must exist; the marker gate reads through it"
);
assert!(
store.contains("pub fn marker_value_for("),
"git::store::marker_value_for must exist; the marker gate computes the expected \
value with it (and the live handler writes the value via the same helper)"
);
// (4) DB-side seams the gate depends on.
assert!(
db.contains("pub async fn mark_request_quarantined"),
"Db::mark_request_quarantined must exist"
);
assert!(
db.contains("pub async fn mark_children_rejected_for_quarantined_parent"),
"Db::mark_children_rejected_for_quarantined_parent must exist"
);
assert!(
db.contains("pub async fn get_receive_pack_requests_by_ids"),
"Db::get_receive_pack_requests_by_ids must exist (avoids N+1 in the marker gate)"
);
assert!(
db.contains("pub const QUARANTINED: &str = \"quarantined\""),
"request_state::QUARANTINED must be defined"
);
// (5) The handler writes the marker ref BEFORE the durability
// boundary (smart_http::receive_pack_raw). Severing the
// ordering re-opens the marker-gate DoS window for live pushes.
let marker_write = production_repos
.find("git::store::marker_value_for")
.expect("U5 gate stale: the live handler no longer computes the marker value");
let receive_raw = production_repos
.find("smart_http::receive_pack_raw")
.expect("U5 gate stale: git_receive_pack no longer calls smart_http::receive_pack_raw");
assert!(
marker_write < receive_raw,
"U5 gate bypassed: the marker ref must be written BEFORE receive_pack_raw so the \
reconcile's gate has evidence of the live push"
);
}
/// Issue #26 split 1 — namespace gate and pre-git refusal lifecycle.
///
/// The push gate (repos.rs) must deny:
/// 1. Any ref-update line that fails UTF-8 decode (the raw body is passed verbatim
/// to git receive-pack, and git accepts refnames with bytes >= 0x80; the gate
/// never sees these refs, no durable child row exists, and under requests/issues/
/// the lossy for-each-ref decode still matches the narrowed exemption).
/// 2. The bare `refs/gitlawb` name (the `starts_with("refs/gitlawb/")` check misses it).
///
/// Every explicit pre-git refusal must call `refuse_receive_pack_before_git` to
/// terminalize the intent aggregate in one transaction, or a dropped future between
/// the intent insert and `receive_pack` strands a `received` + `prepared` aggregate
/// that is never purged, never promotable, and poisons its tuple.
///
/// The visibility exemption (visibility_pack.rs) was narrowed to `requests/` and
/// `issues/` only. The push gate is the premise that narrowing relies on (the comment
/// at visibility_pack.rs:340), so tests must prove:
/// - A `refs/gitlawb/<other>` ref pointing at a blob is denied on push.
/// - A stray `refs/gitlawb/<other>` blob ref planted in the pack path fails closed.
///
/// Source gates below pin the wiring; behavioral proofs live next to the code:
/// `parse_ref_updates_refuses_non_utf8_command_line` and
/// `internal_namespace_gate_covers_bare_name_and_subtree` (api/repos.rs),
/// `planted_non_exempt_gitlawb_blob_ref_fails_closed` (git/visibility_pack.rs),
/// and the mirror import prune
/// `prune_removes_non_exempt_gitlawb_refs_and_keeps_exempt` (sync.rs).
#[test]
fn issue_26_namespace_gate_and_refusal_wiring() {
let repos = src("api/repos.rs");
let vis = src("git/visibility_pack.rs");
// P1: fail the push on ref-update lines the handler cannot decode.
// The gate at parse_ref_updates must refuse when any command line in the
// ref-update section is undecodable UTF-8. The check is an Err arm on
// `from_utf8(data)` that returns BadRequest, so removing it allows non-UTF-8
// refs to bypass the namespace gate.
assert!(
repos.contains("ref-update line is not valid UTF-8; push refused"),
"P1 gate missing: parse_ref_updates must refuse when any ref-update line is \
non-UTF-8 (git accepts bytes >= 0x80; the gate never sees them, no durable \
child exists, and the lossy for-each-ref decode still matches the exemption)"
);
// P1 (cont): the bare `refs/gitlawb` name must also be caught. The
// predicate is extracted as `ref_is_internal_namespace` so it is
// unit-testable; pin the bare-name arm inside it and its use at the
// push gate (a `starts_with("refs/gitlawb/")`-only check misses the
// bare name, and it lands on repos with no internal refs yet).
assert!(
repos.contains("ref_name == \"refs/gitlawb\"")
&& repos.contains(".find(|r| ref_is_internal_namespace(r))"),
"P1 gate missing: ref_is_internal_namespace must match the bare \
'refs/gitlawb' name and the push gate must consult it \
(starts_with('refs/gitlawb/') misses it, and it lands on repos with \
no internal refs yet)"
);
// P1 (cont): every explicit pre-git refusal must call refuse_receive_pack_before_git
// to terminalize the intent. Exactly 6 occurrences: the five inline call
// sites (lease cap, per-source cap, write pool, acquire timeout, acquire
// error) plus the drop-guard's own pre-git spawn. The post-git guard phase
// calls mark_receive_pack_interrupted (a different name), so deleting any
// one inline site drops the count to 5 and turns this red. The parse and
// namespace refusals correctly carry no call: they return before the
// intent insert, so there is no aggregate to terminalize.
assert_eq!(
repos.matches("refuse_receive_pack_before_git(").count(),
6,
"P1 wiring: exactly 6 refuse_receive_pack_before_git calls expected (5 \
inline refusal sites + guard pre-git spawn); deleting any inline site \
must turn this red"
);
// Pin the five inline sites by their reason literals, so a site that stops
// terminalizing (or is renumbered away) fails by name, not just by count.
for reason in [
"repo write-lease waiter cap reached",
"admission per-source cap reached",
"git write pool saturated",
"acquire_write timed out",
"acquire_write failed",
"handler dropped before git started",
] {
assert!(
repos.contains(reason),
"P1 wiring: refusal reason literal missing: {reason}"
);
}
// P1 (cont): the intent drop-guard must be phase-aware. It is armed before
// the insert resolves (covering the cancellation window), flipped to
// git-started immediately before the receive_pack await (a drop from there
// is a mid-git disconnect, terminalized via mark_receive_pack_interrupted
// because the Err arm never runs on a drop), and disarmed once the outcome
// commit resolves. Assert the ordering, not just presence: the phase flip
// must precede the receive_pack call.
assert!(
repos.contains("IntentDropGuard::new") && repos.contains("notify_git_started()"),
"P1 gate missing: IntentDropGuard must exist and flip to git-started phase \
before receive_pack (a dropped future between intent insert and git start \
strands a received+prepared aggregate)"
);
let flip_pos = repos
.find("notify_git_started()")
.expect("notify_git_started call must exist");
let git_pos = repos
.find("receive_pack_raw_with_reflog(")
.expect("receive_pack call must exist");
assert!(
flip_pos < git_pos,
"P1 ordering: notify_git_started() must precede the receive_pack call, \
or a mid-git drop is misclassified as a pre-git shed"
);
assert!(
repos.contains("_intent_guard.disarm()"),
"P1 gate missing: the guard must disarm once the outcome commit resolves, \
or every push spawns a wasteful cleanup txn at scope end"
);
// A sync Drop cannot await: cleanup is spawned via the current runtime
// handle with an off-runtime fallback. A bare tokio::spawn panics
// mid-unwind with no runtime (off-runtime drop, process teardown).
assert!(
repos.contains("Handle::try_current()"),
"P1 gate missing: guard Drop must spawn via Handle::try_current with an \
off-runtime arm, not a bare tokio::spawn"
);
// P2: the post-git drop arm must exist and be wired. The guard's
// GIT_STARTED phase spawns `mark_receive_pack_interrupted` (parent to
// `rejected_at_git`, children to `uncertain` — the Err arm never runs on
// a drop). Deleting that spawn keeps every count above green, so pin the
// method name; its behavior is proved by
// `post_git_interruption_marks_uncertain_and_terminalizes_parent`.
assert!(
repos.contains("mark_receive_pack_interrupted("),
"P2 wiring: the guard's post-git phase must spawn \
mark_receive_pack_interrupted (a mid-git drop otherwise strands \
received+prepared with no Err arm to mark them uncertain)"
);
// The guard must commit computed fates rather than a blanket uncertain
// once the report is parsed, and only disarm on a resolved commit: pin
// both the stash call and the `outcome_commit_ok` gate. Making disarm()
// unconditional strands the commit-failure path and misfiles proven
// rejections as uncertain.
assert!(
repos.contains("set_fates(ComputedFates"),
"P2 wiring: computed fates must be stashed on the guard once parsed, \
or a post-commit drop can only apply a context-free uncertain"
);
assert!(
repos.contains("if outcome_commit_ok"),
"P2 wiring: disarm must be gated on outcome_commit_ok; unconditional \
disarm strands the persistent-commit-failure path"
);
// The live-handler `AwaitingSiblings` arm is unreachable through the
// current outcome builder (documented at the arm itself) and therefore
// has no behavioral test; pin its presence so removing it is conscious.
// The drain consumer IS behaviorally driven (sibling-wait tests).
assert_eq!(
repos.matches("EffectsOutcome::AwaitingSiblings").count(),
1,
"live AwaitingSiblings arm must exist (defensive symmetry with the \
drain consumer); deleting it must turn this red"
);
// P2: pin the mirror prune call sites. The prune helper is called from
// both `clone_repo` and `fetch_repo` (plus its own definition) — deleting
// either call leaves the direct-helper test green while mirrors import
// evil refs again. Count occurrences of the call with its argument.
let sync_src = src("sync.rs");
assert_eq!(
sync_src
.matches("prune_non_exempt_gitlawb_refs(local_path)")
.count(),
2,
"P2 wiring: prune must be called from both clone_repo and fetch_repo; \
deleting either call must turn this red"
);
// Both call sites must route through `wrap_prune_error`, which preserves
// deterministic `PruneInvalid` refusals instead of stringifying every
// error into transient `PruneFailed` (an every-tick deferral spin for
// permanently invalid mirrors). Pin the exact call shape at both sites.
assert_eq!(
sync_src.matches(".map_err(wrap_prune_error)").count(),
2,
"P2 wiring: both prune call sites must use wrap_prune_error; inline \
rewrap must turn this red"
);
// P2: the visibility exemption narrowing removed a serving path. The exemption
// is now scoped to `requests/` and `issues/` only (visibility_pack.rs:371), and
// the comment at :340 states the push gate is the premise. Tests must prove the
// gate works, which the P1 findings above covered (non-UTF-8 line, bare name,
// refusal wiring). This check verifies the exemption narrowing landed.
assert!(
vis.contains(
r#"!r.starts_with("refs/gitlawb/requests/") && !r.starts_with("refs/gitlawb/issues/")"#
),
"P2 gate missing: the visibility exemption must be narrowed to requests/ and \
issues/ only (the push gate is the premise; any other refs/gitlawb/* ref is \
denied on push and must fail closed in the pack path)"
);
}
/// Claimed-webhook delivery must not re-list hooks: the durable path claims
/// synchronously, hands the claimed hook rows to the sender, and a list
/// failure in the send task would silently drop already-claimed deliveries.
#[test]
fn inv26_claimed_hook_delivery_never_relists() {
let webhooks = src("webhooks.rs");
let production = webhooks
.split("\nmod tests {")
.next()
.expect("split always yields a first chunk");
// The claimed entry point exists and passes hook ROWS (not ids) so the
// sender never needs the hook table again.
let entry = production
.find("fire_event_occurrence_with_claimed_hooks(")
.expect("claimed-hooks sender missing");
let claimed_rows = production
.find("claimed_hooks: Vec<crate::db::Webhook>")
.expect("claimed path must carry Webhook rows, not ids");
let list_call = production
.find("db.list_webhooks_for_event(repo_id, event)")
.expect("unclaimed path still lists hooks");
assert!(
entry < claimed_rows && claimed_rows < list_call,
"claimed sender must hand rows through before any list fallback"
);
// Inside the async worker, the claimed branch must bypass the list.
// Inside the async worker (which starts after the claim helper), the
// claimed branch must be selected before the list fallback can run.
let worker = production
.find("async fn fire_event_async_occurrence(")
.expect("occurrence worker missing");
let already_claimed = production[worker..]
.find("let already_claimed = claimed_hooks.is_some();")
.map(|i| worker + i)
.expect("claimed path must short-circuit before re-listing");
let worker_list = production[worker..]
.find("db.list_webhooks_for_event(repo_id, event)")
.map(|i| worker + i)
.expect("unclaimed path still lists hooks");
assert!(
already_claimed < worker_list,
"the claimed branch must be selected before list_webhooks_for_event can run"
);
}
@@ -0,0 +1,177 @@
//! #26 Split PR 4 — `GITLAWB_ENFORCE_OWNER_PUSH` default is consistent across surfaces.
//!
//! Reviewer 2 (2026-08-28) closed PR #224 with a P2 finding: SECURITY.md and
//! README.md both stated the default was `false`, while `Config::enforce_owner_push`
//! declared `default_value_t = true` and the runtime, .env.example, and the env
//! var table on the README all agreed with the code. Two operator-facing
//! surfaces lied about the authorization policy. PR #330 flipped the code
//! default and updated .env.example / README env table / docs/RUN-A-NODE.md,
//! but missed these two prose mentions.
//!
//! This guard pins the documentation and example surfaces together. Each one
//! is read off the working tree at test time, not a copy in the binary, so a
//! revert of the doc edit turns it red:
//!
//! 1. `SECURITY.md` says the default is `true`.
//! 2. `README.md` says the default is `true` in the known-limitations prose.
//! 3. `README.md` says the default is `true` in the env-var table row.
//! 4. `.env.example` sets `GITLAWB_ENFORCE_OWNER_PUSH=true`.
//!
//! The runtime declared default (`Config::enforce_owner_push`'s
//! `default_value_t = true`) is protected by the
//! `enforce_owner_push_is_declared_true_independent_of_the_environment`
//! unit test in `config.rs`, which reads the parser directly and is
//! env-independent. The two checks together cover the surfaces that
//! produced the original doc-vs-code drift; this guard cannot reach into
//! the binary crate's `Config` type (it is private to the bin target,
//! which has no `lib.rs`) without a larger refactor, so the contract is
//! stated as the four documentation/example surfaces above.
//!
//! The four are checked independently. A check that just verified "at least
//! one surface says true" would pass while a stale SECURITY.md lied; a check
//! that only verified the binary would not catch the doc drift that
//! produced this finding in the first place.
use std::path::Path;
fn read(rel: &str) -> String {
let p = Path::new(env!("CARGO_MANIFEST_DIR"))
.join("..")
.join("..")
.join(rel);
std::fs::read_to_string(&p).unwrap_or_else(|e| panic!("read {}: {e}", p.display()))
}
/// Pin the prose mentions of `GITLAWB_ENFORCE_OWNER_PUSH` so a future doc
/// edit cannot reintroduce the "defaults to false" wording the reviewer
/// flagged. The wording in the test is what the docs MUST say about the
/// default; absence of any prose mention is also a fail because operators
/// rely on the docs to learn the default.
#[test]
fn security_md_states_owner_push_default_is_true() {
let security = read("SECURITY.md");
let section = section_after(&security, "### Repository write authorization defaults")
.unwrap_or_else(|| panic!("SECURITY.md is missing the owner-push section"));
assert!(
section.contains("defaults to `true`"),
"SECURITY.md owner-push section must state the default is true. \
Found:\n{section}"
);
assert!(
!section_claims_default_is_false(&section),
"SECURITY.md owner-push section must not claim the default is false. \
Found:\n{section}"
);
}
#[test]
fn readme_known_limitations_states_owner_push_default_is_true() {
let readme = read("README.md");
let limitations = section_after(&readme, "Known limitations:")
.unwrap_or_else(|| panic!("README.md is missing the Known limitations: section"));
// The owner-push line is one bullet in the limitations list. Pull the
// single line that mentions the env var so the assertion is local to
// the claim, not a substring of unrelated text.
let line = limitations
.lines()
.find(|l| l.contains("GITLAWB_ENFORCE_OWNER_PUSH"))
.unwrap_or_else(|| {
panic!(
"README.md Known limitations must mention GITLAWB_ENFORCE_OWNER_PUSH.\n\
Got:\n{limitations}"
)
});
assert!(
line.contains("defaults to `true`") || line.contains("defaults to true"),
"README Known-limitations owner-push bullet must state the default is true. \
Found:\n{line}"
);
assert!(
!section_claims_default_is_false(line),
"README Known-limitations owner-push bullet must not claim the default is false. \
Found:\n{line}"
);
}
#[test]
fn readme_env_var_table_states_owner_push_default_is_true() {
let readme = read("README.md");
let line = readme
.lines()
.find(|l| l.contains("GITLAWB_ENFORCE_OWNER_PUSH") && l.contains("|"))
.unwrap_or_else(|| {
panic!("README.md is missing the env-var-table row for GITLAWB_ENFORCE_OWNER_PUSH")
});
assert!(
line.contains("Defaults to `true`") || line.contains("defaults to `true`"),
"README env-var-table row for GITLAWB_ENFORCE_OWNER_PUSH must state the default is true. \
Found:\n{line}"
);
assert!(
!section_claims_default_is_false(line),
"README env-var-table row for GITLAWB_ENFORCE_OWNER_PUSH must not claim the default is false. \
Found:\n{line}"
);
}
#[test]
fn env_example_sets_owner_push_true() {
let env = read(".env.example");
let line = env
.lines()
.find(|l| l.trim_start().starts_with("GITLAWB_ENFORCE_OWNER_PUSH"))
.unwrap_or_else(|| panic!(".env.example is missing GITLAWB_ENFORCE_OWNER_PUSH"));
assert!(
line.contains("=true"),
".env.example GITLAWB_ENFORCE_OWNER_PUSH must be set to true. Found:\n{line}"
);
}
/// A claim that the default is false can hide in many phrasings. Match the
/// forms that have appeared in the wild on this branch: "defaults to `false`",
/// "defaults to false", "default is false", "is `false` for compatibility",
/// "is false for compatibility". The match is intentionally narrow — false
/// positives would suppress a real doc edit.
fn section_claims_default_is_false(section: &str) -> bool {
let lower = section.to_ascii_lowercase();
lower.contains("defaults to `false`")
|| lower.contains("defaults to false")
|| lower.contains("default is `false`")
|| lower.contains("default is false")
|| (lower.contains("`false`") && lower.contains("for compatibility"))
}
/// Return the prose after a marker up to the next heading boundary. The
/// marker is normally a markdown heading (e.g. `### Section`); if it has
/// no `#` prefix it is treated as a plain-text marker and the slice is
/// bounded by the next heading of any level. This matters because a
/// plain-text marker would otherwise produce `header_level = 0`, making
/// the same-or-higher-level stop condition unsatisfiable and letting the
/// slice run to EOF.
fn section_after(doc: &str, heading: &str) -> Option<String> {
let start = doc.find(heading)?;
let after = &doc[start..];
let header_level = heading.chars().take_while(|c| *c == '#').count();
// Skip past the heading line itself.
let body_start = after.find('\n')? + 1;
let body = &after[body_start..];
// Stop at the next heading of the same or higher level, or at the next
// heading of any level when the marker had no `#` prefix.
let mut end = body.len();
for line in body.lines() {
let level = line.chars().take_while(|c| *c == '#').count();
if level > 0 && (header_level == 0 || level <= header_level) {
end = line.as_ptr() as usize - body.as_ptr() as usize;
break;
}
}
Some(body[..end].to_string())
}
+235
View File
@@ -0,0 +1,235 @@
//! INV-8 deny assertion. A denial must be an actual refusal (4xx) whose body
//! and headers carry none of the data it is withholding, and must never be a 2xx
//! (an empty-200 rendered as success is the denial-as-success bug this guards).
//!
//! The check is split into a pure `check_denied` (unit-tested below with the
//! self-check scenarios) and an async `assert_denied` wrapper that reads a real
//! `reqwest::Response`.
/// Pure deny check. Returns `Err(reason)` when the response is not a clean
/// denial. `withheld` are exact tokens (OIDs, paths, private slugs) that must
/// not appear anywhere in the body; register short-OID prefixes and encoded
/// path forms as separate tokens at the call site (a raw substring scan cannot
/// normalize them).
pub fn check_denied(
status: u16,
body: &str,
expected: u16,
withheld: &[&str],
) -> Result<(), String> {
// The harness only ever expects denials; a non-4xx expectation is a
// programming error in the test, not a node behavior under test.
if !(400..500).contains(&expected) {
return Err(format!(
"test bug: assert_denied expected status {expected} is not a 4xx"
));
}
// INV-8: a denial rendered as success is the headline bug. Catch it
// explicitly so the failure message names it, rather than only failing the
// status-equality check below.
if (200..300).contains(&status) {
return Err(format!(
"denial rendered as success: got {status}, expected denial {expected}. \
body={body:?}"
));
}
if status != expected {
return Err(format!(
"expected denial status {expected}, got {status}. body={body:?}"
));
}
for token in withheld {
if token.is_empty() {
continue;
}
if body.contains(token) {
return Err(format!(
"withheld token {token:?} leaked in denial body: {body:?}"
));
}
}
Ok(())
}
/// Like [`check_denied`] but also scans serialized response headers for withheld
/// tokens. Call sites that only have a body should keep using `check_denied`.
pub fn check_denied_with_headers(
status: u16,
body: &str,
header_text: &str,
expected: u16,
withheld: &[&str],
) -> Result<(), String> {
check_denied(status, body, expected, withheld)?;
for token in withheld {
if token.is_empty() {
continue;
}
if header_text.contains(token) {
return Err(format!(
"withheld token {token:?} leaked in denial headers: {header_text:?}"
));
}
}
Ok(())
}
/// Replace each withheld token in `s` with `[REDACTED]` so a test-failure
/// witness does not itself leak the secret it caught.
fn redact_withheld(s: &str, withheld: &[&str]) -> String {
let mut result = s.to_string();
for token in withheld {
if !token.is_empty() {
result = result.replace(token, "[REDACTED]");
}
}
result
}
/// Drive a real response through the deny check and panic on failure. Reads the
/// full body once.
pub async fn assert_denied(resp: reqwest::Response, expected: u16, withheld: &[&str]) {
let status = resp.status().as_u16();
let header_text = resp
.headers()
.iter()
.filter_map(|(name, value)| {
value
.to_str()
.ok()
.map(|v| format!("{}: {}", name.as_str(), v))
})
.collect::<Vec<_>>()
.join("\n");
// A body-read failure must NOT fold to "" — an empty body passes the withheld
// scan vacuously, so an unreadable denial (a mid-body reset/truncation) would be
// certified leak-free without ever inspecting the bytes. Fail loud instead. A
// legitimately empty body is `Ok("")`, not `Err`, so the honest empty-401 denials
// still pass.
let body = resp
.text()
.await
.expect("read denial body: an unreadable denial cannot be certified leak-free");
if let Err(reason) = check_denied_with_headers(status, &body, &header_text, expected, withheld)
{
// Panic with a fixed string: CodeQL's taint analysis traces withheld
// tokens from check_denied through redact_withheld into the panic sink,
// even though redact_withheld strips them at runtime. A fixed message
// keeps no tainted data on the sink path. The test name identifies
// which probe failed; check_denied's unit tests cover the error detail.
let _ = redact_withheld(&reason, withheld);
panic!("deny assertion failed: status/body/header leak check");
}
}
#[cfg(test)]
mod tests {
use super::{check_denied, check_denied_with_headers, redact_withheld};
#[test]
fn clean_403_with_no_leak_passes() {
let r = check_denied(
403,
r#"{"error":"only the repo owner can do this"}"#,
403,
&["a1b2c3"],
);
assert!(r.is_ok(), "{r:?}");
}
#[test]
fn empty_200_is_flagged_as_denial_rendered_as_success() {
let r = check_denied(200, "", 403, &[]);
assert!(r.is_err());
assert!(r.unwrap_err().contains("rendered as success"));
}
#[test]
fn leaking_403_fails_and_names_the_token() {
let withheld = "deadbeefdeadbeefdeadbeefdeadbeefdeadbeef";
let body = format!("not found near object {withheld}");
let r = check_denied(403, &body, 403, &[withheld]);
assert!(r.is_err());
assert!(r.unwrap_err().contains(withheld));
}
#[test]
fn wrong_status_fails() {
// A 404 when the owner gate's 403 was expected must not pass: it means
// an earlier layer denied, so the gate under test was never exercised.
let r = check_denied(404, "not found", 403, &[]);
assert!(r.is_err());
assert!(r.unwrap_err().contains("expected denial status 403"));
}
#[test]
fn non_4xx_expected_is_a_test_bug() {
// Guards the harness itself: expecting a non-denial status is a test
// authoring error, not a node behavior, and must be rejected.
let r = check_denied(403, "denied", 200, &[]);
assert!(r.is_err());
assert!(r.unwrap_err().contains("not a 4xx"));
}
#[test]
fn empty_withheld_token_never_false_positives() {
// An empty token must be skipped, never treated as "contained in every
// body" — otherwise every denial would spuriously fail as a leak.
let r = check_denied(403, "any body", 403, &[""]);
assert!(r.is_ok(), "{r:?}");
}
#[test]
fn leaking_header_fails_and_names_the_token() {
let withheld = "deadbeefdeadbeefdeadbeefdeadbeefdeadbeef";
let headers = format!("X-Debug: object {withheld}");
let r = check_denied_with_headers(
404,
r#"{"error":"repository not found"}"#,
&headers,
404,
&[withheld],
);
assert!(r.is_err());
let err = r.unwrap_err();
assert!(err.contains("leaked in denial headers"));
assert!(err.contains(withheld));
}
#[test]
fn clean_headers_with_clean_body_passes() {
let withheld = "a1b2c3";
let r = check_denied_with_headers(
404,
r#"{"error":"repository not found"}"#,
"Content-Type: application/json",
404,
&[withheld],
);
assert!(r.is_ok(), "{r:?}");
}
#[test]
fn redact_withheld_replaces_secrets() {
let withheld = "deadbeefdeadbeefdeadbeefdeadbeefdeadbeef";
let msg = format!("withheld token {withheld:?} leaked in body: near {withheld}");
let redacted = redact_withheld(&msg, &[withheld]);
assert!(
!redacted.contains(withheld),
"token must be redacted: {redacted}"
);
assert!(
redacted.contains("[REDACTED]"),
"redaction marker must appear: {redacted}"
);
}
#[test]
fn redact_withheld_leaves_clean_strings() {
let result = redact_withheld(
"no secrets here",
&["deadbeefdeadbeefdeadbeefdeadbeefdeadbeef"],
);
assert_eq!(result, "no secrets here");
}
}
+33
View File
@@ -0,0 +1,33 @@
//! Shared support for the real-node deny harness: the deny assertion (U4), the
//! RFC-9421 signing client (U2), and (via `gitlawb_node::test_harness`) the
//! node spawner (U3).
pub mod assert;
// U2 fixture state matrix + per-gate-class probe generators; driven by U3.
#[allow(dead_code)]
pub mod probe;
// U1 deny-bearing route registry; fields consumed by U2/U3 as they land.
#[allow(dead_code)]
pub mod routes;
pub mod signing;
/// A reqwest client with a bounded request timeout, so a wedged git subprocess or
/// route fails the suite rather than hanging it until CI kills the job (#195, F1).
/// The two real-node probes that drive git subprocesses use this; the inline
/// twins at the upload-pack and registry-sweep sites use the same pattern.
#[allow(dead_code)]
pub fn bounded_client() -> reqwest::Client {
reqwest::Client::builder()
.timeout(std::time::Duration::from_secs(30))
.build()
.unwrap()
}
/// Build an anonymous (unsigned) GET request for `path` under `base_url`.
/// Extracted from inline call sites so URL construction goes through a
/// function boundary, matching the `signed_request` pattern in `support::signing`.
#[allow(dead_code)]
pub fn anon_get(client: &reqwest::Client, base_url: &str, path: &str) -> reqwest::RequestBuilder {
let url = format!("{base_url}{path}");
client.get(url)
}
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
@@ -0,0 +1,32 @@
//! Real RFC-9421 request signing for the deny harness (U2). Wraps
//! `gitlawb_core::http_sig::sign_request` (the same entry point the
//! git-remote-gitlawb helper uses in production) so a test can present a valid
//! signature for any identity, reaching the authenticated-but-wrong-owner path
//! that an injected-DID shortcut cannot.
use gitlawb_core::http_sig::sign_request;
use gitlawb_core::identity::Keypair;
/// Build a reqwest request carrying a valid RFC-9421 signature for `keypair`.
///
/// `path` is both appended to `base_url` to form the request URL and signed as
/// the `@path` component, so the value the server reconstructs and the value
/// that was signed are guaranteed identical. `body` is signed via its
/// content-digest and sent as the request body.
pub fn signed_request(
client: &reqwest::Client,
method: reqwest::Method,
base_url: &str,
path: &str,
body: Vec<u8>,
keypair: &Keypair,
) -> reqwest::RequestBuilder {
let sig = sign_request(keypair, method.as_str(), path, &body);
let url = format!("{base_url}{path}");
client
.request(method, url)
.header("content-digest", sig.content_digest)
.header("signature-input", sig.signature_input)
.header("signature", sig.signature)
.body(body)
}
+3
View File
@@ -26,7 +26,10 @@ clap = { version = "4", features = ["derive", "env"] }
dirs = "5" dirs = "5"
reqwest = { workspace = true } reqwest = { workspace = true }
uuid = { workspace = true } uuid = { workspace = true }
url = { workspace = true }
percent-encoding = { workspace = true }
urlencoding = "2" urlencoding = "2"
libc = "0.2"
alloy = { version = "1", default-features = false, features = [ alloy = { version = "1", default-features = false, features = [
"contract", "contract",
"provider-http", "provider-http",
+1 -1
View File
@@ -88,7 +88,7 @@ async fn cmd_list(node: String, capability: Option<String>) -> Result<()> {
let short = did let short = did
.split(':') .split(':')
.next_back() .next_back()
.map(|s| &s[..s.len().min(16)]) .map(|s| crate::text::truncate(s, 16))
.unwrap_or("?"); .unwrap_or("?");
let trust = agent["trust_score"].as_f64().unwrap_or(0.0); let trust = agent["trust_score"].as_f64().unwrap_or(0.0);
let caps = agent["capabilities"] let caps = agent["capabilities"]
+2 -2
View File
@@ -275,7 +275,7 @@ async fn cmd_list(
let title = b["title"].as_str().unwrap_or("?"); let title = b["title"].as_str().unwrap_or("?");
let amount = b["amount"].as_i64().unwrap_or(0); let amount = b["amount"].as_i64().unwrap_or(0);
let st = b["status"].as_str().unwrap_or("?"); let st = b["status"].as_str().unwrap_or("?");
let short_id = &id[..8.min(id.len())]; let short_id = crate::text::truncate(id, 8);
println!("{short_id} {st:<10} {amount:>12} $GITLAWB {title}"); println!("{short_id} {st:<10} {amount:>12} $GITLAWB {title}");
} }
} }
@@ -403,7 +403,7 @@ async fn cmd_stats(node: String) -> Result<()> {
let did = entry["did"].as_str().unwrap_or("?"); let did = entry["did"].as_str().unwrap_or("?");
let count = entry["completed"].as_i64().unwrap_or(0); let count = entry["completed"].as_i64().unwrap_or(0);
let earned = entry["total_earned"].as_i64().unwrap_or(0); let earned = entry["total_earned"].as_i64().unwrap_or(0);
let short = &did[did.len().saturating_sub(8)..]; let short = crate::text::truncate_suffix(did, 8);
println!( println!(
" {}. ...{short} {count} bounties {earned} $GITLAWB", " {}. ...{short} {count} bounties {earned} $GITLAWB",
i + 1 i + 1
+545 -22
View File
@@ -4,6 +4,7 @@
use anyhow::{Context, Result}; use anyhow::{Context, Result};
use clap::{Args, Subcommand}; use clap::{Args, Subcommand};
use serde_json::Value;
use std::path::PathBuf; use std::path::PathBuf;
use crate::http::NodeClient; use crate::http::NodeClient;
@@ -34,7 +35,12 @@ pub enum CertCmd {
#[arg(long)] #[arg(long)]
dir: Option<PathBuf>, dir: Option<PathBuf>,
}, },
/// Show a specific ref certificate and verify its signature /// Show a specific ref certificate and verify its signature.
///
/// `--verify` requires `--expect-node <did>`: the queried node's
/// self-reported DID is not a trust anchor, because a node serving a
/// forged certificate also controls its own self-report. Callers
/// must name the issuer they trust.
Show { Show {
/// Repository in <owner>/<repo> or <repo> format /// Repository in <owner>/<repo> or <repo> format
repo: String, repo: String,
@@ -49,13 +55,17 @@ pub enum CertCmd {
#[arg(long)] #[arg(long)]
dir: Option<PathBuf>, dir: Option<PathBuf>,
/// Exit non-zero unless the Ed25519 signature verifies AND the /// Exit non-zero unless the Ed25519 signature verifies AND the
/// issuing node matches the queried node (or --expect-node) /// issuing node matches --expect-node (which --verify requires:
/// without a caller-supplied anchor there is no trusted issuer to
/// verify against)
#[arg(long)] #[arg(long)]
verify: bool, verify: bool,
/// Expected issuing node DID for --verify. A valid signature alone /// Expected issuing node DID for --verify. A valid signature alone
/// only proves the cert is internally consistent — signed by whatever /// only proves the cert is internally consistent — signed by whatever
/// key it names — so --verify also anchors the issuer to a DID you /// key it names — so --verify also anchors the issuer to a DID you
/// trust: this value when given, else the queried node's DID. /// trust. The queried node's self-reported DID is deliberately NOT
/// accepted as a fallback anchor: the node that served the cert can
/// serve a matching self-report.
#[arg(long, requires = "verify")] #[arg(long, requires = "verify")]
expect_node: Option<String>, expect_node: Option<String>,
}, },
@@ -117,7 +127,10 @@ async fn cmd_list(repo: String, node: String, dir: Option<PathBuf>) -> Result<()
let id = cert["id"].as_str().unwrap_or("?"); let id = cert["id"].as_str().unwrap_or("?");
let ref_name = cert["ref_name"].as_str().unwrap_or("?"); let ref_name = cert["ref_name"].as_str().unwrap_or("?");
let new_sha = cert["new_sha"].as_str().unwrap_or("?"); let new_sha = cert["new_sha"].as_str().unwrap_or("?");
let issued_at = cert["issued_at"].as_str().map(|s| &s[..19]).unwrap_or("?"); let issued_at = cert["issued_at"]
.as_str()
.map(|s| crate::text::truncate(s, 19))
.unwrap_or("?");
println!(" {id:.8} {issued_at} {ref_name} {new_sha:.12}"); println!(" {id:.8} {issued_at} {ref_name} {new_sha:.12}");
} }
Ok(()) Ok(())
@@ -149,6 +162,24 @@ async fn cmd_show(
let node_did = cert["node_did"].as_str().unwrap_or("?"); let node_did = cert["node_did"].as_str().unwrap_or("?");
let signature = cert["signature"].as_str().unwrap_or("?"); let signature = cert["signature"].as_str().unwrap_or("?");
let issued_at = cert["issued_at"].as_str().unwrap_or("?"); let issued_at = cert["issued_at"].as_str().unwrap_or("?");
// #26 Split PR 3: read the wire-format version. A MISSING key
// selects legacy v1 (an old server that predates the field).
// A PRESENT value must be a JSON integer that fits in u32 and
// names a version this client supports — otherwise the response
// declares a format we cannot represent and we refuse to verify
// rather than guess the payload shape. Reviewer 2 finding:
// collapsing `null`, a string, a float, or an overflow integer
// onto v1 would let a well-signed v1 signature pass `--verify`
// against a server that explicitly said "version 2 (or 99, or
// 4294967297)" — that is the exact mismatch the field exists
// to prevent. parse_cert_version is the load-bearing parser
// that distinguishes missing-key (legacy v1) from invalid-value
// (unsupported).
let parsed_version = parse_cert_version(cert.get("version"));
let version_display: String = match &parsed_version {
Ok(v) => v.to_string(),
Err(reason) => format!("unsupported ({reason})"),
};
println!("Ref Certificate: {cert_id}"); println!("Ref Certificate: {cert_id}");
println!(" Ref: {ref_name}"); println!(" Ref: {ref_name}");
@@ -157,6 +188,7 @@ async fn cmd_show(
println!(" Pusher: {pusher}"); println!(" Pusher: {pusher}");
println!(" Node DID: {node_did}"); println!(" Node DID: {node_did}");
println!(" Issued at: {issued_at}"); println!(" Issued at: {issued_at}");
println!(" Version: {version_display}");
println!(" Signature: {signature}"); println!(" Signature: {signature}");
println!(); println!();
@@ -166,9 +198,31 @@ async fn cmd_show(
// This proves the cert is internally authentic — signed by the key it // This proves the cert is internally authentic — signed by the key it
// names; the node-DID comparison below covers *which* node that is. // names; the node-DID comparison below covers *which* node that is.
let repo_id = cert["repo_id"].as_str().unwrap_or(""); let repo_id = cert["repo_id"].as_str().unwrap_or("");
let verdict = verify_signature( let verdict = match parsed_version {
repo_id, ref_name, old_sha, new_sha, pusher, node_did, issued_at, signature, // v1 is the only version this client verifies. The v1
); // signed payload is the 7-field canonical form with no
// `version` key — see gitlawb-node/src/cert.rs. A future
// v2+ cert has a different signed payload shape (the
// version field becomes part of the signed JSON per
// `v2_signing_payload`), so a v2+ cert from a server this
// client does not know about must NOT be silently verified
// as v1.
// Reviewer 2: refuse rather than guess; the client and
// server must agree on the version.
// parse_cert_version admits exactly one Ok value: 1 (missing key and
// integer 1 both land there; 2, floats, strings, overflow and every
// other spelling come back Err with their own reason). A separate
// `Ok(v != 1)` arm existed here and was DEAD CODE — a test aimed at it
// could never catch drift in this match (review round 2) — so the
// match now mirrors the parser's real contract, which the
// `parse_cert_version_truth_table` test pins.
Ok(_) => verify_signature(
repo_id, ref_name, old_sha, new_sha, pusher, node_did, issued_at, signature,
),
Err(ref reason) => Err(format!(
"cert declared a version this client cannot represent ({reason}); refusing to verify"
)),
};
println!("Signature verification:"); println!("Signature verification:");
match &verdict { match &verdict {
@@ -178,7 +232,11 @@ async fn cmd_show(
); );
} }
Err(reason) => { Err(reason) => {
println!(" INVALID — {reason}"); // A version refusal means no cryptographic check ran, so it must
// not print as an INVALID signature: operators and log scrapers
// would read that as a failed Ed25519 verification. INVALID is
// reserved for failures out of verify_signature itself.
println!(" {} — {reason}", signature_verdict_label(&parsed_version));
} }
} }
@@ -195,29 +253,39 @@ async fn cmd_show(
}, },
Err(e) => Err(e.to_string()), Err(e) => Err(e.to_string()),
}; };
let current_node_did = current.as_ref().ok().cloned();
for line in did_check_report(&current, node_did) { for line in did_check_report(&current, node_did) {
println!("{line}"); println!("{line}");
} }
if require_valid { if require_valid {
if let Err(reason) = verdict { if let Err(reason) = verdict {
anyhow::bail!("certificate signature did not verify: {reason}"); // Round 3: do not claim the signature failed when it was
// never checked. A version refusal means the payload
// shape is unknown, so no signature verification ran.
anyhow::bail!("certificate could not be verified: {reason}");
} }
// A valid signature proves internal consistency only: the payload was // A valid signature proves internal consistency only: the payload was
// signed by whatever key the certificate itself names. A hostile // signed by whatever key the certificate itself names. A hostile
// source can mint a keypair, put its DID in node_did, and self-sign. // source can mint a keypair, put its DID in node_did, and self-sign.
// --verify therefore also anchors the issuer to a trusted DID: // --verify therefore also anchors the issuer — and the anchor must be
// --expect-node when given, else the DID of the node being queried. // a DID the CALLER trusts, which is exactly what --expect-node is.
let expected = expect_node.as_deref().or(current_node_did.as_deref()); // The DID the queried node reports at `/` is NOT an anchor: the same
match expected { // node that served a forged certificate can serve a matching
// self-report, and with that fallback a wholly forged cert printed
// VALID and exited 0 (review round 2, demonstrated by execution). So
// --verify without --expect-node refuses to claim a trusted issuer at
// all, rather than laundering the server's self-assertion into one.
match expect_node.as_deref() {
Some(expected) if expected == node_did => {} Some(expected) if expected == node_did => {}
Some(expected) => anyhow::bail!( Some(expected) => anyhow::bail!(
"certificate is signed by {node_did}, but the expected issuer is {expected} — \ "certificate is signed by {node_did}, but the expected issuer is {expected} — \
a valid signature alone proves internal consistency, not a trusted issuer" a valid signature alone proves internal consistency, not a trusted issuer"
), ),
None => anyhow::bail!( None => anyhow::bail!(
"cannot anchor the issuer: node info is unreachable and no --expect-node was given" "--verify needs --expect-node <did> to anchor the issuer: the signature is \
valid, but it only proves the cert is self-consistent with the DID it \
names ({node_did}); the queried node's self-reported DID is not a trust \
anchor"
), ),
} }
} }
@@ -265,6 +333,18 @@ fn did_check_report(current: &std::result::Result<String, String>, node_did: &st
} }
} }
/// The stdout label for a failed verification verdict. A `parsed_version`
/// error means `verify_signature` never ran, so the refusal must print as
/// UNSUPPORTED VERSION rather than INVALID, which operators and log
/// scrapers would read as a failed Ed25519 check.
fn signature_verdict_label(parsed_version: &std::result::Result<u32, String>) -> &'static str {
if parsed_version.is_err() {
"UNSUPPORTED VERSION"
} else {
"INVALID"
}
}
/// Rebuild the node's canonical signing payload (field order must match /// Rebuild the node's canonical signing payload (field order must match
/// gitlawb-node/src/cert.rs::issue_ref_certificate exactly) and verify the /// gitlawb-node/src/cert.rs::issue_ref_certificate exactly) and verify the
/// certificate's Ed25519 signature against the key embedded in `node_did`. /// certificate's Ed25519 signature against the key embedded in `node_did`.
@@ -311,6 +391,65 @@ fn verify_signature(
.map_err(|_| "Ed25519 signature does not match the signed payload".to_string()) .map_err(|_| "Ed25519 signature does not match the signed payload".to_string())
} }
/// Parse the wire-format `version` field of a certificate response.
///
/// Semantics (Reviewer 2):
/// - MISSING key (`None`) → `Ok(1)`. An old server predates the field, so
/// the cert is by definition the v1 (pre-versioning) shape; the v1
/// verify path is the only path that can possibly match the bytes.
/// - PRESENT key must be a JSON integer that fits losslessly in `u32`
/// and equals a version this client supports. `null`, a string,
/// a float, an overflow integer, or any version other than 1 all
/// yield `Err`. Collapsing any of these onto v1 would let a
/// well-signed v1 signature pass `--verify` against a response
/// that explicitly declared a format this client cannot represent
/// — which is exactly the mismatch the version field exists to
/// prevent.
///
/// The supported-version set is hard-coded to `{1}` because PR 3 ships
/// only v1 verification; bump this set (and the verdict branch) when
/// v2 verification lands.
fn parse_cert_version(value: Option<&Value>) -> Result<u32, String> {
let v = match value {
None => return Ok(1),
Some(v) => v,
};
// serde_json::Value::as_u64 rejects strings, floats, booleans,
// nulls, arrays, and objects — but it silently truncates floats
// that are whole numbers (e.g. 2.0 → 2). We must reject floats
// explicitly so a server cannot smuggle a v2 cert through the
// float shape.
if v.is_f64() {
return Err(format!(
"version is a JSON number with a fractional part ({v}); expected an integer"
));
}
let n = v.as_u64().ok_or_else(|| {
// null / string / bool / array / object — anything that is
// not a JSON integer.
format!("version is {v}; expected a JSON integer")
})?;
// Lossy narrowing: refuse anything that does not fit in u32.
// u32::MAX is 4_294_967_295; serde_json only goes up to u64.
if n > u32::MAX as u64 {
return Err(format!(
"version {n} does not fit in u32 (max {})",
u32::MAX
));
}
let n = n as u32;
if n != 1 {
return Err(format!(
"this client supports cert version 1 only; server returned {n}"
));
}
Ok(1)
}
async fn resolve_cert_id(client: &NodeClient, owner: &str, name: &str, id: &str) -> Result<String> { async fn resolve_cert_id(client: &NodeClient, owner: &str, name: &str, id: &str) -> Result<String> {
if id.len() >= 36 { if id.len() >= 36 {
return Ok(id.to_string()); return Ok(id.to_string());
@@ -808,7 +947,15 @@ mod tests {
let mut server = mockito::Server::new_async().await; let mut server = mockito::Server::new_async().await;
let (cert, _root) = cert_server(&mut server, &id, &body, Some(&info)).await; let (cert, _root) = cert_server(&mut server, &id, &body, Some(&info)).await;
let got = cmd_show("alice/r".to_string(), id, server.url(), None, true, None).await; let got = cmd_show(
"alice/r".to_string(),
id,
server.url(),
None,
true,
Some(kp.did().as_str().to_string()),
)
.await;
assert!(got.is_ok(), "valid sig + matching issuer must pass"); assert!(got.is_ok(), "valid sig + matching issuer must pass");
cert.assert_async().await; cert.assert_async().await;
} }
@@ -824,10 +971,17 @@ mod tests {
let mut server = mockito::Server::new_async().await; let mut server = mockito::Server::new_async().await;
let (cert, _root) = cert_server(&mut server, &id, &body, Some(&info)).await; let (cert, _root) = cert_server(&mut server, &id, &body, Some(&info)).await;
let err = cmd_show("alice/r".to_string(), id, server.url(), None, true, None) let err = cmd_show(
.await "alice/r".to_string(),
.unwrap_err() id,
.to_string(); server.url(),
None,
true,
Some(other.did().as_str().to_string()),
)
.await
.unwrap_err()
.to_string();
assert!(err.contains("expected issuer"), "got: {err}"); assert!(err.contains("expected issuer"), "got: {err}");
cert.assert_async().await; cert.assert_async().await;
} }
@@ -845,7 +999,7 @@ mod tests {
.await .await
.unwrap_err() .unwrap_err()
.to_string(); .to_string();
assert!(err.contains("cannot anchor the issuer"), "got: {err}"); assert!(err.contains("--expect-node"), "got: {err}");
cert.assert_async().await; cert.assert_async().await;
} }
@@ -886,7 +1040,7 @@ mod tests {
.await .await
.unwrap_err() .unwrap_err()
.to_string(); .to_string();
assert!(err.contains("signature did not verify"), "got: {err}"); assert!(err.contains("does not match"), "got: {err}");
cert.assert_async().await; cert.assert_async().await;
} }
@@ -970,4 +1124,373 @@ mod tests {
assert!(report.contains("verify offline"), "got: {report}"); assert!(report.contains("verify offline"), "got: {report}");
assert!(!report.contains("WARNING"), "got: {report}"); assert!(!report.contains("WARNING"), "got: {report}");
} }
/// #26 Split PR 3: a missing `version` field on a cert defaults
/// to 1, so an old server's response is forward-compatible with
/// a new client. The v1 verify path is taken, and a
/// well-signed v1 cert round-trips through `verify_signature`.
///
/// Reviewer 1 finding: the previous shape only parsed the field
/// and never called `verify_signature`, so the test could not
/// fail if the verdict branch was wired to the wrong path. This
/// version signs and verifies the full round-trip.
#[test]
fn missing_version_defaults_to_1_and_verifies() {
let kp = gitlawb_core::identity::Keypair::generate();
let node_did = kp.did().as_str().to_string();
let payload = serde_json::json!({
"repo_id": "repo-1",
"ref": "refs/heads/main",
"old": "0".repeat(40),
"new": "a".repeat(40),
"pusher": "did:key:z6MkPusher",
"node": node_did,
"ts": "2026-07-22T00:00:00+00:00",
});
let sig = kp.sign_b64(&serde_json::to_vec(&payload).unwrap());
// Simulate the JSON a v1 server returns: no `version` key.
let cert_json = serde_json::json!({
"id": "test-id",
"repo_id": "repo-1",
"ref_name": "refs/heads/main",
"old_sha": "0".repeat(40),
"new_sha": "a".repeat(40),
"pusher_did": "did:key:z6MkPusher",
"node_did": node_did,
"signature": sig.clone(),
"issued_at": "2026-07-22T00:00:00+00:00",
// no `version` field
});
// Forward-compat: the parse yields v1 (the only version this
// client verifies), and the verdict arm matching Ok(1) feeds
// the v1 payload through verify_signature end to end.
assert_eq!(parse_cert_version(cert_json.get("version")).unwrap(), 1);
let verdict = verify_signature(
"repo-1",
"refs/heads/main",
&"0".repeat(40),
&"a".repeat(40),
"did:key:z6MkPusher",
&node_did,
"2026-07-22T00:00:00+00:00",
&sig,
);
assert!(
verdict.is_ok(),
"missing version must take the v1 verify path: {verdict:?}"
);
}
/// #26 Split PR 3: an explicit `version: 1` on a cert is the v1
/// verify path. A v1 cert with an explicit version verifies the
/// same as a v1 cert without one — round-trip through
/// `verify_signature`.
#[test]
fn explicit_version_1_takes_v1_path() {
let kp = gitlawb_core::identity::Keypair::generate();
let node_did = kp.did().as_str().to_string();
let payload = serde_json::json!({
"repo_id": "repo-1",
"ref": "refs/heads/main",
"old": "0".repeat(40),
"new": "a".repeat(40),
"pusher": "did:key:z6MkPusher",
"node": node_did,
"ts": "2026-07-22T00:00:00+00:00",
});
let sig = kp.sign_b64(&serde_json::to_vec(&payload).unwrap());
// Simulate a v1 cert with an explicit `version: 1` field.
let cert_json = serde_json::json!({
"id": "test-id",
"repo_id": "repo-1",
"ref_name": "refs/heads/main",
"old_sha": "0".repeat(40),
"new_sha": "a".repeat(40),
"pusher_did": "did:key:z6MkPusher",
"node_did": node_did,
"signature": sig.clone(),
"issued_at": "2026-07-22T00:00:00+00:00",
"version": 1,
});
// The parse arms the Ok(1) verdict branch, which calls
// verify_signature on the v1 payload. A signature mismatch
// here would mean the verdict branch was wired to the
// wrong path or the payload drifted.
assert_eq!(parse_cert_version(cert_json.get("version")).unwrap(), 1);
let verdict = verify_signature(
"repo-1",
"refs/heads/main",
&"0".repeat(40),
&"a".repeat(40),
"did:key:z6MkPusher",
&node_did,
"2026-07-22T00:00:00+00:00",
&sig,
);
assert!(
verdict.is_ok(),
"explicit version: 1 must take the v1 verify path: {verdict:?}"
);
}
/// #26 Split PR 3 + Reviewer 2: parse_cert_version is the
/// load-bearing gate that distinguishes a missing-key (legacy
/// server) from a present-but-malformed value. The four cases
/// below pin every cell of that truth table; collapsing any
/// pair would let a server advertise "version 2" (or 99, or
/// 4294967297, or "two") while the client runs the v1
/// signature path on a different signed payload — a hostile
/// misconfiguration that `--verify` must not silently accept.
#[test]
fn parse_cert_version_truth_table() {
// MISSING key → legacy v1 (the v1 path is the only one that
// could match pre-versioning bytes).
assert_eq!(
parse_cert_version(None).unwrap(),
1,
"a missing version field is the legacy v1 path"
);
// Explicit 1 → v1.
assert_eq!(
parse_cert_version(Some(&serde_json::json!(1))).unwrap(),
1,
"explicit version 1 is the v1 path"
);
// Explicit 2 → Err, NOT Ok(1). Reviewer 2: a v1 signature
// must not verify against a server that said "version 2".
assert!(
parse_cert_version(Some(&serde_json::json!(2))).is_err(),
"explicit version 2 must not collapse to v1"
);
// u32 overflow (2^32 + 1) → Err, NOT Ok(1). The previous
// `.map(|v| v as u32)` silently truncated this to 1.
let overflow = serde_json::json!(u64::from(u32::MAX) + 1);
assert!(
parse_cert_version(Some(&overflow)).is_err(),
"a version that overflows u32 must not truncate to 1"
);
// Non-numeric: a string, a float, null, bool — every shape
// that is not a JSON integer is rejected. None of these
// may collapse to v1.
for bad in [
serde_json::json!("two"),
serde_json::json!(2.0), // serde_json::Value::is_f64 is true for this
serde_json::json!(2.5), // obviously fractional
serde_json::json!(true),
serde_json::json!(null),
serde_json::json!([2]),
serde_json::json!({"v": 2}),
] {
assert!(
parse_cert_version(Some(&bad)).is_err(),
"non-integer version {bad} must not collapse to v1"
);
}
}
/// A version refusal must not print as INVALID — no cryptographic
/// check ran, so labelling it INVALID would tell operators an Ed25519
/// verification failed when the payload shape was the disqualifier.
/// Removing the label branch fails this test.
#[test]
fn signature_verdict_label_distinguishes_refusal_from_bad_signature() {
assert_eq!(
signature_verdict_label(&Err("version 2 not supported".to_string())),
"UNSUPPORTED VERSION"
);
assert_eq!(signature_verdict_label(&Ok(1)), "INVALID");
}
/// #26 Split PR 3, round 2: a `version: 2` certificate must fail
/// `--verify` THROUGH `cmd_show` itself, not through a copy of its
/// match. The prior form of this test duplicated the verdict match in
/// the test body and aimed at an `Ok(v != 1)` arm the real parser can
/// never produce, so gutting `cmd_show`'s rejection kept every test
/// green (demonstrated by the reviewer by execution). Driving the
/// command end to end is what makes this a guard: the mock serves a
/// cert whose SIGNATURE IS VALID for v1 bytes but which declares
/// version 2, and the command must exit Err with the version named —
/// the v1 verify path must not launder it through.
#[tokio::test]
async fn cmd_show_verify_rejects_v2_even_with_valid_v1_signature() {
let kp = gitlawb_core::identity::Keypair::generate();
let node_did = kp.did().as_str().to_string();
// Sign a v1-shaped payload so the ONLY disqualifier is the version.
let payload = serde_json::json!({
"repo_id": "repo-1",
"ref": "refs/heads/main",
"old": "0".repeat(40),
"new": "a".repeat(40),
"pusher": "did:key:z6MkPusher",
"node": node_did,
"ts": "2026-07-22T00:00:00+00:00",
});
let sig = kp.sign_b64(&serde_json::to_vec(&payload).unwrap());
let cert_json = serde_json::json!({
"id": "0123456789abcdef0123456789abcdef0123",
"repo_id": "repo-1",
"ref_name": "refs/heads/main",
"old_sha": "0".repeat(40),
"new_sha": "a".repeat(40),
"pusher_did": "did:key:z6MkPusher",
"node_did": node_did,
"signature": sig,
"issued_at": "2026-07-22T00:00:00+00:00",
"version": 2,
});
let mut server = mockito::Server::new_async().await;
// 36-char id short-circuits resolve_cert_id; "owner/repo" input
// short-circuits resolve_repo — the cert GET is the only required
// route. `/` may be probed for the contextual DID line; serving it
// keeps the test about the verdict, not about network noise.
let _cert = server
.mock(
"GET",
"/api/v1/repos/o/r/certs/0123456789abcdef0123456789abcdef0123",
)
.with_status(200)
.with_header("content-type", "application/json")
.with_body(cert_json.to_string())
.create_async()
.await;
let _root = server
.mock("GET", "/")
.with_status(200)
.with_header("content-type", "application/json")
.with_body(serde_json::json!({ "did": node_did }).to_string())
.create_async()
.await;
let err = cmd_show(
"o/r".to_string(),
"0123456789abcdef0123456789abcdef0123".to_string(),
server.url(),
None,
true,
Some(node_did.clone()),
)
.await
.expect_err("a version-2 cert must fail --verify even with a valid v1 signature");
let msg = format!("{err:#}");
assert!(
msg.contains("version"),
"the failure must name the version mismatch, not a generic invalid signature: {msg}"
);
}
/// Round 2's second demonstrated escape: with no --expect-node, the old
/// fallback anchored the issuer to the DID the QUERIED NODE reports at
/// `/` — which a forger controls along with the cert, so a wholly forged
/// self-signed certificate passed --verify with exit 0. `cmd_show` with
/// --verify and no --expect-node must now refuse to claim a trusted
/// issuer, even when the served cert's signature is valid and the node's
/// self-report matches it.
#[tokio::test]
async fn cmd_show_verify_without_expect_node_refuses_self_asserted_anchor() {
let kp = gitlawb_core::identity::Keypair::generate();
let node_did = kp.did().as_str().to_string();
let payload = serde_json::json!({
"repo_id": "repo-1",
"ref": "refs/heads/main",
"old": "0".repeat(40),
"new": "a".repeat(40),
"pusher": "did:key:z6MkPusher",
"node": node_did,
"ts": "2026-07-22T00:00:00+00:00",
});
let sig = kp.sign_b64(&serde_json::to_vec(&payload).unwrap());
let cert_json = serde_json::json!({
"id": "0123456789abcdef0123456789abcdef0123",
"repo_id": "repo-1",
"ref_name": "refs/heads/main",
"old_sha": "0".repeat(40),
"new_sha": "a".repeat(40),
"pusher_did": "did:key:z6MkPusher",
"node_did": node_did,
"signature": sig,
"issued_at": "2026-07-22T00:00:00+00:00",
});
let mut server = mockito::Server::new_async().await;
// P3 (reviewer round 2): both mocks are required to be
// hit for the test to mean anything. `mockito 1.7`'s
// `assert_on_drop` is off by default, so the previous
// `_cert` and `_root` bindings would silently let a
// regression that skipped the root fetch pass.
// `.expect(2)` pins each mock's call count (the cert mock
// is hit twice because the test runs `cmd_show` twice) and
// `.assert_async()` at the end fires if any are unmet.
let cert_mock = server
.mock(
"GET",
"/api/v1/repos/o/r/certs/0123456789abcdef0123456789abcdef0123",
)
.with_status(200)
.with_header("content-type", "application/json")
.with_body(cert_json.to_string())
.expect(2)
.create_async()
.await;
// The forger's matching self-report — the exact shape
// that used to convert into a trust anchor. The guard's
// whole point is to make the test FAIL if a future
// regression stops calling `/` here: `.expect(2)` +
// `.assert_async()` makes that dependency visible.
let root_mock = server
.mock("GET", "/")
.with_status(200)
.with_header("content-type", "application/json")
.with_body(serde_json::json!({ "did": node_did }).to_string())
.expect(2)
.create_async()
.await;
let err = cmd_show(
"o/r".to_string(),
"0123456789abcdef0123456789abcdef0123".to_string(),
server.url(),
None,
true,
None,
)
.await
.expect_err("--verify without --expect-node must not mint a trust anchor from the node's self-report");
let msg = format!("{err:#}");
assert!(
msg.contains("--expect-node"),
"the refusal must tell the caller how to anchor trust: {msg}"
);
// Positive control: the same cert with the DID explicitly trusted
// passes, so the guard is about the anchor, not the signature.
cmd_show(
"o/r".to_string(),
"0123456789abcdef0123456789abcdef0123".to_string(),
server.url(),
None,
true,
Some(node_did),
)
.await
.expect("explicitly anchored --verify of a valid v1 cert must pass");
// P3 (reviewer round 2): confirm the mocks were actually
// hit. Without this, a regression that skipped the cert or
// root fetches would still produce a passing test (the
// `expect_err` arm succeeds, the positive control succeeds)
// and the test would mean nothing.
cert_mock.assert_async().await;
root_mock.assert_async().await;
}
} }
+2 -2
View File
@@ -75,14 +75,14 @@ pub async fn run(args: ChangelogArgs) -> Result<()> {
for event in &events { for event in &events {
let ts = event["timestamp"].as_str().unwrap_or("?"); let ts = event["timestamp"].as_str().unwrap_or("?");
let date = &ts[..ts.len().min(10)]; let date = crate::text::truncate(ts, 10);
match event["type"].as_str().unwrap_or("") { match event["type"].as_str().unwrap_or("") {
"commit" => { "commit" => {
let sha = event["sha"].as_str().unwrap_or("?"); let sha = event["sha"].as_str().unwrap_or("?");
let msg = event["message"].as_str().unwrap_or("?"); let msg = event["message"].as_str().unwrap_or("?");
let first_line = msg.lines().next().unwrap_or(msg); let first_line = msg.lines().next().unwrap_or(msg);
let short_sha = &sha[..sha.len().min(8)]; let short_sha = crate::text::truncate(sha, 8);
println!(" {date} commit {short_sha} {first_line}"); println!(" {date} commit {short_sha} {first_line}");
} }
"pr_merged" => { "pr_merged" => {
+120 -7
View File
@@ -163,8 +163,8 @@ pub fn setup_partial_clone(
if withheld_globs.is_empty() { if withheld_globs.is_empty() {
match branch { match branch {
Some(b) => git_global(&["clone", "-q", "--branch", b, remote_url, dest_str])?, Some(b) => git_global(&["clone", "-q", "--branch", b, "--", remote_url, dest_str])?,
None => git_global(&["clone", "-q", remote_url, dest_str])?, None => git_global(&["clone", "-q", "--", remote_url, dest_str])?,
} }
return Ok(()); return Ok(());
} }
@@ -174,6 +174,7 @@ pub fn setup_partial_clone(
"-q", "-q",
"--filter=blob:none", "--filter=blob:none",
"--no-checkout", "--no-checkout",
"--",
remote_url, remote_url,
dest_str, dest_str,
])?; ])?;
@@ -266,7 +267,31 @@ async fn fetch_withheld(node: &str, owner: &str, name: &str) -> Result<(Vec<Stri
}; };
let resp = match resp { let resp = match resp {
Ok(r) if r.status().is_success() => r, Ok(r) if r.status().is_success() => r,
Ok(r) if matches!(r.status().as_u16(), 404 | 501) => return Ok((Vec::new(), Vec::new())), Ok(r) if matches!(r.status().as_u16(), 404 | 501) => {
// Two 404s share this status and must be told apart before the
// legacy empty-glob fallback is taken: a route miss on a node that
// predates withheld-paths has an empty body, while a current node
// returns the JSON AppError body for absent/denied/quarantined
// repos. The JSON body proves the route exists, so the repo is
// unavailable to this caller and cloning cannot proceed.
if r.status().as_u16() == 404 {
let status = r.status();
let body = crate::http::read_body_capped(r, 4096).await;
if body.read_failed {
// The read died part-way: we cannot classify the denial,
// and a failed read must not collapse into the legacy
// route-miss arm and start a clone.
bail!("withheld-paths 404 body from this node could not be read");
}
if !body.text.trim().is_empty() {
bail!(
"repository {owner}/{name} is not available on this node ({status}): {}",
sanitize_node_msg(&body.text)
);
}
}
return Ok((Vec::new(), Vec::new()));
}
Ok(r) => bail!("withheld-paths lookup failed: {}", r.status()), Ok(r) => bail!("withheld-paths lookup failed: {}", r.status()),
Err(err) => return Err(err).context("fetching withheld paths"), Err(err) => return Err(err).context("fetching withheld paths"),
}; };
@@ -326,7 +351,12 @@ async fn recover_encrypted_blobs(
.await .await
{ {
Ok(r) if r.status().is_success() => r, Ok(r) if r.status().is_success() => r,
_ => return Ok(vec![]), // Discovery failure is not an empty inventory: a denied, quarantined,
// erroring, or unreachable node must surface so the caller can warn
// and still run the Arweave fallback rather than report a clean clone
// with authorized blobs silently missing.
Ok(r) => bail!("encrypted-blobs inventory unavailable: {}", r.status()),
Err(err) => return Err(err).context("fetching encrypted-blobs inventory"),
}; };
let body: EncryptedBlobsResponse = resp.json().await.context("parsing encrypted-blobs")?; let body: EncryptedBlobsResponse = resp.json().await.context("parsing encrypted-blobs")?;
if body.blobs.is_empty() { if body.blobs.is_empty() {
@@ -388,7 +418,7 @@ async fn recover_encrypted_blobs(
let plaintext = match open_blob(&envelope, keypair) { let plaintext = match open_blob(&envelope, keypair) {
Ok(p) => p, Ok(p) => p,
Err(e) => { Err(e) => {
eprintln!("warning: could not decrypt {oid}: {e}"); emit_warning(&format!("warning: could not decrypt {oid}: {e}"));
continue; continue;
} }
}; };
@@ -406,7 +436,9 @@ async fn recover_encrypted_blobs(
recovered.push(p.clone()); recovered.push(p.clone());
} }
} else { } else {
eprintln!("warning: recovered blob {oid} hashed to {written}; discarding"); emit_warning(&format!(
"warning: recovered blob {oid} hashed to {written}; discarding"
));
} }
} }
Ok(recovered) Ok(recovered)
@@ -804,7 +836,9 @@ async fn recover_from_arweave(
recovered.push(p.clone()); recovered.push(p.clone());
} }
} else { } else {
eprintln!("warning: recovered blob {oid} hashed to {written}; discarding"); emit_warning(&format!(
"warning: recovered blob {oid} hashed to {written}; discarding"
));
} }
} }
Ok(recovered) Ok(recovered)
@@ -1090,6 +1124,85 @@ mod tests {
); );
} }
#[test]
fn setup_partial_clone_plain_without_withheld_paths() {
let (td, url) = bare_remote(&[("file.txt", b"hello\n")]);
let dest = td.path().join("dest");
setup_partial_clone(&dest, &url, &[], &[], None).unwrap();
assert!(dest.join("file.txt").exists());
}
#[test]
fn setup_partial_clone_with_branch_without_withheld_paths() {
let (td, url) = bare_remote(&[("file.txt", b"hello\n")]);
let branch_out = Command::new("git")
.args([
"-C",
td.path().join("origin").to_str().unwrap(),
"branch",
"--show-current",
])
.output()
.unwrap();
let branch = String::from_utf8(branch_out.stdout)
.unwrap()
.trim()
.to_string();
let dest = td.path().join("dest");
setup_partial_clone(&dest, &url, &[], &[], Some(&branch)).unwrap();
assert!(dest.join("file.txt").exists());
}
/// #374: every `setup_partial_clone` shape interpolates `remote_url` into a
/// `git clone` argv. The `--` delimiter is what forces an option-shaped
/// remote to be read as a repository rather than parsed as an option.
///
/// The assertion is on git's own words — `repository '<value>'`, single-quoted
/// — and not merely on the value appearing somewhere in the error. `git_global`
/// formats its failure as `git {args:?} failed: {stderr}`, so the injected
/// string is already present in the debug-printed argv whether or not the
/// delimiter is there; asserting on containment alone would pass with `--`
/// deleted. Undelimited, git consumes the value as an option, leaving the
/// destination as the only positional, and names *that* instead
/// (`repository '<dest>' does not exist`) — so this goes red without the fix.
///
/// The payload carries no `:` or space: a colon would make git read the
/// argument as an ssh-style `host:path` URL and report a hostname, which is a
/// different parse.
fn assert_option_shaped_remote_is_a_repository(withheld: &[String], branch: Option<&str>) {
let td = TempDir::new().unwrap();
let dest = td.path().join("dest");
let injected = "--upload-pack=false";
let err = setup_partial_clone(&dest, injected, withheld, &[], branch)
.expect_err("an option-shaped remote is not a repository and must fail")
.to_string();
assert!(
err.contains(&format!("repository '{injected}'")),
"git must name the argument as the repository, not consume it as an option: {err}"
);
}
#[test]
fn plain_clone_never_parses_a_remote_as_a_git_option() {
assert_option_shaped_remote_is_a_repository(&[], None);
}
#[test]
fn branch_clone_never_parses_a_remote_as_a_git_option() {
// Pins that `--branch <b>` stays ahead of the delimiter: if `--` were moved
// before `--branch`, git would read the branch name as a positional instead.
assert_option_shaped_remote_is_a_repository(&[], Some("main"));
}
#[test]
fn sparse_clone_never_parses_a_remote_as_a_git_option() {
// The withheld-globs arm is a separate argv construction
// (`--filter=blob:none --no-checkout`) and needs its own proof.
assert_option_shaped_remote_is_a_repository(&["/secret/**".to_string()], None);
}
#[test] #[test]
fn sparse_patterns_subtree_and_exact() { fn sparse_patterns_subtree_and_exact() {
assert_eq!(sparse_patterns("/secret/**"), vec!["/secret/".to_string()]); assert_eq!(sparse_patterns("/secret/**"), vec!["/secret/".to_string()]);
Loaded 100 of 130 files, more files were not shown because too many files have changed in this diff. Show more