Compare commits

..
Author SHA1 Message Date
beardthelion e6f4b2fbac fix(node): close review residuals: coalesce cold downloads to one in-flight task, compensate a timed-out upload with the rolled-back tree, aggregate partial ref restores, race-test the anchor lock, fail-closed snapshot test 2026-07-22 17:29:06 -05:00
beardthelion d7551807e6 fix(node): slug-lock arweave anchor writes; fail closed when the pre-push ref snapshot fails 2026-07-22 16:55:28 -05:00
beardthelion 4329b2aa87 Merge origin/main into feat/write-sink-brake-coverage (reintegrate current base) 2026-07-22 16:35:07 -05:00
beardthelion ce25bba091 refactor(node): delegate issue ref helpers to the store ref primitives 2026-07-22 16:21:27 -05:00
beardthelion 1e79ae59a5 fix(node): own cold-download cleanup in a spawned task so handler cancellation cannot resurrect temp dirs 2026-07-22 16:21:27 -05:00
beardthelion 296704f7fa fix(node): slug-lock every peer-ingest writer and drive the real ingest in the purge-race test 2026-07-22 16:21:27 -05:00
beardthelion 9152178109 fix(node): never let a failed ref snapshot become a delete-all rollback; cover merge and push rollbacks 2026-07-22 16:21:27 -05:00
beardthelion 03cd81ee2b chore(node): fmt + clippy cleanup for the #196 fixes
Auto-deref (&mut tx) on the lock_repo_slug calls and rustfmt of the new test
lines. No behavior change.
2026-07-22 12:52:14 -05:00
beardthelion 1d0d3d2dca fix(node): tie cancelled cold-download cleanup to blocking extraction
On a download timeout the TempDownloadDir guard dropped while the detached
spawn_blocking extraction kept running and could rename its output back into
the just-removed temp path; that dir was only reclaimed on a later same-repo
request, so cold reads across distinct repos accumulated extracted repos on
disk. The timeout arm now hands the owned download future to a detached janitor
that awaits extraction completion before removing the temp dir, off the
critical path so waiters are still freed immediately. (F3, #196)
2026-07-22 12:44:17 -05:00
beardthelion 1d8c28113a fix(node): roll back local ref on failed durable upload (close/merge/receive-pack)
close_issue, merge_pr, and the receive-pack task applied a local ref write then
released with an empty failure cleanup, so a failed Tigris upload left stale
state that the local-fast-path read served despite the client getting a 5xx.
Each now captures the prior ref and restores it via release_with_failure_cleanup
when the upload fails, matching create_issue. (F2, #196)
2026-07-22 12:29:44 -05:00
beardthelion 556aa31e2c fix(node): serialize repo purge cascade with mirror ingest via advisory lock
delete_repo_by_id ran a point-in-time sibling check then cascaded slug-scoped
deletes; upsert_mirror_repo (peer sync) did not serialize against that window,
so a colliding mirror could land mid-purge and get half-wiped. Both now take a
transaction-scoped pg_advisory_xact_lock keyed on the normalized slug. (F1, #196)
2026-07-22 12:18:12 -05:00
beardthelion a12f21adb4 fix(node): roll back the fork mirror and archive on late clone/insert failures (#196)
The bounded-clone rework cleaned up the mirror on the timeout and spawn
arms but not on a clone that creates the destination then exits nonzero,
so a retry hit 'destination already exists' and could never complete
that fork name. And because upload_under_guard is now required and runs
before the row insert, a create_repo failure reliably orphaned the
durable archive too, not just the local dir. Both failure arms now
remove the mirror, delete the fork's archive (no-op on a store-less
node), release the lock, and return. RED observed on both new tests (a
non-empty-dest clone-128 repro and a decoy-row insert-failure repro).
2026-07-21 07:47:18 -05:00
beardthelion 4408d27058 fix(node): bound acquire_write's download and free the download-map entry on cancel (#196)
Two gaps a second-model adversarial pass caught that the in-family
review missed. acquire_write's under-lock exists()/download() ran with
no timeout, so a stalled Tigris GET pinned the write lock and its
lock-pool connection forever (the local-copy fallback only fires on an
error, never a hang) - both are now wrapped in
timeout(release_upload_timeout), a timeout mapping to the existing
download-error fallback, which also makes the prior 'bound every
under-lock call' claim true. The download_locks map entry was removed
only on normal return, not on a cancelled cold read, so a disconnecting
caller could grow the map across public repos; a DownloadEntryGuard now
prunes it in Drop (outer map moved to std::sync::Mutex, held only for
map ops, never across an await). RED observed on both new tests,
including a real mid-download cancellation.
2026-07-21 07:47:18 -05:00
beardthelion 03ef9b51bd refactor(node): move read-path download coordination into a repo_store submodule (#196)
repo_store.rs had grown past 2800 lines. The download-coordination code
(DownloadOutcome, TempDownloadDir, download_published,
download_and_publish, remove_download_entry) moves to a private
repo_store/download.rs child module, which reaches the parent's private
fields and methods without widening visibility beyond pub(super) on the
two items the parent names. Pure move: no behavior change, the same 44
git::repo_store tests pass unchanged.
2026-07-21 00:35:28 -05:00
beardthelion c3d6e73e50 fix(node): keep the fork's unbounded work out of the target-lock span (#196)
The fork tail held the target-namespace advisory lock across two
unbounded operations. The source acquire, which on a cold source
triggers a full object-store download plus a nested lock-pool acquire on
the same writer-sized pool, now runs before the target lock is taken (it
is read-only on a different namespace and nothing under the lock depends
on it) so a cold-source fork can no longer demand two connections from a
one-per-writer pool at once. The git clone --mirror now runs via
tokio::process with kill_on_drop under a timeout(git_service_timeout_secs)
ceiling, cleaning the partial mirror and releasing the lock on elapse.
RED observed by executing the pool-size-1 cold-source fork with the
source acquire moved back under the lock: PoolTimedOut, source never
publishes, clone fails.
2026-07-21 00:26:47 -05:00
beardthelion fa4322cb2e fix(node): harden the read-path download against cancellation, stale swaps, and lock contention (#196)
Four read-path gaps the review surfaced. A TempDownloadDir RAII guard
now cleans the temp download dir if the handler future is cancelled on
client disconnect (the read handlers have no spawn shield, and a
post-download disconnect leaked the full populated dir), plus an
entry-time sweep of stale siblings. A cold or refresh reader that
downloaded while a concurrent writer published fresher on-disk state now
discards its stale temp and serves the writer's copy instead of swapping
over it (detected by presence for the cold path, mtime for the refresh
path). The takeover path re-registers its download-mutex entry so late
readers coalesce instead of starting a duplicate download. And the
try_lock Ok(None) degrade arm - the KTD-3 invariant that a reader
degrades rather than resurrecting under a live writer/purge - is now
tested by holding the advisory lock via an out-of-pool observer. RED
observed on both stale-swap tests.
2026-07-21 00:13:35 -05:00
beardthelion b2ca5a7b2d fix(node): bound every under-lock object-store call and dedup the upload path (#196)
Round-two review found the durability fixes reintroduced the
unbounded-work-under-lock class they were meant to close: upload_locked's
PUT (now reachable under the lock via init's wait=true), the read-path
download, and the under-lock exists() re-check all ran with no timeout
while pinning the advisory lock and its lock-pool connection. All three
now wrap the call in timeout(release_upload_timeout) and fail closed on
elapse, matching RepoWriteGuard::release's existing INV-22 pattern.
upload_under_guard and upload_locked's duplicated resolve/recheck/upload
body is extracted into upload_dir_locked, threading the load-bearing
dir-gone signal (fork needs success, lazy migration needs
skip-and-retry) through a parameter rather than aligning it. RED
observed: upload_locked_put_stall_is_bounded failed (lock never freed)
against the un-timeout'd PUT.
2026-07-20 23:55:24 -05:00
beardthelion 2ad1c793ab fix(node): read-path downloads publish under the purge lock, without pinning it across the network (#196)
acquire's cache-miss and every acquire_fresh refresh now coalesce on a
per-repo download mutex, download to a temp sibling with no advisory
lock held, then try-lock, re-check the archive exists under the lock,
and swap - so an in-flight read can no longer resurrect a repository
purge just deleted (RED observed on both the cache-miss and
refresh-over-existing repros), a post-purge reader degrades instead of
publishing, and concurrent cold reads download once. The hit path stays
lock-free (deterministic held-lock fence) and no lock-pool connection is
held while a download is parked.
2026-07-20 23:27:33 -05:00
beardthelion c07435d245 fix(node): fork publishes only after a durable upload, serialized on the target-namespace lock (#196)
fork_repo now takes the per-repo advisory lock before its conflict check
and holds it through clone, upload, and row insert, 503ing on contention
exactly as create_repo does. The upload runs under the held guard via
upload_under_guard, PUT bounded by release_upload_timeout, returning
false only for an attempted upload that failed or timed out (no-store
nodes fork unchanged); on failure the mirror is removed and the fork
500s before any row is built. release_after_write is retired; its tests
migrated onto the new method. RED observed on the serialization,
durability, and contention tests; defect injection (row insert reordered
above the upload) turned the durability test RED and was reverted.
2026-07-20 23:13:06 -05:00
beardthelion 859240b3a3 fix(node): roll back the local issue ref before unlock when the durable upload fails (#196)
The failed-upload 500 previously left refs/gitlawb/issues/{id} on disk,
so a client retry minted a second issue once the store recovered. The
delete now runs via RepoWriteGuard::release_with_failure_cleanup while
the advisory lock is still held, closing the unlock-to-delete window a
concurrent write could have archived the orphan ref through. RED
observed on both new tests (orphan ref survived; retry yielded 2
issues); the retry test pins the store double's exists()=false contract
so it cannot pass via a stale-archive revert.
2026-07-20 23:11:45 -05:00
beardthelion ba572b3b68 fix(node): init's background upload waits out the creator's advisory lock (#196)
create_repo holds the per-repo lock across init, so the wait=false spawn
try-locked against its creator and died permanently, leaving a new repo
with no object-store archive until its first later access. RED observed:
init_upload_waits_out_creators_lock_then_uploads failed (PUT count 0)
against the old code; the bounded-skip negative is retained as
init_upload_gives_up_after_bounded_lock_wait.
2026-07-20 22:50:18 -05:00
beardthelion 16517f85ce refactor(node): route the per-DID 429 through the shared too_many_requests helper 2026-07-20 22:45:35 -05:00
beardthelion 2fa9fa37b3 chore: merge main (iCaptcha PoW + gl retry tests) into feat/write-sink-brake-coverage 2026-07-20 22:41:37 -05:00
beardthelion 5a46ef62cc fix(node): close the durable-upload data-loss class on every write path (#196)
The earlier fix that fails a push when its Tigris upload times out was
wired only into git_receive_pack; the merge, issue-create, and
issue-close write paths still ignored release()'s result and returned
2xx after a failed upload, so the next acquire_write reverted their
change from the stale archive. release() is now #[must_use], which
surfaces every caller: the production upload paths fail with a 5xx on a
false return (so the client retries), and the internal/test callers are
explicitly justified. create_repo's lock-acquire failure now returns a
retryable 503 (new AppError::Unavailable) instead of a 500, matching its
own contract.
2026-07-19 22:16:53 -05:00
beardthelion d2ba14edfe fix(node): correct the create-pool and bounty-tombstone remediations (#196)
A re-run adversarial panel caught three issues in the prior round's own
fixes:

- create_repo took its serialization lock from the app pool and then did
  its DB work on the same pool, so at MAX_CONNECTIONS=1 it self-deadlocked
  (a config finding E requires to work). Reverted to locking on the write
  lock_pool: lock and create-work use separate pools, and the DB-global
  advisory lock still serializes against a purge. The extreme-burst
  lock_pool wait is an accepted retryable-503 residual. Db::pool() is
  test-only again.

- The bounty tombstone matched only one owner form and ran
  unconditionally. It now normalizes BOTH sides (covering did:key-row +
  short-bounty AND bare-row + full-bounty) and is gated on the same
  no-surviving-sibling check as the slug cascade, so it never tombstones
  a bounty a surviving mirror row still serves.

Tests: create-at-1-app-connection (no self-deadlock), bare-row full-form
tombstone, and sibling-survives-keeps-bounty-open, each RED before / GREEN
after.
2026-07-19 21:46:18 -05:00
beardthelion 7dc7be2ecd fix(node): retry-after on the push-advert 429, and create_repo locks on the app pool (#196)
Two review follow-ups:

- The unauthenticated info/refs?service=git-receive-pack limiter still
  returned a bare 429 with no Retry-After while the other limiters
  advertise the window-derived delay; it now uses check_retry and the
  shared too_many_requests helper. Removed the orphaned
  AppError::TooManyRequests variant and gated the now-test-only check().

- create_repo's transient serialization lock drew from the write
  lock_pool, so a burst that pins lock_pool starved repo creation.
  Postgres advisory locks are database-global, so create now takes its
  lock from the app pool (via lock_repo_blocking_on) and still
  serializes against a purge holding the same key from lock_pool.
2026-07-19 21:12:45 -05:00
beardthelion ce5dcc3581 fix(node): fail closed on a symlink at the purge target (#196)
local_refs_on_disk used symlink_metadata only to detect NotFound, then
the HEAD/objects marker checks followed a symlink — so a symlink to a
real empty bare repo outside repos_dir was admitted as an empty
purgeable candidate, and the DB row was deleted before path_within
refused the dir removal. The symlink is now classified present/skipped
before the marker checks. Adds outside-empty-bare and dangling-symlink
tests.
2026-07-19 20:55:34 -05:00
beardthelion d1e596e580 fix(node): tombstone bounties under both owner forms on purge (#196)
The tombstone matched only the repo's full did:key owner, but
create_bounty stores repo_owner verbatim from the URL path — commonly
the bare short form. A short-owner-form bounty therefore stayed open and
re-attached to a recreated repo, the exact hole the tombstone closes for
the full form. The UPDATE now matches repo_owner IN (full, normalized
short) so both forms are tombstoned. Adds short-form + submitted-status
fixtures and asserts dispute cannot reopen a purged bounty.
2026-07-19 20:48:20 -05:00
beardthelion 8f3d643a72 fix(node): fail the push when the durable upload times out, don't 200 (#196)
The bounded release() upload (from the lock-pool fix) abandoned a slow
PUT, freed the lock, and returned no failure signal, so receive-pack
reported 200 while the archive never reached object storage — and the
next acquire_write downloaded the stale pre-push archive over local
disk, reverting the pushed refs. A timed-out or errored durable upload
now returns upload_ok=false; the handler sends a 5xx and skips the
success tail so the client re-pushes idempotently instead of trusting a
lost commit. The lock still frees on every path and the connection-hold
bound is unchanged.
2026-07-19 20:33:20 -05:00
beardthelion 9916bca29b test(node): prove purge deletes at MAX_CONNECTIONS=1 with the split pools (#196)
The split pool from the earlier fix already resolves the one-connection
self-deadlock (the guard holds a lock_pool connection while the delete
runs on the app pool), so this locks that wiring in: a purge at
app-pool MAX_CONNECTIONS=1 deletes the target row. The test is
load-bearing for the wiring - a shared-pool store (guard and delete on
one pool) deadlocks the delete's begin() on the held connection and
deletes nothing. No production change; U2 did the wiring.
2026-07-19 20:14:19 -05:00
beardthelion 3baa9ed4fc fix(node): serialize create_repo against the purge advisory lock (#196)
Purge holds a per-owner/name advisory lock across delete-row +
remove-dir, but create_repo took no synchronous lock, so a concurrent
same-key create slipped into the row-deleted-but-dir-not-removed window
and left a repos row pointing at storage the purge then removed.
create_repo now acquires the same lock (lock_repo_blocking -> the guard
the purge holds, keyed identically) around its existence-check, init,
and insert, releasing on every path. Serializes a same-key create
against a purge; the INV-9 colliding-sibling case stays the accepted
KTD4 residual.
2026-07-19 20:00:13 -05:00
beardthelion c24b5418a8 fix(node): tombstone bounties when their repo is purged (#196)
Bounties are keyed only on (repo_owner, repo_name) with no repo id, and
purge left them intact, so recreating the same name re-attached the old
open bounty and claim_bounty succeeded against it. Purge now transitions
the repo's non-terminal bounties (open/claimed/submitted) to a dead
'purged' status on the delete's own transaction, preserving the
financial record. 'purged' is inert against every mutating bounty
endpoint, including dispute (which would otherwise reset claimed/
submitted back to open).
2026-07-19 19:48:47 -05:00
beardthelion bf2a54e121 fix(node): run the push completion tail in the disconnect-surviving task (#196)
The receive-pack refactor detached only acquire/apply/release; the whole
success tail (push+trust records, ref certificates, webhooks,
replication/pinning, ref-update broadcast, peer-notify) ran inline in
the cancellable request future. A client disconnect mid-apply committed
and uploaded the git state but dropped that tail, leaving accepted git
history with none of the metadata or fan-out every other push produces.
The tail now runs inside the detached task, gated on receive-pack Ok;
report-status still returns to a connected client early over a oneshot
(never awaited on the JoinHandle), and a dropped receiver no longer
aborts the tail. guard.release runs on every path.
2026-07-19 19:35:18 -05:00
beardthelion 86a8228f47 fix(node): fail closed classifying the purge target directory (#196)
local_refs_on_disk returned None for a missing path, an existing
non-bare directory, AND an unreadable path alike; with a store
configured, None promotes the candidate to remote_unverified and the
archive refresh's remove_dir_all replaces whatever sits at the target.
It now returns None only for a verified NotFound (symlink_metadata +
ErrorKind::NotFound, so a dangling symlink counts as present); every
other state fails closed as present so a non-repo directory at the
target is never overwritten.
2026-07-19 19:17:02 -05:00
beardthelion 0c279c9d25 fix(node): skip slug-scoped purge cascade when a mirror row shares the slug (#196)
delete_repo_by_id cascaded branch_cids/sync_queue/received_ref_updates/
arweave_anchors by the normalized owner/name slug, which is non-unique
across repos rows (INV-9): a did:key:zX row and a bare zX mirror of the
same name share the slug. Purging an empty did:key row therefore wiped a
surviving non-empty mirror's branch->CID index and Arweave anchors. The
cascade now runs only when no other repos row resolves to the same slug
(checked with the same OWNER_KEY_CASE_SQL the slug is built from, on the
delete's own transaction); the physical row and id-keyed children are
always removed. The concurrent colliding-sibling-create window stays an
accepted operator-scoped residual.
2026-07-19 18:50:07 -05:00
beardthelion 49346464bf fix(node): advertise a window-derived Retry-After on rate-limit 429s (#196)
Both 429 sites hardcoded Retry-After: 60 while every limiter uses a
one-hour window, so a compliant client retrying at 60s kept hitting 429
for nearly another hour. The limiter now returns the delay from its
oldest live timestamp (window - elapsed, clamped >= 1); a cap-full
rejection with no bucket advertises the full window. The check() -> bool
API is preserved as a wrapper so unrelated callers are untouched.
2026-07-19 18:35:34 -05:00
beardthelion c4b337b194 fix(node): give advisory-lock guards a dedicated pool and bound the release upload (#196)
Repo-write guards pinned a connection from the shared app pool for the
whole receive-pack plus an untimed archive upload, so ~20 concurrent
distinct-repo pushes starved unrelated DB handlers. Guards now draw from
a dedicated lock_pool (GITLAWB_DB_LOCK_POOL_MAX_CONNECTIONS, default 32,
sized to peak writers) separate from the app pool that serves handlers
and the purge delete, and release()'s Tigris upload is wrapped in a
timeout so the connection hold is bounded on every path. The guard still
locks and unlocks on its single held connection (no off-session unlock).

Db::pool() is now test-only since nothing in the bin reads the raw app
pool directly.
2026-07-19 18:25:10 -05:00
beardthelion 2a78524826 docs(node): drop the .env.example shutdown-grace entry #196 added (#196)
The GITLAWB_SHUTDOWN_GRACE_SECS field, its doc-comment, and main.rs's
grace handling pre-date this PR (#22); #196's diff only added the
.env.example entry, whose comment promised a 503-on-expiry the node does
not implement. Remove just that entry. PR #194 owns the real bounded
drain and the config prose in disjoint files.
2026-07-19 17:53:33 -05:00
beardthelion 9f7757cbf5 refactor(node): dedup limiter setup + write-brake test driver; fix .env wording
- .env.example: the write brake wraps the whole /graphql route, so its docs now
  say it covers all GraphQL HTTP requests (queries and the playground GET), not
  only mutations, with WebSocket subscriptions called out as excluded.
- main.rs: collapse the identical create/write/push per-IP limiter setup into
  build_ip_limiter (resolve-env, build bounded, warn-on-zero), preserving the
  exact limits/defaults and the present-but-unparseable warning in
  rate_limit_from_env; each limiter now also info-logs its configured value.
- api/repos.rs: the nine per-IP write-brake tests share a send_from helper
  mirroring rate_limit.rs's post_from/post_with, replacing the repeated
  request-build + ConnectInfo + oneshot + status boilerplate. Behavior identical.
2026-07-16 10:26:42 -05:00
beardthelion 6ba698c99b fix(node): let purge-spam reach repos that exist only as archives
On a multi-machine (object-store) deployment a spam repo can exist only as a
Tigris archive with no local copy. Selection filtered on a ref count that
returns non-empty for a missing local path, so such a repo was never a
candidate and the tool it was hardened for could not reach it. Selection now
distinguishes missing-local (Option::None) from a one-ref repo and admits a
missing-local row as a *remote-unverified* candidate only when an object store
is configured (via RepoStore::has_object_store); partition_for_delete passes
those through rather than re-dropping them on the local ref count. The execute
loop is unchanged — it already refreshes from the archive and rechecks emptiness
under the per-repo lock — so a remote-only empty archive is deleted, one with
refs is refreshed and skipped, and missing-both fails closed (refresh no-ops,
the local recheck reports non-empty). Storeless deployments keep the old
fail-closed behavior. Dry-run marks remote-only candidates distinctly and counts
them in a new PurgeSummary field.

Tests (recording object-store double): remote-only empty archive deleted,
remote-only archive-with-refs skipped, missing-both fail-closed, storeless
missing-local not a candidate. Load-bearing per INV-21 — reverting the
store-configured admission turns the three remote-reach tests RED.
2026-07-16 10:18:15 -05:00
beardthelion e4c4161dbc fix(node): serialize archive uploads under the per-repo advisory lock
The lazy-migration upload in acquire(), init()'s background upload, and
release_after_write() (fork's upload) previously PUT to the object store with
no lock, so an in-flight upload could resurrect an archive purge-spam had just
deleted under the same lock, and init's background upload could clobber a
freshly-pushed archive. All three now go through upload_locked(): take the
per-repo lock, re-check the local dir exists under it (a purge removes the dir
under its lock, so a post-purge uploader finds it gone and skips), upload, then
release. The background migration/init uploads skip on contention and self-heal;
fork's foreground upload waits (bounded) instead, since a skipped fork upload
has no later retry.

Tests (recording object-store double): init upload skips under a held lock,
fork upload waits then PUTs once after release, a late upload after a
purge-style delete does not resurrect the archive, and an uncontended upload
PUTs once. Load-bearing per INV-21 — neutralizing the lock+dir-check turns the
three contention/resurrect tests RED.
2026-07-16 10:06:06 -05:00
beardthelion b94d440e9e fix(node): complete a fully-received push after client disconnect
git_receive_pack now runs the acquire -> receive-pack -> release core in a
spawned task the handler awaits but whose cancellation it does not propagate.
The pack body is fully buffered before the handler runs, so the task is
self-contained: a client that disconnects mid-apply drops the handler future
without cancelling the task, and the push still applies, uploads, and releases
its lock in order. Drop-safety (the guard's close-on-drop) remains the backstop;
this closes the dominant window where a disconnect both bricked the lock and
abandoned a received push. The post-push bookkeeping tail stays in the
cancellable handler (a disconnect still forgoes it, as today).

Settled OQ1 by execution: a guard abandoned inside a detached task when the
tokio runtime tears down does not panic in Drop (5/5 runs), so detached-task
shutdown tracking stays deferred. Kept the check as a regression guard.

Disconnect regression drives a real push with a sleeping pre-receive hook and
drops the handler mid-hook: RED on the inline handler (the git group is killed,
ref absent), GREEN once detached.
2026-07-16 09:53:43 -05:00
beardthelion dde5ea9a3e fix(node): make advisory-lock guards drop-safe and reorder acquire_write
RepoWriteGuard and RepoLockGuard now hold their pooled connection as an
Option: release() takes it (returns it to the pool) while a Drop impl closes
the connection when release() was never reached — an early ?, a panic, or the
handler future cancelled on client disconnect. Closing the connection ends the
Postgres session so the server frees the session-scoped advisory lock, instead
of returning a lock-holding connection to the pool where it would block every
later write to that repo until the connection is reaped (up to ~30min at the
pool's default max_lifetime).

acquire_write now wraps the winning connection in the guard BEFORE the
freshness download, so a cancellation during that await is covered by the
guard's Drop rather than orphaning the lock on a bare connection.

Tests observe the leak by disabling the pool's ambient reaping (which otherwise
masks it ~1.8s after drop), so the lock stays held until the fix's own unlock.
RED before / GREEN after for both guards and the acquire-side cancel; each
protection line proven load-bearing by revert (INV-21).
2026-07-16 09:38:54 -05:00
beardthelion cf669bb8aa fix(review): don't hold/leak the pool connection in acquire_write
Two regressions the connection-pinning fix (010c625) introduced, caught
in pre-push review:

- Pool pressure: acquire_write held its pinned connection through the
  whole 60s lock-retry backoff; pre-change the retry probed via the pool
  and returned the connection between attempts. Under a same-repo write
  burst every spinner now held a pool connection through its sleeps and
  could starve the shared pool. Restructure the loop to return the
  connection between FAILED attempts and keep it only on the winning one.
- Lock leak: on the object-store download-error early-return the pinned
  connection dropped back to the pool with the advisory lock still held
  (no guard Drop runs pre-construction, and sqlx does not release session
  locks on connection return), blocking every future write to that repo
  until the pool reaped the connection. Unlock on the pinned connection
  before bailing.

Add acquire_write_guard_excludes_try_lock_across_connections: a
multi-connection test proving the exclusion holds across sessions (the
production topology), which also catches a pin-but-don't-lock regression
the single-connection test cannot.
2026-07-15 17:37:43 -05:00
beardthelion 3c1ceb8edc fix(node): make purge-spam Tigris-authoritative and archive-deleting
purge-spam built its RepoStore with no object store, so on a Tigris
deployment it rechecked emptiness against possibly-stale local disk
(deleting a repo with live remote refs) and left the archive behind (to
be re-downloaded into a later same-owner/name repo). Give the admin
command the configured object store, refresh the local copy from the
authoritative archive under the per-repo lock before the recheck
(fail-closed on a store error via a new skipped_store_error), and delete
the archive on a successful purge (failures counted in a new
archive_failed, never folded into success). When no bucket is configured
the None path is byte-for-byte unchanged.

Three load-bearing guards verified RED->GREEN on the Tigris-blind purge:
remote-refs-not-deleted, archive-deleted, fail-closed-on-unreachable.
2026-07-15 17:07:47 -05:00
beardthelion 68c1f5c2e6 refactor(node): introduce ObjectStore trait seam over TigrisClient
Extract an async ObjectStore trait (exists/upload/download/delete) that
TigrisClient implements, and hold Option<Arc<dyn ObjectStore>> in
RepoStore instead of Option<TigrisClient>. A dyn async trait needs
async-trait (added as a direct dep; already transitively present), which
E0038 would otherwise reject. Pure refactor — no behavior change; full
suite stays green. Enables an in-test fake for the purge Tigris work (U3).

Cargo.lock also picks up the pre-existing 0.5.0->0.5.1 version sync
(#185: the committed lock was stale vs the manifests) that any cargo run
regenerates.
2026-07-15 16:55:48 -05:00
beardthelion 4df79f22a3 fix(node): pin the writer's advisory-lock connection in RepoWriteGuard
acquire_write took the session-scoped advisory lock through the shared
pool and returned the lock-owning connection to the pool, so release
unlocked on a different connection (silent no-op) and a concurrent
try_lock_repo could be handed the idle lock-owning connection and
reentrantly re-grab the lock, defeating the purge tool's mutual
exclusion. Pin a dedicated PoolConnection for the guard's lifetime and
unlock on it, mirroring RepoLockGuard.

Load-bearing contention test (single-connection pool to force the
reentrant grab deterministic): RED (Ok(Some)) before, GREEN after.
2026-07-15 16:43:59 -05:00
beardthelion cdb102efbd style(node): cargo fmt for the #196 fix set 2026-07-14 22:27:00 -05:00
beardthelion bfbe5026ba test(node): exercise the dry-run guard with a real on-disk candidate present
dry_run_deletes_nothing ran against an empty repos_dir, so select_spam_candidates
found nothing and run_purge_spam hit the no-candidate early return — the
if !execute { return } guard was never exercised with candidates present (a
vacuous guard, INV-21). Materialize a real empty bare repo so a genuine candidate
exists, then assert dry-run leaves both the row and the on-disk dir. Verified
load-bearing: removing the !execute guard turns the test RED (dry-run deletes).
2026-07-14 22:26:29 -05:00
beardthelion 63254029e5 docs(node): document GITLAWB_IPFS_API/TIGRIS_BUCKET/METRICS_ADDR/SHUTDOWN_GRACE_SECS in .env.example
Four config knobs read in config.rs were absent from .env.example, so operators
had no template for them. Add each in its matching section with the config
default (INV-24: operator docs match the config the code reads, in the same PR
that touches the file).
2026-07-14 22:23:31 -05:00
beardthelion d9eebd29c9 fix(node): warn on unparseable rate-limit env vars instead of silently defaulting
GITLAWB_CREATE/WRITE/PUSH_RATE_LIMIT parsed with .ok().and_then(parse).unwrap_or,
so a typo (e.g. 5O with a letter O) silently reverted to the default and left the
operator believing a stricter cap was in force. Route all three through
rate_limit_from_env, which distinguishes absent/empty (default, silent) from
present-but-unparseable (default, WARN). Pure resolve_rate_limit is unit-tested
across absent/empty/valid/zero/unparseable — 0 stays valid (its own ==0 warn
handles the disable case).
2026-07-14 22:22:32 -05:00
beardthelion 1fcca0d433 fix(node): count purge-spam disk-removal failures separately from deletes
The Ok(n) arm incremented the deleted total before remove_dir_all, whose failure
only warn!'d, so the summary reported N deleted even when the on-disk dir
survived (DB/disk drift shown as clean success). run_purge_spam now returns a
PurgeSummary and counts a failed (or containment-refused) on-disk removal in
disk_failed, never folding it into deleted; the summary line surfaces
'M on-disk removal(s) failed'. RED->GREEN: a read-only parent dir forces the
removal to fail; deleted=1 AND disk_failed=1 with the dir surviving (neuter-check
confirms the counter is load-bearing).
2026-07-14 22:20:09 -05:00
beardthelion 15522d5616 fix(node): hold the per-repo advisory lock across purge recheck+delete
purge-spam took no write lock, so a concurrent receive-pack could land a ref
between the emptiness recheck and the delete, silently losing a live push.
Add RepoStore::try_lock_repo — a lock-only, non-blocking counterpart to
acquire_write (no Tigris I/O) whose guard holds a dedicated pooled connection so
the advisory lock and its unlock run on the same session. purge now locks each
candidate, rechecks emptiness UNDER the lock, deletes, and releases; a repo held
by a live writer is skipped, not force-deleted. RED->GREEN: a locked empty repo
survives purge, then deletes once the writer releases (neuter-check confirms the
lock is load-bearing).
2026-07-14 22:14:15 -05:00
beardthelion fde9cfe287 fix(node): reap write_rate_limiter in the periodic cleanup loop
The PR's new write_rate_limiter was the one limiter omitted from main()'s
hand-maintained cleanup list, so its key map never shed expired entries until
the 200k cap. Move the sweep into AppState::cleanup_rate_limiters (co-located
with the limiter fields, single source of truth) and call it from the loop, so
a newly-added limiter is reaped rather than silently dropped. Guard test seeds a
reclaimable entry into write_rate_limiter and asserts the sweep reaps it — RED
if write_rate_limiter is dropped from cleanup_rate_limiters (verified).
2026-07-14 22:03:49 -05:00
beardthelion a0fcbb0813 fix(node): delete child rows transactionally in delete_repo_by_id
delete_repo_by_id was a bare DELETE FROM repos with no cascade and the schema
has no FKs, so every child row (ref_certificates, pull_requests + pr_reviews/
pr_comments, webhooks, push_events, protected_branches, repo_stars, repo_replicas,
repo_labels, visibility_rules, encrypted_blobs, repo_icaptcha_proofs, agent_tasks,
and the slug-keyed branch_cids/sync_queue/received_ref_updates/arweave_anchors)
was orphaned and could re-join a later repo reusing the id. Wrap parent+child
deletes in one transaction, resolving the owner_short/name slug for the repo-TEXT
children. bounties (financial) and issue_comments (unmappable) are left intact by
design. RED->GREEN test seeds a repo_id child, a slug child, and a PR grandchild.
2026-07-14 21:58:31 -05:00
beardthelion 01354199d0 fix(node): reject path-traversal repo names in purge-spam before remove_dir_all
A peer-mirror row skips API name validation, so a burst-owned repo can carry
a '../' name; repo_disk_path joined it verbatim and the emptiness marker check
passed on the traversed target, so purge-spam selected and remove_dir_all'd a
repo OUTSIDE repos_dir. Validate the name via repo_store::validate_repo_name at
selection (fail closed -> non-candidate) and assert canonical containment inside
repos_dir immediately before the destructive remove. Adversarial RED->GREEN: a
victim bare repo reachable via '../../victim' is deleted before, survives after;
symlink-escape covered by path_within.
2026-07-14 21:53:23 -05:00
beardthelion dfd4f6c3d6 test(review): close the reasoned-not-run gaps by execution
- purge-spam execute now re-verifies emptiness immediately before delete (TOCTOU:
  a push between selection and delete no longer risks deleting a now-non-empty
  repo) and removes the on-disk bare repo dir so DB and disk stay consistent; a
  per-repo delete failure warns and continues rather than aborting the batch.
- Tests (RED-verified where they cover new behavior): partition_for_delete skips
  a repo no longer empty; execute removes the on-disk dir (RED without the
  removal); list_repos_by_owner_did finds a short-form burst row (RED without the
  SQL normalization); every braked write group passes under-limit (per-group
  grant path, not just the star representative).
2026-07-13 15:23:19 -05:00
beardthelion 090019deb7 fix(review): normalization-consistent purge, brake helper, CLI + coverage
Addresses the remaining code-review findings:
- purge-spam DID matching now uses did:key normalization on both sides
  (list_repos_by_owner_did via OWNER_KEY_CASE_SQL, is_excluded and the target
  scope via normalize_owner_key), so short/full form is consistent and an
  excluded identity is protected in either form. Adds a short-form exclusion test.
- Extract the per-IP write brake into a chainable WriteBraked helper so a future
  write group can't silently ship unbraked (kept separate from add_auth_layers
  by design; documented why).
- Mark --repos-dir / --database-url global so admin subcommands accept them after
  the subcommand (was an 'unexpected argument' parse error); verified.
- Document GITLAWB_WRITE_RATE_LIMIT in .env.example incl. the per-IP NAT-aggregate
  caveat so operators can size it.
- Coverage: adoption-floor (under-limit write passes), write-limit=0 disables,
  task/bounty/profile route-level 429s, /graphql/ws stays unbraked.

Full suite 927 pass / 0 fail; fmt + clippy -D warnings clean.
2026-07-13 15:03:41 -05:00
beardthelion 285d7c7f7a fix(review): make purge-spam empty-check reject git-discovery reads
P1 from code review: on_disk_ref_count shelled 'git for-each-ref' with only the
repo path as cwd and no --git-dir, so a path that exists but is not a bare repo
let git discover a parent .git (repos_dir may sit inside the operator's checkout)
and read a DIFFERENT repo's refs — possibly 0, deleting a real repo. Now require
the bare-repo markers (HEAD file + objects/ dir) before trusting any count;
anything else fails closed (skipped).

Tests: a non-git dir under a git ancestor (zero-ref) now fails closed — verified
RED without the guard (read the ancestor's 0 refs); target-DID-never-excluded
invariant pinned; write_flood test anchored to a genuinely-drained bucket so its
assert_ne can't pass vacuously.
2026-07-13 14:47:07 -05:00
beardthelion e6324d20df feat(node): purge-spam admin subcommand for the empty-burst cleanup
Adds a purge-spam subcommand (clap #[command(subcommand)] split; no-subcommand
path still runs the node unchanged). Candidate selection is signature-based, not
per-DID: a repo qualifies only if owned by the named burst DID AND verified empty
(zero git refs) per repo. A hard exclusion gate runs BEFORE the empty check, so
an empty repo owned by the content-bearing user or the mirror-bot DID is never a
candidate. Dry-run is the default; deletion needs an explicit --execute flag and
operates per-repo, never 'all repos of DID X'.

Tests (RED->GREEN, exclusion gate confirmed load-bearing by targeting an excluded
DID directly): empty target listed; target-with-refs spared; empty excluded/
intern repos spared (must-assert); dry-run deletes nothing; execute removes only
the empty target on disk.
2026-07-13 14:28:45 -05:00
beardthelion 6fded6ef65 test(node): adversarial TrustedProxy verification through the write brake
Middleware-level tests complementing the client_key unit coverage: a spoofer
varying the client-controlled leftmost X-Forwarded-For hop cannot rotate its
bucket key (keyed on the trusted rightmost hop); distinct trusted client hops
get distinct buckets (no cross-starvation); and a request with no trusted header
in a proxy mode falls back to the socket-peer key — the documented collapse,
asserted by name so any change is deliberate.

Operator-warning on missing-header deferred deliberately: a per-request warning
is a log-noise risk and the security-load-bearing fallback (peer key, never
skip) is what these tests lock down.
2026-07-13 14:14:15 -05:00
beardthelion 74e05be7f7 feat(node): extend the per-IP write brake to the remaining write sinks
Attaches rate_limit_by_ip (write bucket) to task_write_routes,
issue_write_routes, bounty_write_routes, profile_write_routes, and the /graphql
POST/GET surface (KTD-5 — an HTTP-layer brake counts every /graphql request;
/graphql/ws subscriptions stay unbraked). write_ip_limiter is now built once and
shared. Every non-git, non-creation authenticated write group is braked.

Tests (execution-verified against Postgres): /graphql POST is 429'd, and a
representative REST write sink (issue comment) is 429'd, by the write brake.
2026-07-13 14:11:38 -05:00
beardthelion 34df1fddc8 feat(node): add per-IP write-surface rate brake and apply to write_routes
Introduces a dedicated write_rate_limiter bucket (GITLAWB_WRITE_RATE_LIMIT,
default 600/hr, 0 disables), separate from the creation brake so a write flood
cannot drain the creation budget (KTD-1). Attaches rate_limit_by_ip to
write_routes, mirroring creation_routes. Per-DID deliberately not paired (a DID
farm never trips it).

Tests (route-level, execution-verified against Postgres): a write_routes sink
(PUT /star) is 429'd before auth; an exhausted write bucket does not throttle
repo creation (separate-bucket property).
2026-07-13 14:07:31 -05:00
95 changed files with 11472 additions and 66104 deletions

No files matched your search

+36 -175
View File
@@ -13,9 +13,16 @@ GITLAWB_PUBLIC_URL=https://your-node.example.com
# ── Server ────────────────────────────────────────────────────────────────
GITLAWB_HOST=0.0.0.0
GITLAWB_PORT=7545
# Optional address to bind a Prometheus /metrics exposition endpoint on (e.g.
# 127.0.0.1:9091). Leave empty (default) to disable. Bind to localhost or a
# private interface — the metrics endpoint is unauthenticated.
GITLAWB_METRICS_ADDR=
# ── Storage ───────────────────────────────────────────────────────────────
GITLAWB_REPOS_DIR=/data/repos
# Tigris (S3-compatible) bucket for repo storage. Leave empty (default) to
# disable Tigris and use local-only storage.
GITLAWB_TIGRIS_BUCKET=
# PostgreSQL connection URL. Required.
# When using the bundled docker-compose, this is wired automatically.
@@ -25,25 +32,13 @@ DATABASE_URL=postgresql://gitlawb:changeme@localhost:5432/gitlawb
# Maximum connections in the PostgreSQL pool. A cap, not a floor —
# connections open lazily. Size against the DB server's max_connections,
# remembering admin tooling opens its own pool.
GITLAWB_DB_MAX_CONNECTIONS=48
# Maximum connections in the DEDICATED advisory-lock pool, separate from the
# pool above. Every in-flight repo write pins one connection here for its whole
# duration, so this is a hard ceiling on simultaneous writes node-wide: size it
# to expected peak concurrent writers, not small. Keeping it separate is what
# stops a push burst from starving ordinary request handlers. Budget
# (GITLAWB_DB_MAX_CONNECTIONS + this) per node against the server's
# max_connections, times node count, plus admin tooling.
GITLAWB_DB_LOCK_POOL_MAX_CONNECTIONS=40
# Upper bound, in seconds, on any object-storage transfer that runs while a
# per-repo write lock is HELD (the archive download inside acquire_write and the
# upload inside release). These were free before the lock's connection was
# pinned to the guard; now an unbounded stall holds a lock-pool slot, and enough
# stalls deny every write on the node. The bound applies PER SPAN and there are
# two (the acquire-side refresh, which covers the existence check and download
# together, and the release-side upload), so worst-case slot occupancy is about
# twice this value plus the git work between them. Read it together with the pool
# size above and with GITLAWB_GIT_SERVICE_TIMEOUT_SECS.
GITLAWB_LOCK_HELD_TRANSFER_TIMEOUT_SECS=300
GITLAWB_DB_MAX_CONNECTIONS=20
# Maximum connections in the dedicated advisory-lock pool (separate from the
# pool above). Each in-flight repo write pins one connection here for its whole
# lifetime, so size it to peak concurrent distinct-repo writers — keeping it
# separate is what stops a push burst from starving request handlers. Keep
# (main pool + lock pool) within the DB server's max_connections.
GITLAWB_DB_LOCK_POOL_MAX_CONNECTIONS=32
# Seconds a request waits for a pool connection before failing with 503.
GITLAWB_DB_ACQUIRE_TIMEOUT_SECS=5
# Upper bound on each startup connect+migrate attempt, in seconds. Keep it
@@ -57,6 +52,9 @@ GITLAWB_DB_RETRY_INITIAL_SECS=5
GITLAWB_DB_RETRY_MAX_SECS=60
# ── IPFS pinning (Pinata) ─────────────────────────────────────────────────
# URL of a local IPFS/Kubo node HTTP API (e.g. http://127.0.0.1:5001). Leave
# empty (default) to disable local IPFS.
GITLAWB_IPFS_API=
# Get a JWT at https://app.pinata.cloud/developers/api-keys
GITLAWB_PINATA_JWT=
GITLAWB_PINATA_UPLOAD_URL=https://uploads.pinata.cloud/v3/files
@@ -109,14 +107,9 @@ GITLAWB_REQUIRE_SIGNED_PEER_WRITES=false
# Require the authenticated pusher to be the repo owner on git-receive-pack.
# A valid did:key signature is authentication, not authorization: anyone can
# mint a key, derive its DID and sign, so with this off every signed caller may
# push to every repository, private ones included. On by default.
#
# Set to false only for a rolling upgrade whose pushers are not yet the repo
# owner. Note that delegated and CI keys count as non-owners: UCAN git/push is
# verified but not yet honored for authorization, so they cannot push while this
# is on. See docs/RUN-A-NODE.md.
GITLAWB_ENFORCE_OWNER_PUSH=true
# sign as their own DID. When true, pushes from a non-owner DID are rejected.
# Keep false until the repo owner is ready for owner-only writes.
GITLAWB_ENFORCE_OWNER_PUSH=false
# Comma-separated libp2p multiaddrs.
# Example: /ip4/1.2.3.4/udp/7546/quic-v1/p2p/12D3KooW...
@@ -132,161 +125,15 @@ GITLAWB_MAX_PACK_BYTES=2147483648
# Max seconds a served git upload-pack / receive-pack (clone / push) may run
# before it is aborted with a 504. Bounds a hung git that would otherwise pin a
# worker and, on push, the repo write lock. Also bounds both info/refs
# advertisements, the withheld-blob pack build, and the push-side candidate
# discovery (rev-list / cat-file), all reaped via process-group teardown (#174).
# On the path-scoped upload-pack path the withheld-blob classification walk and
# the pack serve share ONE deadline, so this value bounds their COMBINED duration
# rather than giving each stage a full budget: a walk that consumes it leaves the
# serve nothing and the clone gets a 504. Serving large path-scoped repos may
# need a higher value here than when each stage was budgeted separately.
# Must be 1..=3153600000 (100 years): the node derives deadlines from this value,
# and a larger one cannot be represented. Default 600.
# worker and, on push, the repo write lock. Does NOT cover the info/refs
# advertisement or the withheld-blob path, which remain unbounded. Default 600.
GITLAWB_GIT_SERVICE_TIMEOUT_SECS=600
# Max seconds the storage-ACQUISITION phase of a served git op may run before the
# request is shed with a 503, separate from the git-run timeout above. A
# concurrency permit is taken before this phase and GITLAWB_GIT_SERVICE_TIMEOUT_SECS
# only starts once git spawns, so without this a stalled backend (a hung Tigris
# HEAD/GET, or a hung pg advisory-lock iteration on push) pins the permit and drains
# the pool until every later request 503s. On expiry the permit is released
# (fail-closed). Kept separate because acquisition and git execution are distinct
# cost centers. Must be positive; set very large to effectively disable. Default 30.
GITLAWB_GIT_ACQUIRE_TIMEOUT_SECS=30
# Max concurrent git READ ops (upload-pack + the upload-pack info/refs
# advertisement) served at once, a global pool separate from the push pool below.
# The anon receive-pack info/refs advertisement has its OWN pool (see below), not
# this one. Over-cap sheds a clean 503 + Retry-After. Anonymous reads draw from
# here, so pair it with GITLAWB_MAX_CONCURRENT_READS_PER_CALLER (below) so one
# caller cannot monopolize the pool. Default 128.
GITLAWB_MAX_CONCURRENT_GIT_OPS=128
# Max concurrent git-receive-pack (push) POST operations, in a pool separate from
# the read pool (GITLAWB_MAX_CONCURRENT_GIT_OPS) so anonymous reads cannot shed an
# authenticated push at admission. The anon receive-pack info/refs advertisement
# runs in a SEPARATE pool of the same size (disjoint from this one), so an
# advertisement flood cannot shed a push either. Over-cap sheds a 503 +
# Retry-After. Default 32.
GITLAWB_MAX_CONCURRENT_GIT_PUSHES=32
# Max pushes PARKED at once waiting for the per-repo write lease of the SAME repo.
# A parked push holds its already-buffered pack body in memory, so this bounds that
# memory for a hot repo. Past the cap the newest push sheds a clean 503 +
# Retry-After ("repo is busy with another push") instead of joining the queue; a
# push to a DIFFERENT repo is never affected. The lease holder is not counted.
# Raising GITLAWB_MAX_CONCURRENT_GIT_PUSHES does not raise this; set it explicitly.
# Default 8.
GITLAWB_REPO_LEASE_MAX_WAITERS=8
# Max concurrent post-push pin loops (IPFS + Pinata pin_new_objects) across all
# repos. The per-repo encrypt-task set caps the pin-task COUNT; this caps how many
# of those loops RUN CONCURRENTLY, and so how much MB-scale object-id list memory is
# held WHILE BEING PINNED. It does not bound retained memory in general: on the local
# IPFS path the caller builds its list before this pool is entered, so a loop parked
# waiting for a permit still holds its full list, and how many park is capped only
# per repo. Do not size memory from this knob alone. A loop DEFERS (waits) when the
# pool is full, never drops a pin. Default 8.
GITLAWB_MAX_CONCURRENT_PIN_TASKS=8
# Max concurrent read ops (upload-pack + the upload-pack info/refs advertisement)
# a single caller may hold, so one caller cannot monopolize the read pool. Keyed
# on the resolved SOURCE IP, never the DID: a signature does not move a caller off
# this cap. The source-IP key is only as granular as GITLAWB_TRUSTED_PROXY below:
# left unset, a node behind an edge/NAT keys all callers on the edge IP and this
# collapses to one global cap. Set GITLAWB_TRUSTED_PROXY for per-client keying; a
# high-fanout caller (CI behind one NAT) then needs the operator to raise this.
# Default 16.
GITLAWB_MAX_CONCURRENT_READS_PER_CALLER=16
# Two further per-source concurrency caps exist on the PUSH side but have NO
# environment variable of their own: both are DERIVED as
# GITLAWB_MAX_CONCURRENT_GIT_PUSHES / 8, with a floor of 1. They are not settable
# independently, so raising the push pool raises both proportionally, and the
# default push pool of 32 gives each a cap of 4.
# * The anon receive-pack info/refs advertisement cap: one source IP may hold at
# most this many slots in the dedicated advert pool, so saturating that pool
# takes ~8 distinct source IPs.
# * The authenticated receive-pack POST cap: one source IP may hold at most this
# many WRITE-pool slots. This one is load-bearing for push availability, since
# it is acquired before the global write permit: without it, one host minting
# disposable did:key identities could open enough slow pushes to monopolize the
# write pool and 503 every other source (the push rate limiter caps arrival
# rate, not in-flight concurrency, and owner enforcement can be turned off for
# a rolling upgrade).
# Keyed on the resolved source IP, never the DID, so a DID farm does not defeat
# them; keying granularity follows GITLAWB_TRUSTED_PROXY like the read cap above.
# ── Push rate limiting (git-receive-pack flood brake) ─────────────────────
# Max receive-pack requests (info/refs advertisement + push POST) per client
# IP per hour. 0 disables. Default 600.
GITLAWB_PUSH_RATE_LIMIT=600
# ── /ipfs/{cid} visibility-walk admission (#174) ──────────────────────────
# GET /ipfs/{cid} runs a per-repo full-history git walk in a blocking thread to
# decide whether the caller may read a path-scoped blob. It is publicly reachable,
# so it is bounded to keep a permissionless caller from fanning out unbounded
# concurrent walks and exhausting blocking-pool threads + PIDs.
# Max concurrent /ipfs walks across all callers (a pool of its own, disjoint from
# the served-git pools). Over-cap sheds a 503. Default 32.
GITLAWB_MAX_CONCURRENT_IPFS_WALKS=32
# Max concurrent /ipfs walks a single SOURCE IP may hold (keyed like the git
# per-caller caps via GITLAWB_TRUSTED_PROXY; reject-before-insert bounded map).
# Default 4.
GITLAWB_IPFS_WALK_PER_SOURCE=4
# Max legacy (NULL-provenance) repos probed per single /ipfs request, bounding the
# scan-fallback fan-out (git cat-file per candidate repo) for an anonymous caller. A
# truncated scan sheds a retryable 503, never a false 404. Default 256.
GITLAWB_IPFS_MAX_LEGACY_PROBES=256
# Max repo ROWS one /ipfs request's legacy scan may read from the database. The probe
# ceiling above only counts once a probe runs, and quarantined or private repos are
# denied before that, so without this an all-denying inventory paged the whole repo
# table for one anonymous request. A truncated scan sheds a retryable 503 carrying a
# sealed `continuation` token; echo it as ?scan= to resume, so a holder buried past the
# ceiling is reachable in ceil(repos / ceiling) + 1 requests. Raising this also raises
# the per-caller /ipfs work allowance, since each page is charged to it. Lowering it
# sharpens a coarse oracle: laddering to the end reveals the node's total repo count to
# within one ceiling. Default 2048.
GITLAWB_IPFS_MAX_LEGACY_SCAN_ROWS=2048
# Max EXPENSIVE path-scope visibility walks per single /ipfs request (only a
# blob in a path-scoped repo costs a full-history walk). Over-cap repos are
# skipped without a verdict and the scan continues; if the object is then found
# nowhere the request sheds a retryable 503 instead of a false 404. The effective
# cap is the tighter of this value and the node's internal per-request ceiling of
# 17, so values above 17 have no effect; lower values do tighten it. Default 64.
GITLAWB_IPFS_MAX_REPOS_WALKED=64
# Ceiling on repos one /ipfs request may VISIT past the visibility gate. Each
# visit costs a repo acquire — on a Tigris cache miss a full archive download,
# so this is also the worst-case object-store fetch count per request — plus a
# cat-file probe. On exhaustion the scan stops and sheds a retryable 503.
# Default 1024.
GITLAWB_IPFS_MAX_REPO_VISITS=1024
# Absolute wall-clock budget for one admitted /ipfs request's acquire+walk
# lifetime (all stages of the whole scan). Per-stage clamps bound the acquire
# and walk stages to the remaining budget, and no stage starts once it is
# exhausted (the scan then sheds a retryable 503). The object-type probe and
# content-read cat-file subprocesses are budget-checked before starting AND each
# run under their own deadline (the lesser of GITLAWB_GIT_SERVICE_TIMEOUT_SECS and
# the remaining budget), reaped via process-group teardown, so a hung cat-file
# cannot hold the request's walk slot past it. One hang path is still unbounded:
# the probe's object-store readability check is a plain filesystem sweep with
# nothing to reap, so a wedged filesystem can hold the slot past the deadline.
# Must be 1..=3153600000 (100 years): the node derives an Instant deadline from
# this value, and a larger one cannot be represented. Default 600.
GITLAWB_IPFS_REQUEST_BUDGET_SECS=600
# Shorter budget (seconds) for the pre-walk CID resolve: the lookup that maps a
# requested CID to its git oid(s), which runs while the scarce walk admission is
# already held. A well-formed CID with no pin row does no probe and no walk work,
# so without this a stalled lookup could hold a walk slot for the whole request
# budget while nothing walked. The effective deadline is the lesser of this and
# the remaining request budget; walk and probe work stay on the request budget,
# so a slow but progressing scan is never shed by it.
# Must be 1..=3153600000 (100 years). Default 10.
GITLAWB_IPFS_RESOLVE_BUDGET_SECS=10
# Max /ipfs/{cid} requests per client IP per hour (route flood brake, distinct
# from the concurrency caps above). 0 disables. Default 600.
GITLAWB_IPFS_RATE_LIMIT=600
# ── Creation rate limiting (repo/agent/issue/PR flood brake) ──────────────
# Max creation requests (POST /api/v1/repos, /api/register, fork, issues,
# pulls) per client IP per hour, in addition to the per-DID limit. The per-DID
@@ -296,6 +143,20 @@ GITLAWB_IPFS_RATE_LIMIT=600
# the client IP. 0 disables. Default 120.
GITLAWB_CREATE_RATE_LIMIT=120
# ── Write rate limiting (non-creation authenticated writes) ───────────────
# Max non-creation write requests per client IP per hour: issue/PR comments,
# labels, stars, merges, protect/unprotect, replicas, visibility, tasks,
# bounties, profile, and all GraphQL HTTP requests. The brake wraps the whole
# /graphql route, so queries and the playground GET consume this bucket too, not
# only mutations (GraphQL WebSocket subscriptions are excluded). Its own bucket,
# separate from the creation and push brakes. Uses GITLAWB_TRUSTED_PROXY to
# resolve the client IP.
# NOTE: this is a per-IP aggregate across ALL those write actions, so behind a
# shared NAT/egress IP (or with GITLAWB_TRUSTED_PROXY unset) many users collapse
# onto one bucket — raise this for automation-heavy or multi-user single-IP
# deployments. 0 disables. Default 600.
GITLAWB_WRITE_RATE_LIMIT=600
# ── Peer-sync rate limiting (per client IP, uses GITLAWB_TRUSTED_PROXY below) ─
# /api/v1/peers/announce and /api/v1/sync/notify accept unsigned requests from
# known peers and run at higher frequency, so a generous bucket. Separate from
+5 -86
View File
@@ -6,13 +6,6 @@ on:
# result before it lands. Guards against semantic merge races where two
# individually-green PRs break main in combination (#113 + #145 → #229).
merge_group:
# Release-branch heads are created and pushed by GITHUB_TOKEN (release-please
# and the lock-sync job), and GitHub's recursion guard means those pushes
# trigger no workflow runs. workflow_dispatch is the documented carve-out that
# GITHUB_TOKEN CAN trigger, so the release flow dispatches this workflow at
# the release branch to attach check runs to its head. Read-only workflow
# (contents: read), so the dispatch surface adds no privilege.
workflow_dispatch:
push:
branches:
- main
@@ -53,20 +46,7 @@ jobs:
run: cargo fmt --all -- --check
- name: cargo clippy
run: cargo clippy --locked --workspace --all-targets -- -D warnings
release-tag-validation:
name: release tag validation
runs-on: ubuntu-latest
timeout-minutes: 5
steps:
- name: Check out repository
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
persist-credentials: false
- name: Test release tag resolver
run: scripts/test-resolve-release-tag.sh
run: cargo clippy --workspace --all-targets -- -D warnings
test:
name: test (${{ matrix.toolchain }})
@@ -114,7 +94,7 @@ jobs:
- name: cargo test
env:
DATABASE_URL: postgres://postgres:postgres@localhost:5432/gitlawb_test
run: cargo test --locked --workspace
run: cargo test --workspace
build-release:
name: build --release
@@ -137,7 +117,7 @@ jobs:
key: release
- name: cargo build --release
run: cargo build --locked --release --workspace
run: cargo build --release --workspace
audit:
name: cargo audit
@@ -219,7 +199,7 @@ jobs:
key: msrv
- name: cargo check on MSRV
run: cargo check --locked --workspace --all-targets
run: cargo check --workspace --all-targets
docker-build:
name: Docker build smoke test
@@ -266,65 +246,4 @@ jobs:
key: windows
- name: cargo test (shipped Windows crates)
run: cargo test --locked -p gl -p git-remote-gitlawb
# gitlawb-core is embedded by every consumer (gl, git-remote-gitlawb, the node
# daemon), so it must stay lean. This gate fails if gitlawb-core's normal
# (non-dev, non-build) dependency tree gains any crate not on the allowlist in
# ci/gitlawb-core-allowed-deps.txt. Exhaustive-by-construction: a new heavy
# dependency reds CI whether or not anyone thought to ban it, which a denylist
# cannot do. Regen instructions live in the allowlist header.
core-deps-purity:
name: gitlawb-core dependency purity
runs-on: ubuntu-latest
timeout-minutes: 15
steps:
- name: Check out repository
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
persist-credentials: false
- name: Set up Rust toolchain
uses: dtolnay/rust-toolchain@b3b07ba8b418998c39fb20f53e8b695cdcc8de1b # stable
with:
toolchain: stable
- name: Cache cargo
uses: Swatinem/rust-cache@98c8021b550208e191a6a3145459bfc9fb29c4c0 # v2.8.0
with:
key: core-deps-purity
- name: Check gitlawb-core dependency allowlist
run: bash scripts/check-gitlawb-core-deps.sh
# release-please only rewrites a crate's version when the line carries the
# `# x-release-please-version` annotation AND the manifest is listed under
# extra-files in release-please-config.json. Miss either half and the crate
# is skipped silently at release time, freezing at its creation version:
# icaptcha-client was added as a workspace member after the config was last
# written and sat at 0.4.0 through four releases before anyone noticed.
# Exhaustive-by-construction, like the allowlist gate above: the required set
# is derived from `cargo metadata`, so a newly added crate reds CI whether or
# not its author knew this gate existed.
release-versioning:
name: release versioning coverage
runs-on: ubuntu-latest
timeout-minutes: 15
steps:
- name: Check out repository
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
persist-credentials: false
- name: Set up Rust toolchain
uses: dtolnay/rust-toolchain@b3b07ba8b418998c39fb20f53e8b695cdcc8de1b # stable
with:
toolchain: stable
- name: Cache cargo
uses: Swatinem/rust-cache@98c8021b550208e191a6a3145459bfc9fb29c4c0 # v2.8.0
with:
key: release-versioning
- name: Check every workspace crate is release-managed
run: bash scripts/check-release-versioning.sh
run: cargo test -p gl -p git-remote-gitlawb
+1 -7
View File
@@ -70,15 +70,9 @@ jobs:
const changedRust = names.some(n => n.startsWith("crates/") && n.endsWith(".rs"));
// A test can live in a tests/ dir, a *_test.rs file, or (the common Rust
// pattern) an inline #[test] / #[cfg(test)] block added inside the source file.
// Async tests use a namespaced attribute (#[tokio::test], #[async_std::test],
// possibly with args like #[tokio::test(flavor = ...)]) rather than the bare
// #[test]. Match only on added lines, with the attribute at the start of the
// line after horizontal whitespace, covering bare, cfg(test), namespaced,
// optional leading ::, and raw-identifier (r#test) forms. No head-file fetch
// or lexer: needs-tests is advisory, not a merge gate.
const addsInlineTest = files.some(f =>
f.filename.endsWith(".rs") && f.patch &&
/^\+[ \t]*#\[[ \t]*(?:cfg[ \t]*\([ \t]*test[ \t]*\)|(?:::[ \t]*)?(?:[\w-]+[ \t]*::[ \t]*)*(?:r#)?test\b)/m.test(f.patch));
/^\+.*#\[(test\]|cfg\(test\))/m.test(f.patch));
const touchedTests =
names.some(n => n.includes("/tests/") || n.endsWith("_test.rs")) || addsInlineTest;
if (changedRust && !touchedTests) want.add("needs-tests");
+22 -158
View File
@@ -41,7 +41,6 @@ jobs:
release_created: ${{ steps.release.outputs.release_created }}
tag_name: ${{ steps.release.outputs.tag_name }}
version: ${{ steps.release.outputs.version }}
pr: ${{ steps.release.outputs.pr }}
steps:
- name: Run release-please
id: release
@@ -49,143 +48,6 @@ jobs:
with:
token: ${{ secrets.GITHUB_TOKEN }}
# The release PR bumps the workspace crate versions in the Cargo.tomls but
# release-please cannot update Cargo.lock, so without this job every release
# merge ships a tag whose lockfile disagrees with its manifests (a --locked
# build from the tag fails, and pr-checks builds --locked). Sync the lock on
# the release branch so the tagged tree is internally consistent.
#
# Runs on every push-triggered release run, not only when release-please
# reports the PR: the `pr` output is set only on runs that CREATE or UPDATE
# the release PR, so gating on it would leave an already-open release PR
# (opened before this job existed, or untouched by a non-releasable push)
# permanently without the sync and without check runs. The resolve step
# discovers any open release-please PR itself and no-ops when there is none.
sync-release-lock:
name: Sync Cargo.lock on the release PR
needs: release-please
if: ${{ !cancelled() && github.repository == 'Gitlawb/node' && github.event_name != 'workflow_dispatch' }}
runs-on: ubuntu-latest
permissions:
contents: write
pull-requests: read
# actions: write lets the final step dispatch pr-checks on the release
# branch. Note this grant is repo-wide dispatch rights (GitHub cannot
# scope it to one workflow); acceptable here because reaching this job's
# steps already requires write to main. Needed because every push to the
# release branch (release-please's and the lock sync below) is made with
# GITHUB_TOKEN, which triggers no workflow runs, so without an explicit
# dispatch the release PR's head carries no check runs and the --locked
# gate never validates it.
actions: write
steps:
# Prefer the fresh `pr` output when this run just created/updated the
# release PR; otherwise fall back to querying for an open release-please
# branch (prefix match: the branch carries `--components--` suffixes).
- name: Resolve the open release PR branch
id: resolve
env:
GH_TOKEN: ${{ github.token }}
OWNER: ${{ github.repository_owner }}
BRANCH_FROM_OUTPUT: ${{ needs.release-please.outputs.pr && fromJSON(needs.release-please.outputs.pr).headBranchName || '' }}
run: |
branch="$BRANCH_FROM_OUTPUT"
if [ -z "$branch" ]; then
# The branch name alone is not a trust signal. `gh pr list` includes
# fork PRs, and a fork's headRefName is bare (no owner prefix), so a
# prefix match by itself lets any fork nominate the branch this job
# checks out, commits to, and dispatches checks on. Require the head
# repo to be THIS repository and the author to be the release-please
# app, so neither a fork nor a same-repo human branch named
# `release-please--*` can be selected. Then take the highest PR
# number: `[0]` depended on gh's default ordering, which is not a
# documented guarantee, and picking a stable one keeps a second
# (component) release PR from making the choice flap between runs.
branch=$(gh pr list --repo "$GITHUB_REPOSITORY" --state open \
--json headRefName,headRepositoryOwner,number,author \
--jq '[ .[]
| select(.headRepositoryOwner.login == env.OWNER)
| select(.author.login == "app/github-actions")
| select(.headRefName | startswith("release-please--"))
] | max_by(.number).headRefName // empty')
fi
if [ -z "$branch" ]; then
echo "no open release PR; nothing to sync"
fi
echo "branch=$branch" >> "$GITHUB_OUTPUT"
- name: Checkout release branch
if: ${{ steps.resolve.outputs.branch != '' }}
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
ref: ${{ steps.resolve.outputs.branch }}
# Resolve on a declared toolchain instead of whatever Rust the runner
# image happens to ship, so the lock this job writes comes off the same
# stable channel the other cargo jobs use. Not a claim of identical
# resolution: `stable` still moves between runs, and the MSRV gate
# checks this same lock on 1.91 (pr-checks.yml). It only removes the
# runner image as an undeclared input.
- name: Install Rust toolchain
if: ${{ steps.resolve.outputs.branch != '' }}
uses: dtolnay/rust-toolchain@b3b07ba8b418998c39fb20f53e8b695cdcc8de1b # stable
with:
toolchain: stable
- name: Sync workspace crate versions into Cargo.lock
if: ${{ steps.resolve.outputs.branch != '' }}
run: cargo update --workspace
- name: Commit and push when the lock changed
if: ${{ steps.resolve.outputs.branch != '' }}
run: |
if git diff --quiet Cargo.lock; then
echo "Cargo.lock already in sync"
exit 0
fi
git config user.name "github-actions[bot]"
git config user.email "41898282+github-actions[bot]@users.noreply.github.com"
git add Cargo.lock
git commit -m "chore: sync Cargo.lock with the release version bump"
git push
# Every push to the release branch is made with GITHUB_TOKEN (release-
# please's own pushes and the lock sync above), and events created by
# GITHUB_TOKEN start no workflow runs. workflow_dispatch is the documented
# exception GITHUB_TOKEN may trigger, so dispatch pr-checks at the release
# branch to attach check runs to its current head. Unconditional (also on
# the already-in-sync path) because the branch's ORIGINAL head has the
# same no-runs problem. Loud on failure by design: a silent fallback
# would recreate the checkless-head problem, and a job rerun is safe
# (sync no-ops, dispatch retries). Loop-safe: pr-checks is read-only and
# dispatches nothing. Runs after the push ack, so the dispatched ref tip
# is the sync commit. Merge-queue note: this validates the branch tip;
# the enforced merge queue's merge_group run still re-validates the true
# merged result before landing.
- name: Dispatch PR checks on the release branch
if: ${{ steps.resolve.outputs.branch != '' }}
env:
GH_TOKEN: ${{ github.token }}
BRANCH: ${{ steps.resolve.outputs.branch }}
run: |
# Every dispatch failure fails the job, including the 422 a release
# branch cut before pr-checks gained its workflow_dispatch trigger
# returns ("does not have a workflow_dispatch trigger", reported
# against the dispatched ref's copy of the file). An earlier revision
# tolerated exactly that 422 on the theory that release-please would
# rewrite the branch and heal it, but a rewrite only happens when a
# later releasable commit lands: an open release PR nobody touches is
# never rewritten. Tolerating it therefore left the lock-sync commit
# this job had just pushed sitting on the release head with no
# --locked validation, which is the precise state this workflow exists
# to prevent. The tolerance is also no longer needed: once this lands,
# every release branch is cut from a main that already carries the
# trigger, so the only branch that could 422 is one open at merge time
# (there is none), and a job rerun after a manual rebase is safe (the
# sync no-ops, the dispatch retries).
gh api "repos/${GITHUB_REPOSITORY}/actions/workflows/pr-checks.yml/dispatches" \
-f "ref=$BRANCH"
# Each architecture builds NATIVELY (amd64 on ubuntu-latest, arm64 on
# ubuntu-24.04-arm) and pushes by digest; docker-manifest below stitches the
# digests into the tagged multi-arch image. No QEMU: emulating the arm64
@@ -216,11 +78,6 @@ jobs:
contents: read
packages: write
steps:
- name: Check out workflow scripts
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
persist-credentials: false
- name: Resolve release tag
id: rel
env:
@@ -228,7 +85,13 @@ jobs:
RELEASE_TAG: ${{ needs.release-please.outputs.tag_name }}
run: |
set -euo pipefail
scripts/resolve-release-tag.sh "${DISPATCH_TAG:-$RELEASE_TAG}"
TAG="${DISPATCH_TAG:-$RELEASE_TAG}"
case "$TAG" in
v[0-9]*) ;;
*) echo "::error::'$TAG' does not look like a release tag (vX.Y.Z)"; exit 1 ;;
esac
echo "tag=$TAG" >> "$GITHUB_OUTPUT"
echo "version=${TAG#v}" >> "$GITHUB_OUTPUT"
# ghcr requires a lowercase repository path, and unlike metadata-action,
# buildx's `--output name=` does no lowercasing — a mixed-case owner
# makes the digest push fail with "invalid reference format".
@@ -310,14 +173,8 @@ jobs:
if: ${{ !cancelled() && needs.docker.result == 'success' }}
runs-on: ubuntu-latest
permissions:
contents: read
packages: write
steps:
- name: Check out workflow scripts
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
persist-credentials: false
- name: Resolve release tag
id: rel
env:
@@ -325,7 +182,13 @@ jobs:
RELEASE_TAG: ${{ needs.release-please.outputs.tag_name }}
run: |
set -euo pipefail
scripts/resolve-release-tag.sh "${DISPATCH_TAG:-$RELEASE_TAG}"
TAG="${DISPATCH_TAG:-$RELEASE_TAG}"
case "$TAG" in
v[0-9]*) ;;
*) echo "::error::'$TAG' does not look like a release tag (vX.Y.Z)"; exit 1 ;;
esac
echo "tag=$TAG" >> "$GITHUB_OUTPUT"
echo "version=${TAG#v}" >> "$GITHUB_OUTPUT"
- name: Download digests
uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4.3.0
@@ -445,7 +308,7 @@ jobs:
set -euo pipefail
args=""
for b in $BINS; do args="$args -p $b"; done
cargo build --release --locked --target ${{ matrix.target }} $args
cargo build --release --target ${{ matrix.target }} $args
# Run each packaged binary's --version so a broken release artifact fails the
# build instead of shipping. Every target builds on a matching-arch runner,
@@ -541,11 +404,6 @@ jobs:
contents: read
id-token: write # npm trusted publishing (OIDC) + provenance
steps:
- name: Check out workflow scripts
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
persist-credentials: false
- name: Resolve release tag
id: rel
env:
@@ -553,7 +411,13 @@ jobs:
RELEASE_TAG: ${{ needs.release-please.outputs.tag_name }}
run: |
set -euo pipefail
scripts/resolve-release-tag.sh "${DISPATCH_TAG:-$RELEASE_TAG}"
TAG="${DISPATCH_TAG:-$RELEASE_TAG}"
case "$TAG" in
v[0-9]*) ;;
*) echo "::error::'$TAG' does not look like a release tag (vX.Y.Z)"; exit 1 ;;
esac
echo "tag=$TAG" >> "$GITHUB_OUTPUT"
echo "version=${TAG#v}" >> "$GITHUB_OUTPUT"
- name: Checkout release tag
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
+1 -1
View File
@@ -1,3 +1,3 @@
{
".": "0.7.1"
".": "0.6.0"
}
-44
View File
@@ -1,44 +0,0 @@
# Contributor guide for coding agents
This file carries the rules a patch author cannot recover from the code alone, for humans and coding agents alike. For build, test, and PR basics, read `CONTRIBUTING.md` first.
## Authorization invariants (the rules PRs most often break)
Every repo-scoped endpoint must bind the caller to an authorization decision before serving or mutating anything. The `known_ungated` allowlist in `authz_guard::every_repo_scoped_handler_is_gated` is empty on main: `list_webhooks`, `list_replicas`, and `list_protected_branches` are gated (`list_webhooks` with read-visibility then owner; the other two with `authorize_repo_read`). Do not re-open those surfaces as exceptions.
- Owner-only mutations (visibility, webhooks, protected branches, merges) are gated against the repo owner. Three gate forms are in use and all are recognized: webhooks and merges call `require_repo_owner` (`crates/gitlawb-node/src/api/mod.rs`), visibility uses its module-local `require_owner`, and protected branches compare inline with `did_matches`. Use the form the surrounding module already uses.
- Not every write is owner-only, and that is by design: `star_repo`/`unstar_repo` bind to the signer (they still require repo read access), `register_replica`/`unregister_replica` bind to the replica's own DID, the bounty actions (claim, submit, approve, cancel, dispute) have their own multi-party rules, and closing a PR or issue is owner-or-author. Do not add owner gates to these.
- Content-serving reads (blobs, trees, raw files) gate through `authorize_repo_read` with the specific path being served, so a withheld subtree is denied even on an otherwise-public repo; repo-level reads (listings) pass `"/"` by the helper's own contract. The source scans below cannot check this argument's value, so review will.
- A route that reaches repo data through a global id, with no repo in its path, or through an extractor shape the scans do not recognize still needs the same gate; the scans cannot see those.
- Read-surface denials must not reveal existence: a caller who may not read a repo (or a withheld subtree) gets the same 404 as a missing repo, never a 403. (Owner-gated mutations currently return 403 after the repo lookup; do not widen that shape to read paths.)
- Two tests in module `authz_guard` (`crates/gitlawb-node/src/api/mod.rs`) scan handler sources and fail on a repo-scoped handler without a recognized gate: `every_in_scope_mutation_has_its_gate` and `every_repo_scoped_handler_is_gated`. If they fail on your handler, add the gate. The second test's `known_ungated` allowlist is empty; never add an entry to it without an open issue and review.
## Adding or changing routes
- A new gated handler needs tests proving its route-specific gate fires: exercise an unauthorized authenticated caller and an anonymous caller where applicable, assert the exact denial status each way, and assert the body leaks nothing. The non-owner tests in `crates/gitlawb-node/src/test_support.rs` show the shape for owner-gated routes.
- A new mutation handler also gets a row in `every_in_scope_mutation_has_its_gate` naming its gate type; the per-handler row is the point of that test.
- A table-driven deny suite covering every deny-bearing route is in review (#194, #195). If it is present in your checkout, add a probe row there for any new deny-bearing route.
## Verifying signatures and certificates
- Derive the verifying key from something outside the artifact you are checking: the node's own DID, a pinned or configured value, a peers lookup. A key read from a field of the object being verified (`Did::from_str(&cert.node_did)`) only proves the artifact is self-consistent, and identities here are permissionless, so anyone can mint a keypair, name it in the artifact, self-sign, and be told it is valid. A mismatch against that anchor has to fail the result, not log a warning.
- A handler whose correct answer is "valid" needs a positive case driven by a trusted, independently anchored artifact, and a rejection case for a well-formed, fully populated forged artifact. Invalid-only suites miss an implementation that rejects everything; a forged artifact that still returns valid is the regression that matters.
- The fields a signature covers are a versioned format. Adding one invalidates every artifact already signed, so give the payload a version, keep a path that verifies the older form, and add a test that verifies an artifact signed before your change.
## Removing a serving path
When a fix removes a path that used to serve content, the same PR must add a test asserting the removed path now denies. The deletion is what turns that test green; without it the path can quietly come back.
## Database migrations
Schema changes are code-defined in `MIGRATIONS` (`crates/gitlawb-node/src/db/mod.rs`). Always append a new versioned entry; never edit an entry that has merged (operators deploy from main). Test runs build the schema from scratch, so editing an applied version stays green in CI while breaking every existing deployment.
## Client behavior (`gl`, `git-remote-gitlawb`)
When a node denies a request, surface the denial to the user. Never render a denial as an empty list or a silent success; that turns an authorization boundary into invisible data loss.
## Toolchain and CI reality
- MSRV is Rust 1.91 (`rust-version` in the workspace `Cargo.toml`). The MSRV CI job runs `cargo check --workspace --all-targets` on exactly 1.91, so check that an API you rely on is stable there; the full test suite runs on stable.
- The PR workflow runs these jobs on every PR: `cargo fmt --check`, `cargo clippy --workspace --all-targets -- -D warnings`, `cargo test --workspace` (against Postgres), a release build, `cargo audit`, the MSRV check, and a Docker build. The `main` branch ruleset currently requires one approving review but does not require these checks to pass before merge; a failing job should block review approval in practice, but nothing enforces that mechanically today.
- Conventional commit titles (`feat:`, `fix:`, `docs:`, ...) are required by convention and drive releases, but no CI job checks them; a bad title fails review, not the pipeline.
-44
View File
@@ -1,49 +1,5 @@
# Changelog
## [0.7.1](https://github.com/Gitlawb/node/compare/v0.7.0...v0.7.1) (2026-08-03)
### Bug Fixes
* **ci:** make the docker dependency cache work and unbreak Dockerfile.bins ([99b2b9a](https://github.com/Gitlawb/node/commit/99b2b9a16662761a3959a2941798e6823621bda5))
* **ci:** make the docker dependency cache work and unbreak Dockerfile.bins ([a49e04d](https://github.com/Gitlawb/node/commit/a49e04ddcd23f3b666421486ae60c3c4329e2242)), closes [#265](https://github.com/Gitlawb/node/issues/265) [#266](https://github.com/Gitlawb/node/issues/266)
* **db:** pin profile DID matching with lone non-key fixture and boundary matrix ([b4016dc](https://github.com/Gitlawb/node/commit/b4016dce9d1fe450603173f9d92bd880127620fa)), closes [#298](https://github.com/Gitlawb/node/issues/298)
* **node:** add a shared owner/name slug validator ([#272](https://github.com/Gitlawb/node/issues/272)) ([51d82de](https://github.com/Gitlawb/node/commit/51d82ded57a776d9c7e2fe65895efdbf9e227dc8))
* **node:** bound the slug owner by NAME_MAX, not the DID limit ([#272](https://github.com/Gitlawb/node/issues/272)) ([27b0495](https://github.com/Gitlawb/node/commit/27b0495a6f7edeecddbe62b0f075a7e3b1ef5053))
* **node:** fail a sync row terminally when a non-directory holds the owner path ([2e664b7](https://github.com/Gitlawb/node/commit/2e664b7885036ee72ce175af95bb567b18222583))
* **node:** make a stalled sync queue visible, and fix two test claims ([9aa4c17](https://github.com/Gitlawb/node/commit/9aa4c17aefd1885d134b1334b54551366cdd8ef0))
* **node:** move the attempted_at migration to v17, clear of two open branches ([fbc5868](https://github.com/Gitlawb/node/commit/fbc5868bd16946ea184dad26abcdb0e89d68e99a))
* **node:** prove the mirror path is inside repos_dir before running git ([#272](https://github.com/Gitlawb/node/issues/272)) ([3ca9473](https://github.com/Gitlawb/node/commit/3ca94739ed7b90e4c265a0947025dc3b50e7583b))
* **node:** reject a malformed slug at the notify boundary ([#272](https://github.com/Gitlawb/node/issues/272)) ([c587a40](https://github.com/Gitlawb/node/commit/c587a40a508acea9f74c3468d42ff037397c75d0))
* **node:** reset a peer's reachability gate when an announce changes its http_url ([#270](https://github.com/Gitlawb/node/issues/270)) ([1964f88](https://github.com/Gitlawb/node/commit/1964f88378fe6942ffca5836de3a092b5b8f67b4))
* **node:** reset a peer's reachability gate when an announce changes its http_url ([#270](https://github.com/Gitlawb/node/issues/270)) ([e2560ee](https://github.com/Gitlawb/node/commit/e2560ee46c28e4e966d87867a86dc4bad4ca7a3a))
* **node:** stamp every dequeued sync row so a stuck one yields the batch ([6e64d0d](https://github.com/Gitlawb/node/commit/6e64d0d0a7fdd38ef3a56820c1808dd7c9bbd658))
* **node:** stop a peer-supplied repo slug from escaping repos_dir ([#272](https://github.com/Gitlawb/node/issues/272)) ([c70b1df](https://github.com/Gitlawb/node/commit/c70b1df42d5c3f70ec4c272886c85fcc16ed92d1))
* **node:** validate the queued slug before the worker derives a path ([#272](https://github.com/Gitlawb/node/issues/272)) ([9f00e24](https://github.com/Gitlawb/node/commit/9f00e24aec217e4b8256dd437cff4741b1478b96))
* **review:** correct two comment claims and guard two more wrong shapes ([#270](https://github.com/Gitlawb/node/issues/270)) ([811ad50](https://github.com/Gitlawb/node/commit/811ad504bc268c8e9cddac93822adcd00bfe9ff8))
* **review:** stop the dequeue overclaiming, and pin the starvation premise ([457be11](https://github.com/Gitlawb/node/commit/457be11d728a2a34775cf96be2bc5af9a58cb250))
* strict DID matching in get_profile ([acb4714](https://github.com/Gitlawb/node/commit/acb47145beec8ce7b20b7a2479f7a55ed68e2a89))
* strict DID matching in get_profile ([13b565b](https://github.com/Gitlawb/node/commit/13b565bcac09f075a7fd555a6e9acdd27de6c7bb))
* target stored did on profile update aliases ([9836c3a](https://github.com/Gitlawb/node/commit/9836c3a21bc96a2ce95cb1fd12bd84b22142787d))
## [0.7.0](https://github.com/Gitlawb/node/compare/v0.6.0...v0.7.0) (2026-07-22)
### Features
* **gl:** actually verify certificate signatures in gl cert show ([b0a2ed8](https://github.com/Gitlawb/node/commit/b0a2ed830821b5a77b0dafdcf5ce4b1ad904ec25))
* **gl:** doctor warns when a shell alias shadows the gl binary ([0765c42](https://github.com/Gitlawb/node/commit/0765c42feff47b29054b9a277881a7cd5bfd9135))
### Bug Fixes
* **gl:** address review findings — non-fatal DID check, exact-host loopback, detached HEAD, multi-URL remotes, command-aware unalias ([3fa38dc](https://github.com/Gitlawb/node/commit/3fa38dce4e435ef017f607d355266c2892161f8d))
* **gl:** anchor cert --verify to a trusted issuer; pin canonical payload form ([c681af7](https://github.com/Gitlawb/node/commit/c681af70dba3a76b11795d324f2ca0fecf7e42fd))
* **gl:** detect gitlawb remotes beyond origin in gl status ([2d88937](https://github.com/Gitlawb/node/commit/2d88937fbabf21013f28debb4d4f93270a4bb8cd))
* **gl:** doctor treats a reachable local node as configuration, not failure ([94c61f7](https://github.com/Gitlawb/node/commit/94c61f79421fa26ecc1bdf7c255e0c48e66a2048))
* **gl:** make gl init's push guidance match the repo's actual state ([cb42a48](https://github.com/Gitlawb/node/commit/cb42a4830d0d070165e2212f316d009ec716dbee))
* warn about the oh-my-zsh gl alias at install time ([e3df51d](https://github.com/Gitlawb/node/commit/e3df51d76c0275fde33954ff0af0aab95fd0cb7d))
## [0.6.0](https://github.com/Gitlawb/node/compare/v0.5.1...v0.6.0) (2026-07-22)
+2 -5
View File
@@ -31,9 +31,7 @@ crates/
├── gitlawb-core/ crypto primitives (DID, CID, HTTP sigs, UCAN, ref certs)
├── gitlawb-node/ axum HTTP server, git smart HTTP, P2P, GraphQL
├── gl/ CLI — identity, repos, MCP server, Base L2 names
├── git-remote-gitlawb/ git remote helper for gitlawb:// URLs
├── gitlawb-attest/ provenance attestations for ref-update certificates
└── icaptcha-client/ client for the iCaptcha proof-of-intelligence service
└── git-remote-gitlawb/ git remote helper for gitlawb:// URLs
docs/ Operator guides
scripts/ Build helpers
```
@@ -56,7 +54,6 @@ Smart contracts live in a separate repo: [github.com/Gitlawb/contracts](https://
3. **No breaking changes without discussion.** Open an issue first for protocol-level changes.
4. **Conventional commits.** Use `feat:`, `fix:`, `docs:`, `refactor:`, `test:`, `chore:`. Releases are automated by [release-please](https://github.com/googleapis/release-please) — your commit prefixes drive the next version bump.
5. **Format and lint.** Run `cargo fmt --all` and `cargo clippy --workspace --all-targets -- -D warnings` before submitting. CI will reject anything that fails these.
6. **Security-surface rules.** [`AGENTS.md`](AGENTS.md) carries the authorization invariants PRs most often break; it is written for coding agents and humans alike, so read it before touching endpoints, visibility, or migrations.
## What gets merged, what gets closed
@@ -108,7 +105,7 @@ cargo test -p gitlawb-node
- **TypeScript SDK** (`@gitlawb/sdk`) — client library for the HTTP API
- **Python SDK** (`gitlawb`) — for ML/agent pipeline integration
- **UCAN authorization** — add trusted issuer anchoring, capability checks, and authorization-aware revocation
- **UCAN chain validation** — complete the auth middleware
- **Filecoin storage tier** — wire up cold storage deals
- **Documentation** — guides, tutorials, API examples
- **Node operators** — run a public node and report issues
Generated
+13 -91
View File
@@ -770,23 +770,6 @@ dependencies = [
"zeroize",
]
[[package]]
name = "ark-ff"
version = "0.6.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f7a806ac6c8307b929df4645776290a50ee2aac754ad09d8bdf73391309e43af"
dependencies = [
"ark-ff-asm 0.6.0",
"ark-ff-macros 0.6.0",
"ark-serialize 0.6.0",
"ark-std 0.6.0",
"digest 0.10.7",
"educe",
"num-bigint",
"num-traits",
"zeroize",
]
[[package]]
name = "ark-ff-asm"
version = "0.3.0"
@@ -817,16 +800,6 @@ dependencies = [
"syn 2.0.117",
]
[[package]]
name = "ark-ff-asm"
version = "0.6.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1479009684adc073dff49a1025d3a7065b317a9ead25aaaca38cdc70058ba8a2"
dependencies = [
"quote",
"syn 2.0.117",
]
[[package]]
name = "ark-ff-macros"
version = "0.3.0"
@@ -865,19 +838,6 @@ dependencies = [
"syn 2.0.117",
]
[[package]]
name = "ark-ff-macros"
version = "0.6.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "4a0691ed21ef00ef89c1e9bda832eba493dda3ec2f8d892fb25b705f73f06bb8"
dependencies = [
"num-bigint",
"num-traits",
"proc-macro2",
"quote",
"syn 2.0.117",
]
[[package]]
name = "ark-serialize"
version = "0.3.0"
@@ -911,30 +871,6 @@ dependencies = [
"num-bigint",
]
[[package]]
name = "ark-serialize"
version = "0.6.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a74dd304fd536fb95d0a328e72be759209cc496a9da094c5bc56e5fea4f9e86b"
dependencies = [
"ark-serialize-derive",
"ark-std 0.6.0",
"digest 0.10.7",
"num-bigint",
"serde_with",
]
[[package]]
name = "ark-serialize-derive"
version = "0.6.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "4f153690697a2b91e5e1251ff98411ee5371500a111a0fd317a70e588eb300f9"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.117",
]
[[package]]
name = "ark-std"
version = "0.3.0"
@@ -965,16 +901,6 @@ dependencies = [
"rand 0.8.6",
]
[[package]]
name = "ark-std"
version = "0.6.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "367c9c827ed431bff6868b7aa926e05b16eb46603cc8b6e768e4a5553fa1d155"
dependencies = [
"num-traits",
"rand 0.8.6",
]
[[package]]
name = "arrayref"
version = "0.3.9"
@@ -3374,22 +3300,19 @@ dependencies = [
[[package]]
name = "git-remote-gitlawb"
version = "0.7.1"
version = "0.6.0"
dependencies = [
"anyhow",
"gitlawb-core",
"libc",
"mockito",
"reqwest",
"tempfile",
"tracing",
"tracing-subscriber",
"windows-sys 0.61.2",
]
[[package]]
name = "gitlawb-attest"
version = "0.7.1"
version = "0.6.0"
dependencies = [
"base64",
"ed25519-dalek",
@@ -3406,7 +3329,7 @@ dependencies = [
[[package]]
name = "gitlawb-core"
version = "0.7.1"
version = "0.6.0"
dependencies = [
"anyhow",
"base64",
@@ -3427,20 +3350,20 @@ dependencies = [
"sha2",
"thiserror 2.0.18",
"tokio",
"url",
"uuid",
"zeroize",
]
[[package]]
name = "gitlawb-node"
version = "0.7.1"
version = "0.6.0"
dependencies = [
"alloy",
"anyhow",
"async-compression",
"async-graphql",
"async-graphql-axum",
"async-trait",
"aws-config",
"aws-sdk-s3",
"axum",
@@ -3490,7 +3413,7 @@ dependencies = [
[[package]]
name = "gl"
version = "0.7.1"
version = "0.6.0"
dependencies = [
"alloy",
"anyhow",
@@ -3543,9 +3466,9 @@ dependencies = [
[[package]]
name = "h2"
version = "0.4.18"
version = "0.4.13"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "839c0e8a181239723652be9062bb56ca5bf5f64011f73b623f6f4fc59086a228"
checksum = "2f44da3a8150a6703ed5d34e164b875fd14c2cdab9af1252a9a1020bde2bdc54"
dependencies = [
"atomic-waker",
"bytes",
@@ -3864,7 +3787,7 @@ dependencies = [
"libc",
"percent-encoding",
"pin-project-lite",
"socket2 0.5.10",
"socket2 0.6.3",
"tokio",
"tower-service",
"tracing",
@@ -5432,7 +5355,7 @@ dependencies = [
"quinn-udp",
"rustc-hash",
"rustls",
"socket2 0.5.10",
"socket2 0.6.3",
"thiserror 2.0.18",
"tokio",
"tracing",
@@ -5469,7 +5392,7 @@ dependencies = [
"cfg_aliases",
"libc",
"once_cell",
"socket2 0.5.10",
"socket2 0.6.3",
"tracing",
"windows-sys 0.52.0",
]
@@ -5819,15 +5742,14 @@ dependencies = [
[[package]]
name = "ruint"
version = "1.20.0"
version = "1.17.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f5e99bff0393163bb25029a6af25d3d8d202ba5b5438a74d1bd8789f5c822970"
checksum = "c141e807189ad38a07276942c6623032d3753c8859c146104ac2e4d68865945a"
dependencies = [
"alloy-rlp",
"ark-ff 0.3.0",
"ark-ff 0.4.2",
"ark-ff 0.5.0",
"ark-ff 0.6.0",
"bytes",
"fastrlp 0.3.1",
"fastrlp 0.4.0",
-3
View File
@@ -53,9 +53,6 @@ chrono = { version = "0.4", features = ["serde"] }
uuid = { version = "1", features = ["v4"] }
# http client
reqwest = { version = "0.12", features = ["blocking", "json", "multipart", "rustls-tls"], default-features = false }
# URL parsing (what reqwest::Url re-exports, so the shared redirect predicate can
# take a parsed URL without pulling reqwest into gitlawb-core)
url = "2"
# HMAC
hmac = "0.12"
+5 -18
View File
@@ -8,33 +8,20 @@ FROM rust:1.91-bookworm AS builder
WORKDIR /build
# Cache dependencies first for faster rebuilds.
# EVERY workspace member's manifest must be copied here, not just the ones this
# image ships: cargo loads the whole workspace before it resolves anything, so a
# missing member's Cargo.toml (or a path dependency pointing at one) aborts the
# layer before a single crate is fetched.
# Cache dependencies first for faster rebuilds
COPY Cargo.toml Cargo.lock ./
COPY crates/gitlawb-core/Cargo.toml crates/gitlawb-core/
COPY crates/gitlawb-node/Cargo.toml crates/gitlawb-node/
COPY crates/gl/Cargo.toml crates/gl/
COPY crates/git-remote-gitlawb/Cargo.toml crates/git-remote-gitlawb/
COPY crates/gitlawb-attest/Cargo.toml crates/gitlawb-attest/
COPY crates/icaptcha-client/Cargo.toml crates/icaptcha-client/
# Fetch deps (this layer caches until Cargo.{toml,lock} change).
# No `|| true`: this layer is expected to succeed, and swallowing its exit code
# is what let it sit dead from the moment gl gained a path dependency on
# icaptcha-client. A failure here should stop the build, not silently turn every
# image build into a cold one.
RUN mkdir -p crates/gitlawb-core/src crates/gitlawb-node/src crates/gl/src \
crates/git-remote-gitlawb/src crates/gitlawb-attest/src crates/icaptcha-client/src && \
# Fetch deps (this layer caches until Cargo.{toml,lock} change)
RUN mkdir -p crates/gitlawb-core/src crates/gitlawb-node/src crates/gl/src crates/git-remote-gitlawb/src && \
echo 'fn main() {}' > crates/gitlawb-node/src/main.rs && \
echo 'fn main() {}' > crates/gl/src/main.rs && \
echo 'fn main() {}' > crates/git-remote-gitlawb/src/main.rs && \
echo '' > crates/gitlawb-core/src/lib.rs && \
echo '' > crates/gitlawb-attest/src/lib.rs && \
echo '' > crates/icaptcha-client/src/lib.rs && \
cargo build --release --locked -p gitlawb-node -p gl -p git-remote-gitlawb
cargo build --release -p gitlawb-node -p gl -p git-remote-gitlawb || true
# Now copy real sources and build for real.
# Force-bump mtimes so cargo's fingerprint check rebuilds — without this,
@@ -47,7 +34,7 @@ RUN find crates -name "*.rs" -exec touch {} + && \
rm -rf target/release/.fingerprint/gitlawb-node-* \
target/release/.fingerprint/gl-* \
target/release/.fingerprint/git-remote-gitlawb-* && \
cargo build --release --locked -p gitlawb-node -p gl -p git-remote-gitlawb && \
cargo build --release -p gitlawb-node -p gl -p git-remote-gitlawb && \
strip target/release/gitlawb-node target/release/gl target/release/git-remote-gitlawb
# ── Runtime stage ───────────────────────────────────────────────────────────
+2 -5
View File
@@ -3,10 +3,7 @@
# Used by scripts/build-bins.sh — not used for node deployment.
ARG TARGET=x86_64-unknown-linux-musl
# Must stay at or above the workspace `rust-version` in Cargo.toml. An unmet
# MSRV is a hard cargo error, and nothing in CI builds this file, so a stale
# base image here fails only when someone runs scripts/build-bins.sh.
FROM rust:1.91-bookworm AS builder
FROM rust:1.85-bookworm AS builder
ARG TARGET
WORKDIR /build
@@ -26,7 +23,7 @@ COPY crates/ crates/
# Configure cross-linker for aarch64
ENV CARGO_TARGET_AARCH64_UNKNOWN_LINUX_MUSL_LINKER=aarch64-linux-gnu-gcc
RUN cargo build --release --locked --target "$TARGET" -p gl -p git-remote-gitlawb
RUN cargo build --release --target "$TARGET" -p gl -p git-remote-gitlawb
FROM debian:bookworm-slim
ARG TARGET
+22 -92
View File
@@ -4,7 +4,7 @@
Gitlawb Node is the open-source node software behind the Gitlawb network. It lets anyone run a self-hosted node, publish repositories under a DID, sign writes with Ed25519 HTTP signatures, replicate git activity across peers, and move toward a resilient app-delivery network where code and build assets can be served closer to users.
Gitlawb is not trying to be only "another git host." The long-term direction is:
Gitlawb is not trying to be only “another git host.” The long-term direction is:
```txt
Decentralized GitHub
@@ -19,7 +19,7 @@ The mission is simple: once code is pushed to the network, it should not disappe
## What is in this repository?
This is a Rust workspace with six crates:
This is a Rust workspace with four crates:
| Crate | Purpose |
|---|---|
@@ -27,8 +27,6 @@ This is a Rust workspace with six crates:
| `gl` | The Gitlawb CLI for identity, repos, issues, PRs, bounties, tasks, peers, node status, MCP, and setup flows. |
| `git-remote-gitlawb` | Git remote helper for `gitlawb://` URLs, so normal `git clone`, `git fetch`, and `git push` can talk to Gitlawb nodes. |
| `gitlawb-core` | Shared primitives: Ed25519 identities, `did:key`, CIDs, RFC 9421 HTTP signatures, certificates, and UCAN tokens. |
| `gitlawb-attest` | Pluggable external provenance attestations for ref-update certificates. |
| `icaptcha-client` | Client for the iCaptcha proof-of-intelligence flow used to protect spam-prone writes. |
---
@@ -36,12 +34,12 @@ This is a Rust workspace with six crates:
Most git hosting today depends on a small number of centralized platforms. Gitlawb Nodes are designed for a different model:
- **Own your identity**: every user, agent, and node is an Ed25519 keypair represented as `did:key:z6Mk...`.
- **Signed writes by default**: write requests use RFC 9421 HTTP Signatures instead of passwords.
- **Git-native transport**: repositories are still real git repositories served over smart HTTP.
- **Agent-native workflows**: the `gl` CLI and MCP server expose repo, issue, task, PR, and UCAN flows to AI agents.
- **Peer-aware delivery**: nodes can announce, discover, gossip, and sync with each other.
- **App CDN direction**: the network can evolve from decentralized code storage into code + asset + app delivery.
- **Own your identity** — every user, agent, and node is an Ed25519 keypair represented as `did:key:z6Mk...`.
- **Signed writes by default** — write requests use RFC 9421 HTTP Signatures instead of passwords.
- **Git-native transport** — repositories are still real git repositories served over smart HTTP.
- **Agent-native workflows** — the `gl` CLI and MCP server expose repo, issue, task, PR, and UCAN flows to AI agents.
- **Peer-aware delivery** — nodes can announce, discover, gossip, and sync with each other.
- **App CDN direction** — the network can evolve from decentralized code storage into code + asset + app delivery.
---
@@ -56,7 +54,6 @@ Good today:
- Bare git repository storage.
- Git smart-HTTP clone/fetch/push.
- RFC 9421-signed writes.
- Repository and path-scoped visibility enforcement for repository and Git content reads, with 404-shaped repository denials.
- DID identities.
- `gl` CLI workflows.
- libp2p peer discovery/gossip foundation.
@@ -66,13 +63,11 @@ Good today:
Known limitations:
- Repository write authorization is not secure by default: `GITLAWB_ENFORCE_OWNER_PUSH` defaults to `false` for compatibility, so a valid HTTP Signature identifies a pusher but does not enforce owner-only pushes.
- UCAN proof chains are validated when supplied, but UCAN capabilities are not consulted by write authorization and the root issuer is not independently trust-anchored. UCANs therefore do not yet grant scoped collaborator access.
- Agent lifecycle revocation is not enforced by HTTP Signature authorization; do not rely on removing or revoking an agent record to block a compromised signer.
- Read visibility is not a blanket data-classification boundary: task, IPFS-pin, and Arweave-anchor listings are not repository-gated; withheld path names can be visible to a root reader; and later visibility changes cannot retract content already announced or externally anchored.
- Private repository read enforcement is not wired yet. Treat public nodes as public infrastructure unless you restrict access at your proxy/firewall.
- UCAN chain validation and revocation are not complete.
- Repository write authorization is not capability-complete yet; HTTP signatures prove identity, not full authorization policy.
- Peer writes are signed by upgraded nodes, but strict signed-peer enforcement is opt-in during rolling upgrades.
- Current GraphQL mutations require an authenticated signer, but there is no mutation-specific guardrail that prevents a future mutation from omitting that check.
- Pull-request review comments do not yet have threaded line-level anchors, and merges do not enforce approval requirements.
- GraphQL mutations need mutation-aware auth before becoming a public write surface.
See:
@@ -194,14 +189,14 @@ For public-network use, make sure `GITLAWB_NODE` points to the node you want. Th
Public nodes (e.g. `node.gitlawb.com`) require two things on writes:
1. **RFC 9421 HTTP Signatures**: every write is signed by your identity key. `gl`
1. **RFC 9421 HTTP Signatures** — every write is signed by your identity key. `gl`
and the `git-remote-gitlawb` helper do this automatically. An old/unsigned CLI
fails with `401 not_an_agent`; `gl` will tell you to upgrade and register.
2. **An iCaptcha proof** on the spam-gated writes (**repo create, fork, register**).
`gl` solves this for you: on the node's `403 icaptcha_proof_required` it reads the
`x-icaptcha-url` / `x-icaptcha-level` hints, requests a challenge, solves it
locally (arithmetic / algebra / sequence), and **retries the same signed request**
with the `x-icaptcha-proof` header. No manual steps, no env vars.
with the `x-icaptcha-proof` header — no manual steps, no env vars.
```bash
gl identity new # create did:key identity
@@ -220,61 +215,14 @@ Notes:
- **Proofs are short-lived (~5 min TTL) and single-use.** If one expires between
solving and use, the client transparently solves a fresh one and retries.
- **What needs what:** create / fork / register are signed **and** iCaptcha-gated;
`git push` is **signed-only** (owner signature is the gate, no per-push challenge);
`git push` is **signed-only** (owner signature is the gate — no per-push challenge);
reads (clone / fetch / `repo info`) need no proof. A non-existent repo returns a
clear `404`, never a placeholder.
- **API-key iCaptcha deployments:** set `GITLAWB_ICAPTCHA_URL` to your iCaptcha
origin and `GITLAWB_ICAPTCHA_API_KEY` to its key. The client only talks to an
`https` origin whose host is allowlisted (that URL or the public default), and
sends the bearer token **only** to your configured origin, never to a URL a
node advertises, so a hostile node can't capture the key or redirect the solve.
### Fetching an object by CID
```bash
gl ipfs list # CIDs this node has pinned
gl ipfs get bafkrei... > object.bin # object bytes on stdout
```
Objects that were pinned before the node started recording which repo they came
from are found by scanning its repo inventory, and that scan stops at the
per-request ceilings in the [Configuration](#configuration) table. A stopped scan
answers 503 with a resume token instead of a false "not found", and `gl ipfs get`
follows the token automatically: up to 8 resumes after the first request, so at
most 9 calls to the node, waiting between attempts for as long as the node's
`Retry-After` asks and never longer than 5 seconds.
The whole ladder runs under a 60 second wall-clock deadline. The deadline bounds
the search, not the download: each attempt gets the time left on it to produce
response headers, and once an object is found its bytes stream outside that
deadline. They are not unbounded, though. The client's own 30 second HTTP timeout
is a total request timeout, running from the start of a request until its body
has finished, so a transfer still going 30 seconds after its request began is cut
off. Waits between attempts never run past the deadline either, so a single run
spends at most around 90 seconds on the network: the deadline plus the 30 second
timeout covering the last attempt. Writing the object out sits outside both
bounds, so piping into a reader that stops reading can hold the command open
longer than that.
Two node-side brakes end a ladder early and are reported rather than retried
around. A 429 is terminal, because the node's rate-limit window is an hour and
that wait cannot be honored inside one invocation; a transient overload (a 503
carrying no incomplete-scan code) is retried on the token already held, under the
same cap, clamp and deadline. The per-IP fanout brake can also stop a ladder well
short of the 9 calls, so automatic resumption is not a guarantee of reaching the
object.
When a bound stops the ladder with a usable token in hand, the command prints the
token and the invocation that continues from it before exiting nonzero:
```txt
resume from where this stopped: gl ipfs get bafkrei... --scan <token>
```
Run that to carry on from where the scan stopped. Re-running without `--scan`
restarts at the first row, reproduces the same truncation and spends the node's
per-IP budget again, so the token is the only thing that makes progress. Tokens
are valid for an hour.
sends the bearer token **only** to your configured origin — never to a URL a
node advertises — so a hostile node can't capture the key or redirect the solve.
---
@@ -361,7 +309,7 @@ POST /api/v1/peers/announce
POST /api/v1/sync/notify
```
When `GITLAWB_REQUIRE_SIGNED_PEER_WRITES=false`, unsigned legacy peers are accepted on those two routes, but signed requests are verified when signature headers are present. Staged rollout relaxes who may announce, not who owns a peer row. An unsigned announce may register a previously unseen peer, and it may refresh an existing row whose `http_url` is unchanged, but changing an existing peer's `http_url` requires an RFC 9421 signature from that peer's own DID and is refused with 403 otherwise. An unsigned announce can only register a `did:key`, since that is the only method whose verifying key can be resolved and therefore the only kind of row anyone could ever correct through the signed path; any other DID is refused with 400. Once all live peers upgrade, operators can set:
When `GITLAWB_REQUIRE_SIGNED_PEER_WRITES=false`, unsigned legacy peers are accepted on those two routes, but signed requests are verified when signature headers are present. Once all live peers upgrade, operators can set:
```bash
GITLAWB_REQUIRE_SIGNED_PEER_WRITES=true
@@ -392,34 +340,16 @@ Important node settings:
| `GITLAWB_BOOTSTRAP_PEERS` | Comma-separated HTTP peer URLs. |
| `GITLAWB_P2P_BOOTSTRAP` | Comma-separated libp2p multiaddrs. |
| `GITLAWB_BOOTSTRAP_DISABLE_SEEDS` | Disable embedded seed peers for isolated dev/test networks. |
| `GITLAWB_REQUIRE_SIGNED_PEER_WRITES` | Require signed peer announce/sync writes. Defaults to `false` during the staged rollout below. |
| `GITLAWB_ENFORCE_OWNER_PUSH` | Require the authenticated pusher to be the repo owner on `git-receive-pack`. **Defaults to `true`.** A `did:key` signature is authentication, not authorization — anyone can mint a key and sign — so with this off every signed caller may push to every repository, private ones included. Delegated and CI keys count as non-owners: a UCAN `git/push` capability is verified but not yet honored for authorization, so they cannot push while this is on. Set `false` only for a rolling upgrade; see [`docs/RUN-A-NODE.md`](docs/RUN-A-NODE.md). |
| `GITLAWB_REQUIRE_SIGNED_PEER_WRITES` | Require signed peer announce/sync writes. |
| `GITLAWB_AUTO_SYNC` | Enable automatic sync from known peers. |
| `GITLAWB_MAX_PACK_BYTES` | Max git pack body size for smart-HTTP routes. |
| `GITLAWB_GIT_SERVICE_TIMEOUT_SECS` | Max seconds a served git upload-pack, receive-pack, or `info/refs` advertisement may run before it is aborted (504). Default 600. Also bounds the withheld-blob classification walk (on both the upload-pack serve and receive-pack replication paths) and the push-side pin-candidate discovery (`rev-list` / `cat-file`), each reaped via process-group teardown at the deadline. On the path-scoped upload-pack path the classification walk and the pack serve share ONE deadline, so this value bounds their combined duration rather than granting each stage a full budget: a walk that consumes it leaves the serve nothing and the clone gets a 504. Serving large path-scoped repos may therefore need a higher value than they did when each stage was budgeted separately. Accepted range is 1 to 3153600000 (100 years), since the node derives deadlines from this value and a larger one cannot be represented. |
| `GITLAWB_GIT_ACQUIRE_TIMEOUT_SECS` | Max seconds the storage-acquisition phase (Tigris HEAD/GET, push advisory-lock) of a served git op may run before the request is shed with a 503, separate from the git-run timeout. The concurrency permit is released on expiry so a stalled backend cannot pin the pool. Default 30. |
| `GITLAWB_MAX_CONCURRENT_GIT_OPS` | Max concurrent served git READ ops (upload-pack and its `info/refs` advertisement) across all callers; over-cap sheds a 503 + Retry-After. Anonymous reads draw from this pool, so pair it with `GITLAWB_MAX_CONCURRENT_READS_PER_CALLER`. Pushes and the receive-pack advertisement have their own pools, so a read flood cannot shed an authenticated push. Default 128. |
| `GITLAWB_MAX_CONCURRENT_GIT_PUSHES` | Max concurrent `git-receive-pack` POST operations, in a pool separate from the read pool. The anon receive-pack `info/refs` advertisement runs in a third pool of the same size, disjoint from both, so an advertisement flood cannot shed a push either. Two per-source push caps are derived from this value (`/8`, floor 1) and have no env var of their own. Over-cap sheds a 503 + Retry-After. Default 32. |
| `GITLAWB_MAX_CONCURRENT_READS_PER_CALLER` | Max concurrent read ops a single caller may hold, so one caller cannot monopolize the read pool. Keyed on the resolved source IP, never the DID, and only as granular as `GITLAWB_TRUSTED_PROXY`: left unset, a node behind an edge or NAT keys every caller on the edge IP and this collapses to one global cap. Default 16. |
| `GITLAWB_MAX_CONCURRENT_PIN_TASKS` | Max post-push pin loops (IPFS + Pinata) running concurrently across all repos. This caps how many loops RUN at once, not how much object-id list memory the node retains: on the local IPFS path a loop parked waiting for a permit still holds its full list. Do not size memory from this knob alone. A loop over cap waits, never drops a pin. Default 8. |
| `GITLAWB_REPO_LEASE_MAX_WAITERS` | Max pushes parked at once waiting for the same repo's write lease. Each waiter pins its buffered pack body, so this bounds that memory for a hot repo; past the cap the newest push sheds a 503 + Retry-After instead of queueing. Pushes to other repos are unaffected, and the lease holder is not counted. Default 8. |
| `GITLAWB_MAX_CONCURRENT_IPFS_WALKS` | Max concurrent `GET /ipfs/{cid}` visibility walks across all callers (own pool, disjoint from the served-git pools); over-cap sheds 503. Default 32. |
| `GITLAWB_IPFS_WALK_PER_SOURCE` | Max concurrent `/ipfs` walks a single source IP may hold. Default 4. |
| `GITLAWB_IPFS_MAX_LEGACY_PROBES` | Max legacy (NULL-provenance) repos probed per `/ipfs/{cid}` request, bounding the scan-fallback fan-out. A truncated scan returns a retryable 503, not a false 404. Default 256. |
| `GITLAWB_IPFS_MAX_LEGACY_SCAN_ROWS` | Max repo rows one `/ipfs/{cid}` request's legacy scan may read from the database. The probe ceiling above only starts counting once a probe runs, and quarantined or private repos are denied before that, so this is what bounds a scan over an inventory that denies the caller everywhere. A truncated scan sheds a retryable 503 carrying an opaque `continuation` token; echoing it as `?scan=` resumes the scan where it stopped. Every per-request ceiling on this path (rows, probes, visits, retained rule bytes) mints one, so each request advances the ladder by at least the rows it read and a holder buried past a ceiling is reached in a bounded number of requests, `ceil(repos / ceiling) + 1` when this row ceiling is the one that binds. No ceiling ever produces a 404. Every page is charged to the caller's `/ipfs` work allowance, so raising this raises that allowance too. Lowering it sharpens an oracle: because a truncation emits a token and a completed wrap does not, laddering to the end reveals the node's total repo count (private and quarantined included) to within one ceiling. Default 2048. |
| `GITLAWB_IPFS_MAX_REPOS_WALKED` | Max expensive path-scope visibility walks a `/ipfs/{cid}` request may run per phase; over-cap repos are skipped and the scan continues, shedding a retryable 503 (not a false 404) if the object is then found nowhere. The effective cap is the tighter of this knob and the node's internal history-walk ceiling of 17 (`MAX_PIN_SOURCES + 1`), so a value above 17 has no effect while a value below it does tighten the cap. It is charged per phase: the provenance lookup and the legacy-scan fallback get separate equal budgets, so one request can run up to twice the cap in total. Default 64. |
| `GITLAWB_IPFS_MAX_REPO_VISITS` | Ceiling on repos one `/ipfs/{cid}` request may visit (acquire + probe) past the visibility gate. Also the worst-case per-request Tigris fetch count. On exhaustion the scan stops with a retryable 503. Default 1024. |
| `GITLAWB_IPFS_REQUEST_BUDGET_SECS` | Absolute wall-clock budget for one admitted `/ipfs/{cid}` request's acquire+walk lifetime. Per-stage clamps bound the acquire and walk stages to the remaining budget, and no stage starts once it is exhausted; the scan then stops with a retryable 503. The object-type probe and content-read `cat-file` subprocesses are budget-checked before starting and each also run under their own deadline (the lesser of `GITLAWB_GIT_SERVICE_TIMEOUT_SECS` and the remaining budget), reaped via process-group teardown, so a hung `cat-file` cannot hold the request's walk slot past it. One hang path is still unbounded: the probe's object-store readability check is a plain filesystem sweep with nothing to reap, so a wedged filesystem can hold the slot past the deadline. Default 600. Accepted range is 1 to 3153600000 (100 years), since the node derives a deadline from this value and a larger one cannot be represented. |
| `GITLAWB_IPFS_RESOLVE_BUDGET_SECS` | Shorter budget for the pre-walk CID resolve inside an admitted `/ipfs/{cid}` request: the lookup that maps the requested CID to its git oid(s), which runs while the scarce walk admission is already held. A well-formed CID with no pin row does no probe and no walk work, so without this it could hold a walk slot for the whole request budget while nothing walked, and enough such requests shed every real retrieval at admission. The effective deadline is the lesser of this and the remaining request budget, so a value above `GITLAWB_IPFS_REQUEST_BUDGET_SECS` degrades to the request budget. Only the resolve is on this clock; walk and probe work stay on the request budget, so a slow but progressing scan is never shed by it. Default 10. Accepted range is 1 to 3153600000 (100 years). |
| `GITLAWB_IPFS_RATE_LIMIT` | Max `/ipfs/{cid}` requests per client IP per hour (route flood brake). 0 disables. Default 600. |
| `GITLAWB_GIT_SERVICE_TIMEOUT_SECS` | Max seconds a served git upload-pack/receive-pack may run before it is aborted (504). Default 600. Does not bound `info/refs` or the withheld-blob path. |
| `GITLAWB_TIGRIS_BUCKET` | Optional S3/Tigris shared repo storage bucket. |
| `GITLAWB_PINATA_JWT` | Optional Pinata/IPFS warm-storage pinning. |
| `GITLAWB_IRYS_URL` | Optional Irys/Arweave permanent anchoring. |
Production note: change the default Postgres password before exposing a node publicly.
Legacy-pin window: releases before the CID-resolver work stored the provider CID (Kubo dag-pb / Pinata) as a pinned object's resolver key. The `/ipfs/{cid}` resolver now recomputes the raw-content CID from the object bytes and refuses to serve a key that does not match, so `GET /api/v1/ipfs/pins` can still advertise an unrepaired legacy CID that 404s. Such a row is repaired opportunistically the next time a push carries the object again (its key is rewritten to the raw CID, the old value kept in `legacy_provider_cid`), but git negotiation omits objects the node already has, so most legacy rows never re-enter a push delta. A deferred one-shot startup sweep, not this opportunistic path, is what fully retires the advertise-then-404 window. Rows whose object bytes are gone stay withheld.
---
## Optional node staking
@@ -524,8 +454,8 @@ Short-term priorities:
2. Add Docker and installer smoke tests.
3. Improve operator docs and `gl doctor` checks.
4. Harden peer writes and publish the signed-peer rollout plan.
5. Close default-open write authorization and wire trusted UCAN delegation into repository permissions.
6. Add threaded, line-level pull-request discussions and enforce approval requirements on merges.
5. Implement repo write authorization: owner checks, protected branches, and UCAN capability checks.
6. Implement private-read enforcement or remove private repo affordances until it exists.
7. Add metrics for pushes, fetches, pack sizes, peer sync, failed auth, and webhooks.
Product direction:
+22 -32
View File
@@ -28,11 +28,10 @@ We will acknowledge receipt within 48 hours and aim to release a fix within 14 d
- Every git object is content-addressed via CIDv1 (SHA-256)
- Tamper-evident by construction — a modified object changes its CID
**UCAN token validation**
- Bootstrap UCAN tokens are issued at registration.
- A supplied token's signature, audience, expiry, and proof-chain attenuation are validated.
- Tokens use a signed JSON wire format with expiry.
- Capability grants are not yet consulted by repository write authorization; see the limitations below.
**UCAN capability tokens**
- Bootstrap UCAN tokens issued at registration
- Capability-scoped: `git:push`, `git:fetch`, `issue:create`, `pr:open`
- JWT-format tokens with expiry
**Smart contracts (Base Sepolia testnet)**
- `GitlawbDIDRegistry` — on-chain DID → document registry
@@ -49,20 +48,21 @@ We will acknowledge receipt within 48 hours and aim to release a fix within 14 d
---
## Known Limitations
## Known Limitations (Planned for v0.2)
These are documented limitations of the current live release. They should be prioritized without breaking existing nodes during rolling upgrades.
These are **documented, accepted limitations** for the current live release and should be prioritized without breaking existing nodes during rolling upgrades.
### Repository write authorization defaults
- `git-receive-pack` verifies HTTP Signatures, but `GITLAWB_ENFORCE_OWNER_PUSH` defaults to `false` for compatibility during rollout.
- **Impact:** With the default setting, a valid signature authenticates the pusher but does not require that DID to be the repository owner.
- **Mitigation:** Set `GITLAWB_ENFORCE_OWNER_PUSH=true` on nodes where owner-only pushes are required. Confirm that every legitimate pusher uses the owner DID before enabling it.
### UCAN chain validation
- The auth middleware verifies HTTP Signatures and token structure, but does not yet walk the full UCAN delegation chain.
- **Impact:** A node cannot yet enforce fine-grained capability delegation. Currently, any registered agent with a valid HTTP Signature can push.
- **Mitigation:** Keep write endpoints signed, treat public nodes as public infrastructure, and treat trust scores as soft rate-limiting signals rather than authorization.
- **Fix target:** v0.2
### UCAN delegation and revocation
- The middleware validates a supplied UCAN's complete proof chain, but a root token is accepted without an independently trusted issuer anchor. `Ucan::can` is not yet used by write handlers, so a UCAN does not grant scoped repository access.
- Agent lifecycle revocation is not checked by HTTP Signature authorization. Removing or revoking an agent record does not itself block a compromised DID from authenticating.
- **Impact:** Do not use UCANs for collaborator permissions or rely on agent-record revocation as a key-compromise response.
- **Mitigation:** Keep sensitive deployments behind operational network controls and enable owner-only push enforcement where it fits the deployment until trusted delegation and authorization-aware revocation are implemented.
### UCAN revocation
- Issued UCAN tokens cannot be revoked before expiry.
- **Impact:** If a keypair is compromised, the attacker retains access until the UCAN expires (default: 30 days).
- **Mitigation:** Regenerate your identity (`gl identity new --force`) and re-register to issue a new UCAN. Until revocation/blocklisting is implemented, operators should remove compromised DIDs directly from their local database.
- **Fix target:** v0.2
### git-receive-pack authentication
- The `git-receive-pack` endpoint enforces HTTP Signature auth. Plain Git smart-HTTP clients do not generate those headers, so the `git-remote-gitlawb` helper is required for pushes.
@@ -71,20 +71,10 @@ These are documented limitations of the current live release. They should be pri
- **Fix target:** v0.2
### Private repository reads
- Repository and path-scoped visibility checks are enforced for repository API and Git content reads. A denied whole-repository or root read returns the same 404 shape as a missing repository, so the denial does not reveal private-repository existence.
- Sparse-clone support exposes withheld path globs to callers who may read the repository root. Do not put sensitive information in withheld path names.
- `GET /api/v1/tasks`, `/api/v1/ipfs/pins`, and `/api/v1/arweave/anchors` are not repository-gated. Task records include a UCAN token; pin and anchor listings expose object and ref metadata.
- Changing a repository's visibility controls future serving, but cannot retract ref metadata or configured external pins and anchors already announced while the repository was public. Do not push secrets to an announceable repository.
- **Impact:** Visibility policies protect the repository and Git content routes they gate, not every metadata endpoint or previously published content.
- **Remaining boundary:** This read control does not address the independent write-authorization and UCAN-delegation limitations described above.
### Pull-request review enforcement
- Pull-request review comments are not yet threaded or line-anchored, and merges do not enforce required approvals.
- **Impact:** Teams must use their own review policy or external controls for merge approval requirements.
### GraphQL mutation coverage
- Existing GraphQL mutations require an authenticated signer, but a mutation-specific source-level guardrail has not yet been added for future mutations.
- **Impact:** A new mutation could accidentally omit its signer check without an explicit test fence.
- Repository records have an `is_public` field and the node exposes `GITLAWB_PUBLIC_READ`, but per-repository private-read enforcement is not wired in the current live release.
- **Impact:** Do not store private repositories or secrets on public nodes.
- **Mitigation:** Run isolated nodes for non-public data and restrict network access at the reverse proxy or firewall layer.
- **Fix target:** v0.2
### Peer route hardening rollout
- Peer announce and sync notification routes accept signed requests and verify DID matches when a signature is present.
@@ -111,7 +101,7 @@ These are documented limitations of the current live release. They should be pri
| Key storage | PKCS#8 PEM, 0600 permissions |
| Content hashing | SHA-256 via CIDv1 |
| HTTP Signatures | RFC 9421 (Ed25519 + SHA-256 Content-Digest) |
| UCAN tokens | Signed JSON object (Ed25519 signature) |
| UCAN tokens | JWT (Ed25519 signatures) |
| On-chain | ECDSA secp256k1 (Base L2 / Ethereum) |
---
@@ -119,7 +109,7 @@ These are documented limitations of the current live release. They should be pri
## Threat Model
gitlawb is designed to be secure against:
- **Unauthenticated writes** — HTTP Signature auth on protected write endpoints; see Known Limitations for owner and capability authorization gaps
- **Unauthorized writes** — HTTP Signature auth on all write endpoints
- **Tampered git objects** — CIDv1 content addressing detects modification
- **Identity spoofing** — DIDs derived from public keys, unforgeable without the private key
- **Centralized takedown** — no single point of control; data on IPFS + Arweave
-112
View File
@@ -1,112 +0,0 @@
# Dependency allowlist for the `gitlawb-core` crate.
#
# gitlawb-core is the shared-primitives crate every consumer embeds (the `gl`
# CLI, git-remote-gitlawb, the node daemon). It must stay lean so those
# consumers do not inherit daemon-weight dependencies. This file is the source
# of truth for the crates gitlawb-core's NORMAL (non-dev, non-build) transitive
# dependency closure is allowed to contain. CI (`scripts/check-gitlawb-core-deps.sh`,
# wired into the `core-deps-purity` job in .github/workflows/pr-checks.yml)
# fails if gitlawb-core's tree contains any crate not listed here.
#
# The allowlist is exhaustive-by-construction: a NEW dependency reds CI whether
# or not anyone thought to ban it, which a denylist cannot do.
#
# Measured in-workspace, so feature unification across the workspace can, in
# rare cases, surface a crate here that gitlawb-core would not pull standalone.
# An unexpected new entry is a prompt to check what pulled it in, not noise to
# rubber-stamp.
#
# Regenerate (from repo root, after an INTENTIONAL dependency change):
# cargo tree -p gitlawb-core --edges normal --prefix none \
# | sed -E 's/ v[0-9].*$//' \
# | grep -v '^gitlawb-core$' \
# | sort -u
# then paste the result below this header block. (The checker snapshots and
# restores Cargo.lock; a manual regen may refresh it — `git checkout Cargo.lock`
# afterward if you did not intend that.)
#
# Lines starting with `#` and blank lines are ignored by the checker.
aead
anyhow
base-x
base256emoji
base64
base64ct
block-buffer
cfg-if
chacha20
chacha20poly1305
chrono
cid
cipher
const-oid
const-str
core2
cpufeatures
crypto-common
crypto_box
crypto_secretbox
curve25519-dalek
curve25519-dalek-derive
data-encoding
data-encoding-macro
data-encoding-macro-internal
der
digest
ed25519
ed25519-dalek
equivalent
generic-array
getrandom
hashbrown
hex
iana-time-zone
indexmap
inout
itoa
libc
match-lookup
memchr
multibase
multihash
multihash-codetable
multihash-derive
multihash-derive-impl
num-traits
opaque-debug
pem-rfc7468
pkcs8
poly1305
ppv-lite86
proc-macro-crate
proc-macro2
quote
rand
rand_chacha
rand_core
salsa20
serde
serde_core
serde_derive
serde_json
sha2
signature
spki
subtle
syn
synstructure
thiserror
thiserror-impl
toml_datetime
toml_edit
toml_parser
typenum
unicode-ident
universal-hash
unsigned-varint
uuid
winnow
zerocopy
zeroize
zeroize_derive
zmij
+2 -19
View File
@@ -1,7 +1,7 @@
[package]
name = "git-remote-gitlawb"
description = "Git remote helper — enables 'git clone gitlawb://did:gitlawb:...'"
version = "0.7.1" # x-release-please-version
version = "0.6.0" # x-release-please-version
edition.workspace = true
rust-version.workspace = true
license.workspace = true
@@ -11,7 +11,7 @@ name = "git-remote-gitlawb"
path = "src/main.rs"
[dependencies]
gitlawb-core = { path = "../gitlawb-core", features = ["redirect"] }
gitlawb-core = { path = "../gitlawb-core" }
anyhow = { workspace = true }
reqwest = { workspace = true }
tracing = { workspace = true }
@@ -19,20 +19,3 @@ tracing-subscriber = { workspace = true }
[dev-dependencies]
mockito = "1"
tempfile = "3"
[target.'cfg(unix)'.dev-dependencies]
libc = "0.2"
[target.'cfg(windows)'.dev-dependencies]
# Job Object bindings for the real_git_fetch harness: the Windows analog of the
# unix process-group teardown, so the whole `git fetch` tree can be killed
# together on both the timeout and clean-exit paths (INV-22). Win32_Security is
# required by CreateJobObjectW, and Win32_System_Threading by
# JOBOBJECT_EXTENDED_LIMIT_INFORMATION (it embeds IO_COUNTERS).
windows-sys = { version = "0.61", features = [
"Win32_Foundation",
"Win32_Security",
"Win32_System_JobObjects",
"Win32_System_Threading",
] }
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
+1 -1
View File
@@ -1,7 +1,7 @@
[package]
name = "gitlawb-attest"
description = "External Attestation v1: pluggable provenance attachments for gitlawb ref-update certs"
version = "0.7.1" # x-release-please-version
version = "0.6.0" # x-release-please-version
edition.workspace = true
rust-version.workspace = true
license.workspace = true
+2 -37
View File
@@ -16,7 +16,7 @@
//! by exact match.
use base64::{engine::general_purpose::URL_SAFE_NO_PAD as B64U, Engine};
use ed25519_dalek::{Signature, Signer, SigningKey, VerifyingKey};
use ed25519_dalek::{Signature, Signer, SigningKey, Verifier, VerifyingKey};
use serde::{de::DeserializeOwned, Deserialize, Serialize};
use crate::error::{Error, Result};
@@ -111,7 +111,7 @@ impl Attestation {
.try_into()
.map_err(|_| Error::Signature("signature must be 64 bytes".to_string()))?;
let sig = Signature::from_bytes(&sig_bytes);
vk.verify_strict(&bytes, &sig)
vk.verify(&bytes, &sig)
.map_err(|e| Error::Signature(format!("ed25519: {e}")))?;
Ok(vk)
@@ -554,39 +554,4 @@ mod tests {
let err = tampered.verify_signature(cert_hash).unwrap_err();
assert!(matches!(err, Error::Signature(_)));
}
/// Regression guard for strict verification: a signature forged under a
/// weak (small-order) public key satisfies the verification equation
/// but must be rejected. The identity point is such a key: with R the
/// identity and S = 0, [S]B - [k]A is the identity for any message.
#[test]
fn verify_rejects_weak_key_signature() {
let mut weak_key_bytes = [0u8; 32];
weak_key_bytes[0] = 1; // compressed identity point (y = 1, x = 0)
let weak_vk = VerifyingKey::from_bytes(&weak_key_bytes).unwrap();
assert!(weak_vk.is_weak(), "identity point must be a weak key");
let cert_hash = sample_cert_hash();
let mut forged = [0u8; 64];
forged[0] = 1; // R = identity point, S = 0
let forged_sig = B64U.encode(forged);
// Build an attestation with the weak key as signer and the forged
// signature. The weak-key check happens at verify time.
let mut att = dummy_attestation(&SigningKey::generate(&mut OsRng), cert_hash);
let mut buf = Vec::with_capacity(ED25519_MULTICODEC.len() + 32);
buf.extend_from_slice(&ED25519_MULTICODEC);
buf.extend_from_slice(&weak_key_bytes);
att.signer = format!(
"did:key:{}",
multibase::encode(multibase::Base::Base58Btc, &buf)
);
att.sig = forged_sig;
let err = att.verify_signature(cert_hash).unwrap_err();
assert!(
matches!(err, Error::Signature(_)),
"signature under a weak (small-order) public key must be rejected"
);
}
}
+1 -8
View File
@@ -1,7 +1,7 @@
[package]
name = "gitlawb-core"
description = "Core cryptographic primitives for the gitlawb network: DIDs, CIDs, UCAN, HTTP Signatures, ref-update certificates"
version = "0.7.1" # x-release-please-version
version = "0.6.0" # x-release-please-version
edition.workspace = true
rust-version.workspace = true
license.workspace = true
@@ -22,7 +22,6 @@ multihash-codetable = { workspace = true }
cid = { workspace = true }
chrono = { workspace = true }
uuid = { workspace = true }
url = { workspace = true, optional = true }
zeroize = { version = "1", features = ["derive"] }
pkcs8 = { version = "0.10", features = ["pem", "std"] }
curve25519-dalek = "4"
@@ -31,9 +30,3 @@ chacha20poly1305 = "0.10"
[dev-dependencies]
tokio = { workspace = true }
# Not optional here: the redirect matrix must run under a bare
# `cargo test -p gitlawb-core`, with no feature selected.
url = { workspace = true }
[features]
redirect = ["dep:url"]
+2 -35
View File
@@ -6,8 +6,6 @@
//! The schema is frozen at v1. All fields are mandatory for forward compatibility.
//! Nodes that receive a certificate with an unknown version MUST reject it.
use std::collections::HashSet;
use chrono::{DateTime, Utc};
use serde::{Deserialize, Serialize};
use uuid::Uuid;
@@ -137,14 +135,10 @@ impl RefUpdateCert {
/// Check if this certificate satisfies a threshold of valid signatures
/// from the provided set of authorized maintainer DIDs.
///
/// Counts distinct signer DIDs, not signature entries: a repeated
/// signature from the same maintainer counts once.
pub fn satisfies_threshold(&self, maintainers: &[Did], threshold: usize) -> Result<bool> {
let valid = self.verify_all()?;
let distinct_signers: HashSet<&Did> =
valid.iter().filter(|d| maintainers.contains(d)).collect();
Ok(distinct_signers.len() >= threshold)
let count = valid.iter().filter(|d| maintainers.contains(d)).count();
Ok(count >= threshold)
}
/// Validate the certificate structure (not signatures).
@@ -350,33 +344,6 @@ mod tests {
assert!(!cert.satisfies_threshold(&maintainers, 1).unwrap());
}
#[test]
fn satisfies_threshold_rejects_duplicated_signature() {
let kp1 = Keypair::generate();
let kp2 = Keypair::generate();
let kp3 = Keypair::generate();
let repo_did = kp1.did();
let mut cert = RefUpdateCert::new(
repo_did,
"refs/heads/main".to_string(),
dummy_hash('0'),
dummy_hash('a'),
1,
&kp1,
)
.unwrap();
// Copy-paste the only real signature onto the certificate a second
// time. Same signer, same valid signature, still one real signer.
let dup = cert.signatures[0].clone();
cert.signatures.push(dup);
let maintainers = vec![kp1.did(), kp2.did(), kp3.did()];
// Two signature entries but a single distinct signer must not
// satisfy a 2-of-3 threshold.
assert!(!cert.satisfies_threshold(&maintainers, 2).unwrap());
}
#[test]
fn threshold_zero_is_always_satisfied() {
let kp = Keypair::generate();
-45
View File
@@ -64,19 +64,6 @@ impl Cid {
}
}
/// True when `s` parses as a CIDv1 with the raw codec — the exact shape
/// [`Cid::from_git_object_bytes`] produces and the `/ipfs` resolver looks up.
/// A legacy provider CID (Kubo dag-pb, Pinata CIDv0) parses to a different
/// version or codec and returns `false`, marking it an opportunistic-repair
/// candidate. Decidable from the string alone (no object bytes), so the pin path
/// can gate the byte-read/recompute cost on it and leave non-legacy rows at the
/// existing DB-only skip cost. An unparseable string is non-canonical (`false`).
pub fn is_raw_cidv1(s: &str) -> bool {
s.parse::<CidGeneric<64>>()
.map(|c| c.version() == cid::Version::V1 && c.codec() == RAW)
.unwrap_or(false)
}
impl fmt::Display for Cid {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
write!(f, "{}", self.0)
@@ -190,38 +177,6 @@ mod tests {
assert!(result.is_err());
}
#[test]
fn is_raw_cidv1_classifies_codec_from_string() {
// The canonical resolver key: CIDv1 + raw codec → not a repair candidate.
let raw = Cid::from_git_object_bytes(b"blob 5\0hello");
assert!(
is_raw_cidv1(raw.as_str()),
"from_git_object_bytes output is CIDv1/raw"
);
// A CIDv0 (Pinata dag-pb legacy shape) → repair candidate.
assert!(
!is_raw_cidv1("QmYwAPJzv5CZsnA625s3Xf2nemtYgPpHdWEz79ojWnPbdG"),
"a CIDv0 dag-pb value is a legacy-repair candidate"
);
// A CIDv1 with the dag-pb codec (the Kubo above-block-size root) over the
// same multihash → still a repair candidate (codec, not just version).
let parsed = raw.as_str().parse::<CidGeneric<64>>().unwrap();
const DAG_PB: u64 = 0x70;
let dagpb = CidGeneric::<64>::new_v1(DAG_PB, *parsed.hash()).to_string();
assert!(
!is_raw_cidv1(&dagpb),
"a CIDv1 dag-pb value is a legacy-repair candidate"
);
// Garbage is non-canonical.
assert!(
!is_raw_cidv1("not-a-cid"),
"an unparseable string is non-canonical"
);
}
#[test]
fn sha256_hex_of_empty_input_is_well_known() {
// SHA-256("") is a fixed constant; verifies the hasher is wired correctly.
-61
View File
@@ -80,19 +80,6 @@ impl Did {
)));
}
// Refuse an oversized id before decoding. base58 decoding is quadratic
// in its input, and both callers of this function take the string from
// an untrusted request: the unauthenticated peer-announce gate, and the
// HTTP-signature keyid. An ed25519 did:key method-id is a fixed 48
// characters, so this bound is slack rather than a behavior change, and
// it has to sit ahead of the decode to be worth anything.
const MAX_METHOD_ID_LEN: usize = 64;
if self.method_id().len() > MAX_METHOD_ID_LEN {
return Err(Error::InvalidDid(
"did:key method-specific id too long".to_string(),
));
}
let (_, bytes) =
multibase::decode(self.method_id()).map_err(|e| Error::InvalidDid(e.to_string()))?;
@@ -321,52 +308,4 @@ mod tests {
assert_eq!(Did::web("example.com").method_id(), "example.com");
assert_eq!(Did::gitlawb("z6MkSomeKey").method_id(), "z6MkSomeKey");
}
/// An oversized method-specific id is refused on its length, before the
/// base58 decode runs.
///
/// base58 decoding is quadratic in the input, and `to_verifying_key` is
/// reached from two places that take an attacker-supplied string: the
/// unauthenticated peer-announce gate, and the HTTP-signature keyid. A
/// 64k-character DID measured 8s through the router without this guard
/// against 4.6ms with it, on a tokio worker with no spawn_blocking, so a
/// single caller could buy CPU-seconds by the thousand.
///
/// Asserting the MESSAGE is what makes this load-bearing: an oversized id
/// errors either way, so a test that only checked `is_err()` would pass
/// with the guard removed, having merely paid the cost it exists to avoid.
#[test]
fn to_verifying_key_rejects_an_oversized_method_id_before_decoding() {
let oversized = format!("did:key:z{}", "z".repeat(64_000));
let did: Did = oversized.parse().expect("method is still `key`");
let start = std::time::Instant::now();
let err = did.to_verifying_key().expect_err("must be refused");
let elapsed = start.elapsed();
assert!(
err.to_string().contains("too long"),
"must be refused on length, not after the decode: {err}"
);
assert!(
elapsed < std::time::Duration::from_secs(1),
"refusal must not pay the quadratic decode: took {elapsed:?}"
);
}
/// The bound must not reject a real key. Every `did:key` this project
/// mints is a fixed 48 characters, so the limit is slack, not a behavior
/// change.
#[test]
fn to_verifying_key_still_accepts_a_real_did_key() {
use ed25519_dalek::SigningKey;
let signing = SigningKey::from_bytes(&[7u8; 32]);
let did = Did::from_verifying_key(&signing.verifying_key());
assert_eq!(
did.method_id().len(),
48,
"did:key method-id is fixed-width"
);
did.to_verifying_key().expect("a real did:key must resolve");
}
}
+2 -23
View File
@@ -78,9 +78,10 @@ impl Keypair {
/// Verify an Ed25519 signature.
pub fn verify(verifying_key: &VerifyingKey, msg: &[u8], sig_bytes: &[u8; 64]) -> Result<()> {
use ed25519_dalek::Verifier;
let sig = Signature::from_bytes(sig_bytes);
verifying_key
.verify_strict(msg, &sig)
.verify(msg, &sig)
.map_err(|_| Error::SignatureInvalid)
}
@@ -208,28 +209,6 @@ mod tests {
);
}
#[test]
fn verify_rejects_weak_key_signature() {
// Regression guard for strict verification: a signature forged under a
// weak (small-order) public key satisfies the verification equation
// but must be rejected. The identity point is such a key: with R the
// identity and S = 0, [S]B - [k]A is the identity for any message.
let mut weak_key_bytes = [0u8; 32];
weak_key_bytes[0] = 1; // compressed identity point (y = 1, x = 0)
let weak_vk = VerifyingKey::from_bytes(&weak_key_bytes).unwrap();
assert!(weak_vk.is_weak(), "identity point must be a weak key");
let msg = b"forged message";
let mut forged = [0u8; 64];
forged[0] = 1; // R = identity point, S = 0
let result = verify(&weak_vk, msg, &forged);
assert!(
result.is_err(),
"signature under a weak (small-order) public key must be rejected"
);
}
#[test]
fn signed_payload_round_trip() {
let kp = Keypair::generate();
-9
View File
@@ -5,16 +5,7 @@ pub mod encrypt;
pub mod error;
pub mod http_sig;
pub mod identity;
// `url` is the one dependency here that drags a tail (idna, then the icu
// crates), and gitlawb-core is allowlisted to stay embeddable. Every client that
// needs this predicate already parses URLs, so they opt in and nothing else
// pays. `test` is in the cfg so `cargo test -p gitlawb-core` still compiles and
// runs the matrix below with no feature selected; without it the tests would
// silently not run, which is the failure this module exists to prevent.
#[cfg(any(feature = "redirect", test))]
pub mod redirect;
pub mod sanitize;
pub mod scan_token;
pub mod ucan;
pub use error::Error;
-241
View File
@@ -1,241 +0,0 @@
//! Redirect policy shared by every gitlawb HTTP client that signs its requests.
//!
//! Both `gl` (async) and `git-remote-gitlawb` (blocking) attach RFC 9421
//! `Signature` and `Signature-Input` headers, and reqwest strips only
//! `Authorization`, `Cookie`, `Proxy-Authorization` and `WWW-Authenticate` when a
//! redirect crosses hosts. A signature would survive that hop, and it binds
//! `@method`, `@path` and `content-digest` with no authority component, so a node
//! answering 302 could hand a working credential to a host of its choosing and read
//! as the caller anywhere until the clock-skew window closes. On a 307/308 the
//! request body goes along with it, which for the remote helper is the pack.
//!
//! That same missing authority component is why the predicate also pins the
//! request-target: `@path` is signed as the client sent it and verified as the node
//! received it, so a same-origin hop that rewrites the path or the query (a
//! trailing-slash or query normalization) makes the signature cover a target the
//! node never saw and the read 401s. Only a hop that re-issues the identical target,
//! an http-to-https upgrade being the one that matters in practice, is followed.
//!
//! The decision lives here rather than in either client because the two used to
//! disagree: `gl` was scoped to the origin while the remote helper, the binary that
//! actually runs `git clone gitlawb://`, still ran reqwest's default and followed
//! anywhere. One predicate is what keeps a future third client from repeating that.
//!
//! The type is `url::Url`, which is what `reqwest::Url` re-exports, so both clients
//! pass their attempt URLs straight in.
/// Longest redirect chain followed. `reqwest::redirect::Policy::custom` replaces
/// reqwest's built-in limit, so the bound has to be restated by every client that
/// installs a custom policy; the value is reqwest's own default.
///
/// This counts FOLLOWS, matching `Policy::limited`: reqwest pushes the redirecting
/// URL onto `previous` before consulting the policy, and `Limit(max)` refuses once
/// `previous.len() > max`, so a caller comparing against this constant must use `>`
/// too or it permits one hop fewer than it says.
pub const MAX_REDIRECTS: usize = 10;
/// Follow a redirect only when it stays on the origin that issued it AND re-issues
/// the identical request-target.
///
/// `Policy::none()` would have been the simpler answer, but one same-origin redirect
/// shape is legitimate here: a node fronted by a proxy that upgrades http to https,
/// or otherwise re-issues the same path and query, so the policy is scoped rather
/// than switched off.
///
/// Path and query must match exactly, and that is the request-target clause rather
/// than an origin one. `@path` is signed as the client sent it and verified as the
/// node received it, so a hop that rewrites either half leaves a signature covering
/// a target nobody asked for and the node answers 401. Refusing the hop turns a
/// confusing 401 into the 3xx that names what actually happened. One policy covers
/// signed and unsigned callers alike, for the same reason the predicate is shared:
/// two rules would drift.
///
/// The clause pins `@path`, and only `@path`. A gitlawb signature also covers
/// `@method` and `content-digest`, and both of those are still open on a followed
/// hop: on a 301, 302 or 303, reqwest 0.12.28 delegates to tower-http's
/// `FollowRedirect`, which rewrites a POST to a GET and empties the body
/// (tower-http-0.6.8 `src/follow_redirect/mod.rs:273-285`), while its
/// `drop_payload_headers` removes only `Content-Type`, `Content-Length`,
/// `Content-Encoding` and `Transfer-Encoding`. So `Signature`, `Signature-Input` and
/// `Content-Digest` ride along on a request that no longer has the method or the body
/// they were computed over. Only 307 and 308 preserve both. This predicate returning
/// true therefore makes a GET-shaped hop safe to replay against the node's verifier
/// and says nothing about a bodied one: a signed write must not rely on it alone.
///
/// `Url::query` is `None` for `/a` and `Some("")` for `/a?`, and those are two
/// different request-targets on the node side too, so the comparison is strict and
/// needs no special case.
///
/// Host and port must match exactly. Port is compared as `Url::port`, which is
/// `None` for a scheme's default port, so http -> https on the same host compares
/// equal while http -> http on a different port does not. A downgrade from https to
/// http is refused as well: the target is the same host, but the signature would go
/// out in cleartext, which is the same credential leak by a slower route.
///
/// Host comparison rides on `url`'s parse-time normalization (lowercasing and IDN
/// -> punycode), so the spellings an attacker reaches for do not open a gap. That
/// is a property of the parsed `Url`, not of this function, which is why the test
/// matrix pins it: a move to raw string comparison would silently lose it.
pub fn may_follow(previous: &url::Url, next: &url::Url) -> bool {
let same_origin = next.host_str() == previous.host_str() && next.port() == previous.port();
let same_target = next.path() == previous.path() && next.query() == previous.query();
let downgraded = previous.scheme() == "https" && next.scheme() != "https";
same_origin && same_target && !downgraded
}
#[cfg(test)]
mod tests {
use super::*;
/// Every branch of the decision, both ways, plus the spellings that would slip
/// past a comparison less careful than `Url`'s own normalization.
#[test]
fn may_follow_covers_each_origin_branch() {
let url = |s: &str| url::Url::parse(s).unwrap();
let cases: &[(&str, &str, bool, &str)] = &[
(
"http://node.example/a",
"http://node.example/b",
false,
"same origin but the path changed: @path is signed as sent and verified as received",
),
(
"http://node.example/a",
"https://node.example/a",
true,
"http to https on one host with an identical request-target: both ports are \
the scheme default, so the proxy upgrade is still followed",
),
(
"https://node.example/a",
"https://node.example/a/",
false,
"trailing-slash normalization changes the request-target, so it is refused",
),
(
"https://node.example:8443/a",
"https://node.example:8443/a",
true,
"same explicit port",
),
(
"http://node.example/a",
"http://attacker.example/a",
false,
"different host",
),
(
"http://node.example/a",
"http://node.example:8080/a",
false,
"same host, different port",
),
(
"https://node.example/a",
"http://node.example/a",
false,
"https downgraded to cleartext on the same host",
),
(
"https://node.example/a",
"http://node.example:443/a",
false,
"a downgrade dressed up as the https port",
),
// The rows below pass today because `Url::parse` normalizes the host, not
// because anything here compares case-insensitively or decodes IDN. They
// are the variants an attacker reaches for, so they are pinned: swapping
// this predicate for a raw string comparison must break the suite.
// Each of these pairs an IDENTICAL path on both sides, deliberately. The
// request-target clause below would make every one of them false on the
// path alone, and a row that is false for two reasons has stopped pinning
// either. With the paths equal, the host or port comparison is the only
// thing left that can decide them.
(
"https://node.example/a",
"https://NODE.EXAMPLE/a",
true,
"same host in a different case: parse lowercases it",
),
(
"https://node.example/a",
"https://node.example./a",
false,
"a trailing dot is a different host to url, so the redirect is refused",
),
(
"https://exämple.test/a",
"https://xn--exmple-cua.test/a",
true,
"unicode host and its punycode spelling are one host after parse",
),
(
"https://node.example/a",
"https://user:[email protected]/a",
true,
"userinfo is not part of the origin: same host, still followed",
),
(
"https://node.example/a",
"https://[email protected]/a",
false,
"the node's name smuggled into userinfo: the host is the attacker's",
),
(
"https://node.example/a",
"https://attacker.example/a#node.example",
false,
"the node's name pushed into the fragment: the host is the attacker's",
),
// The request-target clause, both directions. `@path` is the only thing a
// gitlawb signature binds the request to, so a hop that rewrites it hands
// the node a signature over a target it never received.
(
"https://node.example/a?x=1",
"https://node.example/a?x=1",
true,
"identical request-target: the same-origin hop that is still followed",
),
(
"https://node.example/a?x=1",
"https://node.example/a?x=2",
false,
"same path but a different query: the request-target covers the query too",
),
(
"https://node.example/a",
"https://node.example/a?x=1",
false,
"a query added where there was none",
),
(
"http://node.example/a",
"http://node.example/a?",
false,
"an empty query added where there was none: a missing query and an empty \
one are different request-targets",
),
(
"https://node.example/a?x=1",
"https://node.example/a",
false,
"the query dropped where there was one: the comparison is symmetric, and \
nothing else in the matrix pins that direction",
),
(
"https://node.example/a#x",
"https://node.example/a#y",
true,
"a fragment-only difference is still followed: a fragment never reaches \
the wire, so it is no part of the request-target the node verifies",
),
];
for (previous, next, expected, why) in cases {
assert_eq!(
may_follow(&url(previous), &url(next)),
*expected,
"{previous} -> {next} ({why})"
);
}
}
}
-499
View File
@@ -1,499 +0,0 @@
//! Sealed continuation token for the node's bounded legacy CID scan (INV-13).
//!
//! The `/ipfs/{cid}` resolver's legacy scan stops at a row ceiling and sheds a
//! retryable 503. To let a holder buried past that ceiling still be reached, the
//! shed carries the scan position so the caller can echo it back and resume. The
//! whole point of the design is that the node keeps NO server-side scan state: the
//! position rides in the caller's token.
//!
//! That makes the token an EMITTED continuation derived from a FETCHED row, and on
//! a scan that served nothing every fetched row is by construction a private or
//! quarantined repo the caller may not read. The row's `created_at` leaks its
//! creation time and its `id` carries the owner's DID, so both halves are withheld
//! fields and the token must be CONFIDENTIAL, not merely tamper-evident:
//!
//! * AEAD-sealed (XChaCha20-Poly1305), never base64-of-plaintext and never
//! signed plaintext. Integrity is not confidentiality.
//! * A fresh `OsRng` nonce on EVERY seal. Under a stream cipher a repeated nonce
//! means repeated keystream, and an attacker who can force the node to seal a
//! position whose plaintext they know XORs two tokens and recovers a withheld
//! row's fields in full, strictly worse than emitting plaintext.
//! * FIXED-WIDTH plaintext. AEAD ciphertext is plaintext-length plus the tag, and
//! every field of a scan position varies in length, so a variable encoding would
//! make token LENGTH a side channel for the sealed row (a short name under a
//! short owner vs a long one) and for the candidate oid's width (40 hex on a
//! sha1 repo, 64 on a sha256 one). Every token this module mints is byte-identical
//! in length. Each field is padded to its own separate width, which is a per-field
//! constant and so still leaks nothing about a given row or candidate.
//! * The canonical CID as associated data, so a token minted while scanning for
//! one CID does not authenticate when replayed against another.
//!
//! Every failure to open (wrong key, tampered bytes, wrong CID, expired, malformed)
//! returns the same `None`. The caller treats that as "no token" and starts at the
//! front, so no failure class is distinguishable and the token is no oracle.
use base64::{engine::general_purpose::URL_SAFE_NO_PAD as B64URL, Engine};
use chacha20poly1305::{
aead::{Aead, KeyInit, OsRng, Payload},
XChaCha20Poly1305, XNonce,
};
use rand::RngCore;
/// Keyset position of the last row a truncated scan fetched.
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct ScanPosition {
/// The row's raw stored `created_at` text, the first half of the keyset cursor.
pub created_at_key: String,
/// The row's `id`, the tiebreaking half of the keyset cursor.
pub id: String,
/// The oid hex of the CANDIDATE this position resumes. One CID can map to several
/// git oids, so a bare row cursor names a row without naming whose walk it belongs
/// to. The candidate is named by IDENTITY rather than by position in the candidate
/// list: that list is ordered by hex and mutates between rungs, so an index would
/// name a different candidate the moment anything is pinned or unpinned.
pub sha256_hex: String,
}
/// Plaintext version byte, so a future layout change is a clean open-failure
/// (treated as absent) rather than a misparse.
///
/// Bumped to 2 when the two halves stopped sharing one width (see [`ID_WIDTH`]), and
/// to 3 when the position gained the candidate oid it resumes (see [`OID_WIDTH`]).
/// A token minted under an earlier layout is a different length and a different
/// framing, so it opens to `None` and the caller restarts at the front, which is the
/// safe direction: a misparse would resume at a fabricated row and skip coverage.
const VERSION: u8 = 3;
/// Byte width the `created_at` half is padded to. Every value stored here is a
/// serialized timestamp, about 30 bytes, so 64 is roomy for the field's whole domain.
/// It is deliberately NOT widened to match [`ID_WIDTH`]: padding both halves to the id
/// width would nearly double every token for a field that can never use the space.
const CREATED_WIDTH: usize = 64;
/// Byte width the `id` half is padded to.
///
/// The bound is set by the WRITERS, not by what a typical id happens to look like.
/// `upsert_mirror_repo` builds `repos.id` as `{owner}/{name}`, and the slug validators
/// in the node's `repo_store` admit an owner of up to 255 bytes and a name of up to
/// 100, so 356 bytes is reachable through the ordinary write path and repo names are
/// peer-controllable. 384 clears that with margin.
///
/// Under-sizing this is not a cosmetic bug. A row at a truncation boundary whose id
/// exceeds the width fails the seal, the handler sheds a 503 with no continuation, and
/// a tokenless shed is byte-identical to the wrapped-scan response whose contract is
/// "the absence of a token means the ladder is over". The boundary row is deterministic
/// for a stable inventory, so every retry reproduces it and every row past it becomes
/// permanently unreachable. Anything past the width still fails loudly rather than
/// silently truncating a cursor into one that resumes at the wrong row.
const ID_WIDTH: usize = 384;
/// Byte width the candidate oid half is padded to.
///
/// Git mints exactly two oid widths and this field carries BOTH. A production repo is
/// created by `store::init_bare` with `git init --bare --object-format=sha1`, so its
/// oids are 40 hex; only the sha256 test fixtures mint 64. The field is therefore
/// length-prefixed like the two row halves rather than a bare fixed 64: a 64-only
/// field would fail every seal on a real deployment, and a failed seal sheds a
/// tokenless 503 that is byte-identical to "your ladder is over".
///
/// The padding to 64 is what keeps the WIDTH off the wire. Without it a 40-hex token
/// is 24 bytes shorter than a 64-hex one, and token length would say which object
/// format the holder's repo uses.
const OID_WIDTH: usize = 64;
/// `version | created_len:u16 | created[CREATED_WIDTH] | id_len:u16 | id[ID_WIDTH] | oid_len:u16 | oid[OID_WIDTH] | expires:i64`
const PLAINTEXT_LEN: usize = 1 + 2 + CREATED_WIDTH + 2 + ID_WIDTH + 2 + OID_WIDTH + 8;
/// Nonce width for XChaCha20-Poly1305.
const NONCE_LEN: usize = 24;
/// A fresh random 32-byte sealing key from the OS CSPRNG.
pub fn new_key() -> [u8; 32] {
let mut key = [0u8; 32];
OsRng.fill_bytes(&mut key);
key
}
// The two halves below are deliberately separate and adjacent. The framing pair owns
// "every token is the same length"; the AEAD pair owns "the contents are confidential
// and CID-bound". Keeping them apart is what lets each property be exercised (and
// broken) without disturbing the other.
/// Encode a position into the FIXED-WIDTH plaintext:
/// `version | created_len:u16 | created[CREATED_WIDTH] | id_len:u16 | id[ID_WIDTH] | oid_len:u16 | oid[OID_WIDTH] | expires:i64`
///
/// The padding is the point. AEAD ciphertext is plaintext-length plus the tag, and every
/// field of a scan position varies in length, so a length-prefixed encoding with no
/// padding would make token LENGTH a side channel for the sealed row and for the
/// candidate's object format. Each field is padded to its OWN fixed width, which keeps
/// every minted token the same length while letting the id half carry the range the
/// write path actually admits.
fn encode_position(pos: &ScanPosition, expires_at_unix: i64) -> anyhow::Result<Vec<u8>> {
let mut out = vec![0u8; PLAINTEXT_LEN];
out[0] = VERSION;
let mut at = 1;
for (field, width) in [
(pos.created_at_key.as_bytes(), CREATED_WIDTH),
(pos.id.as_bytes(), ID_WIDTH),
(pos.sha256_hex.as_bytes(), OID_WIDTH),
] {
if field.len() > width {
// Loud rather than truncating: a clipped cursor resumes at the wrong row and
// silently skips coverage, which is the availability half of the bug this
// token exists to fix.
anyhow::bail!(
"scan token field is {} bytes, over the {width}-byte fixed width",
field.len()
);
}
out[at..at + 2].copy_from_slice(&(field.len() as u16).to_le_bytes());
at += 2;
out[at..at + field.len()].copy_from_slice(field);
at += width;
}
out[at..at + 8].copy_from_slice(&expires_at_unix.to_le_bytes());
Ok(out)
}
/// Decode what [`encode_position`] wrote. `None` on any structural mismatch.
fn decode_position(bytes: &[u8]) -> Option<(ScanPosition, i64)> {
if bytes.len() != PLAINTEXT_LEN || bytes[0] != VERSION {
return None;
}
let mut at = 1;
let mut fields = [const { String::new() }; 3];
for (slot, width) in fields.iter_mut().zip([CREATED_WIDTH, ID_WIDTH, OID_WIDTH]) {
let len = u16::from_le_bytes([bytes[at], bytes[at + 1]]) as usize;
at += 2;
if len > width {
return None;
}
*slot = String::from_utf8(bytes[at..at + len].to_vec()).ok()?;
at += width;
}
let expires_at = i64::from_le_bytes(bytes[at..at + 8].try_into().ok()?);
let [created_at_key, id, sha256_hex] = fields;
// A zero-length candidate is a third state the encoder never mints. The front-of-table
// sentinel is empty ROW halves with a real oid, so an empty oid would hand the resume
// path a candidate that names nothing; refuse it like every other malformed frame.
if sha256_hex.is_empty() {
return None;
}
Some((
ScanPosition {
created_at_key,
id,
sha256_hex,
},
expires_at,
))
}
/// AEAD-seal `plaintext` under `key`, bound to `cid`, framed as `nonce || ciphertext`.
fn seal_bytes(key: &[u8; 32], cid: &str, plaintext: &[u8]) -> anyhow::Result<Vec<u8>> {
let cipher = XChaCha20Poly1305::new_from_slice(key)
.map_err(|e| anyhow::anyhow!("scan token key: {e}"))?;
// A FRESH nonce per seal, from the OS CSPRNG. Under a stream cipher a repeated nonce
// repeats the keystream, and two tokens sealed under one nonce XOR to the difference
// of their plaintexts, which recovers a withheld row in full when the attacker can
// force one of the two positions. This draw is the property the whole confidentiality
// claim rests on.
let mut nonce = [0u8; NONCE_LEN];
OsRng.fill_bytes(&mut nonce);
let sealed = cipher
.encrypt(
XNonce::from_slice(&nonce),
Payload {
msg: plaintext,
// The canonical CID as associated data: a token minted while scanning for
// one CID does not authenticate against another, so it cannot be replayed
// to seed a different scan.
aad: cid.as_bytes(),
},
)
.map_err(|e| anyhow::anyhow!("scan token seal: {e}"))?;
let mut out = Vec::with_capacity(NONCE_LEN + sealed.len());
out.extend_from_slice(&nonce);
out.extend_from_slice(&sealed);
Ok(out)
}
/// Open what [`seal_bytes`] framed. `None` on any failure, including a wrong `cid`.
fn open_bytes(key: &[u8; 32], cid: &str, raw: &[u8]) -> Option<Vec<u8>> {
if raw.len() <= NONCE_LEN + 16 {
return None;
}
let (nonce, sealed) = raw.split_at(NONCE_LEN);
let cipher = XChaCha20Poly1305::new_from_slice(key).ok()?;
cipher
.decrypt(
XNonce::from_slice(nonce),
Payload {
msg: sealed,
aad: cid.as_bytes(),
},
)
.ok()
}
/// Seal `pos` under `key`, bound to `cid`, expiring at `expires_at_unix`.
///
/// Returns the base64url (no pad) token. Errors only when a field exceeds
/// its own fixed width ([`CREATED_WIDTH`], [`ID_WIDTH`], [`OID_WIDTH`]) or the AEAD
/// itself fails, never silently truncates.
pub fn seal_scan_token(
key: &[u8; 32],
cid: &str,
pos: &ScanPosition,
expires_at_unix: i64,
) -> anyhow::Result<String> {
let plaintext = encode_position(pos, expires_at_unix)?;
Ok(B64URL.encode(seal_bytes(key, cid, &plaintext)?))
}
/// Open a token minted by [`seal_scan_token`] under the same key and CID.
///
/// `None` for every failure class alike (wrong key, tampered, foreign CID, expired,
/// malformed, wrong version), so the caller can treat all of them as "absent" without
/// leaking which one occurred.
pub fn open_scan_token(
key: &[u8; 32],
cid: &str,
token: &str,
now_unix: i64,
) -> Option<ScanPosition> {
let raw = B64URL.decode(token).ok()?;
let plaintext = open_bytes(key, cid, &raw)?;
let (pos, expires_at) = decode_position(&plaintext)?;
if now_unix >= expires_at {
return None;
}
Some(pos)
}
#[cfg(test)]
mod tests {
use super::*;
/// A production-shaped candidate: `git init --bare --object-format=sha1`, so 40 hex.
const OID_40: &str = "a1b2c3d4e5f60718293a4b5c6d7e8f9012345678";
/// A test-fixture-shaped candidate: the sha256 repos the suite creates mint 64 hex.
const OID_64: &str = "0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef";
fn pos(created: &str, id: &str) -> ScanPosition {
pos_for(created, id, OID_40)
}
fn pos_for(created: &str, id: &str, sha256_hex: &str) -> ScanPosition {
ScanPosition {
created_at_key: created.to_string(),
id: id.to_string(),
sha256_hex: sha256_hex.to_string(),
}
}
/// Seal a hand-built plaintext through the AEAD half, so a test can frame bytes the
/// encoder would never mint (an old version byte, a zero-length oid) and still exercise
/// the real open path.
fn seal_raw(key: &[u8; 32], cid: &str, plaintext: &[u8]) -> String {
B64URL.encode(seal_bytes(key, cid, plaintext).unwrap())
}
/// Byte offset of the oid length prefix inside the plaintext, derived from the widths
/// rather than hardcoded so a width change moves it with the layout.
const OID_LEN_AT: usize = 1 + 2 + CREATED_WIDTH + 2 + ID_WIDTH;
/// Scenario 1: both of git's oid widths round trip. The 40-hex case is the PRODUCTION
/// shape (`git init --bare --object-format=sha1`), and an all-64 fixture suite would
/// never exercise it, which is exactly how a fixed-64 field would ship broken.
#[test]
fn round_trips_at_both_oid_widths() {
let key = new_key();
for hex in [OID_40, OID_64] {
let p = pos_for("2020-01-01T00:00:03+00:00", "z6MkOwner/private-repo", hex);
let t =
seal_scan_token(&key, "bafkcid", &p, 1 << 40).expect("both oid widths must seal");
assert_eq!(
open_scan_token(&key, "bafkcid", &t, 0),
Some(p),
"a {}-hex candidate must open to the identical position",
hex.len()
);
}
}
/// Scenario 2: a token framed under the OLD version opens to `None`, never a misparse.
/// Both legs matter: the version-2 plaintext was a different LENGTH, and a future
/// same-length layout would only be caught by the version byte itself.
#[test]
fn a_prior_layout_version_opens_to_none() {
let key = new_key();
// The version-2 layout verbatim: no oid field, so 461 bytes.
let mut old = vec![0u8; 1 + 2 + CREATED_WIDTH + 2 + ID_WIDTH + 8];
old[0] = 2;
assert_eq!(old.len(), 461, "the version-2 plaintext was 461 bytes");
assert_eq!(
open_scan_token(&key, "bafkcid", &seal_raw(&key, "bafkcid", &old), 0),
None,
"a version-2 token must open to None so the caller restarts at the front"
);
// Same length, stale version byte: only the version check can refuse this one.
let mut stamped = encode_position(
&pos("2020-01-01T00:00:03+00:00", "z6MkOwner/private-repo"),
1 << 40,
)
.unwrap();
stamped[0] = VERSION - 1;
assert_eq!(
open_scan_token(&key, "bafkcid", &seal_raw(&key, "bafkcid", &stamped), 0),
None,
"a stale version byte must open to None even at the current width"
);
}
/// Scenario 3: token length is invariant across candidate VALUE and candidate WIDTH.
/// The padding is what keeps the oid width off the wire: without it a 40-hex token is
/// 24 bytes shorter than a 64-hex one and the length says which repo format the
/// holder uses.
#[test]
fn token_length_is_invariant_across_oid_widths() {
let key = new_key();
let created = "2020-01-01T00:00:00+00:00";
let id = "z6MkOwner/private-repo";
let short =
seal_scan_token(&key, "bafkcid", &pos_for(created, id, OID_40), 1 << 40).unwrap();
let long =
seal_scan_token(&key, "bafkcid", &pos_for(created, id, OID_64), 1 << 40).unwrap();
assert_eq!(
short.len(),
long.len(),
"a 40-hex and a 64-hex candidate must mint tokens of identical length, or the \
oid width is a side channel"
);
// The absolute width, pinned by execution rather than by arithmetic on paper. The
// gl client's mock fixtures hardcode this number (`TOKEN_LEN` in
// crates/gl/src/ipfs_cmd.rs), and nothing in that crate seals a real token, so this
// assertion is the only executable check that the two agree.
assert_eq!(
short.len(),
756,
"24 nonce + {PLAINTEXT_LEN} plaintext + 16 tag, base64url no pad"
);
}
/// Scenario 4: the front-of-table sentinel. Empty row halves with a REAL candidate
/// round trip, which is what lets a seal say "this candidate, no row cursor yet".
#[test]
fn empty_row_fields_round_trip_with_a_real_candidate() {
let key = new_key();
let p = pos_for("", "", OID_40);
let t =
seal_scan_token(&key, "bafkcid", &p, 1 << 40).expect("the front sentinel must seal");
assert_eq!(open_scan_token(&key, "bafkcid", &t, 0), Some(p));
}
/// Scenario 6, third leg: a zero-length oid is a distinguishable third state that the
/// encoder never mints, and accepting it would hand the sentinel machinery a candidate
/// naming nothing. The decode path refuses it.
#[test]
fn a_zero_length_candidate_opens_to_none() {
let key = new_key();
let mut plaintext = encode_position(
&pos("2020-01-01T00:00:03+00:00", "z6MkOwner/private-repo"),
1 << 40,
)
.unwrap();
plaintext[OID_LEN_AT..OID_LEN_AT + 2].copy_from_slice(&0u16.to_le_bytes());
assert_eq!(
open_scan_token(&key, "bafkcid", &seal_raw(&key, "bafkcid", &plaintext), 0),
None,
"a zero-length candidate must open to None, not to a position naming nothing"
);
}
/// Scenario 6, second leg: an oid past the 64-byte width fails the seal loudly rather
/// than being clipped into a hex that names a different candidate.
#[test]
fn an_oid_over_the_fixed_width_fails_loudly() {
let key = new_key();
let p = pos_for(
"2020-01-01T00:00:03+00:00",
"z6MkOwner/private-repo",
&"a".repeat(OID_WIDTH + 1),
);
assert!(
seal_scan_token(&key, "bafkcid", &p, 1 << 40).is_err(),
"an over-wide candidate must fail the seal, never be truncated into a hex that \
resumes the wrong candidate"
);
}
#[test]
fn round_trips_under_the_same_key_and_cid() {
let key = new_key();
let p = pos("2020-01-01T00:00:03+00:00", "z6MkOwner/private-repo");
let t = seal_scan_token(&key, "bafkcid", &p, 1 << 40).unwrap();
assert_eq!(open_scan_token(&key, "bafkcid", &t, 0), Some(p));
}
#[test]
fn every_failure_class_opens_to_none() {
let key = new_key();
let other = new_key();
let p = pos("2020-01-01T00:00:03+00:00", "z6MkOwner/private-repo");
let t = seal_scan_token(&key, "bafkcid", &p, 1 << 40).unwrap();
assert_eq!(open_scan_token(&other, "bafkcid", &t, 0), None, "wrong key");
assert_eq!(
open_scan_token(&key, "bafkOTHER", &t, 0),
None,
"foreign CID"
);
assert_eq!(
open_scan_token(&key, "bafkcid", &t, 1 << 41),
None,
"expired"
);
assert_eq!(open_scan_token(&key, "bafkcid", "!!not b64", 0), None);
assert_eq!(open_scan_token(&key, "bafkcid", "", 0), None);
let mut flipped: Vec<u8> = t.bytes().collect();
let last = flipped.len() - 1;
flipped[last] = if flipped[last] == b'A' { b'B' } else { b'A' };
assert_eq!(
open_scan_token(&key, "bafkcid", &String::from_utf8(flipped).unwrap(), 0),
None,
"tampered"
);
}
/// The id half must clear the LARGEST repo id the node's own write path admits,
/// not merely a typical one. `upsert_mirror_repo` builds `repos.id` as
/// `{owner}/{name}`, and the slug validators in `repo_store` admit 255 bytes of
/// owner and 100 of name, so 356 is reachable. A width under that turns the
/// boundary row into a seal failure, which sheds a tokenless 503 that is
/// byte-identical to "your ladder is over" and strands every row past it forever.
#[test]
fn round_trips_a_repo_id_at_the_write_paths_maximum() {
let key = new_key();
let id = format!("{}/{}", "o".repeat(255), "n".repeat(100));
assert_eq!(id.len(), 356, "255 owner + '/' + 100 name");
let p = pos("2020-01-01T00:00:03+00:00", &id);
let t = seal_scan_token(&key, "bafkcid", &p, 1 << 40)
.expect("a repo id the write path admits must seal, never fail the width");
assert_eq!(open_scan_token(&key, "bafkcid", &t, 0), Some(p));
}
#[test]
fn a_field_over_the_fixed_width_fails_loudly() {
let key = new_key();
let p = pos("2020-01-01T00:00:03+00:00", &"x".repeat(ID_WIDTH + 1));
assert!(
seal_scan_token(&key, "bafkcid", &p, 1 << 40).is_err(),
"an over-wide field must fail the seal, never be truncated into a cursor \
that resumes at the wrong row"
);
}
}
+2 -4
View File
@@ -1,7 +1,7 @@
[package]
name = "gitlawb-node"
description = "The gitlawb node daemon — git hosting over HTTP with DID auth"
version = "0.7.1" # x-release-please-version
version = "0.6.0" # x-release-please-version
edition.workspace = true
rust-version.workspace = true
license.workspace = true
@@ -12,6 +12,7 @@ path = "src/main.rs"
[dependencies]
gitlawb-core = { path = "../gitlawb-core" }
async-trait = "0.1"
ed25519-dalek = { workspace = true }
base64 = { workspace = true }
tokio = { workspace = true }
@@ -77,6 +78,3 @@ libp2p-dns = { version = "0.44.0", features = ["tokio"] }
[dev-dependencies]
mockito = "1"
tempfile = "3"
# `test-util` for `#[tokio::test(start_paused = true)]`: lets the deadline tests assert
# on which timer fired instead of on wall clock.
tokio = { workspace = true, features = ["test-util"] }
File diff suppressed because it is too large. Load diff
+2 -49
View File
@@ -26,61 +26,14 @@ pub async fn list_anchors(
Query(q): Query<ListAnchorsQuery>,
) -> Result<Json<serde_json::Value>> {
let limit = q.limit.min(200);
// Bare `?` so connection-class sqlx failures downcast to `AppError::Db` and
// map to 503 `db_unavailable` (not 500 via `.map_err(AppError::Internal)`) (#251).
let anchors = state
.db
.list_arweave_anchors(q.repo.as_deref(), limit)
.await?;
.await
.map_err(crate::error::AppError::Internal)?;
Ok(Json(serde_json::json!({
"anchors": anchors,
"count": anchors.len(),
})))
}
#[cfg(test)]
mod closed_pool_tests {
use super::*;
use axum::http::{Request, StatusCode};
use axum::Router;
use serde_json::Value;
use sqlx::PgPool;
use tower::ServiceExt;
/// #251: a closed pool on /api/v1/arweave/anchors must be 503 db_unavailable.
#[sqlx::test]
async fn list_anchors_closed_pool_returns_503_db_unavailable(pool: PgPool) {
let state = crate::test_support::test_state(pool.clone()).await;
pool.close().await;
let resp = Router::new()
.route("/api/v1/arweave/anchors", axum::routing::get(list_anchors))
.with_state(state)
.oneshot(
Request::builder()
.uri("/api/v1/arweave/anchors")
.body(axum::body::Body::empty())
.unwrap(),
)
.await
.unwrap();
assert_eq!(
resp.status(),
StatusCode::SERVICE_UNAVAILABLE,
"closed-pool outage must be retryable 503, not 500"
);
let bytes = axum::body::to_bytes(resp.into_body(), usize::MAX)
.await
.expect("read body");
let v: Value = serde_json::from_slice(&bytes).expect("json body");
assert_eq!(
v,
serde_json::json!({
"error": crate::error::DB_UNAVAILABLE_CODE,
"message": crate::error::DB_UNAVAILABLE_MESSAGE,
})
);
}
}
+1 -14
View File
@@ -1440,8 +1440,7 @@ mod ref_updates_feed_tests {
// A DB error in the gate fails closed as 500, not swallowed into an empty 200 (the
// regression the old get_repo().ok().flatten() allowed). Inject by dropping a
// column get_repo selects so its query errors. Also pin the anonymous body to
// the exact opaque object (#226).
// column get_repo selects so its query errors.
#[sqlx::test]
async fn repo_events_db_error_fails_closed_500(pool: PgPool) {
let state = test_state(pool.clone()).await;
@@ -1464,18 +1463,6 @@ mod ref_updates_feed_tests {
StatusCode::INTERNAL_SERVER_ERROR,
"a DB error must fail closed (500), never serve an empty 200"
);
let body = axum::body::to_bytes(resp.into_body(), usize::MAX)
.await
.expect("read body");
let body = String::from_utf8_lossy(&body);
let v: serde_json::Value = serde_json::from_str(&body).expect("json body");
assert_eq!(
v,
serde_json::json!({
"error": "db_error",
"message": crate::error::DB_ERROR_MESSAGE,
})
);
}
// Symmetric to the gate DB-error test: a DB error in the CERT fetch (after the gate
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
+5 -25
View File
@@ -208,15 +208,11 @@ mod authz_guard {
(issues, "create_issue", "authorize_repo_read("),
(bounties, "create_bounty", "authorize_repo_read("),
(repos, "fork_repo", "authorize_repo_read("),
// get_by_cid resolves each candidate (provenance path + legacy scan) through
// the shared `gate_and_serve`; the gate markers themselves are asserted
// below. This row proves the delegation is real, so the gate is actually
// reached rather than dead code. The delegated gate still calls
// `visibility_check` directly and never `authorize_repo_read`, so it keeps
// the property the pre-merge marker enforced: no fuzzy re-resolve.
(ipfs, "get_by_cid", "gate_and_serve("),
// Sibling read surfaces: gate private-repo metadata on read visibility
// (public repos stay anonymous; private repos 404).
// get_by_cid gates each iterated repo row directly via visibility_check
// (KTD2a: it must NOT route through authorize_repo_read's fuzzy re-resolve).
(ipfs, "get_by_cid", "visibility_check("),
// #94 sibling read surfaces: gate private-repo metadata on read
// visibility (public repos stay anonymous; private repos 404).
(replicas, "list_replicas", "authorize_repo_read("),
(protect, "list_protected_branches", "authorize_repo_read("),
(labels, "list_labels", "authorize_repo_read("),
@@ -254,22 +250,6 @@ mod authz_guard {
"visibility::require_owner must use did_matches for DID-safe owner matching"
);
// The CID read surface (#173) enforces its gate inside the shared
// `gate_and_serve`, which BOTH the provenance path and the legacy scan call, so
// the markers must live there (the get_by_cid row above only proves delegation).
// The repo's own "/" visibility check (KTD2a — never authorize_repo_read's fuzzy
// re-resolve) and the quarantine hard-drop BEFORE visibility (INV-11) are both
// load-bearing: removing either re-opens a leak on the provenance path.
let gate_body = fn_body(ipfs, "gate_and_serve");
assert!(
gate_body.contains("visibility_check("),
"gate_and_serve must gate the CID read surface via visibility_check (KTD2a)"
);
assert!(
gate_body.contains("if quarantined"),
"gate_and_serve must hard-drop a quarantined repo before the visibility gate (INV-11)"
);
for (src, func, marker) in rows {
let body = fn_body(src, func);
assert!(
File diff suppressed because it is too large. Load diff
+167 -72
View File
@@ -213,9 +213,17 @@ pub async fn merge_pr(
.repo_store
.acquire_write(&record.owner_did, &record.name)
.await
.map_err(|e| crate::api::repos::acquire_write_app_error(&e, &name))?;
.map_err(|e| AppError::Git(e.to_string()))?;
let disk_path = guard.path().to_path_buf();
let merger_did = auth.0;
// Snapshot the target branch's pre-merge tip so a failed durable upload can
// roll the merge commit back on local disk (mirrors create_issue). Without
// this the local fast path in RepoStore::acquire serves the merged tree until
// a later write refreshes the archive, even though the client got a 5xx.
let target_ref = format!("refs/heads/{}", pr.target_branch);
let prior_target = store::ref_oid(&disk_path, &target_ref).ok().flatten();
let merge_result = store::merge_branch(
&disk_path,
&pr.target_branch,
@@ -224,14 +232,37 @@ pub async fn merge_pr(
&pr.title,
);
// Always release the advisory lock — even on error; upload to Tigris only on success.
// Short-circuit on a refused publish before the PR is marked merged and
// before the webhook fires. Both are irreversible announcements of a merge
// commit that only exists on this node's disk.
guard.release(merge_result.is_ok()).await.into_result()?;
// Always release the advisory lock, even on error; upload to Tigris only on
// success. A false return means the durable upload failed/timed out: the merge
// commit applied to local disk but never reached object storage, so the next
// acquire_write reverts it from the stale archive. Report 5xx so the client
// retries rather than marking the PR merged in the DB while the merged tree
// was silently lost. Fail BEFORE merge_pr / touch_repo / webhooks. (P1
// data-loss guard, same as receive-pack.) On that failure, roll the local
// merge back to the captured tip BEFORE the lock releases.
let upload_ok = guard
.release_with_failure_cleanup(merge_result.is_ok(), |path| {
if let Some(oid) = &prior_target {
if let Err(e) = store::set_ref(path, &target_ref, oid) {
tracing::warn!(repo = %record.name, err = %e,
"failed to roll back local merge commit after failed durable upload; \
a local-fast-path read may serve the un-uploaded merged tree");
}
}
})
.await;
let merge_sha = merge_result.map_err(|e| AppError::Git(e.to_string()))?;
if !upload_ok {
tracing::error!(repo = %record.name,
"durable upload failed after merge; reporting failure so the client retries");
return Err(AppError::Git(format!(
"durable storage upload failed for {}",
record.name
)));
}
state.db.merge_pr(&pr.id, &merger_did).await?;
let _ = state.db.touch_repo(&record.id).await;
@@ -428,57 +459,133 @@ pub async fn list_comments(
Ok(Json(serde_json::json!({ "comments": comments })))
}
/// #173 F1 follow-up: merge holds no admission permit, so an exhausted write-lock
/// pool must shed 503 + Retry-After.
#[cfg(test)]
mod lock_pool_shed_tests {
mod tests {
// super::* brings in the handler plus the axum Extension/Path/State
// extractors and the Utc/Uuid/PullRequest types the seed uses.
use super::*;
use axum::response::IntoResponse;
use sqlx::PgPool;
use std::path::Path as StdPath;
use std::time::Duration;
fn seed_repo(owner_did: &str, name: &str) -> crate::db::RepoRecord {
let now = Utc::now();
crate::db::RepoRecord {
id: Uuid::new_v4().to_string(),
name: name.to_string(),
owner_did: owner_did.to_string(),
description: None,
is_public: true,
default_branch: "main".to_string(),
created_at: now,
updated_at: now,
disk_path: format!("/tmp/{name}"),
forked_from: None,
machine_id: None,
// Object store whose upload() parks forever, so release()'s bounded timeout
// is the only thing that completes it, modeling a stalled durable PUT.
// exists() is false so acquire_write never downloads over the local bare repo.
struct StallStore;
#[async_trait::async_trait]
impl crate::git::tigris::ObjectStore for StallStore {
async fn exists(&self, _o: &str, _r: &str) -> anyhow::Result<bool> {
Ok(false)
}
async fn upload(&self, _o: &str, _r: &str, _p: &StdPath) -> anyhow::Result<()> {
std::future::pending::<()>().await;
Ok(())
}
async fn download(&self, _o: &str, _r: &str, _p: &StdPath) -> anyhow::Result<()> {
Ok(())
}
async fn delete(&self, _o: &str, _r: &str) -> anyhow::Result<()> {
Ok(())
}
}
// The failed-upload rollback on the merge path. merge_pr writes the merge
// commit to LOCAL disk, then release() uploads to durable storage. When
// that upload times out the handler must 5xx (before merge_pr/webhooks)
// AND roll the target branch back to its pre-merge tip while the advisory
// lock is held, so a local-fast-path read never serves the un-uploaded
// merged tree. RED with the cleanup closure at ~pulls.rs:244 reverted to
// `|_| {}`: main stays at the merge commit.
#[sqlx::test]
async fn merge_pr_lock_pool_exhaustion_sheds_503_not_500(pool: PgPool) {
let owner = "did:key:zMERGELOCKPOOLOWNERAAAAAAAAAAAAAAAAAAAAA";
async fn merge_failed_upload_rolls_back_target_ref(pool: sqlx::PgPool) {
use axum::response::IntoResponse;
use std::process::Command;
let mut state = crate::test_support::test_state(pool.clone()).await;
let repos_dir = tempfile::TempDir::new().unwrap();
state.repo_store = crate::git::repo_store::RepoStore::new(
std::path::PathBuf::from("/tmp/gitlawb-pulls-lockpool"),
None,
crate::git::repo_store::build_lock_pool(&pool, 1, std::time::Duration::from_secs(1)),
std::time::Duration::from_secs(300),
repos_dir.path().to_path_buf(),
Some(std::sync::Arc::new(StallStore)),
pool.clone(),
)
.with_release_upload_timeout(Duration::from_millis(200));
let owner = "z6mergerollbackowner";
let name = "mergerollback";
let owner_slug = owner.replace([':', '/'], "_");
let bare = repos_dir
.path()
.join(&owner_slug)
.join(format!("{name}.git"));
state
.db
.upsert_mirror_repo(owner, name, &bare.to_string_lossy(), None, false)
.await
.unwrap();
fn git(args: &[&str], dir: &StdPath) -> String {
let out = Command::new("git")
.args(args)
.current_dir(dir)
.output()
.unwrap();
assert!(
out.status.success(),
"git {args:?} failed: {}",
String::from_utf8_lossy(&out.stderr)
);
String::from_utf8_lossy(&out.stdout).trim().to_string()
}
// Scratch repo: c1 on main, c2 on feature; the bare clone carries both
// branches, so the merge of feature into main has real work to do.
let work = tempfile::TempDir::new().unwrap();
git(&["init", "-q", "-b", "main", "."], work.path());
git(&["config", "user.email", "t@t"], work.path());
git(&["config", "user.name", "t"], work.path());
std::fs::write(work.path().join("f.txt"), "one").unwrap();
git(&["add", "f.txt"], work.path());
git(&["commit", "-q", "-m", "c1"], work.path());
git(&["checkout", "-q", "-b", "feature"], work.path());
std::fs::write(work.path().join("g.txt"), "two").unwrap();
git(&["add", "g.txt"], work.path());
git(&["commit", "-q", "-m", "c2"], work.path());
std::fs::create_dir_all(bare.parent().unwrap()).unwrap();
let out = Command::new("git")
.args([
"clone",
"--bare",
"-q",
&work.path().to_string_lossy(),
&bare.to_string_lossy(),
])
.output()
.unwrap();
assert!(
out.status.success(),
"clone --bare failed: {}",
String::from_utf8_lossy(&out.stderr)
);
let repo = seed_repo(owner, "lp-merge");
let repo_id = repo.id.clone();
state.db.create_repo(&repo).await.expect("seed repo");
let pre_merge_tip = store::ref_oid(&bare, "refs/heads/main")
.unwrap()
.expect("main exists before the merge");
// Seed the open PR row the handler looks up.
let record = state.db.get_repo(owner, name).await.unwrap().unwrap();
let now = Utc::now().to_rfc3339();
state
.db
.create_pr(&PullRequest {
id: Uuid::new_v4().to_string(),
repo_id: repo_id.clone(),
repo_id: record.id.clone(),
number: 1,
title: "lp".to_string(),
title: "t".into(),
body: None,
author_did: owner.to_string(),
source_branch: "feature".to_string(),
target_branch: "main".to_string(),
source_branch: "feature".into(),
target_branch: "main".into(),
status: "open".to_string(),
merged_by_did: None,
merged_at: None,
@@ -486,46 +593,34 @@ mod lock_pool_shed_tests {
updated_at: now,
})
.await
.expect("seed open PR");
.unwrap();
let held = state
.repo_store
.acquire_write(owner, "other-repo")
.await
.expect("the first write takes the only lock-pool connection");
let shed = merge_pr(
// Merge as the owner: the merge commit applies locally, the durable
// upload stalls and times out, and the handler must report 5xx.
let resp = super::merge_pr(
State(state.clone()),
Extension(AuthenticatedDid(owner.to_string())),
Path((owner.to_string(), "lp-merge".to_string(), 1)),
)
.await;
let err = shed.expect_err("an exhausted lock pool must fail the call");
let resp = err.into_response();
assert_eq!(
resp.status(),
StatusCode::SERVICE_UNAVAILABLE,
"merge_pr: an exhausted write-lock pool must shed 503, not a 500 git error"
);
assert_eq!(
resp.headers()
.get("retry-after")
.map(|v| v.to_str().unwrap()),
Some("1"),
"merge_pr: a capacity shed must tell the client when to retry"
);
let _ = held.release(false).await;
let admitted = merge_pr(
State(state.clone()),
Extension(AuthenticatedDid(owner.to_string())),
Path((owner.to_string(), "lp-merge".to_string(), 1)),
Path((owner.to_string(), name.to_string(), 1)),
)
.await;
let status = match resp {
Ok(_) => StatusCode::OK,
Err(e) => e.into_response().status(),
};
assert!(
!matches!(admitted, Err(AppError::Overloaded(_))),
"with the lock pool free, merge_pr must not be shed as capacity; got {:?}",
admitted.err()
status.is_server_error(),
"a stalled durable upload on merge must 5xx, got {status}"
);
// The target branch must still point at the pre-merge tip: the failed
// upload rolled the merge commit back while the lock was held.
let tip = store::ref_oid(&bare, "refs/heads/main")
.unwrap()
.expect("main still exists after the rollback");
assert_eq!(
tip, pre_merge_tip,
"a failed durable upload must roll the target branch back to its \
pre-merge tip; a local-fast-path read served the un-uploaded merge"
);
}
}
File diff suppressed because it is too large. Load diff
+1 -27
View File
@@ -490,7 +490,6 @@ mod tests {
use clap::Parser;
let keypair = Keypair::generate();
let scan_token_key = crate::state::AppState::derive_scan_token_key(&keypair);
let (ref_tx, _) = tokio::sync::broadcast::channel(1);
let (task_tx, _) = tokio::sync::broadcast::channel(1);
let pool = sqlx::postgres::PgPoolOptions::new()
@@ -516,37 +515,12 @@ mod tests {
repo_store: crate::git::repo_store::RepoStore::for_testing(PathBuf::from("/tmp"), pool),
rate_limiter: RateLimiter::new(100, Duration::from_secs(60)),
create_ip_rate_limiter: RateLimiter::new(1000, Duration::from_secs(3600)),
write_rate_limiter: RateLimiter::new(1000, Duration::from_secs(3600)),
push_rate_limiter: RateLimiter::new(600, Duration::from_secs(3600)),
close_issue_rate_limiter: RateLimiter::new(120, Duration::from_secs(3600)),
ipfs_rate_limiter: RateLimiter::new(600, Duration::from_secs(3600)),
ipfs_work_rate_limiter: RateLimiter::new(600, Duration::from_secs(3600)),
ipfs_max_history_walks: crate::api::ipfs::MAX_HISTORY_WALKS_PER_REQUEST,
ipfs_max_legacy_probes: crate::api::ipfs::MAX_LEGACY_PROBES_PER_REQUEST,
ipfs_legacy_scan_page_rows: crate::api::ipfs::LEGACY_SCAN_PAGE_ROWS,
ipfs_max_legacy_scan_rows: crate::api::ipfs::MAX_LEGACY_SCAN_ROWS_PER_REQUEST,
ipfs_max_legacy_scan_rule_bytes:
crate::api::ipfs::MAX_LEGACY_SCAN_RULE_BYTES_PER_REQUEST,
ipfs_scan_token_key: Arc::new(scan_token_key),
ipfs_max_served_object_bytes: crate::api::ipfs::MAX_SERVED_OBJECT_BYTES,
push_limiter_trust: crate::rate_limit::TrustedProxy::None,
sync_trigger_rate_limiter: RateLimiter::new(60, Duration::from_secs(3600)),
peer_write_rate_limiter: RateLimiter::new(600, Duration::from_secs(3600)),
shutdown_tx: tokio::sync::watch::channel(false).0,
git_read_semaphore: Arc::new(tokio::sync::Semaphore::new(64)),
git_write_semaphore: Arc::new(tokio::sync::Semaphore::new(64)),
git_push_advert_semaphore: Arc::new(tokio::sync::Semaphore::new(64)),
git_encrypt_semaphore: Arc::new(tokio::sync::Semaphore::new(64)),
pin_semaphore: Arc::new(tokio::sync::Semaphore::new(64)),
encrypt_inflight: crate::state::EncryptInflight::new(),
repo_write_leases: crate::state::RepoWriteLeases::new(8),
git_read_per_caller: crate::rate_limit::PerCallerConcurrency::with_default_max_keys(16),
git_push_advert_per_caller:
crate::rate_limit::PerCallerConcurrency::with_default_max_keys(8),
git_write_per_caller: crate::rate_limit::PerCallerConcurrency::with_default_max_keys(8),
git_ipfs_walk_semaphore: Arc::new(tokio::sync::Semaphore::new(64)),
git_ipfs_walk_per_caller:
crate::rate_limit::PerCallerConcurrency::with_default_max_keys(16),
git_bin: "git".to_string(),
}
}
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
+1 -1
View File
@@ -170,7 +170,7 @@ pub async fn encrypt_and_pin(
continue;
}
};
let cid = match crate::ipfs_pin::pin_git_object(ipfs_api, oid, &envelope, None).await {
let cid = match crate::ipfs_pin::pin_git_object(ipfs_api, oid, &envelope).await {
Ok(c) if !c.is_empty() => c,
Ok(_) => {
tracing::warn!(oid = %oid, "pin_git_object returned no cid; skipping");
+27 -273
View File
@@ -35,48 +35,17 @@ pub enum AppError {
#[error("invalid request: {0}")]
BadRequest(String),
/// A DID was well-formed enough to carry to a resolver but no verifying key
/// could be derived from it. Its own code rather than plain `bad_request`
/// because the auth middleware already answers `unresolvable_did` for the
/// same failure on a request's keyid, and a client should not have to
/// substring-match a message to tell this apart from the other validation
/// failures on the same route.
#[error("unresolvable did: {0}")]
UnresolvableDid(String),
#[error("too many requests: {0}")]
TooManyRequests(String),
#[error("incomplete: {0}")]
Incomplete(String),
/// A bounded search that could not complete. `continuation`, when present, is the
/// sealed scan position the caller echoes as `?scan=` to resume where the search
/// stopped (#173 round 13, F2). It is AEAD-sealed at the mint site, never plaintext:
/// the row it names is by construction one the caller was denied (INV-13).
#[error("search incomplete: {message}")]
SearchIncomplete {
message: String,
continuation: Option<String>,
},
#[error("git error: {0}")]
Git(String),
#[error("git service timed out: {0}")]
Timeout(String),
#[error("server overloaded: {0}")]
Overloaded(String),
#[error("repository is busy")]
RepoBusy,
#[error("repository is temporarily unavailable")]
RepoUnavailable,
#[error("repository write was fenced by a concurrent publish")]
RepoWriteFenced,
#[error("service unavailable: {0}")]
Unavailable(String),
#[error("database error: {0}")]
Db(#[from] sqlx::Error),
@@ -91,14 +60,6 @@ pub enum AppError {
pub const DB_UNAVAILABLE_CODE: &str = "db_unavailable";
pub const DB_UNAVAILABLE_MESSAGE: &str = "database is temporarily unavailable";
/// Generic client-facing message for `AppError::Internal`. The real error is
/// logged server-side; never put sqlx/anyhow detail in the HTTP body (#226).
pub const INTERNAL_ERROR_MESSAGE: &str = "an internal error occurred";
/// Generic client-facing message for non-unavailable `AppError::Db`. Query /
/// schema errors stay in logs; the HTTP body must not leak them (#226).
pub const DB_ERROR_MESSAGE: &str = "a database error occurred";
/// Connection-level sqlx failures that mean the database is unreachable right
/// now (retryable, 503), as opposed to server-reported query errors.
fn db_unavailable(e: &sqlx::Error) -> bool {
@@ -119,31 +80,7 @@ impl From<anyhow::Error> for AppError {
fn from(err: anyhow::Error) -> Self {
match err.downcast::<sqlx::Error>() {
Ok(sql) => AppError::Db(sql),
// Lock contention is transient and ordinary, so it must not land as a
// 500. The internal message names the owner slug and repo, so the
// variant carries nothing: the detail stays in the log at the raise
// site and the client gets a fixed retryable body.
Err(err) => match err.downcast::<crate::git::repo_store::LockPoolBusy>() {
Ok(_) => AppError::Overloaded("git write locks at capacity, retry shortly".into()),
Err(err) => match err.downcast::<crate::git::repo_store::RepoBusy>() {
Ok(_) => AppError::RepoBusy,
// Same reasoning one rung down: a refused under-lock refresh is a
// transient storage condition, and its internal message names the
// owner slug and repo, so the variant carries nothing.
Err(err) => match err.downcast::<crate::git::repo_store::RepoUnavailable>() {
Ok(_) => AppError::RepoUnavailable,
// And one more rung: a publish the store refused twice is
// transient in the same way, and the retry is the client's
// to make. The variant carries nothing for the same reason
// as the two above.
Err(err) => match err.downcast::<crate::git::repo_store::RepoWriteFenced>()
{
Ok(_) => AppError::RepoWriteFenced,
Err(err) => AppError::Internal(err),
},
},
},
},
Err(err) => AppError::Internal(err),
}
}
}
@@ -195,123 +132,40 @@ impl IntoResponse for AppError {
// IcaptchaProofRequired is handled above (it carries extra headers/fields).
AppError::IcaptchaProofRequired { .. } => unreachable!("handled before this match"),
AppError::BadRequest(msg) => (StatusCode::BAD_REQUEST, "bad_request", msg.clone()),
AppError::UnresolvableDid(msg) => {
(StatusCode::BAD_REQUEST, "unresolvable_did", msg.clone())
}
AppError::TooManyRequests(msg) => {
(StatusCode::TOO_MANY_REQUESTS, "rate_limited", msg.clone())
}
AppError::Incomplete(msg) => {
(StatusCode::UNPROCESSABLE_ENTITY, "incomplete", msg.clone())
}
// A bounded search that could not complete (the CID resolver hit its
// legacy-probe or walk ceiling), distinct from the 404 that asserts a
// definitive not-found: absence was NOT proven, so the caller should
// retry rather than treat it as gone (#173, F2). 503, retryable.
AppError::SearchIncomplete { message, .. } => (
StatusCode::SERVICE_UNAVAILABLE,
"search_incomplete",
message.clone(),
),
AppError::Git(msg) => (StatusCode::INTERNAL_SERVER_ERROR, "git_error", msg.clone()),
// 504, distinct from the 500 git_error and from the read-gate's 404 /
// the auth 401, so the client can tell a deadline from a failure.
AppError::Timeout(msg) => (StatusCode::GATEWAY_TIMEOUT, "git_timeout", msg.clone()),
// 503 with a FIXED body: the caller should retry, and must not be told
// which repo is contended or for how long.
AppError::RepoBusy => (
// 503, retryable: a transient resource contention (e.g. the repo lock
// pool is pinned by a write burst or a purge holds the key), distinct
// from the 500 git_error so the client knows to back off and retry.
AppError::Unavailable(msg) => (
StatusCode::SERVICE_UNAVAILABLE,
"repo_busy",
"repository is busy — retry".into(),
),
// 503 with a FIXED body for the same reason: the caller should retry, and
// must not be told which repo could not be refreshed or why.
AppError::RepoUnavailable => (
StatusCode::SERVICE_UNAVAILABLE,
"repo_unavailable",
"repository is temporarily unavailable, retry".into(),
),
// 503 with a FIXED body again, and its own code: the caller should
// retry, but the condition is not contention, so a client that
// distinguishes them should be able to. The body must not say which
// repo lost its publish or to whom.
AppError::RepoWriteFenced => (
StatusCode::SERVICE_UNAVAILABLE,
"repo_write_fenced",
"repository changed underneath this write, retry".into(),
"service_unavailable",
msg.clone(),
),
AppError::Db(e) if db_unavailable(e) => (
StatusCode::SERVICE_UNAVAILABLE,
DB_UNAVAILABLE_CODE,
DB_UNAVAILABLE_MESSAGE.into(),
),
// 503 with a Retry-After (attached after this match — the shared tail
// can't carry per-variant headers). This is the single place Overloaded
// becomes a response, so it can never ship a 503 without the retry hint.
AppError::Overloaded(msg) => {
(StatusCode::SERVICE_UNAVAILABLE, "overloaded", msg.clone())
}
// Opaque body + server log: bare `?` on sqlx paths becomes `AppError::Db`
// via `From`, so this arm (not `Internal`) is the common leak for open
// routes like GET /api/v1/repos and GET /api/v1/peers (#226).
AppError::Db(e) => {
tracing::error!(error = %e, "database error");
(
StatusCode::INTERNAL_SERVER_ERROR,
"db_error",
DB_ERROR_MESSAGE.into(),
)
}
// Opaque body: handlers that map with `.map_err(AppError::Internal)`
// (e.g. GET /ipfs/{cid}) land here; other DB failures usually hit `Db`.
// Log `{e:#}` so context-wrapped anyhow chains keep the leaf cause
// (Display alone is only the outermost layer; see api/repos.rs).
AppError::Internal(e) => {
tracing::error!(error = %format!("{e:#}"), "internal error");
(
StatusCode::INTERNAL_SERVER_ERROR,
"internal_error",
INTERNAL_ERROR_MESSAGE.into(),
)
}
AppError::Db(e) => (StatusCode::INTERNAL_SERVER_ERROR, "db_error", e.to_string()),
AppError::Internal(e) => (
StatusCode::INTERNAL_SERVER_ERROR,
"internal_error",
e.to_string(),
),
};
let mut body = json!({
let body = Json(json!({
"error": code,
"message": message,
});
// A truncated CID search may carry the sealed position the caller echoes as
// `?scan=` to resume. Rendered as a third body field, present only when the
// shed actually left something to resume: a wrapped scan and a throttled
// request both omit it, and its ABSENCE is what tells a caller the ladder is
// over. It is opaque ciphertext; see `gitlawb_core::scan_token`.
if let AppError::SearchIncomplete {
continuation: Some(token),
..
} = &self
{
body["continuation"] = json!(token);
}
}));
let mut resp = (status, Json(body)).into_response();
// Both retryable 503s advertise when to retry: Overloaded (capacity shed) and
// SearchIncomplete (a bounded CID search cut short by a cap — retry may complete
// it). They ride the shared tail above for body/status, so the header is attached
// here rather than in bespoke early returns, keeping each variant handled once.
if matches!(
self,
AppError::Overloaded(_)
| AppError::SearchIncomplete { .. }
| AppError::RepoBusy
| AppError::RepoUnavailable
| AppError::RepoWriteFenced
) {
resp.headers_mut().insert(
axum::http::header::RETRY_AFTER,
axum::http::HeaderValue::from_static("1"),
);
}
resp
(status, body).into_response()
}
}
@@ -335,118 +189,18 @@ mod tests {
}
#[test]
fn overloaded_maps_to_503_with_retry_after() {
let resp = AppError::Overloaded("x".into()).into_response();
assert_eq!(resp.status(), StatusCode::SERVICE_UNAVAILABLE);
fn unavailable_maps_to_503_distinct_from_git_500() {
// The create-repo lock-acquire failure returns Unavailable so a client sees
// a retryable 503, not a terminal 500 git_error.
assert_eq!(
resp.headers().get("retry-after").unwrap().to_str().unwrap(),
"1"
AppError::Unavailable("lock pool pinned".into())
.into_response()
.status(),
StatusCode::SERVICE_UNAVAILABLE
);
}
/// #226: raw sqlx/DB detail must never appear in the Internal 500 body.
#[tokio::test]
async fn internal_error_body_is_opaque() {
use serde_json::{json, Value};
let leak = "error returned from database: relation \"repos\" does not exist";
let resp = AppError::Internal(anyhow::anyhow!("{leak}")).into_response();
assert_eq!(resp.status(), StatusCode::INTERNAL_SERVER_ERROR);
let bytes = axum::body::to_bytes(resp.into_body(), usize::MAX)
.await
.expect("read body");
let v: Value = serde_json::from_slice(&bytes).expect("json body");
// Exact object: a new `detail` field with different sensitive text must
// also fail, not only a repeat of the original error string.
assert_eq!(
v,
json!({
"error": "internal_error",
"message": INTERNAL_ERROR_MESSAGE,
})
AppError::Git("x".into()).into_response().status(),
StatusCode::INTERNAL_SERVER_ERROR
);
}
/// #226: `AppError::Db` query errors (the common `?` path) must also be opaque.
#[tokio::test]
async fn db_error_body_is_opaque() {
use serde_json::{json, Value};
let resp = AppError::Db(sqlx::Error::Protocol(
"error returned from database: column \"is_public\" does not exist".into(),
))
.into_response();
assert_eq!(resp.status(), StatusCode::INTERNAL_SERVER_ERROR);
let bytes = axum::body::to_bytes(resp.into_body(), usize::MAX)
.await
.expect("read body");
let v: Value = serde_json::from_slice(&bytes).expect("json body");
assert_eq!(
v,
json!({
"error": "db_error",
"message": DB_ERROR_MESSAGE,
})
);
}
/// Connection-level failures must stay 503 `db_unavailable`, not collapse
/// into the opaque 500 `db_error` arm if `db_unavailable` loses a variant.
#[tokio::test]
async fn db_pool_timeout_stays_503_unavailable() {
use serde_json::{json, Value};
let resp = AppError::Db(sqlx::Error::PoolTimedOut).into_response();
assert_eq!(resp.status(), StatusCode::SERVICE_UNAVAILABLE);
let bytes = axum::body::to_bytes(resp.into_body(), usize::MAX)
.await
.expect("read body");
let v: Value = serde_json::from_slice(&bytes).expect("json body");
assert_eq!(
v,
json!({
"error": DB_UNAVAILABLE_CODE,
"message": DB_UNAVAILABLE_MESSAGE,
})
);
}
/// #251: bare `?` on anyhow-wrapped sqlx relies on this downcast so a
/// closed pool becomes 503 `db_unavailable`, not 500 `internal_error`.
#[test]
fn pool_closed_via_anyhow_from_is_503_db_unavailable() {
let err: AppError = anyhow::Error::from(sqlx::Error::PoolClosed).into();
let resp = err.into_response();
assert_eq!(resp.status(), StatusCode::SERVICE_UNAVAILABLE);
}
#[test]
fn lock_pool_busy_via_anyhow_from_is_503_overloaded() {
let err: AppError = anyhow::Error::new(crate::git::repo_store::LockPoolBusy).into();
let resp = err.into_response();
assert_eq!(resp.status(), StatusCode::SERVICE_UNAVAILABLE);
assert_eq!(
resp.headers().get("retry-after").unwrap().to_str().unwrap(),
"1"
);
}
#[test]
fn repo_busy_unavailable_and_fenced_advertise_retry_after() {
for err in [
AppError::RepoBusy,
AppError::RepoUnavailable,
AppError::RepoWriteFenced,
] {
let resp = err.into_response();
assert_eq!(resp.status(), StatusCode::SERVICE_UNAVAILABLE);
assert_eq!(
resp.headers().get("retry-after").unwrap().to_str().unwrap(),
"1"
);
}
}
}
+41 -38
View File
@@ -61,27 +61,51 @@ pub fn create_issue(repo_path: &Path, issue_id: &str, json: &str) -> Result<()>
Ok(())
}
/// Remove a single issue ref while the write guard still holds the advisory lock,
/// after a definite publish refusal. Best-effort: a failed delete leaves a
/// local-only orphan, which is still better than telling the client to retry into a
/// duplicate id. Not used on `UploadUnknowable`, where the PUT may still land.
pub fn delete_issue_ref(
git_bin: &str,
repo_path: &Path,
issue_id: &str,
deadline: std::time::Instant,
) -> Result<()> {
/// Delete an issue ref. Used to roll back a locally written issue whose
/// durable upload failed, so a retry does not duplicate it. The dangling
/// blob object is harmless.
pub fn delete_issue_ref(repo_path: &Path, issue_id: &str) -> Result<()> {
let ref_name = format!("refs/gitlawb/issues/{issue_id}");
crate::git::visibility_pack::run_bounded_git(
git_bin,
&["update-ref", "-d", &ref_name],
repo_path,
b"",
deadline,
)?;
let output = Command::new("git")
.args(["update-ref", "-d", &ref_name])
.current_dir(repo_path)
.output()
.context("failed to run git update-ref -d")?;
if !output.status.success() {
let stderr = String::from_utf8_lossy(&output.stderr);
anyhow::bail!("git update-ref -d failed: {stderr}");
}
Ok(())
}
/// Read the object id an issue ref currently points to (resolving an 8-char
/// prefix), returning `(full_id, oid)`. Returns None if no such issue exists.
/// Used to snapshot an issue ref's pre-mutation state so a failed durable
/// upload can restore it, mirroring `create_issue`'s rollback.
pub fn issue_ref_oid(repo_path: &Path, issue_id: &str) -> Result<Option<(String, String)>> {
let full_id = match resolve_issue_id(repo_path, issue_id)? {
Some(id) => id,
None => return Ok(None),
};
let ref_name = format!("refs/gitlawb/issues/{full_id}");
let oid = match crate::git::store::ref_oid(repo_path, &ref_name)? {
Some(oid) => oid,
None => return Ok(None),
};
Ok(Some((full_id, oid)))
}
/// Reset an issue ref back to a captured object id — rolls back a local
/// mutation (e.g. a close) whose durable upload failed, so a local-fast-path
/// read does not serve the un-uploaded state. `full_id` must be the resolved
/// UUID (from [`issue_ref_oid`]). The now-dangling object is harmless.
pub fn restore_issue_ref(repo_path: &Path, full_id: &str, oid: &str) -> Result<()> {
let ref_name = format!("refs/gitlawb/issues/{full_id}");
crate::git::store::set_ref(repo_path, &ref_name, oid)
}
/// List all issue refs and return their JSON content.
pub fn list_issues(repo_path: &Path) -> Result<Vec<String>> {
// List all refs under refs/gitlawb/issues/
@@ -299,27 +323,6 @@ mod tests {
assert!(result.unwrap_err().to_string().contains("ambiguous"));
}
#[test]
fn delete_issue_ref_removes_a_created_ref() {
let dir = TempDir::new().unwrap();
init_repo(&dir);
let full_id = "eee88888-0000-0000-0000-000000000000";
create_issue(
dir.path(),
full_id,
r#"{"id":"eee88888-0000-0000-0000-000000000000","status":"open"}"#,
)
.unwrap();
delete_issue_ref(
"git",
dir.path(),
full_id,
std::time::Instant::now() + std::time::Duration::from_secs(30),
)
.unwrap();
assert_eq!(resolve_issue_id(dir.path(), full_id).unwrap(), None);
}
#[test]
fn test_close_issue_via_prefix() {
let dir = TempDir::new().unwrap();
+96 -458
View File
@@ -29,9 +29,9 @@
use std::collections::HashSet;
use std::path::{Path, PathBuf};
use std::time::{Duration, Instant};
use std::process::Command;
use anyhow::Result;
use anyhow::{bail, Context, Result};
/// Env var that forces the push path to always full-scan, bypassing the delta
/// optimization (KTD7 kill-switch). Reuses the already-tested fallback branch
@@ -64,21 +64,11 @@ pub enum PinCandidates {
/// Fail-closed: any condition where the introduced set cannot be safely
/// determined returns [`PinCandidates::FullScanRequired`] rather than a partial
/// set, so the caller full-scans instead of silently under-pinning.
pub fn resolve_push_delta(
repo_path: &Path,
new_tips: &[&str],
old_tips: &[&str],
git_bin: &str,
deadline: Instant,
) -> PinCandidates {
resolve_push_delta_inner(
repo_path,
new_tips,
old_tips,
force_full_scan(),
git_bin,
deadline,
)
pub fn resolve_push_delta(repo_path: &Path, new_tips: &[&str], old_tips: &[&str]) -> PinCandidates {
// KTD7 kill-switch: force the (already-tested) full-scan fallback. The env
// read is split out from the pure logic so the resolver stays unit-testable
// without touching process-global state.
resolve_push_delta_inner(repo_path, new_tips, old_tips, force_full_scan())
}
/// Whether the force-full-scan kill-switch env var is set.
@@ -93,8 +83,6 @@ fn resolve_push_delta_inner(
new_tips: &[&str],
old_tips: &[&str],
force_full_scan: bool,
git_bin: &str,
deadline: Instant,
) -> PinCandidates {
if force_full_scan {
tracing::debug!("{FORCE_FULL_SCAN_ENV} set — forcing full scan");
@@ -114,7 +102,7 @@ fn resolve_push_delta_inner(
// commit). Bare `cat-file -t` returns `tag` for an annotated tag, and
// `for-each-ref %(*objecttype)` peels only one level — neither is correct.
for tip in new_tips {
match peeled_object_type(repo_path, tip, git_bin, deadline) {
match peeled_object_type(repo_path, tip) {
Some(t) if t == "commit" => {}
other => {
tracing::debug!(
@@ -127,7 +115,7 @@ fn resolve_push_delta_inner(
}
}
match rev_list_delta(repo_path, new_tips, old_tips, git_bin, deadline) {
match rev_list_delta(repo_path, new_tips, old_tips) {
Ok(oids) => PinCandidates::Delta(oids),
Err(e) => {
tracing::debug!(err = %e, "push-delta rev-list failed — forcing full scan");
@@ -138,43 +126,41 @@ fn resolve_push_delta_inner(
/// Return the fully-peeled object type of `sha` (e.g. `commit`, `tree`,
/// `blob`), or `None` if the object is missing/unpeelable or git errored.
fn peeled_object_type(
repo_path: &Path,
sha: &str,
git_bin: &str,
deadline: Instant,
) -> Option<String> {
let peel = format!("{sha}^{{}}");
let out = crate::git::visibility_pack::run_bounded_git(
git_bin,
&["cat-file", "-t", &peel],
repo_path,
b"",
deadline,
)
.ok()?;
Some(String::from_utf8_lossy(&out).trim().to_string())
fn peeled_object_type(repo_path: &Path, sha: &str) -> Option<String> {
let output = Command::new("git")
.args(["cat-file", "-t", &format!("{sha}^{{}}")])
.current_dir(repo_path)
.output()
.ok()?;
if !output.status.success() {
return None;
}
Some(String::from_utf8_lossy(&output.stdout).trim().to_string())
}
/// Run `git rev-list --objects --no-object-names <new> --not <old>` and return
/// the bare OID set. Decides on `status.success()` *before* parsing stdout, so
/// a walk that prints a valid prefix then errors mid-walk is discarded.
fn rev_list_delta(
repo_path: &Path,
new_tips: &[&str],
old_tips: &[&str],
git_bin: &str,
deadline: Instant,
) -> Result<Vec<String>> {
fn rev_list_delta(repo_path: &Path, new_tips: &[&str], old_tips: &[&str]) -> Result<Vec<String>> {
let mut args: Vec<&str> = vec!["rev-list", "--objects", "--no-object-names"];
args.extend_from_slice(new_tips);
if !old_tips.is_empty() {
args.push("--not");
args.extend_from_slice(old_tips);
}
let out =
crate::git::visibility_pack::run_bounded_git(git_bin, &args, repo_path, b"", deadline)?;
let stdout = String::from_utf8_lossy(&out);
let output = Command::new("git")
.args(&args)
.current_dir(repo_path)
.output()
.context("failed to run git rev-list for push delta")?;
if !output.status.success() {
let stderr = String::from_utf8_lossy(&output.stderr);
bail!("git rev-list failed: {stderr}");
}
let stdout = String::from_utf8_lossy(&output.stdout);
Ok(stdout
.lines()
.map(|l| l.trim().to_string())
@@ -189,19 +175,23 @@ fn rev_list_delta(
/// reconciliation sweep relies on. It returns *all* objects (including
/// unreachable/dangling ones), which is what the sweep needs to catch
/// stragglers — do not swap it for a reachability walk.
pub fn list_all_objects(repo_path: &Path, git_bin: &str, deadline: Instant) -> Result<Vec<String>> {
let out = crate::git::visibility_pack::run_bounded_git(
git_bin,
&[
pub fn list_all_objects(repo_path: &Path) -> Result<Vec<String>> {
let output = Command::new("git")
.args([
"cat-file",
"--batch-all-objects",
"--batch-check=%(objectname)",
],
repo_path,
b"",
deadline,
)?;
let stdout = String::from_utf8_lossy(&out);
])
.current_dir(repo_path)
.output()
.context("failed to run git cat-file")?;
if !output.status.success() {
let stderr = String::from_utf8_lossy(&output.stderr);
bail!("git cat-file failed: {stderr}");
}
let stdout = String::from_utf8_lossy(&output.stdout);
Ok(stdout
.lines()
.map(|l| l.trim().to_string())
@@ -213,23 +203,23 @@ pub fn list_all_objects(repo_path: &Path, git_bin: &str, deadline: Instant) -> R
/// `--batch-check='%(objectname) %(objecttype)'`. The pin path's fail-closed
/// filter needs to tell blobs (content, withholdable) from commits/trees
/// (structural, never withheld) without typing the candidate list itself.
pub fn list_all_objects_with_type(
repo_path: &Path,
git_bin: &str,
deadline: Instant,
) -> Result<Vec<(String, String)>> {
let out = crate::git::visibility_pack::run_bounded_git(
git_bin,
&[
pub fn list_all_objects_with_type(repo_path: &Path) -> Result<Vec<(String, String)>> {
let output = Command::new("git")
.args([
"cat-file",
"--batch-all-objects",
"--batch-check=%(objectname) %(objecttype)",
],
repo_path,
b"",
deadline,
)?;
let stdout = String::from_utf8_lossy(&out);
])
.current_dir(repo_path)
.output()
.context("failed to run git cat-file")?;
if !output.status.success() {
let stderr = String::from_utf8_lossy(&output.stderr);
bail!("git cat-file failed: {stderr}");
}
let stdout = String::from_utf8_lossy(&output.stdout);
Ok(stdout
.lines()
.filter_map(|l| {
@@ -245,12 +235,8 @@ pub fn list_all_objects_with_type(
/// fail-closed pin filter drops any candidate blob absent from the reachable,
/// visibility-allowed set; a dangling private blob is in this set but not the
/// allowed set, so it never replicates (#99).
pub fn all_blob_oids(
repo_path: &Path,
git_bin: &str,
deadline: Instant,
) -> Result<HashSet<String>> {
Ok(list_all_objects_with_type(repo_path, git_bin, deadline)?
pub fn all_blob_oids(repo_path: &Path) -> Result<HashSet<String>> {
Ok(list_all_objects_with_type(repo_path)?
.into_iter()
.filter(|(_, ty)| ty == "blob")
.map(|(oid, _)| oid)
@@ -282,68 +268,22 @@ pub struct PinCandidateSet {
/// it can never leak because the withheld/fail-closed filter still runs on
/// whatever set is returned. `full_scan` rides on the returned set so the caller
/// knows when the dangling-inclusive filter is required.
///
/// `scan_sem` is the post-receive scan admission pool (`git_encrypt_semaphore`,
/// #174 F4): both git-spawning stages — the per-tip `cat-file` probe + delta
/// rev-list, and the full-scan fallback — run under ONE permit held for the
/// whole blocking scan. The deletion-only fast path (no new tips) spawns no git
/// and never parks.
///
/// `force_full_scan` forces the full-scan fallback regardless of the tips — the
/// coalesced-drain `PendingWork::FullScan` marker (#174 F5). It composes with
/// the env kill-switch (either forces), and it disqualifies the empty-tips fast
/// path: a forced-scan call with no tips must still enumerate the repo, never
/// silently resolve to an empty delta (which would pin nothing and lose the
/// drained pushes' work — the F5 bug shape).
pub async fn resolve_candidates_for_push(
scan_sem: std::sync::Arc<tokio::sync::Semaphore>,
repo_path: PathBuf,
new_tips: Vec<String>,
old_tips: Vec<String>,
git_bin: String,
timeout: Duration,
force_full_scan: bool,
) -> PinCandidateSet {
let force = force_full_scan || self::force_full_scan();
// No-git fast path: a deletion-only push resolves to an empty delta without
// spawning any git child, so it must not park on the scan pool. A forced
// full scan (caller flag or kill-switch) disqualifies it — the scan spawns git.
if new_tips.is_empty() && !force {
tracing::info!(delta = 0usize, repo = %repo_path.display(), "pin candidate set from push delta");
return PinCandidateSet {
candidates: Vec::new(),
full_scan: false,
};
}
// Scan admission (#174 F4): DEFER, never shed — a dropped scan would
// silently under-pin this push. The permit moves into the blocking closure
// so a started scan always completes holding it. Residuals at
// `acquire_scan_permit`.
let permit =
crate::state::acquire_scan_permit(scan_sem, &repo_path, "pin-candidate scan").await;
tokio::task::spawn_blocking(move || {
let _permit = permit;
// ONE shared deadline for the whole scan, per jatmn ("the same deadline").
let deadline = Instant::now() + timeout;
let new_refs: Vec<&str> = new_tips.iter().map(String::as_str).collect();
let old_refs: Vec<&str> = old_tips.iter().map(String::as_str).collect();
// `force` already folds in the env kill-switch, so the forced arm skips the
// delta machinery outright; the unforced arm goes through the normal
// resolver (whose own env read is false here by construction).
let resolved = if force {
tracing::debug!("full scan forced (coalesced-drain marker or kill-switch)");
PinCandidates::FullScanRequired
} else {
resolve_push_delta(&repo_path, &new_refs, &old_refs, &git_bin, deadline)
};
match resolved {
match resolve_push_delta(&repo_path, &new_refs, &old_refs) {
PinCandidates::Delta(objs) => {
tracing::info!(delta = objs.len(), repo = %repo_path.display(), "pin candidate set from push delta");
PinCandidateSet { candidates: objs, full_scan: false }
}
PinCandidates::FullScanRequired => {
tracing::warn!(repo = %repo_path.display(), "pin delta unavailable (non-commit tip, git error, or force-full-scan) — full-scan fallback");
match list_all_objects(&repo_path, &git_bin, deadline) {
match list_all_objects(&repo_path) {
Ok(objs) => PinCandidateSet { candidates: objs, full_scan: true },
Err(e) => {
tracing::warn!(repo = %repo_path.display(), err = %e, "full-scan fallback failed; pinning nothing this push (reconciliation sweep backstops)");
@@ -364,13 +304,8 @@ pub async fn resolve_candidates_for_push(
mod tests {
use super::*;
use std::collections::HashSet;
use std::process::Command;
use tempfile::TempDir;
fn td() -> std::time::Instant {
std::time::Instant::now() + std::time::Duration::from_secs(600)
}
/// Minimal git helper for building test repos.
struct Repo {
_td: TempDir,
@@ -428,10 +363,9 @@ mod tests {
let repo = Repo::new();
let c1 = repo.commit_file("a.txt", "one\n");
let c2 = repo.commit_file("b.txt", "two\n");
let got: HashSet<String> =
delta(resolve_push_delta(&repo.path, &[&c2], &[&c1], "git", td()))
.into_iter()
.collect();
let got: HashSet<String> = delta(resolve_push_delta(&repo.path, &[&c2], &[&c1]))
.into_iter()
.collect();
// The new blob b.txt and commit c2 are in the delta; the old blob a.txt
// and commit c1 are not.
let new_blob = repo.rev("HEAD:b.txt");
@@ -449,7 +383,7 @@ mod tests {
// genuinely new objects, never fewer.
let repo = Repo::new();
let c1 = repo.commit_file("a.txt", "one\n");
let got: HashSet<String> = delta(resolve_push_delta(&repo.path, &[&c1], &[], "git", td()))
let got: HashSet<String> = delta(resolve_push_delta(&repo.path, &[&c1], &[]))
.into_iter()
.collect();
assert!(got.contains(&c1));
@@ -465,15 +399,9 @@ mod tests {
// Rewrite history: reset to base, commit a different file.
repo.git(&["reset", "-q", "--hard", &base]);
let new_tip = repo.commit_file("c.txt", "three\n");
let got: HashSet<String> = delta(resolve_push_delta(
&repo.path,
&[&new_tip],
&[&old_tip],
"git",
td(),
))
.into_iter()
.collect();
let got: HashSet<String> = delta(resolve_push_delta(&repo.path, &[&new_tip], &[&old_tip]))
.into_iter()
.collect();
assert!(got.contains(&new_tip), "new tip in delta");
assert!(got.contains(&repo.rev(&format!("{new_tip}:c.txt"))));
// No error; force-push computes new-minus-old cleanly.
@@ -485,7 +413,7 @@ mod tests {
repo.commit_file("a.txt", "one\n");
// All updates were deletions => new_tips empty after the caller strips zeros.
assert_eq!(
resolve_push_delta(&repo.path, &[], &[ZERO], "git", td()),
resolve_push_delta(&repo.path, &[], &[ZERO]),
PinCandidates::Delta(Vec::new())
);
}
@@ -496,7 +424,7 @@ mod tests {
repo.commit_file("a.txt", "one\n");
let blob = repo.rev("HEAD:a.txt");
assert_eq!(
resolve_push_delta(&repo.path, &[&blob], &[], "git", td()),
resolve_push_delta(&repo.path, &[&blob], &[]),
PinCandidates::FullScanRequired,
"a blob tip must force full scan (rev-list would exit 0 and walk it)"
);
@@ -508,7 +436,7 @@ mod tests {
repo.commit_file("a.txt", "one\n");
let tree = repo.rev("HEAD^{tree}");
assert_eq!(
resolve_push_delta(&repo.path, &[&tree], &[], "git", td()),
resolve_push_delta(&repo.path, &[&tree], &[]),
PinCandidates::FullScanRequired
);
}
@@ -521,7 +449,7 @@ mod tests {
repo.git(&["tag", "-a", "treetag", "-m", "x", &tree]);
let tag = repo.rev("treetag");
assert_eq!(
resolve_push_delta(&repo.path, &[&tag], &[], "git", td()),
resolve_push_delta(&repo.path, &[&tag], &[]),
PinCandidates::FullScanRequired,
"annotated tag peeling to a tree must force full scan"
);
@@ -537,7 +465,7 @@ mod tests {
repo.git(&["tag", "-a", "t2", "-m", "x", &t1]);
let t2 = repo.rev("t2");
assert_eq!(
resolve_push_delta(&repo.path, &[&t2], &[], "git", td()),
resolve_push_delta(&repo.path, &[&t2], &[]),
PinCandidates::FullScanRequired
);
}
@@ -552,7 +480,7 @@ mod tests {
let c1 = repo.commit_file("a.txt", "one\n");
repo.git(&["tag", "-a", "rel", "-m", "release", &c1]);
let tag = repo.rev("rel");
let got: HashSet<String> = delta(resolve_push_delta(&repo.path, &[&tag], &[], "git", td()))
let got: HashSet<String> = delta(resolve_push_delta(&repo.path, &[&tag], &[]))
.into_iter()
.collect();
assert!(
@@ -573,7 +501,7 @@ mod tests {
let t1 = repo.rev("t1");
repo.git(&["tag", "-a", "t2", "-m", "x", &t1]);
let t2 = repo.rev("t2");
let got: HashSet<String> = delta(resolve_push_delta(&repo.path, &[&t2], &[], "git", td()))
let got: HashSet<String> = delta(resolve_push_delta(&repo.path, &[&t2], &[]))
.into_iter()
.collect();
assert!(
@@ -588,16 +516,9 @@ mod tests {
let repo = Repo::new();
let c1 = repo.commit_file("a.txt", "one\n");
let c2 = repo.commit_file("b.txt", "two\n");
let set = resolve_candidates_for_push(
std::sync::Arc::new(tokio::sync::Semaphore::new(64)),
repo.path.clone(),
vec![c2.clone()],
vec![c1.clone()],
"git".to_string(),
std::time::Duration::from_secs(600),
false,
)
.await;
let set =
resolve_candidates_for_push(repo.path.clone(), vec![c2.clone()], vec![c1.clone()])
.await;
assert!(!set.full_scan, "happy-path delta is not a full scan");
let got: HashSet<String> = set.candidates.into_iter().collect();
let new_blob = repo.rev("HEAD:b.txt");
@@ -615,206 +536,20 @@ mod tests {
let repo = Repo::new();
repo.commit_file("a.txt", "one\n");
let blob = repo.rev("HEAD:a.txt");
let all: HashSet<String> = list_all_objects(&repo.path, "git", td())
.unwrap()
.into_iter()
.collect();
let set = resolve_candidates_for_push(
std::sync::Arc::new(tokio::sync::Semaphore::new(64)),
repo.path.clone(),
vec![blob],
vec![],
"git".to_string(),
std::time::Duration::from_secs(600),
false,
)
.await;
let all: HashSet<String> = list_all_objects(&repo.path).unwrap().into_iter().collect();
let set = resolve_candidates_for_push(repo.path.clone(), vec![blob], vec![]).await;
assert!(set.full_scan, "non-commit tip is signalled as a full scan");
let got: HashSet<String> = set.candidates.into_iter().collect();
assert_eq!(got, all, "non-commit tip falls back to full repo scan");
}
/// F4 defer proof 2: `resolve_candidates_for_push`'s git stages (the per-tip
/// cat-file type probe + delta rev-list, and the full-scan fallback) run under a
/// scan-admission permit: with a zero-permit pool the call parks and spawns no
/// git; once a permit is available the SAME call runs (defer, not shed). On
/// ungated code the git runs regardless of the pool (RED).
#[cfg(unix)]
#[tokio::test]
async fn resolve_candidates_defers_when_scan_pool_exhausted() {
use std::os::unix::fs::PermissionsExt;
use std::sync::Arc;
use std::time::Duration;
use tokio::sync::Semaphore;
let dir = tempfile::TempDir::new().unwrap();
let marker = dir.path().join("git.ran");
// Fake git records ANY invocation; cat-file reports a commit tip so the
// delta stage proceeds, rev-list yields an empty delta.
let fake = dir.path().join("fakegit");
std::fs::write(
&fake,
format!(
"#!/bin/sh\necho ran >> \"{}\"\ncase \"$1\" in\n cat-file) echo commit ;;\n *) : ;;\nesac\nexit 0\n",
marker.display()
),
)
.unwrap();
let mut perm = std::fs::metadata(&fake).unwrap().permissions();
perm.set_mode(0o755);
std::fs::set_permissions(&fake, perm).unwrap();
let git_bin = fake.to_str().unwrap().to_string();
let tip = "deadbeefdeadbeefdeadbeefdeadbeefdeadbeef".to_string();
let sem: Arc<Semaphore> = Arc::new(Semaphore::new(0));
let blocked = tokio::time::timeout(
Duration::from_millis(500),
resolve_candidates_for_push(
sem.clone(),
dir.path().to_path_buf(),
vec![tip.clone()],
vec![],
git_bin.clone(),
Duration::from_secs(5),
false,
),
)
.await;
assert!(
blocked.is_err(),
"the pin-candidate scan must defer (park on admission) when the pool is exhausted"
);
assert!(
!marker.exists(),
"the scan's git must not spawn while its admission permit is unavailable (F4)"
);
// Release admission: the SAME scan now runs (defer, not shed).
sem.add_permits(1);
let set = resolve_candidates_for_push(
sem,
dir.path().to_path_buf(),
vec![tip],
vec![],
git_bin,
Duration::from_secs(5),
false,
)
.await;
assert!(
marker.exists(),
"once admission is available the deferred scan runs its git"
);
assert!(!set.full_scan, "commit tip + empty rev-list is a delta");
assert!(set.candidates.is_empty());
}
/// F4 fast-path negative arm: a deletion-only push (no new tips, kill-switch
/// off) computes its empty delta without spawning ANY git child, so it must
/// complete without acquiring from the scan pool — even at zero permits. The
/// per-tip cat-file probe means every push with a non-empty new tip DOES spawn
/// git, so this is the only genuinely git-free stage.
#[cfg(unix)]
#[tokio::test]
async fn resolve_candidates_no_git_fast_path_skips_admission() {
use std::os::unix::fs::PermissionsExt;
use std::sync::Arc;
use std::time::Duration;
use tokio::sync::Semaphore;
let dir = tempfile::TempDir::new().unwrap();
let marker = dir.path().join("git.ran");
let fake = dir.path().join("fakegit");
std::fs::write(
&fake,
format!("#!/bin/sh\necho ran >> \"{}\"\nexit 0\n", marker.display()),
)
.unwrap();
let mut perm = std::fs::metadata(&fake).unwrap().permissions();
perm.set_mode(0o755);
std::fs::set_permissions(&fake, perm).unwrap();
let set = tokio::time::timeout(
Duration::from_millis(500),
resolve_candidates_for_push(
Arc::new(Semaphore::new(0)),
dir.path().to_path_buf(),
vec![],
vec!["deadbeefdeadbeefdeadbeefdeadbeefdeadbeef".to_string()],
fake.to_str().unwrap().to_string(),
Duration::from_secs(5),
false,
),
)
.await
.expect("the no-new-tips fast path must not park on the scan pool");
assert_eq!(
set,
PinCandidateSet {
candidates: Vec::new(),
full_scan: false
}
);
assert!(!marker.exists(), "the fast path must spawn no git at all");
}
/// #174 F5: the coalesced-drain FullScan marker is signalled via the explicit
/// `force_full_scan` flag, and a flagged call with NO tips must still enumerate
/// the whole repo (full_scan=true, non-empty candidates). The RED arm of the
/// encoding question: if the marker were encoded as a plain empty-tips call
/// (flag off — the fast path), the drain would resolve to an empty delta and
/// pin nothing, silently losing the coalesced pushes' work.
#[tokio::test]
async fn forced_full_scan_with_no_tips_enumerates_the_repo() {
let repo = Repo::new();
repo.commit_file("a.txt", "one\n");
let all: HashSet<String> = list_all_objects(&repo.path, "git", td())
.unwrap()
.into_iter()
.collect();
assert!(!all.is_empty(), "fixture repo has objects");
let set = resolve_candidates_for_push(
std::sync::Arc::new(tokio::sync::Semaphore::new(64)),
repo.path.clone(),
vec![],
vec![],
"git".to_string(),
std::time::Duration::from_secs(600),
true,
)
.await;
assert!(set.full_scan, "the forced call is signalled as a full scan");
let got: HashSet<String> = set.candidates.into_iter().collect();
assert_eq!(
got, all,
"a forced full scan with no tips enumerates the repo — never an empty delta"
);
// The discriminator: the SAME empty-tips call without the flag is the
// deletion-only fast path (empty delta, pin nothing). The two must differ,
// or the marker encoding has collapsed into the silent-loss shape.
let unforced = resolve_candidates_for_push(
std::sync::Arc::new(tokio::sync::Semaphore::new(64)),
repo.path.clone(),
vec![],
vec![],
"git".to_string(),
std::time::Duration::from_secs(600),
false,
)
.await;
assert!(!unforced.full_scan);
assert!(unforced.candidates.is_empty());
}
#[test]
fn missing_oid_tip_forces_full_scan() {
let repo = Repo::new();
repo.commit_file("a.txt", "one\n");
let bogus = "deadbeefdeadbeefdeadbeefdeadbeefdeadbeef";
assert_eq!(
resolve_push_delta(&repo.path, &[bogus], &[], "git", td()),
resolve_push_delta(&repo.path, &[bogus], &[]),
PinCandidates::FullScanRequired,
"a missing/corrupt tip OID must force full scan"
);
@@ -829,7 +564,7 @@ mod tests {
let c1 = repo.commit_file("a.txt", "one\n");
let c2 = repo.commit_file("b.txt", "two\n");
let old_tree = repo.rev(&format!("{c1}^{{tree}}"));
let result = resolve_push_delta(&repo.path, &[&c2], &[&old_tree], "git", td());
let result = resolve_push_delta(&repo.path, &[&c2], &[&old_tree]);
match result {
PinCandidates::FullScanRequired => {} // safe: caller full-scans
PinCandidates::Delta(objs) => {
@@ -848,15 +583,10 @@ mod tests {
// branch2 from base advances independently
repo.git(&["checkout", "-q", "-b", "branch2", &base]);
let b2 = repo.commit_file("c.txt", "three\n");
let got: HashSet<String> = delta(resolve_push_delta(
&repo.path,
&[&b1, &b2],
&[&base, &base],
"git",
td(),
))
.into_iter()
.collect();
let got: HashSet<String> =
delta(resolve_push_delta(&repo.path, &[&b1, &b2], &[&base, &base]))
.into_iter()
.collect();
assert!(got.contains(&b1), "branch1 new commit");
assert!(got.contains(&b2), "branch2 new commit");
}
@@ -865,7 +595,7 @@ mod tests {
fn empty_repo_no_tips_yields_empty_delta() {
let repo = Repo::new();
assert_eq!(
resolve_push_delta(&repo.path, &[], &[], "git", td()),
resolve_push_delta(&repo.path, &[], &[]),
PinCandidates::Delta(Vec::new())
);
}
@@ -879,12 +609,12 @@ mod tests {
let repo = Repo::new();
let c1 = repo.commit_file("a.txt", "one\n");
assert_eq!(
resolve_push_delta_inner(&repo.path, &[&c1], &[], true, "git", td()),
resolve_push_delta_inner(&repo.path, &[&c1], &[], true),
PinCandidates::FullScanRequired
);
// And with the flag off, the same push yields a Delta.
assert!(matches!(
resolve_push_delta_inner(&repo.path, &[&c1], &[], false, "git", td()),
resolve_push_delta_inner(&repo.path, &[&c1], &[], false),
PinCandidates::Delta(_)
));
}
@@ -894,7 +624,7 @@ mod tests {
let repo = Repo::new();
repo.commit_file("a.txt", "one\n");
repo.commit_file("b.txt", "two\n");
let all = list_all_objects(&repo.path, "git", td()).unwrap();
let all = list_all_objects(&repo.path).unwrap();
// 2 commits + 2 trees + 2 blobs = 6 objects.
assert_eq!(all.len(), 6, "got: {all:?}");
}
@@ -912,7 +642,7 @@ mod tests {
std::fs::write(repo.path.join("orphan.bin"), b"dangling\n").unwrap();
let dangling = repo.git(&["hash-object", "-w", "orphan.bin"]);
let blobs = all_blob_oids(&repo.path, "git", td()).unwrap();
let blobs = all_blob_oids(&repo.path).unwrap();
assert!(blobs.contains(&reachable_blob), "reachable blob present");
assert!(
blobs.contains(&dangling),
@@ -922,7 +652,7 @@ mod tests {
assert!(!blobs.contains(&tree), "tree is not a blob");
// The typed lister tags each object; spot-check the dangling blob's type.
let typed = list_all_objects_with_type(&repo.path, "git", td()).unwrap();
let typed = list_all_objects_with_type(&repo.path).unwrap();
assert!(
typed
.iter()
@@ -930,96 +660,4 @@ mod tests {
"dangling object is typed as a blob"
);
}
#[cfg(unix)]
#[test]
fn list_all_objects_times_out_on_a_hung_git_instead_of_blocking() {
use std::os::unix::fs::PermissionsExt;
use std::time::{Duration, Instant};
let dir = tempfile::TempDir::new().unwrap();
// Fake git: hang on cat-file (bounded to 30s so a broken test can't wedge).
let fake = dir.path().join("fakegit");
std::fs::write(
&fake,
"#!/bin/sh\ncase \"$1\" in\n cat-file) i=0; while [ $i -lt 30 ]; do sleep 1; i=$((i+1)); done ;;\n *) : ;;\nesac\n",
)
.unwrap();
let mut perm = std::fs::metadata(&fake).unwrap().permissions();
perm.set_mode(0o755);
std::fs::set_permissions(&fake, perm).unwrap();
let git_bin = fake.to_str().unwrap().to_string();
let (tx, rx) = std::sync::mpsc::channel();
let path = dir.path().to_path_buf();
std::thread::spawn(move || {
let _ = tx.send(list_all_objects(
&path,
&git_bin,
Instant::now() + Duration::from_millis(150),
));
});
let res = rx.recv_timeout(Duration::from_secs(10)).expect(
"list_all_objects must return within the watchdog budget, not block on a hung git",
);
assert!(
res.is_err(),
"a hung git must make list_all_objects error out, not hang"
);
}
/// #174 finding 2: the DELTA-path children — `peeled_object_type` (cat-file),
/// `rev_list_delta` (rev-list) — and `list_all_objects_with_type` (cat-file) are
/// the remaining candidate-discovery execs (the common push path). Each must
/// return within the watchdog budget on a hung git rather than block and pin the
/// write permit. Revert any one to a bare `Command::output()` and its arm blocks
/// past the recv budget (RED).
#[cfg(unix)]
#[test]
fn delta_path_exec_fns_time_out_on_a_hung_git() {
use std::os::unix::fs::PermissionsExt;
use std::time::{Duration, Instant};
let dir = tempfile::TempDir::new().unwrap();
// Fake git hangs on BOTH cat-file and rev-list (bounded 30s so a broken
// test can't leak a permanent orphan or wedge the suite).
let fake = dir.path().join("fakegit");
std::fs::write(
&fake,
"#!/bin/sh\ncase \"$1\" in\n cat-file|rev-list) i=0; while [ $i -lt 30 ]; do sleep 1; i=$((i+1)); done ;;\n *) : ;;\nesac\n",
)
.unwrap();
let mut perm = std::fs::metadata(&fake).unwrap().permissions();
perm.set_mode(0o755);
std::fs::set_permissions(&fake, perm).unwrap();
let git_bin = fake.to_str().unwrap().to_string();
let path = dir.path().to_path_buf();
// Run `f` on a thread and require it to RETURN (not block) within 10s.
fn returns_within<T: Send + 'static>(f: impl FnOnce() -> T + Send + 'static) -> T {
let (tx, rx) = std::sync::mpsc::channel();
std::thread::spawn(move || {
let _ = tx.send(f());
});
rx.recv_timeout(Duration::from_secs(10)).expect(
"a delta-path exec fn must return within the watchdog budget, not block on a hung git",
)
}
let dl = || Instant::now() + Duration::from_millis(150);
let sha = "deadbeefdeadbeefdeadbeefdeadbeefdeadbeef";
let (p, g, d) = (path.clone(), git_bin.clone(), dl());
assert!(
returns_within(move || peeled_object_type(&p, sha, &g, d)).is_none(),
"peeled_object_type must time out to None on a hung cat-file"
);
let (p, g, d) = (path.clone(), git_bin.clone(), dl());
assert!(
returns_within(move || rev_list_delta(&p, &[sha], &[], &g, d)).is_err(),
"rev_list_delta must error out on a hung rev-list"
);
let (p, g, d) = (path.clone(), git_bin.clone(), dl());
assert!(
returns_within(move || list_all_objects_with_type(&p, &g, d)).is_err(),
"list_all_objects_with_type must error out on a hung cat-file"
);
}
}
File diff suppressed because it is too large. Load diff
@@ -0,0 +1,431 @@
//! Read-path download coordination (KTD-3) for [`RepoStore`].
//!
//! The read path's cache-miss handling lives here: the per-repo async mutex
//! that serializes concurrent readers of the same repo, the holder's fetch +
//! extract + under-lock publish, and the RAII temp-dir cleanup. `acquire` and
//! `acquire_fresh` stay in the parent module and call into `download_published`.
//! These methods reach the parent's private fields (`download_locks`) and
//! private methods (`try_lock_repo`) directly, because a child module can see
//! its parent's private items.
use super::*;
/// Outcome of a coordinated read-path download (KTD-3).
pub(super) enum DownloadOutcome {
/// The local dir is present: this reader downloaded and published it under
/// the advisory lock, or a concurrent reader did and this one served it.
Published,
/// The download was discarded without publishing: the advisory lock was
/// contended (a live writer or purge holds it), the lock pool errored, or
/// the archive vanished under the lock (purged mid-download). The caller
/// degrades to its missing-path or serve-local-copy outcome.
Skipped,
}
/// RAII cleanup for a read-path download temp dir (finding 1). Its `Drop`
/// removes the dir (best effort), so a handler future cancelled mid-download
/// (axum drops it on client disconnect, the same hazard `RepoWriteGuard` /
/// `RepoLockGuard` Drop impls exist for) cannot leak a fully-populated repo
/// dir. `disarm` forgets the dir once the publish rename has consumed it.
struct TempDownloadDir {
path: PathBuf,
armed: bool,
}
impl TempDownloadDir {
fn new(path: PathBuf) -> Self {
Self { path, armed: true }
}
fn path(&self) -> &Path {
&self.path
}
/// Disarm: the temp has been consumed (renamed into place), so `Drop` must
/// not try to remove it.
fn disarm(mut self) {
self.armed = false;
}
}
impl Drop for TempDownloadDir {
fn drop(&mut self) {
if self.armed {
let _ = std::fs::remove_dir_all(&self.path);
}
}
}
/// RAII removal of a download-coordination map entry (finding 4). Its `Drop`
/// removes the entry from `download_locks` whenever the map still holds THIS
/// Arc (ptr_eq-guarded so a newer entry inserted after an earlier removal is
/// never clobbered). Removal is guaranteed on every path (a normal return or
/// `?` drops it in the holder; once a download is in flight it travels with
/// the spawned task and drops at settle, covering holder timeout and a
/// handler future cancelled mid cold-read, which axum triggers on client
/// disconnect), which is what keeps the map from growing per arbitrary
/// requested name: the pre-fix explicit removals ran only at the labelled
/// return points, which a cancelled future skipped, leaking the entry. The
/// outer map is a std::sync::Mutex so
/// this Drop can prune it synchronously (Drop cannot be async); it is only ever
/// held briefly for a get/insert/remove, never across an await.
struct DownloadEntryGuard {
locks: Arc<std::sync::Mutex<HashMap<String, Arc<Mutex<()>>>>>,
key: String,
entry: Arc<Mutex<()>>,
}
impl Drop for DownloadEntryGuard {
fn drop(&mut self) {
// A poisoned map is still safe to prune — recover the guard and remove.
let mut map = self.locks.lock().unwrap_or_else(|p| p.into_inner());
if map
.get(&self.key)
.is_some_and(|cur| Arc::ptr_eq(cur, &self.entry))
{
map.remove(&self.key);
}
}
}
impl RepoStore {
/// Coordinated read-path download (KTD-3). Serializes concurrent readers
/// of the same repo on a per-repo async mutex: the first one in becomes
/// the holder and runs [`download_and_publish`](Self::download_and_publish);
/// a contended reader awaits the holder, re-checks the local dir on wake,
/// and serves what the holder published instead of re-downloading. Only
/// callers that already confirmed the archive exists reach this, and the
/// map entry is removed on completion, so the map cannot grow per
/// arbitrary requested name.
pub(super) async fn download_published(
&self,
owner_did: &str,
repo_name: &str,
owner_slug: &str,
local_path: &Path,
store: &Arc<dyn ObjectStore>,
local_existed_at_entry: bool,
) -> Result<DownloadOutcome> {
let map_key = format!("{owner_slug}/{repo_name}");
let entry = {
let mut map = self.download_locks.lock().unwrap();
Arc::clone(
map.entry(map_key.clone())
.or_insert_with(|| Arc::new(Mutex::new(()))),
)
};
let held = match Arc::clone(&entry).try_lock_owned() {
Ok(g) => g,
Err(_) => {
// Bounded coalescing wait: once a holder's download is in
// flight, the spawned task owns this lock (and the map entry)
// until it settles, whatever happens to the holder (timeout or
// cancellation), so an unbounded lock().await here would park
// this reader for as long as a stall lasts. On elapse, report
// the same timeout error the holder would. Do NOT touch the
// map entry on this path: pruning it while the download is in
// flight would let the next arrival start a duplicate download.
let waited = tokio::time::timeout(
self.release_upload_timeout,
Arc::clone(&entry).lock_owned(),
)
.await;
let g = match waited {
Ok(g) => g,
Err(_) => {
if local_path.exists() {
// A concurrent reader published while we waited.
return Ok(DownloadOutcome::Published);
}
return Err(anyhow::anyhow!("read download timed out"));
}
};
if local_path.exists() {
// A concurrent reader published while we waited — serve it.
// The holder's own guard prunes the map entry on its exit.
return Ok(DownloadOutcome::Published);
}
// The holder degraded without publishing; this reader takes over.
// The prior holder removed the map entry on its way out, so
// re-register THIS entry before downloading, so readers arriving
// now coalesce onto it instead of starting a duplicate download
// (finding 3). Narrows the duplicate-download window; publishes
// stay serialized by the advisory lock regardless.
{
let mut map = self.download_locks.lock().unwrap();
map.entry(map_key.clone())
.or_insert_with(|| Arc::clone(&entry));
}
g
}
};
// RAII: remove the map entry when this holder's work is done (finding
// 4). Created only once THIS reader holds the per-repo lock, so a
// parked or timed-out waiter never prunes the entry out from under a
// live holder; its Drop is ptr_eq-guarded, so it never clobbers a
// newer entry. Ownership moves into download_and_publish and from
// there into the spawned download task itself, so on EVERY settle path
// (success, download error, holder timeout, holder cancellation) the
// entry stays registered and the lock stays held until the in-flight
// download settles: waiters queue on the ONE download instead of each
// timeout or cancel wave starting a fresh full download of the same
// object.
let entry_guard = DownloadEntryGuard {
locks: Arc::clone(&self.download_locks),
key: map_key,
entry: Arc::clone(&entry),
};
self.download_and_publish(
owner_did,
repo_name,
owner_slug,
local_path,
store,
local_existed_at_entry,
entry_guard,
held,
)
.await
}
/// The holder's half of the coordinated download: fetch and extract the
/// archive into a temp sibling with NO advisory lock held (the network
/// phase must never pin a lock-pool connection — a cold-read burst across
/// distinct repos must not drain the writer-sized pool), then take the
/// per-repo advisory lock only around the publish: re-check the archive
/// still exists under the lock (a purge deletes it under this same lock,
/// so a post-purge downloader discards rather than resurrecting), swap
/// the temp copy into place, release. Advisory contention or a lock-pool
/// error discards the temp copy and degrades — no blocking lock or pool
/// wait exists anywhere on the read path.
///
/// `local_existed_at_entry` selects the stale-swap guard (finding 2): a cold
/// `acquire` passes `false`, an `acquire_fresh` refresh passes `true`. Under
/// the lock, before the swap, we detect a writer that published a fresher
/// copy during our unlocked download window and serve theirs instead of
/// clobbering it with our now-stale temp.
///
/// `entry_guard` and `held` are this holder's per-repo coordination (the
/// download_locks map entry and the locked per-repo mutex). Both move into
/// the spawned download task, which returns them through its JoinHandle on
/// success; on any other settle path (download error, task panic, holder
/// timeout, holder cancellation) they drop when the task settles, so
/// waiters keep queueing on the SAME in-flight download until it settles
/// rather than each starting a duplicate one.
#[allow(clippy::too_many_arguments)]
async fn download_and_publish(
&self,
owner_did: &str,
repo_name: &str,
owner_slug: &str,
local_path: &Path,
store: &Arc<dyn ObjectStore>,
local_existed_at_entry: bool,
entry_guard: DownloadEntryGuard,
held: tokio::sync::OwnedMutexGuard<()>,
) -> Result<DownloadOutcome> {
let parent = local_path.parent().context("repo path has no parent")?;
std::fs::create_dir_all(parent).context("creating repo parent dir")?;
let file_name = local_path
.file_name()
.context("repo path has no file name")?
.to_string_lossy();
// Best-effort sweep of leftover temp siblings from a prior download whose
// cleanup never ran (a process crash or kill mid-download, where no Drop
// and no owning task survive to remove the temp). Serialized per
// repo by the download mutex, so no concurrent same-repo download owns a
// live temp here; scoped to THIS repo's prefix, so other repos are
// untouched.
let tmp_prefix = format!(".{file_name}.tmp-download.");
if let Ok(entries) = std::fs::read_dir(parent) {
for entry in entries.flatten() {
if entry.file_name().to_string_lossy().starts_with(&tmp_prefix) {
let _ = std::fs::remove_dir_all(entry.path());
}
}
}
// Stale-swap signal (finding 2): capture the local dir's mtime BEFORE the
// unlocked download. Under the lock, a changed mtime (acquire_fresh,
// refreshing over an existing copy) or the dir's mere appearance (cold
// acquire) means a writer published a fresher copy during our download
// window, so we serve theirs rather than swap our stale temp over it.
let entry_mtime = std::fs::metadata(local_path)
.and_then(|m| m.modified())
.ok();
// Unique per-download temp target (same parent as local_path so the
// publish rename stays on one filesystem); mirrors the extract temp
// naming in tigris.rs. Wrapped in an RAII guard so a handler future
// cancelled mid-download (axum drops it on client disconnect) cannot
// leak the populated temp dir (the same hazard the guard Drop impls
// below exist for, finding 1).
let temp = TempDownloadDir::new(parent.join(format!(
".{file_name}.tmp-download.{}",
uuid::Uuid::new_v4()
)));
// Ownership of the download AND the temp-dir guard moves into a spawned
// task, so timeout and handler cancellation share ONE cleanup path
// (finding F3 and its cancellation twin). `store.download` runs its tar
// extraction in `spawn_blocking`, and neither a tokio timeout nor a
// dropped handler future cancels a spawn_blocking task: pre-fix,
// dropping the download future (a timed-out await, or axum dropping the
// whole handler on client disconnect) detached the extraction, whose
// later rename resurrected `temp.path()` AFTER the guard's Drop had
// removed it, leaving an orphan only a later same-repo download sweeps.
// The spawned task is never dropped mid-flight, so the download future
// always runs to settle. On success it returns the still-armed guard
// through the JoinHandle: the caller consumes it and publishes below.
// If the caller timed out or was cancelled, the JoinHandle is gone, so
// the runtime drops the returned guard when the task settles and its
// Drop removes the temp dir; cleanup is tied to the extraction's
// completion, never racing it (this subsumes the previous detached
// janitor). A download error drops the guard inside the task with the
// same at-settle timing.
let store_dl = Arc::clone(store);
let (dl_owner, dl_repo, dl_target) = (
owner_slug.to_string(),
repo_name.to_string(),
temp.path().to_path_buf(),
);
// The per-repo coordination (`held`, `entry_guard`) moves into the task
// alongside the temp guard: whatever happens to THIS caller (timeout
// below, or a handler future cancelled by axum on client disconnect),
// the lock stays held and the map entry stays registered until the
// in-flight download settles, so waiters and new arrivals queue on the
// ONE download instead of starting duplicates. On success the tuple
// returns through the JoinHandle: the caller publishes under `held` as
// before. If the caller timed out or was cancelled, the runtime drops
// the returned tuple when the task settles, in field order: `temp`
// (dir removed), then `held` (lock released), then `entry_guard` (map
// entry pruned), the same unlock-before-prune order as a normal
// return.
let dl_task = tokio::spawn(async move {
// The guards travel as one `(held, entry_guard)` tuple so that
// wherever it drops (here on error, in the runtime at settle after
// a caller timeout or cancellation, or in the caller after a
// normal publish), its field order gives unlock-before-prune.
match store_dl.download(&dl_owner, &dl_repo, &dl_target).await {
Ok(()) => Ok((temp, (held, entry_guard))),
// The download settled, so cleanup is safe. Explicit drops pin
// the order (async-block captures have no guaranteed one):
// temp removed, lock released, map entry pruned last.
Err(e) => {
drop(temp);
drop(held);
drop(entry_guard);
Err(e)
}
}
});
// Bound the wait, not the work: a stalled GET would park this reader
// indefinitely. On timeout the caller gets the same error as before,
// but the coordination does NOT unwind: it lives in the task (see
// above) and is released only at settle. Waiters therefore stay queued
// on the SAME in-flight download (each bounded by its own timeout in
// download_published) instead of each timeout wave starting a fresh
// full download of the same object; the task keeps running and cleans
// up its temp when the download settles.
// `_coordination` is `(held, entry_guard)`, kept alive to the end of
// this function exactly as the separate guards were before: the
// publish below still runs with the per-repo lock held, and the map
// entry is pruned only on exit (unlock first, prune second, per the
// tuple's field order).
let (temp, _coordination) =
match tokio::time::timeout(self.release_upload_timeout, dl_task).await {
Ok(Ok(Ok(returned))) => returned,
Ok(Ok(Err(e))) => return Err(e),
// The task panicked; its unwind dropped the guards, removing
// the dir, releasing the lock, and pruning the map entry.
Ok(Err(join_err)) => {
return Err(anyhow::Error::from(join_err).context("read download task failed"))
}
Err(_) => {
warn!(repo = %repo_name, timeout_secs = self.release_upload_timeout.as_secs(),
"read download timed out — discarding");
// Dropping the JoinHandle detaches the task; the runtime
// frees temp dir, lock, and map entry at settle (see the
// spawn comment above).
return Err(anyhow::anyhow!("read download timed out"));
}
};
let guard = match self.try_lock_repo(owner_did, repo_name).await {
Ok(Some(g)) => g,
Ok(None) => {
debug!(repo = %repo_name,
"read download discarded — repo lock held by a live writer or purge");
return Ok(DownloadOutcome::Skipped);
}
Err(e) => {
warn!(repo = %repo_name, err = %e,
"read download discarded — could not acquire repo lock");
return Ok(DownloadOutcome::Skipped);
}
};
// Re-check under the lock: the archive gone here means a purge won the
// key mid-download — discard, never publish (the read-side twin of the
// upload paths' under-lock dir re-check). Bounded so a stalled HEAD
// cannot pin the advisory lock + its lock-pool connection. Both a
// timeout and an exists() error collapse to "not present" and take the
// discard arm: fail closed rather than publish unconfirmed state.
let archive_present = matches!(
tokio::time::timeout(
self.release_upload_timeout,
store.exists(owner_slug, repo_name),
)
.await,
Ok(Ok(true))
);
if !archive_present {
warn!(repo = %repo_name,
"read download discarded — archive gone under lock (purged?)");
guard.release().await;
return Ok(DownloadOutcome::Skipped);
}
// Stale-swap guard (finding 2): a writer that locked, published a fresher
// copy, and released during our unlocked download window must not have
// its work clobbered by our now-stale temp. The only concurrent mutator
// of local_path is a writer or a purge (readers of the same repo
// serialize on the download mutex), so the signals below unambiguously
// mean "a writer published here."
let writer_published = match (local_existed_at_entry, entry_mtime) {
// acquire_fresh over an existing copy: presence is always true, so it
// cannot signal a writer; a CHANGED mtime can (the writer replaced
// the dir via rename, minting a fresh mtime).
(true, Some(entry_m)) => std::fs::metadata(local_path)
.and_then(|m| m.modified())
.map(|now| now != entry_m)
.unwrap_or(false),
// Cold acquire, or acquire_fresh whose dir was absent at entry: the
// dir being present now means a writer published it during the window.
_ => local_path.exists(),
};
if writer_published {
warn!(repo = %repo_name,
"read download discarded: a writer published a fresher copy during the download window");
guard.release().await;
// temp guard's Drop removes the stale temp; serve the fresher local copy.
return Ok(DownloadOutcome::Published);
}
let swapped = (|| -> std::io::Result<()> {
if local_path.exists() {
std::fs::remove_dir_all(local_path)?;
}
std::fs::rename(temp.path(), local_path)
})();
guard.release().await;
match swapped {
Ok(()) => {
temp.disarm(); // the rename consumed the temp; nothing to clean up
Ok(DownloadOutcome::Published)
}
// temp guard's Drop removes the temp if the rename left it behind.
Err(e) => Err(e).context("publishing downloaded repo into place"),
}
}
}
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
+48 -471
View File
@@ -5,39 +5,25 @@
use std::collections::HashMap;
use std::path::{Path, PathBuf};
use std::sync::atomic::AtomicBool;
use std::sync::{Arc, Mutex, OnceLock};
use anyhow::{Context, Result};
use aws_sdk_s3::Client as S3Client;
use tracing::{debug, info};
/// The precondition an upload is fenced on.
///
/// Object storage is the only place a fence can hold. Dropping the future of an
/// in-flight PUT does not cancel the request the server is already processing,
/// so no amount of local locking stops an abandoned writer's bytes from landing
/// after a successor has published. A conditional PUT the store itself refuses
/// is what actually stops it.
#[derive(Clone, Debug)]
pub enum UploadPrecondition {
/// Publish only if the stored object is still the generation we observed.
IfMatch(String),
/// Publish only if nothing is stored under the key yet.
IfAbsent,
/// No fence. Last writer wins.
Unconditional,
}
/// Why an upload failed, split so a caller can tell "someone else already
/// published this key" (expected, and dropping our bytes is the correct
/// outcome) from a real storage failure.
#[derive(Debug, thiserror::Error)]
pub enum UploadError {
#[error("upload precondition lost (HTTP {status})")]
PreconditionLost { status: u16 },
#[error(transparent)]
Other(#[from] anyhow::Error),
/// Object storage for git bare-repo archives. The seam that lets `RepoStore`
/// (and tests) depend on storage behavior without a live bucket. `TigrisClient`
/// is the production implementation.
#[async_trait::async_trait]
pub trait ObjectStore: Send + Sync {
/// Whether an archive exists for this repo.
async fn exists(&self, owner_slug: &str, repo_name: &str) -> Result<bool>;
/// Upload a local bare repo directory as an archive.
async fn upload(&self, owner_slug: &str, repo_name: &str, local_path: &Path) -> Result<()>;
/// Download and extract a repo archive to local disk.
async fn download(&self, owner_slug: &str, repo_name: &str, local_path: &Path) -> Result<()>;
/// Delete a repo archive.
async fn delete(&self, owner_slug: &str, repo_name: &str) -> Result<()>;
}
/// Wrapper around the S3 client with the configured bucket.
@@ -60,40 +46,16 @@ impl TigrisClient {
})
}
/// Build a client pointed at an arbitrary endpoint, for tests.
///
/// The production constructor reads the endpoint and credentials from the
/// environment, which a test cannot steer without mutating process-global
/// state. This takes both explicitly so a test can aim the client at a
/// closed port and get a prompt transport error out of `exists()`.
///
/// `RetryConfig::disabled()` is load-bearing, not tidiness: the SDK's default
/// policy retries a connection refusal with backoff, which turns each failing
/// call into seconds of waiting.
#[cfg(test)]
pub fn for_testing_with_endpoint(bucket: &str, endpoint: &str) -> Self {
use aws_sdk_s3::config::{retry::RetryConfig, Credentials, Region};
let config = aws_sdk_s3::config::Config::builder()
.endpoint_url(endpoint)
.credentials_provider(Credentials::new("test", "test", None, None, "test"))
.region(Region::new("auto"))
.retry_config(RetryConfig::disabled())
.behavior_version_latest()
.build();
Self {
s3: S3Client::from_conf(config),
bucket: bucket.to_string(),
}
}
/// S3 key for a given repo: `repos/v1/{owner_slug}/{repo_name}.tar.zst`
fn repo_key(owner_slug: &str, repo_name: &str) -> String {
format!("repos/v1/{owner_slug}/{repo_name}.tar.zst")
}
}
#[async_trait::async_trait]
impl ObjectStore for TigrisClient {
/// Check if a repo archive exists in Tigris.
pub async fn exists(&self, owner_slug: &str, repo_name: &str) -> Result<bool> {
async fn exists(&self, owner_slug: &str, repo_name: &str) -> Result<bool> {
let key = Self::repo_key(owner_slug, repo_name);
match self
.s3
@@ -114,47 +76,8 @@ impl TigrisClient {
}
}
/// Read the ETag of a repo archive, or `None` when nothing is stored under
/// the key. The ETag identifies the generation a later conditional upload
/// can fence itself on.
///
/// Separate from `exists` rather than folded into it: `exists` has callers
/// that only want the boolean, and widening its return type would churn
/// every one of them for no benefit.
pub async fn head_etag(&self, owner_slug: &str, repo_name: &str) -> Result<Option<String>> {
let key = Self::repo_key(owner_slug, repo_name);
match self
.s3
.head_object()
.bucket(&self.bucket)
.key(&key)
.send()
.await
{
Ok(out) => Ok(Some(
out.e_tag()
.context(format!("tigris HEAD {key}: hit carried no ETag"))?
.to_string(),
)),
Err(e) => {
if e.as_service_error().is_some_and(|e| e.is_not_found()) {
Ok(None)
} else {
Err(anyhow::anyhow!("tigris HEAD {key}: {e}"))
}
}
}
}
/// Upload a local bare repo directory to Tigris as a tar.zst archive,
/// fenced by `precondition`.
pub async fn upload(
&self,
owner_slug: &str,
repo_name: &str,
local_path: &Path,
precondition: UploadPrecondition,
) -> std::result::Result<(), UploadError> {
/// Upload a local bare repo directory to Tigris as a tar.zst archive.
async fn upload(&self, owner_slug: &str, repo_name: &str, local_path: &Path) -> Result<()> {
let key = Self::repo_key(owner_slug, repo_name);
debug!(key = %key, path = %local_path.display(), "uploading repo to tigris");
@@ -169,93 +92,24 @@ impl TigrisClient {
let body = aws_sdk_s3::primitives::ByteStream::from(archive_bytes);
let mut req = self
.s3
self.s3
.put_object()
.bucket(&self.bucket)
.key(&key)
.body(body)
.content_type("application/zstd");
match &precondition {
UploadPrecondition::IfMatch(etag) => req = req.if_match(etag),
UploadPrecondition::IfAbsent => req = req.if_none_match("*"),
UploadPrecondition::Unconditional => {}
}
if let Err(e) = req.send().await {
// `PutObjectError` models no PreconditionFailed variant (its arms are
// EncryptionTypeMismatch, InvalidRequest, InvalidWriteOffset,
// TooManyParts, Unhandled), so a refused precondition arrives as
// `Unhandled` and matching the enum would classify it as a generic
// failure. The raw HTTP status off the response is the only place the
// answer actually lives.
//
// Read it via `raw_response()`, not a `ServiceError`-only match: the
// SDK exposes the raw response for BOTH `ServiceError` and
// `ResponseError`, and a refused conditional PUT whose error body the
// SDK cannot parse (malformed XML, premature close) surfaces as
// `ResponseError`. Matching only `ServiceError` would classify that
// unparsable 409/412 as a generic failure, and `RepoWriteGuard::release`
// would log-and-succeed instead of taking the supersede retry,
// acknowledging a write that was definitively not published.
let status = e.raw_response().map(|raw| raw.status().as_u16());
// 412 is always a lost precondition. 409 is one only when we asked
// for create-only, which is how S3-compatible stores report "the key
// already exists". Everything else, 404 included, is a real failure:
// archive keys are never deleted (`delete` has no callers), so a 404
// here means something permanent like a missing bucket or a
// misrouted endpoint, and reporting that as a lost precondition
// would tell a caller to expect a successor that does not exist.
let lost = match status {
Some(412) => true,
Some(409) => matches!(precondition, UploadPrecondition::IfAbsent),
_ => false,
};
if lost {
return Err(UploadError::PreconditionLost {
status: status.expect("a lost precondition came from a status"),
});
}
return Err(UploadError::Other(
anyhow::Error::new(e).context(format!("tigris PUT {key}")),
));
}
.content_type("application/zstd")
.send()
.await
.context(format!("tigris PUT {key}"))?;
info!(key = %key, "uploaded repo to tigris");
Ok(())
}
/// Download a repo archive from Tigris and extract to local disk.
pub async fn download(
&self,
owner_slug: &str,
repo_name: &str,
local_path: &super::repo_store::ValidatedRepoDiskPath,
swap_authority: Option<Arc<AtomicBool>>,
) -> Result<()> {
self.download_to(owner_slug, repo_name, local_path, true, swap_authority)
.await
.map(|_| ())
}
/// Download a repo archive from Tigris and extract it, returning the
/// directory that was populated.
///
/// `publish` controls whether the extract is swapped into `target` in place
/// (the live-path mutation used by writes; returns `target`) or unpacked
/// into a fresh temp directory under `target`'s parent (a non-mutating
/// snapshot read; returns the temp dir, which the caller owns and cleans
/// up). The snapshot form never touches the live repo path.
pub async fn download_to(
&self,
owner_slug: &str,
repo_name: &str,
target: &super::repo_store::ValidatedRepoDiskPath,
publish: bool,
swap_authority: Option<Arc<AtomicBool>>,
) -> Result<DownloadExtract> {
async fn download(&self, owner_slug: &str, repo_name: &str, local_path: &Path) -> Result<()> {
let key = Self::repo_key(owner_slug, repo_name);
debug!(key = %key, path = %target.as_path().display(), "downloading repo from tigris");
debug!(key = %key, path = %local_path.display(), "downloading repo from tigris");
let resp = self
.s3
@@ -273,63 +127,21 @@ impl TigrisClient {
.context("reading tigris response body")?
.into_bytes();
// The snapshot temp dir is decided HERE, in the async layer, before the
// extraction runs. Cleanup ownership moves into the blocking task so a
// cancelled async future cannot drop it while extraction is still running.
let snapshot_tmp = if publish {
None
} else {
let parent = target.parent().context("snapshot path has no parent")?;
std::fs::create_dir_all(parent).context("creating parent dir")?;
let file_name = target
.file_name()
.context("snapshot path has no file name")?
.to_string_lossy();
Some(parent.join(format!(
".{file_name}.tmp-snapshot.{}",
uuid::Uuid::new_v4()
)))
};
// Extract tar.zst to a directory.
let extracted = tokio::task::spawn_blocking({
let target = target.clone();
let snapshot_tmp = snapshot_tmp.clone();
move || -> Result<DownloadExtract> {
let result = (|| -> Result<DownloadExtract> {
if publish {
decompress_repo(&data, &target, swap_authority.as_ref())?;
return Ok(DownloadExtract::Published(()));
}
// Non-mutating snapshot: unpack into the temp dir decided above.
// The live repo path is never touched.
let tmp_dir = snapshot_tmp.expect("snapshot path was decided above");
std::fs::create_dir_all(&tmp_dir).context("creating temp extract dir")?;
let unpack = (|| -> Result<()> {
let decoder = zstd::stream::Decoder::new(&data[..])?;
let mut archive = tar::Archive::new(decoder);
archive.unpack(&tmp_dir).context("unpacking tar.zst")?;
Ok(())
})();
if let Err(e) = unpack {
let _ = std::fs::remove_dir_all(&tmp_dir);
return Err(e);
}
Ok(DownloadExtract::Snapshot(TempSnapshotDir { path: tmp_dir }))
})();
result
}
// Extract tar.zst to local path
tokio::task::spawn_blocking({
let local_path = local_path.to_path_buf();
move || decompress_repo(&data, &local_path)
})
.await
.context("extract task panicked")?
.context("extracting repo")?;
info!(key = %key, path = %target.as_path().display(), "downloaded repo from tigris");
Ok(extracted)
info!(key = %key, path = %local_path.display(), "downloaded repo from tigris");
Ok(())
}
/// Delete a repo archive from Tigris.
pub async fn delete(&self, owner_slug: &str, repo_name: &str) -> Result<()> {
async fn delete(&self, owner_slug: &str, repo_name: &str) -> Result<()> {
let key = Self::repo_key(owner_slug, repo_name);
self.s3
.delete_object()
@@ -361,7 +173,7 @@ fn compress_repo(repo_path: &Path) -> Result<Vec<u8>> {
/// `decompress_repo`. Concurrent extractions unpack into isolated temp dirs in
/// parallel, but the final `remove_dir_all` + `rename` must not interleave for
/// the same `local_path`, or they race to a nondeterministic overwrite/failure.
pub(crate) fn publish_lock(local_path: &Path) -> Arc<Mutex<()>> {
fn publish_lock(local_path: &Path) -> Arc<Mutex<()>> {
// KNOWN LIMITATION: this map is never evicted — one (PathBuf, Arc<Mutex>)
// entry accrues per distinct repo path for the process lifetime. Bounded by
// the number of repos a node hosts, so it's negligible for normal use, but
@@ -374,35 +186,6 @@ pub(crate) fn publish_lock(local_path: &Path) -> Arc<Mutex<()>> {
.clone()
}
/// Owns a snapshot temp dir from extraction through handoff to [`RepoSnapshot`].
/// Dropped when the `spawn_blocking` join result is abandoned, so cancellation
/// before the outer future resumes still removes the directory.
pub(crate) struct TempSnapshotDir {
path: PathBuf,
}
impl TempSnapshotDir {
pub(crate) fn into_repo_snapshot(self) -> super::repo_store::RepoSnapshot {
let path = self.path.clone();
std::mem::forget(self);
super::repo_store::RepoSnapshot::from_owned_path(path)
}
}
impl Drop for TempSnapshotDir {
fn drop(&mut self) {
let _ = std::fs::remove_dir_all(&self.path);
}
}
/// What [`TigrisClient::download_to`] produced.
pub enum DownloadExtract {
/// Published into the validated live repo path.
Published(()),
/// Unpacked into a throwaway temp dir that cleans up on drop until adopted.
Snapshot(TempSnapshotDir),
}
/// Decompress a tar.zst byte vector into a local directory.
///
/// Extraction is atomic with respect to `local_path`: the archive is unpacked
@@ -410,16 +193,11 @@ pub enum DownloadExtract {
/// fully succeeds. A corrupt or truncated archive therefore can never clobber a
/// good existing copy at `local_path` — on failure we discard the temp dir and
/// leave `local_path` exactly as it was.
fn decompress_repo(
data: &[u8],
local_path: &super::repo_store::ValidatedRepoDiskPath,
swap_authority: Option<&Arc<AtomicBool>>,
) -> Result<()> {
let live = local_path.as_path();
let parent = live.parent().context("repo path has no parent")?;
fn decompress_repo(data: &[u8], local_path: &Path) -> Result<()> {
let parent = local_path.parent().context("repo path has no parent")?;
std::fs::create_dir_all(parent).context("creating parent dir")?;
let file_name = live
let file_name = local_path
.file_name()
.context("repo path has no file name")?
.to_string_lossy();
@@ -444,218 +222,17 @@ fn decompress_repo(
return Err(e);
}
// Swap through the validated-path helper so CodeQL sees the barrier before the
// remove/rename sink (`rust/path-injection`).
super::repo_store::swap_extracted_into_validated_repo(local_path, &tmp_dir, swap_authority)?;
// Swap the freshly-extracted repo into place. rename within the same parent
// is effectively atomic, but most platforms refuse to rename onto a
// non-empty dir, so remove the old copy first. Serialize this per repo path:
// concurrent extractions unpack into isolated temp dirs, but their swaps
// must not interleave or they race to a nondeterministic overwrite/failure.
let lock = publish_lock(local_path);
let _publish = lock.lock().expect("publish lock poisoned");
if local_path.exists() {
std::fs::remove_dir_all(local_path).context("removing stale repo dir")?;
}
std::fs::rename(&tmp_dir, local_path).context("swapping extracted repo into place")?;
Ok(())
}
#[cfg(test)]
mod tests {
use super::*;
use aws_sdk_s3::primitives::ByteStream;
use futures::FutureExt;
/// The envs the probe needs, all of them, or it does not run.
///
/// `AWS_ENDPOINT_URL_S3` is included on purpose: without it the SDK resolves
/// to real AWS S3, and a probe that passed there would say nothing about
/// Tigris.
fn probe_env() -> Option<String> {
if std::env::var("GITLAWB_TIGRIS_PROBE").ok().as_deref() != Some("1") {
return None;
}
for name in [
"AWS_ACCESS_KEY_ID",
"AWS_SECRET_ACCESS_KEY",
"AWS_ENDPOINT_URL_S3",
] {
if std::env::var(name).is_err() {
eprintln!("tigris conditional-write probe: {name} is unset, skipping");
return None;
}
}
match std::env::var("GITLAWB_TIGRIS_BUCKET") {
Ok(b) if !b.is_empty() => Some(b),
_ => {
eprintln!(
"tigris conditional-write probe: GITLAWB_TIGRIS_BUCKET is unset, skipping"
);
None
}
}
}
/// One conditional PUT, reported as the status that REFUSED it, or `None`
/// when the store accepted the write.
///
/// Accepted is the interesting answer here, not an error: it means the
/// endpoint ignored the header we fenced on.
async fn conditional_put(
s3: &S3Client,
bucket: &str,
key: &str,
body: &'static [u8],
if_match: Option<&str>,
if_none_match: Option<&str>,
) -> Result<Option<u16>, String> {
let mut req = s3
.put_object()
.bucket(bucket)
.key(key)
.body(ByteStream::from_static(body));
if let Some(v) = if_match {
req = req.if_match(v);
}
if let Some(v) = if_none_match {
req = req.if_none_match(v);
}
match req.send().await {
Ok(_) => Ok(None),
Err(e) => match e.raw_response() {
// Same raw-response rule as `upload`: a refused conditional PUT
// whose body the SDK cannot parse surfaces as `ResponseError`, and
// the status has to come off the raw response for both variants.
// Without it, an unparsable 409/412 would report "no HTTP
// response" here and skip the supersede retry.
Some(raw) => Ok(Some(raw.status().as_u16())),
None => Err(format!("conditional PUT {key}: no HTTP response: {e}")),
},
}
}
/// The probe body, written to RETURN its failures rather than panic on
/// them, so the caller's cleanup is reached on every arm.
async fn conditional_write_probe(s3: &S3Client, bucket: &str, key: &str) -> Result<(), String> {
// 1. A plain PUT under a throwaway key.
let seeded = s3
.put_object()
.bucket(bucket)
.key(key)
.body(ByteStream::from_static(b"probe-one"))
.send()
.await
.map_err(|e| format!("seeding PUT {key}: {e}"))?;
// 2. Its ETag, which is the generation the next arm fences against.
let etag = seeded
.e_tag()
.ok_or_else(|| format!("seeding PUT {key} returned no ETag"))?
.to_string();
// 3. A deliberately wrong If-Match. A store honoring it answers 412.
let wrong = format!("\"{}\"", "0".repeat(32));
if etag.trim_matches('"') == wrong.trim_matches('"') {
return Err(format!(
"the seeded ETag {etag} collides with the deliberately wrong one, \
so this arm would prove nothing"
));
}
match conditional_put(s3, bucket, key, b"probe-two", Some(&wrong), None).await? {
Some(412) => {}
Some(status) => {
return Err(format!(
"a stale If-Match must be refused with 412, the endpoint answered {status}"
))
}
None => {
return Err(
"a stale If-Match was ACCEPTED: this endpoint does not honor If-Match, so \
the release fence cannot hold here"
.to_string(),
)
}
}
// 4. If-None-Match `*` over the object that now exists. This arm matters
// MORE than the one above. An ignored If-Match eventually surfaces as
// odd behavior, because a stale writer overwrites and someone notices
// the lost tree. An ignored If-None-Match just returns 200, so a publish
// that should have been fenced lands with no error anywhere: the silent
// no-op the bucket-type caveat on this test describes.
match conditional_put(s3, bucket, key, b"probe-three", None, Some("*")).await? {
// Either status is a pass, and the asymmetry with the If-Match arm
// above mirrors `upload`'s classifier exactly: 412 is always a lost
// precondition, and 409 is one too when we asked for create-only.
// AWS documents 409 for a create-only conflict racing a delete, so a
// store answering it is enforcing the precondition and we already
// handle it. Pinning 412 alone here would fail the probe against a
// backend that is behaving correctly, which sends whoever runs it
// chasing a fault that is not there.
Some(412) | Some(409) => {}
Some(status) => {
return Err(format!(
"create-only over an existing object must be refused with 412 or 409, \
the endpoint answered {status}"
))
}
None => {
return Err(
"If-None-Match * was ACCEPTED over an existing object: this endpoint does \
not honor create-only, so a fenced publish lands silently"
.to_string(),
)
}
}
Ok(())
}
/// Probe the REAL Tigris endpoint for the conditional-write semantics the
/// release fence depends on.
///
/// UNTIL THIS IS RUN AGAINST REAL CREDENTIALS, the fence is verified against
/// vendor documentation and an in-process mock, not against the backend it
/// runs on. The mock implements the semantics we believe Tigris has; it
/// cannot tell us whether Tigris actually has them.
///
/// The bucket matters, not just the endpoint. Tigris documents conditional
/// operations as supported on Single-region and Multi-region buckets only.
/// Global and Dual-region buckets are eventually consistent, and a
/// conditional PUT evaluated against a stale replica would make the fence a
/// silent no-op rather than an error. So point `GITLAWB_TIGRIS_BUCKET` at a
/// throwaway bucket of the SAME type production uses.
///
/// Ignored by default and additionally gated on `GITLAWB_TIGRIS_PROBE=1`,
/// because it writes to a real bucket and costs real requests. Run with:
/// `GITLAWB_TIGRIS_PROBE=1 cargo test -p gitlawb-node --bin gitlawb-node
/// tigris_honors_conditional_writes -- --ignored --nocapture`
#[tokio::test]
#[ignore = "writes to a real Tigris bucket; needs GITLAWB_TIGRIS_PROBE=1 plus credentials"]
async fn tigris_honors_conditional_writes() {
let Some(bucket) = probe_env() else {
eprintln!(
"tigris conditional-write probe: skipped. Set GITLAWB_TIGRIS_PROBE=1, \
GITLAWB_TIGRIS_BUCKET, AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY and \
AWS_ENDPOINT_URL_S3 to run it."
);
return;
};
let config = aws_config::load_defaults(aws_config::BehaviorVersion::latest()).await;
let s3 = S3Client::new(&config);
// A fresh key per run, so a probe that somehow orphaned an object on an
// earlier run cannot change what this one observes.
let key = format!("probe/conditional-write-{}.bin", uuid::Uuid::new_v4());
// CLEANUP MUST RUN ON EVERY ARM, and a failing assertion is precisely
// the case this probe exists to catch, so the delete cannot sit after
// the checks. The body returns its failures rather than panicking, and
// `catch_unwind` covers the panic an SDK call could still raise; either
// way the delete below is reached before the verdict is re-raised.
let outcome = std::panic::AssertUnwindSafe(conditional_write_probe(&s3, &bucket, &key))
.catch_unwind()
.await;
if let Err(e) = s3.delete_object().bucket(&bucket).key(&key).send().await {
eprintln!("tigris conditional-write probe: cleanup of {key} failed: {e}");
}
match outcome {
Ok(Ok(())) => {}
Ok(Err(msg)) => panic!("tigris conditional-write probe: {msg}"),
Err(payload) => std::panic::resume_unwind(payload),
}
}
}
File diff suppressed because it is too large. Load diff
-164
View File
@@ -14,68 +14,6 @@ use subscription::SubscriptionRoot;
pub type GitlawbSchema = Schema<QueryRoot, MutationRoot, SubscriptionRoot>;
/// Client-facing message for GraphQL resolver failures that wrap a real
/// `sqlx::Error`. The real error is logged server-side; never put sqlx/Postgres
/// detail in the GraphQL `errors` array (#250).
///
/// Kept as its own constant on this PR's base (main still renders
/// `AppError::Db` with `e.to_string()`). If/when #247's `DB_ERROR_MESSAGE`
/// lands, fold this into that shared constant.
pub const GRAPHQL_DB_ERROR_MESSAGE: &str = "a database error occurred";
fn anyhow_has_sqlx(e: &anyhow::Error) -> bool {
e.chain().any(|c| c.downcast_ref::<sqlx::Error>().is_some())
}
/// Map an `anyhow` failure from the db layer to a GraphQL error.
///
/// - Real DB faults (`sqlx::Error` anywhere in the chain) → opaque client
/// message + `error!` log with the full `{e:#}` cause chain.
/// - Application/business errors (e.g. claim race, not-in-claimed-state) →
/// keep the actionable message; log at `warn!` so they are not mistaken for
/// infrastructure failures (#250 review).
pub(crate) fn graphql_db_err(e: anyhow::Error) -> async_graphql::Error {
if anyhow_has_sqlx(&e) {
tracing::error!(error = %format!("{e:#}"), "graphql database error");
async_graphql::Error::new(GRAPHQL_DB_ERROR_MESSAGE)
} else {
tracing::warn!(error = %format!("{e:#}"), "graphql application error");
async_graphql::Error::new(e.to_string())
}
}
/// Map an `AppError` from a shared collector (e.g. ref-update feed) to a
/// GraphQL error.
///
/// Fail closed: only explicitly curated variants surface their `Display`
/// text. Unnamed variants (including `Git`, which may embed on-disk paths)
/// render opaque so a future addition cannot leak by default (#255 review).
pub(crate) fn graphql_app_err(e: crate::error::AppError) -> async_graphql::Error {
match e {
crate::error::AppError::Db(sql) => graphql_db_err(sql.into()),
crate::error::AppError::Internal(err) => {
tracing::error!(error = %format!("{err:#}"), "graphql internal error");
async_graphql::Error::new(GRAPHQL_DB_ERROR_MESSAGE)
}
// Curated client-safe variants — `Display` is intentional API text.
safe @ (crate::error::AppError::RepoNotFound(_)
| crate::error::AppError::RepoExists(_)
| crate::error::AppError::NotFound(_)
| crate::error::AppError::Unauthorized(_)
| crate::error::AppError::Forbidden(_)
| crate::error::AppError::BadRequest(_)
| crate::error::AppError::TooManyRequests(_)
| crate::error::AppError::Incomplete(_)) => {
tracing::warn!(error = %safe, "graphql application error");
async_graphql::Error::new(safe.to_string())
}
other => {
tracing::error!(error = %other, "graphql unclassified AppError (opaque)");
async_graphql::Error::new(GRAPHQL_DB_ERROR_MESSAGE)
}
}
}
pub fn build_schema(
db: Arc<Db>,
ref_update_tx: tokio::sync::broadcast::Sender<RefUpdateBroadcast>,
@@ -87,105 +25,3 @@ pub fn build_schema(
.data(task_event_tx)
.finish()
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn graphql_db_err_opaques_sqlx_chain() {
let leak = "error returned from database: column \"is_public\" does not exist";
// Context layer must not hide sqlx from the chain walk (db helpers
// wrap with `.context(...)` in several places).
let err = graphql_db_err(
anyhow::Error::from(sqlx::Error::Protocol(leak.into())).context("loading repos"),
);
assert_eq!(err.message, GRAPHQL_DB_ERROR_MESSAGE);
assert!(!err.message.contains("is_public"));
assert!(!err.message.contains(leak));
assert!(!err.message.contains("loading repos"));
}
#[test]
fn graphql_db_err_keeps_business_message() {
let msg = "task not claimable: not found or already claimed";
let err = graphql_db_err(anyhow::anyhow!("{msg}"));
assert_eq!(err.message, msg);
}
#[test]
fn graphql_app_err_opaques_db_and_internal() {
let leak = "column \"is_public\" does not exist";
let db_err = graphql_app_err(crate::error::AppError::Db(sqlx::Error::Protocol(
leak.into(),
)));
assert_eq!(db_err.message, GRAPHQL_DB_ERROR_MESSAGE);
assert!(!db_err.message.contains("is_public"));
let internal = graphql_app_err(crate::error::AppError::Internal(anyhow::anyhow!(
"loading repo: {leak}"
)));
assert_eq!(internal.message, GRAPHQL_DB_ERROR_MESSAGE);
assert!(!internal.message.contains("is_public"));
}
#[test]
fn graphql_app_err_keeps_safe_variant_messages() {
let err = graphql_app_err(crate::error::AppError::NotFound("widget".into()));
assert!(
err.message.contains("widget"),
"safe NotFound message must reach the client: {}",
err.message
);
assert_ne!(err.message, GRAPHQL_DB_ERROR_MESSAGE);
let err = graphql_app_err(crate::error::AppError::BadRequest("bad cid".into()));
assert!(
err.message.contains("bad cid"),
"safe BadRequest message must reach the client: {}",
err.message
);
assert_ne!(err.message, GRAPHQL_DB_ERROR_MESSAGE);
}
#[test]
fn graphql_app_err_opaques_unclassified_variants() {
// `Git` may embed on-disk paths from libgit2; fail closed.
let path = "/var/lib/gitlawb/repos/owner/secret.git";
let err = graphql_app_err(crate::error::AppError::Git(format!(
"failed to open '{path}'"
)));
assert_eq!(err.message, GRAPHQL_DB_ERROR_MESSAGE);
assert!(!err.message.contains(path));
assert!(!err.message.contains("failed to open"));
}
/// Every `.map_err(` in the GraphQL query/mutation resolvers must route
/// through the opaque helpers, or discard the error (`|_|`). Same source-
/// scrape pattern as `api::authz_guard` (#255 review).
#[test]
fn every_graphql_map_err_uses_opaque_helpers() {
for (file, src) in [
("query.rs", include_str!("query.rs")),
("mutation.rs", include_str!("mutation.rs")),
] {
for (lineno, line) in src.lines().enumerate() {
let code = line.split("//").next().unwrap_or(line);
let Some(idx) = code.find(".map_err(") else {
continue;
};
let after = code[idx + ".map_err(".len()..].trim_start();
let ok = after.starts_with("crate::graphql::graphql_db_err")
|| after.starts_with("crate::graphql::graphql_app_err")
|| after.starts_with("|_|")
|| after.starts_with("|_ ");
assert!(
ok,
"{file}:{}: `.map_err(` must use graphql_db_err / graphql_app_err \
or discard (`|_|`): {line}",
lineno + 1
);
}
}
}
}
+8 -50
View File
@@ -54,7 +54,7 @@ impl MutationRoot {
};
db.create_task(&task)
.await
.map_err(crate::graphql::graphql_db_err)?;
.map_err(|e| async_graphql::Error::new(e.to_string()))?;
Ok(AgentTaskType::from(task))
}
@@ -76,7 +76,7 @@ impl MutationRoot {
let task = db
.claim_task(&id, &assignee_did)
.await
.map_err(crate::graphql::graphql_db_err)?;
.map_err(|e| async_graphql::Error::new(e.to_string()))?;
let _ = tx.send(TaskEventBroadcast {
task_id: id,
old_status: "pending".to_string(),
@@ -108,7 +108,7 @@ impl MutationRoot {
let existing = db
.get_task(&id)
.await
.map_err(crate::graphql::graphql_db_err)?
.map_err(|e| async_graphql::Error::new(e.to_string()))?
.ok_or_else(|| async_graphql::Error::new("task not found"))?;
if !crate::api::did_matches(caller, existing.assignee_did.as_deref().unwrap_or_default()) {
return Err(async_graphql::Error::new(
@@ -118,7 +118,7 @@ impl MutationRoot {
let task = db
.finish_task(&id, "completed", input.result.as_deref())
.await
.map_err(crate::graphql::graphql_db_err)?;
.map_err(|e| async_graphql::Error::new(e.to_string()))?;
let _ = tx.send(TaskEventBroadcast {
task_id: id,
old_status: "claimed".to_string(),
@@ -149,7 +149,7 @@ impl MutationRoot {
let existing = db
.get_task(&id)
.await
.map_err(crate::graphql::graphql_db_err)?
.map_err(|e| async_graphql::Error::new(e.to_string()))?
.ok_or_else(|| async_graphql::Error::new("task not found"))?;
if !crate::api::did_matches(caller, existing.assignee_did.as_deref().unwrap_or_default()) {
return Err(async_graphql::Error::new(
@@ -160,7 +160,7 @@ impl MutationRoot {
let task = db
.finish_task(&id, "failed", Some(&reason))
.await
.map_err(crate::graphql::graphql_db_err)?;
.map_err(|e| async_graphql::Error::new(e.to_string()))?;
let _ = tx.send(TaskEventBroadcast {
task_id: id,
old_status: "claimed".to_string(),
@@ -218,9 +218,8 @@ mod tests {
errors(&resp)
);
// 3. Signed as the claimed assignee → passes the auth gate. The missing
// task is a business error from claim_task, not a sqlx fault, so the
// actionable message must survive (not the opaque DB string) (#250).
// 3. Signed as the claimed assignee → passes the auth gate (any remaining
// error is the missing task, not an auth error).
let resp = schema
.execute(Request::new(&q).data(AuthenticatedDid(assignee.into())))
.await;
@@ -229,47 +228,6 @@ mod tests {
!errs.contains("authentication required") && !errs.contains("authenticated signer"),
"matching signer must pass the auth gate: {errs}"
);
assert!(
errs.contains("task not claimable"),
"claim race / missing task must keep its business message: {errs}"
);
assert!(
!errs.contains(crate::graphql::GRAPHQL_DB_ERROR_MESSAGE),
"business error must not be rewritten as opaque DB error: {errs}"
);
}
/// #250: mutation DB faults must be opaque; create_task hits agent_tasks.
#[sqlx::test]
async fn create_task_db_error_message_is_opaque(pool: PgPool) {
let state = crate::test_support::test_state(pool.clone()).await;
sqlx::query("ALTER TABLE agent_tasks DROP COLUMN status")
.execute(&pool)
.await
.unwrap();
let delegator = "did:key:zGQLDELEGATORAAAAAAAAAAAAAAAAAAAAAAAAAA";
let q = format!(
r#"mutation {{
createTask(
delegatorDid: "{delegator}",
input: {{ kind: "build", capability: "repo:write" }}
) {{ id }}
}}"#
);
let resp = state
.graphql_schema
.execute(Request::new(&q).data(AuthenticatedDid(delegator.into())))
.await;
let errs = errors(&resp);
assert!(
errs.contains(crate::graphql::GRAPHQL_DB_ERROR_MESSAGE),
"sqlx fault must be opaque: {errs}"
);
assert!(
!errs.contains("column") && !errs.contains("status"),
"schema text leaked: {errs}"
);
}
/// Adversarial-review GATE-1 (GraphQL): completing a task requires being its
+7 -102
View File
@@ -14,7 +14,7 @@ impl QueryRoot {
let repos = db
.list_all_repos_deduped()
.await
.map_err(crate::graphql::graphql_db_err)?;
.map_err(|e| async_graphql::Error::new(e.to_string()))?;
// Apply the same "/" visibility gate the REST/per-repo endpoints use so
// this surface does not enumerate private repos (#97). The caller DID is
@@ -27,7 +27,7 @@ impl QueryRoot {
let rules_by_repo = db
.list_visibility_rules_for_repos(&ids)
.await
.map_err(crate::graphql::graphql_db_err)?;
.map_err(|e| async_graphql::Error::new(e.to_string()))?;
Ok(repos
.into_iter()
@@ -71,7 +71,7 @@ impl QueryRoot {
let updates =
crate::api::events::collect_visible_ref_updates(db, repo.as_deref(), limit, caller)
.await
.map_err(crate::graphql::graphql_app_err)?;
.map_err(|e| async_graphql::Error::new(e.to_string()))?;
// Resolve the trusted display owner_did per row, identical to the REST
// feed: the stored wire value is untrusted, so it is echoed only when it
@@ -86,7 +86,7 @@ impl QueryRoot {
let owner_dids = db
.resolve_ref_update_owner_dids(&pairs)
.await
.map_err(crate::graphql::graphql_db_err)?;
.map_err(|e| async_graphql::Error::new(e.to_string()))?;
let resolved: Vec<RefUpdateType> = updates
.into_iter()
@@ -110,21 +110,13 @@ impl QueryRoot {
ctx: &Context<'_>,
status: Option<String>,
assignee_did: Option<String>,
#[graphql(
default = 50,
desc = "Max 200; larger requests are clamped to 200 (no error). Negative values clamp to 0."
)]
limit: i64,
#[graphql(default = 50)] limit: i64,
) -> Result<Vec<AgentTaskType>> {
let db = ctx.data_unchecked::<Arc<Db>>();
// Clamp before SQL: a negative LIMIT is a client fault that Postgres
// rejects with 2201W, which would otherwise trip the opaque DB path
// and write an error-level log on every probe (#250 review).
let limit = limit.clamp(0, 200);
let tasks = db
.list_tasks(status.as_deref(), assignee_did.as_deref(), limit)
.await
.map_err(crate::graphql::graphql_db_err)?;
.map_err(|e| async_graphql::Error::new(e.to_string()))?;
Ok(tasks.into_iter().map(AgentTaskType::from).collect())
}
@@ -133,7 +125,7 @@ impl QueryRoot {
let t = db
.get_task(&id)
.await
.map_err(crate::graphql::graphql_db_err)?;
.map_err(|e| async_graphql::Error::new(e.to_string()))?;
Ok(t.map(AgentTaskType::from))
}
}
@@ -427,91 +419,4 @@ mod tests {
let q = r#"{ refUpdates { repo } }"#;
assert_eq!(count(&authed(&schema, q, "did:key:z6MkQuar").await), 0);
}
/// #250: anonymous GraphQL query DB failures must not leak sqlx/schema text.
#[sqlx::test]
async fn repos_query_db_error_message_is_opaque(pool: PgPool) {
let db = db(pool.clone()).await;
db.create_repo(&repo("r1", OWNER, "widget", true))
.await
.unwrap();
sqlx::query("ALTER TABLE repos DROP COLUMN is_public")
.execute(&pool)
.await
.unwrap();
let schema = schema(db);
let resp = anon(&schema, "{ repos { name ownerDid } }").await;
assert!(
!resp.errors.is_empty(),
"DB failure must surface as a GraphQL error"
);
for err in &resp.errors {
assert_eq!(
err.message,
crate::graphql::GRAPHQL_DB_ERROR_MESSAGE,
"raw DB detail leaked into GraphQL error: {}",
err.message
);
assert!(
!err.message.contains("is_public") && !err.message.contains("column"),
"schema text leaked: {}",
err.message
);
}
}
/// #250: negative tasks(limit) must not hit Postgres (and must not 500-log).
#[sqlx::test]
async fn tasks_negative_limit_clamped(pool: PgPool) {
let db = db(pool).await;
let schema = schema(db);
let resp = anon(&schema, "{ tasks(limit: -1) { id } }").await;
assert!(
resp.errors.is_empty(),
"negative limit must clamp, not fail: {:?}",
resp.errors
);
assert_eq!(count_tasks(&resp), 0);
}
/// #255: ceiling of the tasks(limit) clamp must be held (schema promises Max 200).
#[sqlx::test]
async fn tasks_limit_ceiling_clamped_to_200(pool: PgPool) {
let db = db(pool).await;
let now = Utc::now().to_rfc3339();
for i in 0..201 {
db.create_task(&crate::db::AgentTask {
id: format!("task-ceil-{i}"),
repo_id: None,
kind: "build".into(),
status: "pending".into(),
delegator_did: OWNER.into(),
assignee_did: None,
capability: "repo:write".into(),
ucan_token: None,
payload: None,
result: None,
created_at: now.clone(),
updated_at: now.clone(),
deadline: None,
})
.await
.unwrap();
}
let schema = schema(db);
let resp = anon(&schema, "{ tasks(limit: 5000) { id } }").await;
assert_eq!(count_tasks(&resp), 200, "limit above 200 must clamp to 200");
}
fn count_tasks(resp: &async_graphql::Response) -> usize {
assert!(resp.errors.is_empty(), "graphql errors: {:?}", resp.errors);
let async_graphql::Value::Object(obj) = &resp.data else {
panic!("data not an object: {:?}", resp.data);
};
let async_graphql::Value::List(rows) = obj.get("tasks").expect("tasks key") else {
panic!("tasks not a list");
};
rows.len()
}
}
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
+9 -47
View File
@@ -57,15 +57,10 @@ static PEERS_CONNECTED: OnceLock<IntGauge> = OnceLock::new();
/// more than once is a silent no-op. MUST be called from `main()` after
/// the node DID is known.
pub fn init(version: &str, node_did: &str) {
// Guard with Once so two concurrent callers cannot both pass an unsynchronized
// `REGISTRY.get()` check and then race on `OnceLock::set(...).expect(...)`,
// panicking the loser (#192 F4). Once runs the body exactly once, so the
// `.expect("set X once")` calls below can never observe an already-set slot.
static INIT: std::sync::Once = std::sync::Once::new();
INIT.call_once(|| init_inner(version, node_did));
}
if REGISTRY.get().is_some() {
return;
}
fn init_inner(version: &str, node_did: &str) {
let registry = Registry::new();
let info = IntGaugeVec::new(
@@ -223,17 +218,6 @@ pub fn record_fetch(repo: &str) {
}
}
/// Test-only: current `gitlawb_fetches_total` value for `repo` (0 if the registry
/// is not initialized). Lets api-layer tests assert the completed-fetch count with
/// a unique label instead of scraping the encoded text.
#[cfg(test)]
pub fn fetch_count_for_test(repo: &str) -> u64 {
FETCHES
.get()
.map(|c| c.with_label_values(&[repo]).get())
.unwrap_or(0)
}
/// Record one HTTP signature check that passed.
#[allow(dead_code)] // wired in a follow-up; helpers are part of the public metrics surface
pub fn record_auth_success(route: &str) {
@@ -251,11 +235,7 @@ pub fn record_auth_failure(route: &str, reason: &str) {
}
}
/// Record one sync_queue item outcome. `status` ∈ {done, failed, skipped,
/// rejected, deferred}. `rejected` is terminal and caused by a guard (a
/// malformed slug, or a path resolving outside repos_dir); `deferred` leaves
/// the row pending for a later pass, so a queue stalled on an operator
/// condition is visible rather than looking idle.
/// Record one sync_queue item outcome. `status` ∈ {done, failed, skipped}.
pub fn record_sync_processed(status: &str) {
if let Some(c) = SYNC_PROCESSED.get() {
c.with_label_values(&[status]).inc();
@@ -302,11 +282,11 @@ pub fn encode() -> Result<String, prometheus::Error> {
mod tests {
use super::*;
// Note: cargo runs tests in a binary in parallel via threads, and several
// tests here call init(). init() is guarded by std::sync::Once, so concurrent
// callers run the body exactly once and never race on the OnceLock setters
// (#192 F4). The encode test below is structured so it's safe to run alongside
// other tests in the same binary regardless of ordering.
// Note: these tests are not run in parallel by default (cargo runs
// tests in a binary in parallel via threads but the OnceLock guard
// means only the first init() call succeeds; subsequent ones are
// no-ops). The encode test below is structured so it's safe to run
// alongside other tests in the same binary.
#[test]
fn encode_after_init_returns_prometheus_text() {
@@ -337,24 +317,6 @@ mod tests {
);
}
/// #192 F4: `init` is idempotent and safe to call repeatedly. The panic that
/// hit concurrent callers (both passing the old unsynchronized `REGISTRY.get()`
/// check, then racing on `OnceLock::set(...).expect(...)`) is fixed by the
/// `Once` guard. That race is inherently non-deterministic to assert in the
/// shared-binary suite — a 32-thread barrier stress-test reproduces it only in
/// isolation (verified: RED before the `Once` fix, GREEN after). This guard
/// covers the deterministic half of the contract: a repeat call is a no-op,
/// never a panic, and the registry stays usable.
#[test]
fn init_is_idempotent_no_panic_on_repeat() {
init("0.0.0-a", "did:key:a");
init("0.0.0-b", "did:key:b");
assert!(
encode().is_ok(),
"registry must stay usable after a repeated init"
);
}
#[test]
fn record_helpers_are_noops_before_init() {
// These don't panic. They also don't show up in encode() output
File diff suppressed because it is too large. Load diff
+265 -202
View File
@@ -70,10 +70,23 @@ impl RateLimiter {
self.window.min(Duration::from_secs(1))
}
/// Returns `true` if the request is allowed. Thin wrapper over
/// [`check_retry`](Self::check_retry) for callers that only need the
/// allow/deny decision, not the rejection's Retry-After delay. Test-only now:
/// production 429 sites use `check_retry` so they can advertise Retry-After.
#[cfg(test)]
pub(crate) async fn check(&self, key: &str) -> bool {
self.check_retry(key).await.is_none()
}
/// Record a request against `key`. Returns `None` when it is allowed, or
/// `Some(retry_after)` when it is rejected — the delay a client should wait
/// before retrying, so a 429 can advertise a value consistent with the
/// actual window instead of a constant.
pub(crate) async fn check_retry(&self, key: &str) -> Option<Duration> {
// max_requests == 0 means the limiter is disabled, not "block all".
if self.max_requests == 0 {
return true;
return None;
}
let now = Instant::now();
let mut state = self.state.lock().await;
@@ -85,10 +98,25 @@ impl RateLimiter {
.timestamps
.retain(|t| now.duration_since(*t) < self.window);
if window.timestamps.len() >= self.max_requests {
return false;
// Insertion order is preserved, so after the retain the first
// element is the oldest live request. It ages out (freeing a
// slot) one window after it landed, so that remaining time is
// the delay to advertise. Clamp to >= 1 so a 429 never tells a
// client to retry immediately.
let retry = window
.timestamps
.first()
.map(|oldest| {
self.window
.as_secs()
.saturating_sub(now.duration_since(*oldest).as_secs())
})
.unwrap_or(0)
.max(1);
return Some(Duration::from_secs(retry));
}
window.timestamps.push(now);
return true;
return None;
}
// New key. Enforce the cap BEFORE inserting so a flood of distinct keys
@@ -109,7 +137,11 @@ impl RateLimiter {
}
drop(last_sweep);
if state.len() >= self.max_keys {
return false;
// The key is untracked (never inserted), so there is no oldest
// entry to derive a delay from. Advertise the full window as a
// conservative bound: capacity frees as other keys expire, at
// the latest one window from now.
return Some(self.window.max(Duration::from_secs(1)));
}
}
state.insert(
@@ -118,29 +150,7 @@ impl RateLimiter {
timestamps: vec![now],
},
);
true
}
/// Non-consuming check: is this key ALREADY at its limit for the current window?
/// Unlike [`check`], it records nothing and never inserts a new key — used to shed
/// expensive preparatory work (e.g. the `/ipfs/{cid}` legacy scan's O(repos) DB
/// preload) BEFORE it runs, without perturbing the per-unit budget the consuming
/// `check` maintains (#173, F3). An unknown key or a disabled limiter is not
/// throttled. Prunes the key's expired timestamps as a side effect (keeps state
/// tidy) but adds none, so it cannot itself fill or grow the map.
pub(crate) async fn is_throttled(&self, key: &str) -> bool {
if self.max_requests == 0 {
return false;
}
let now = Instant::now();
let mut state = self.state.lock().await;
if let Some(window) = state.get_mut(key) {
window
.timestamps
.retain(|t| now.duration_since(*t) < self.window);
return window.timestamps.len() >= self.max_requests;
}
false
None
}
pub async fn cleanup(&self) {
@@ -153,116 +163,39 @@ impl RateLimiter {
});
}
/// Number of distinct keys currently tracked. Test-only introspection so a
/// cross-module test can assert that a sweep actually evicted expired entries
/// and observe what it reclaimed. There is no production reader.
/// Test-only: insert a fully-expired entry (no live timestamps) that the next
/// `cleanup()` must reclaim. Models a key whose window has aged out, without
/// depending on wall-clock sleeps or a short window. Used cross-module by the
/// AppState cleanup guard test (H2).
#[cfg(test)]
pub(crate) async fn insert_reclaimable_for_test(&self, key: &str) {
self.state
.lock()
.await
.insert(key.to_string(), Window { timestamps: vec![] });
}
/// Test-only: number of distinct keys currently tracked.
#[cfg(test)]
pub(crate) async fn tracked_keys(&self) -> usize {
self.state.lock().await.len()
}
}
/// Per-source concurrency cap derived from the write-pool size: one resolved client
/// key (see [`client_key`]) may hold at most an eighth of the pool, so saturating it
/// takes ~8 distinct keys. Real for an IPv4 or single-address caller; a caller with a
/// routed IPv6 /64 has 2^64 keys, because `client_key` returns the full address with
/// no prefix folding. Narrowing that keying is a deferred decision, so this cap is a
/// bound per key, not a bound per operator.
///
/// Floored at 1 because the value feeds [`PerCallerConcurrency`], where a cap of 0
/// would shed EVERY receive-pack advertisement and break all pushes. The floor is
/// load-bearing at the minimum write-pool size (1), which integer-divides to 0.
pub(crate) fn per_source_push_cap(max_concurrent_git_pushes: usize) -> usize {
(max_concurrent_git_pushes / 8).max(1)
}
/// A bounded per-caller CONCURRENCY limiter — distinct from [`RateLimiter`], which
/// caps request RATE. Each caller key may hold at most `per_caller` in-flight
/// permits at once; beyond that [`try_acquire`](Self::try_acquire) returns `None`
/// and the caller sheds. Used to stop one caller (a single anonymous source-IP or
/// DID) monopolizing the served-git read pool (#174).
///
/// The key map is self-bounding: a key is removed the instant its in-flight count
/// reaches zero, so it never holds more keys than there are concurrently-active
/// callers (itself bounded by the read semaphore). A `max_keys` reject-before-insert
/// backstop guarantees a key farm can never grow the map even if that invariant
/// weakened — a NEW key at the cap is rejected WITHOUT allocating an entry (INV-15).
///
/// Uses a `std::sync::Mutex` (not the file's `tokio::sync::Mutex`) because the
/// permit's `Drop` must release synchronously; the critical section holds no await.
#[derive(Clone)]
pub struct PerCallerConcurrency {
state: Arc<std::sync::Mutex<HashMap<String, usize>>>,
per_caller: usize,
max_keys: usize,
}
/// RAII permit from [`PerCallerConcurrency::try_acquire`]. On drop it decrements
/// the caller's in-flight count and removes the key when it reaches zero.
pub struct PerCallerPermit {
state: Arc<std::sync::Mutex<HashMap<String, usize>>>,
key: String,
}
impl PerCallerConcurrency {
pub fn new(per_caller: usize, max_keys: usize) -> Self {
Self {
state: Arc::new(std::sync::Mutex::new(HashMap::new())),
per_caller: per_caller.max(1),
max_keys: max_keys.max(1),
}
}
/// Convenience constructor with the default key bound.
pub fn with_default_max_keys(per_caller: usize) -> Self {
Self::new(per_caller, DEFAULT_MAX_KEYS)
}
/// `Some(permit)` when the caller is under its cap and the map has room;
/// `None` (shed) otherwise. Reject-before-insert: a new key at `max_keys` is
/// rejected without allocating.
pub fn try_acquire(&self, key: &str) -> Option<PerCallerPermit> {
// Recover from a poisoned lock rather than panicking: the critical section
// is pure counter arithmetic and cannot itself panic, so a poisoned mutex
// would only ever come from an unrelated abort, and a slightly-off count
// self-heals as permits drop. A panic here would instead brick the limiter
// for every caller (each subsequent lock re-panics).
let mut map = self.state.lock().unwrap_or_else(|e| e.into_inner());
match map.get_mut(key) {
Some(count) => {
if *count >= self.per_caller {
return None;
}
*count += 1;
}
None => {
if map.len() >= self.max_keys {
return None;
}
map.insert(key.to_string(), 1);
}
}
Some(PerCallerPermit {
state: self.state.clone(),
key: key.to_string(),
})
}
/// Test-only: seed a key's bucket with entries of the given ages (how long
/// ago each request landed), so a test can drive the Retry-After computation
/// off a controlled oldest-entry age without wall-clock sleeps. Ages that
/// would underflow the monotonic clock collapse to `now`.
#[cfg(test)]
pub fn tracked_keys(&self) -> usize {
self.state.lock().unwrap().len()
}
}
impl Drop for PerCallerPermit {
fn drop(&mut self) {
let mut map = self.state.lock().unwrap_or_else(|e| e.into_inner());
if let Some(count) = map.get_mut(&self.key) {
*count -= 1;
if *count == 0 {
map.remove(&self.key);
}
}
pub(crate) async fn seed_aged_entry_for_test(&self, key: &str, ages: &[Duration]) {
let now = Instant::now();
let timestamps = ages
.iter()
.map(|a| now.checked_sub(*a).unwrap_or(now))
.collect();
self.state
.lock()
.await
.insert(key.to_string(), Window { timestamps });
}
}
@@ -275,13 +208,8 @@ pub async fn rate_limit_by_did(request: Request, next: Next) -> Response {
.map(|a| a.0.clone());
if let (Some(limiter), Some(did)) = (limiter, did) {
if !limiter.check(&did).await {
return (
StatusCode::TOO_MANY_REQUESTS,
[("retry-after", "60")],
"rate limit exceeded — try again later",
)
.into_response();
if let Some(retry_after) = limiter.check_retry(&did).await {
return too_many_requests(retry_after);
}
}
@@ -384,13 +312,13 @@ impl<S: Send + Sync> axum::extract::FromRequestParts<S> for PeerAddr {
}
}
/// The shared 429 response for the per-IP flood brakes. Route-agnostic: this
/// middleware now serves the push path AND the peer-sync routes, so the message
/// stays generic (the offending path is recorded in the warn log below).
pub fn too_many_requests() -> Response {
/// The shared 429 response for the rate-limit middlewares (per-DID and per-IP
/// alike). Route-agnostic: the message stays generic (the offending path, when
/// logged, is recorded at the call site).
pub fn too_many_requests(retry_after: Duration) -> Response {
(
StatusCode::TOO_MANY_REQUESTS,
[("retry-after", "60")],
[("retry-after", retry_after.as_secs().to_string())],
"rate limit exceeded — try again later",
)
.into_response()
@@ -408,9 +336,9 @@ pub async fn rate_limit_by_ip(request: Request, next: Next) -> Response {
if let Some(limiter) = limiter {
if let Some(key) = client_key(request.headers(), peer, limiter.trust) {
if !limiter.limiter.check(&key).await {
if let Some(retry_after) = limiter.limiter.check_retry(&key).await {
tracing::warn!(key = %key, path = %request.uri().path(), "per-IP rate limit exceeded");
return too_many_requests();
return too_many_requests(retry_after);
}
}
}
@@ -422,63 +350,6 @@ pub async fn rate_limit_by_ip(request: Request, next: Next) -> Response {
mod tests {
use super::*;
#[test]
fn per_caller_concurrency_caps_one_caller_and_frees_on_drop() {
let lim = PerCallerConcurrency::new(2, 100);
let p1 = lim.try_acquire("did:key:zA").expect("first under cap");
let p2 = lim.try_acquire("did:key:zA").expect("second under cap");
assert!(
lim.try_acquire("did:key:zA").is_none(),
"a third in-flight op for the same caller sheds (over the per-caller cap)"
);
// A DIFFERENT caller is unaffected — the cap is per-caller, not global.
let _other = lim
.try_acquire("did:key:zB")
.expect("a different caller has its own budget");
drop(p1);
assert!(
lim.try_acquire("did:key:zA").is_some(),
"freeing one in-flight slot lets the same caller back in"
);
drop(p2);
}
#[test]
fn per_caller_concurrency_map_is_self_bounding_and_reject_before_insert() {
// Self-bounding: acquire+drop many distinct keys — the map never grows
// because a key is removed the instant its in-flight count hits zero.
let lim = PerCallerConcurrency::new(4, 3);
for i in 0..50 {
let _p = lim.try_acquire(&format!("k{i}"));
}
assert_eq!(
lim.tracked_keys(),
0,
"keys with zero in-flight ops are removed, so an acquire+drop flood leaves the map empty"
);
// Reject-before-insert: HOLD max_keys distinct keys, then a new key sheds
// WITHOUT growing the map past the cap (INV-15 — a rejected request never
// allocates an entry).
let held: Vec<_> = (0..3)
.map(|i| lim.try_acquire(&format!("h{i}")).unwrap())
.collect();
assert_eq!(
lim.tracked_keys(),
3,
"three distinct callers held concurrently"
);
assert!(
lim.try_acquire("h3").is_none(),
"a new key at max_keys is rejected"
);
assert_eq!(
lim.tracked_keys(),
3,
"the rejected new key did not allocate an entry (reject-before-insert)"
);
drop(held);
}
#[tokio::test]
async fn allows_within_limit() {
let limiter = RateLimiter::new(3, Duration::from_secs(60));
@@ -796,4 +667,196 @@ mod tests {
"an exhausted IP must be braked with 429 before auth runs, not leak to 401"
);
}
// ── Adversarial TrustedProxy verification through the middleware (U5) ──
// These complement the client_key unit tests above by driving hostile
// headers through rate_limit_by_ip on a real router: the property under test
// is that the *bucket key* cannot be rotated by a spoofer.
async fn post_with(
router: &axum::Router,
peer: SocketAddr,
hdrs: &[(&str, &str)],
) -> StatusCode {
use tower::ServiceExt;
let mut b = axum::http::Request::builder()
.method(axum::http::Method::POST)
.uri("/api/v1/repos");
for (k, v) in hdrs {
b = b.header(*k, *v);
}
let mut req = b.body(axum::body::Body::empty()).unwrap();
req.extensions_mut().insert(ConnectInfo(peer));
router.clone().oneshot(req).await.unwrap().status()
}
#[tokio::test]
async fn xff_spoofed_leftmost_hop_cannot_rotate_bucket() {
// XForwardedFor mode, budget 1. A spoofer varies the client-controlled
// leftmost hop every request but the trusted proxy always appends the
// same rightmost hop. All requests must key to the rightmost hop, so the
// second is braked despite a fresh leftmost value.
let router = ip_limited_over_auth_router(IpRateLimiter {
limiter: RateLimiter::new(1, Duration::from_secs(60)),
trust: TrustedProxy::XForwardedFor,
});
let p1: SocketAddr = "10.0.0.1:1".parse().unwrap();
let p2: SocketAddr = "10.0.0.2:1".parse().unwrap();
// First request passes the brake, reaches (failing) auth.
assert_eq!(
post_with(&router, p1, &[("x-forwarded-for", "9.9.9.1, 203.0.113.5")]).await,
StatusCode::UNAUTHORIZED
);
// Different leftmost + different socket peer, SAME rightmost trusted hop:
// braked, because the key is the rightmost hop, not the spoofed value.
assert_eq!(
post_with(&router, p2, &[("x-forwarded-for", "9.9.9.2, 203.0.113.5")]).await,
StatusCode::TOO_MANY_REQUESTS,
"a spoofer varying the leftmost XFF hop must not rotate its bucket key"
);
}
#[tokio::test]
async fn xff_distinct_real_clients_get_distinct_buckets() {
// Distinct trusted (rightmost) hops are distinct clients: neither throttles
// the other, so a busy legitimate client cannot starve a different one.
let router = ip_limited_over_auth_router(IpRateLimiter {
limiter: RateLimiter::new(1, Duration::from_secs(60)),
trust: TrustedProxy::XForwardedFor,
});
let peer: SocketAddr = "10.0.0.9:1".parse().unwrap();
assert_eq!(
post_with(
&router,
peer,
&[("x-forwarded-for", "1.1.1.1, 203.0.113.5")]
)
.await,
StatusCode::UNAUTHORIZED
);
assert_eq!(
post_with(
&router,
peer,
&[("x-forwarded-for", "1.1.1.1, 203.0.113.6")]
)
.await,
StatusCode::UNAUTHORIZED,
"a different trusted client hop must get its own bucket"
);
}
#[tokio::test]
async fn absent_trusted_header_collapses_to_socket_peer() {
// The documented fallback: in a trusted-proxy mode, a request with no
// forwarded header keys on the socket peer. Behind a real edge every
// request shares the proxy's socket, so they collapse onto one bucket —
// asserted here by name so any change to this behavior is deliberate.
let router = ip_limited_over_auth_router(IpRateLimiter {
limiter: RateLimiter::new(1, Duration::from_secs(60)),
trust: TrustedProxy::XForwardedFor,
});
let proxy: SocketAddr = "172.16.0.1:1".parse().unwrap();
assert_eq!(
post_with(&router, proxy, &[]).await,
StatusCode::UNAUTHORIZED
);
assert_eq!(
post_with(&router, proxy, &[]).await,
StatusCode::TOO_MANY_REQUESTS,
"with no trusted header, requests fall back to the socket peer key (collapse)"
);
}
// ── Retry-After derived from the window, not a constant 60 (U5) ───────
// A 429 must advertise the delay until the oldest live request ages out of
// the limiter's window, so the value tracks the real window instead of the
// hardcoded 60 that contradicted every limiter's 3600s window.
async fn resp_from(router: &axum::Router, peer: SocketAddr) -> Response {
use tower::ServiceExt;
let mut req = axum::http::Request::builder()
.method(axum::http::Method::POST)
.uri("/o/r/git-receive-pack")
.body(axum::body::Body::empty())
.unwrap();
req.extensions_mut().insert(ConnectInfo(peer));
router.clone().oneshot(req).await.unwrap()
}
fn retry_after_secs(resp: &Response) -> u64 {
resp.headers()
.get("retry-after")
.expect("429 must carry a retry-after header")
.to_str()
.unwrap()
.parse()
.expect("retry-after must be an integer number of seconds")
}
#[tokio::test]
async fn retry_after_reflects_full_window_for_freshly_filled_bucket() {
// A just-filled bucket's oldest entry is ~now, so the advertised delay
// must be close to the whole window (100s), NOT the old constant 60.
let router = ip_limited_router(IpRateLimiter {
limiter: RateLimiter::new(1, Duration::from_secs(100)),
trust: TrustedProxy::None,
});
let peer: SocketAddr = "203.0.113.50:6000".parse().unwrap();
assert_eq!(resp_from(&router, peer).await.status(), StatusCode::OK);
let resp = resp_from(&router, peer).await;
assert_eq!(resp.status(), StatusCode::TOO_MANY_REQUESTS);
let retry = retry_after_secs(&resp);
assert!(
(95..=100).contains(&retry),
"freshly-filled bucket must advertise ~window (95..=100s), got {retry} (constant-60 bug if 60)"
);
}
#[tokio::test]
async fn retry_after_shrinks_as_oldest_entry_ages() {
// Seed the sole slot with an entry already 95s into a 100s window: the
// next request is rejected and must advertise only the ~5s remaining,
// not 60.
let limiter = RateLimiter::new(1, Duration::from_secs(100));
let peer: SocketAddr = "203.0.113.51:6000".parse().unwrap();
limiter
.seed_aged_entry_for_test(&peer.ip().to_string(), &[Duration::from_secs(95)])
.await;
let router = ip_limited_router(IpRateLimiter {
limiter,
trust: TrustedProxy::None,
});
let resp = resp_from(&router, peer).await;
assert_eq!(resp.status(), StatusCode::TOO_MANY_REQUESTS);
let retry = retry_after_secs(&resp);
assert!(
(1..=10).contains(&retry),
"an aged bucket must advertise the small remaining delay (1..=10s), got {retry}"
);
}
#[tokio::test]
async fn retry_after_clamps_to_at_least_one_second() {
// Oldest entry is 99s into a 100s window: the raw remainder rounds toward
// 1s and must never be advertised as 0 (a 0/blank retry lets a rejected
// client hammer immediately).
let limiter = RateLimiter::new(1, Duration::from_secs(100));
let peer: SocketAddr = "203.0.113.52:6000".parse().unwrap();
limiter
.seed_aged_entry_for_test(&peer.ip().to_string(), &[Duration::from_secs(99)])
.await;
let router = ip_limited_router(IpRateLimiter {
limiter,
trust: TrustedProxy::None,
});
let resp = resp_from(&router, peer).await;
assert_eq!(resp.status(), StatusCode::TOO_MANY_REQUESTS);
let retry = retry_after_secs(&resp);
assert!(retry >= 1, "retry-after must clamp to >= 1, got {retry}");
assert!(
retry <= 2,
"with 99s of a 100s window elapsed, the remaining delay is ~1s, got {retry}"
);
}
}
+52 -20
View File
@@ -55,18 +55,50 @@ fn add_auth_layers(router: Router<AppState>, state: AppState) -> Router<AppState
.layer(middleware::from_fn(auth::require_signature))
}
/// Attach the shared per-IP write brake to an authenticated write-route group,
/// outermost so a flood is rejected before signature verification. Every
/// authenticated non-creation write group MUST call this or it silently ships
/// unbraked with no compile error. Deliberately a separate call, NOT folded into
/// `add_auth_layers`: the creation routes use a different bucket
/// (`create_ip_rate_limiter`) and the graphql group is not built through
/// `add_auth_layers`, so a helper keyed on auth would attach the wrong limiter.
trait WriteBraked {
fn write_braked(self, limiter: &rate_limit::IpRateLimiter) -> Self;
}
impl WriteBraked for Router<AppState> {
fn write_braked(self, limiter: &rate_limit::IpRateLimiter) -> Self {
self.layer(middleware::from_fn(rate_limit::rate_limit_by_ip))
.layer(axum::Extension(limiter.clone()))
}
}
pub fn build_router(state: AppState) -> Router {
// ── GraphQL routes ─────────────────────────────────────────────────────
let schema = state.graphql_schema.as_ref().clone();
// Per-IP write brake, shared across every authenticated non-creation write
// group (see `AppState::write_rate_limiter`). Built once here so each group
// clones the same bucket; attached outermost on each group so a flood is
// rejected before auth runs. Separate bucket from the creation brake.
let write_ip_limiter = rate_limit::IpRateLimiter {
limiter: state.write_rate_limiter.clone(),
trust: state.push_limiter_trust,
};
let graphql_routes = Router::new()
.route("/graphql", get(graphql_playground).post(graphql_handler))
// Attach the verified DID to /graphql when a signature is present. The
// layer covers only routes added before it, so /graphql/ws (added after,
// read-only subscriptions) stays open.
// read-only subscriptions) stays open. The per-IP write brake covers the
// same /graphql (KTD-5: an HTTP-layer brake cannot tell a query POST from
// a mutation POST, so it counts every /graphql request — acceptable, both
// cost work); /graphql/ws subscriptions stay unbraked.
.layer(middleware::from_fn(auth::optional_signature))
.write_braked(&write_ip_limiter)
.route_service("/graphql/ws", GraphQLSubscription::new(schema));
// ── Task routes (write — require HTTP Signature) ───────────────────────
// ── Task routes (write — require HTTP Signature + per-IP write brake) ──
let task_write_routes = add_auth_layers(
Router::new()
.route("/api/v1/tasks", post(tasks::create_task))
@@ -74,7 +106,8 @@ pub fn build_router(state: AppState) -> Router {
.route("/api/v1/tasks/{id}/complete", post(tasks::complete_task))
.route("/api/v1/tasks/{id}/fail", post(tasks::fail_task)),
state.clone(),
);
)
.write_braked(&write_ip_limiter);
// ── Task routes (read — open) ──────────────────────────────────────────
let task_read_routes = Router::new()
@@ -111,7 +144,13 @@ pub fn build_router(state: AppState) -> Router {
.layer(middleware::from_fn(rate_limit::rate_limit_by_ip))
.layer(axum::Extension(create_ip_limiter));
// ── Write routes — require HTTP Signature (no rate limit) ─────────────
// ── Write routes — require HTTP Signature + per-IP write brake ────────
// Same per-IP flood defense as the creation routes, on its own `write`
// bucket (see `AppState::write_rate_limiter`). Per-DID is deliberately not
// paired (a DID farm never trips it; busy legitimate agents would false-
// positive). The bundled visibility GET is a read that also draws from the
// write bucket — harmless, since the bucket is sized far above any legit
// per-IP read rate.
let write_routes = add_auth_layers(
Router::new()
.route(
@@ -181,7 +220,8 @@ pub fn build_router(state: AppState) -> Router {
axum::routing::delete(agents::deregister_agent),
),
state.clone(),
);
)
.write_braked(&write_ip_limiter);
// Body limit is raised to GITLAWB_MAX_PACK_BYTES (default 2 GB) for git
// routes only — all other API routes keep axum's default 2 MB cap.
@@ -214,20 +254,9 @@ pub fn build_router(state: AppState) -> Router {
// identity and can apply per-repo visibility (#110); anonymous callers stay
// anonymous and still read genuinely public content. `/api/v1/ipfs/pins`
// stays unsigned — gating the pin index is tracked separately (#121).
// `/ipfs/{cid}` also carries a per-IP flood brake: it is anon-reachable and each
// request can drive a full-history git walk, so the per-IP rate limiter is the
// outermost layer (rejects a flood before the walk-admission work), mirroring the
// push/create routers. The extension MUST be attached or rate_limit_by_ip is a
// silent no-op. `/api/v1/ipfs/pins` (no walk) is merged in unbraked, as before.
let ipfs_limiter = rate_limit::IpRateLimiter {
limiter: state.ipfs_rate_limiter.clone(),
trust: state.push_limiter_trust,
};
let ipfs_routes = Router::new()
.route("/ipfs/{cid}", get(ipfs::get_by_cid))
.layer(middleware::from_fn(auth::optional_signature))
.layer(middleware::from_fn(rate_limit::rate_limit_by_ip))
.layer(axum::Extension(ipfs_limiter))
.merge(Router::new().route("/api/v1/ipfs/pins", get(ipfs::list_pins)));
// ── Arweave permanent anchors ──────────────────────────────────────────
@@ -258,7 +287,8 @@ pub fn build_router(state: AppState) -> Router {
post(bounties::dispute_bounty),
),
state.clone(),
);
)
.write_braked(&write_ip_limiter);
// ── Bounty routes (read — open) ──────────────────────────────────────
let bounty_read_routes = Router::new()
@@ -279,9 +309,10 @@ pub fn build_router(state: AppState) -> Router {
let profile_write_routes = add_auth_layers(
Router::new().route("/api/v1/profile", axum::routing::put(profiles::set_profile)),
state.clone(),
);
)
.write_braked(&write_ip_limiter);
// ── Issue routes (write — require HTTP Signature, no rate limit) ─────
// ── Issue routes (write — require HTTP Signature + per-IP write brake) ─
let issue_write_routes = add_auth_layers(
Router::new()
.route(
@@ -293,7 +324,8 @@ pub fn build_router(state: AppState) -> Router {
post(issues::create_issue_comment),
),
state.clone(),
);
)
.write_braked(&write_ip_limiter);
// ── Peer discovery routes ─────────────────────────────────────────────
// Peer writes accept signatures when present and can require them after a
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
-25
View File
@@ -437,31 +437,6 @@ mod tests {
);
}
// #135 T1: a Mode-B rule on `/secret/**` must DENY the withheld directory's
// OWN path `/secret` (the `path == prefix` arm), not just strict descendants —
// otherwise get_by_cid's tree gate would serve the /secret tree object and leak
// its children. Pins parity with get_tree, which denies the /secret path.
#[test]
fn subtree_rule_denies_the_withheld_directory_itself() {
let reader = "did:key:z6MkReader";
let rules = [rule("/secret/**", VisibilityMode::B, &[reader])];
assert_eq!(
visibility_check(&rules, true, OWNER, None, "/secret"),
Decision::Deny,
"anon denied at the withheld directory's OWN path /secret"
);
assert_eq!(
visibility_check(&rules, true, OWNER, None, "/public"),
Decision::Allow,
"anon allowed at a sibling path outside the withheld subtree"
);
assert_eq!(
visibility_check(&rules, true, OWNER, Some(reader), "/secret"),
Decision::Allow,
"listed reader allowed at the withheld directory (caller-aware)"
);
}
// #153 regression: cross-method DID must still be denied even when the
// trailing segment collides with a bare owner key.
#[test]
-574
View File
@@ -1,574 +0,0 @@
//! #174 U6 — INV-22 completeness guard (rung-raising).
//!
//! INV-22: a permit held per op recovers only if every path that holds it is also
//! duration-bounded and reaps the process group before releasing admission, and every
//! detached git/blocking task carries its own admission. PR #174 fixed five paths
//! (U1-U5) that violated this. Each fix has a per-unit RED/GREEN regression; together
//! those form the five-revert matrix. This guard adds the missing piece: a source-scan
//! tripwire that fails when a NEW site reintroduces the class, or when one of the five
//! gates is removed.
//!
//! It lives in `tests/` (a separate crate) on purpose: a guard that scanned the same
//! file it lives in would match its own identifier literals and pass vacuously. Here
//! the scanned `src/` files never contain this file's literals, so each check is
//! load-bearing — reverting the named gate turns the assertion red.
//!
//! These are deliberately coarse structural checks, not a parser. They cannot prove a
//! gate is *correct* (the per-unit tests do that); they prove a gate is *present and
//! not bypassed*, which is what stops the class from silently regressing.
use std::path::Path;
fn src(rel: &str) -> String {
let p = Path::new(env!("CARGO_MANIFEST_DIR")).join("src").join(rel);
std::fs::read_to_string(&p).unwrap_or_else(|e| panic!("read {}: {e}", p.display()))
}
#[test]
fn inv22_concurrency_gates_present_and_not_bypassed() {
let repos = src("api/repos.rs");
let smart_http = src("git/smart_http.rs");
let vis = src("git/visibility_pack.rs");
let ipfs = src("api/ipfs.rs");
// U1 / P1-a: run_bounded_git stands the watchdog down only after confirming the
// child actually terminated (WNOWAIT), not on the raw stdout-drain EOF — otherwise
// a child that closes stdout then hangs pins the permit past the deadline. The
// probe is defined and called, so >= 2 occurrences; reverting the fix removes both.
assert!(
vis.matches("child_terminated_without_reaping").count() >= 2,
"U1/P1-a gate missing: run_bounded_git must confirm child exit via \
child_terminated_without_reaping before signalling the watchdog"
);
// U2 / P1-c: on client disconnect KillGroupOnDrop must launch a detached reaper
// that runs the full TERM/grace/KILL/reap, not a lone SIGTERM. The reaper is spawned
// via a runtime handle in Drop; `Handle::try_current` is unique to that launch (the
// timeout path already has an async context and never calls it).
assert!(
smart_http.contains("Handle::try_current"),
"U2/P1-c gate missing: KillGroupOnDrop::drop must launch the full \
TERM/grace/KILL reaper on disconnect (Handle::try_current), not a lone SIGTERM"
);
// U4 / P1-d: git_receive_pack must acquire the per-source write sub-cap before the
// global write permit. The acquire reads `state.git_write_per_caller`; comments name
// the field without the `state.` prefix, so this targets the real acquire site.
assert!(
repos.contains("state.git_write_per_caller"),
"U4/P1-d gate missing: git_receive_pack must acquire the per-source write cap \
(state.git_write_per_caller) before the global write permit"
);
// U5 / P1-e: the detached post-push encryption walk must run through the
// admission-gated helper, which is wired to the shared encrypt pool.
assert!(
repos.contains("fn withheld_recipients_gated")
&& repos.contains("state.git_encrypt_semaphore"),
"U5/P1-e gate missing: the encryption walk must run through \
withheld_recipients_gated, which acquires git_encrypt_semaphore"
);
// U1 / R2 (#173 round-10): the path-scoped filtered-pack serve must thread the
// caller's AdmissionGuard through BOTH git stages so read + per-caller admission is
// held until the pack-objects group is reaped on disconnect, closing the cap bypass
// the plain upload_pack path already fixed. Two load-bearing markers: rev-list hands
// the disarmed guard back (its tuple return type), and build_filtered_pack forwards
// that guard into the pack-objects stage (the `admission` arg after the
// "pack-objects" label). Reverting either — dropping the guard between stages, or
// passing `None` to pack-objects — trips this.
assert!(
smart_http.contains("Result<(Vec<String>, Option<AdmissionGuard>)>")
&& smart_http.contains("\"pack-objects\",\n admission,"),
"U1/R2 gate missing: build_filtered_pack must thread the AdmissionGuard through \
rev-list -> pack-objects so the permits are held until the pack-objects group \
is reaped on disconnect (the path-scoped half of #174 P1-a)"
);
// U2 / R1: a cancelled or timed-out `GET /ipfs/{cid}` must release admission only
// after the blocking work it admitted has finished, not the instant the handler
// future drops (the /ipfs half of #174 P1-a). The mechanism is the shared
// `Arc<WalkAdmission>`: both walk permits are moved into it once per request and a
// clone rides every `spawn_blocking`, so the permits release when the LAST holder
// drops. Reverting to handler-local permits trips this; the per-site clones are
// bound separately by `inv22_ipfs_walk_admission_reaches_every_blocking_site`.
//
// This previously required the permits to be owned by a detached `tokio::spawn`
// running the whole pipeline. That closed the same bypass but kept an abandoned
// request's full legacy scan running against a held slot, so the merge of #173 and
// #174 kept the Arc and dropped the detached task.
assert!(
ipfs.contains("struct WalkAdmission")
&& ipfs.contains("let admission = std::sync::Arc::new(WalkAdmission {"),
"U2/R1 gate missing: get_by_cid must move both /ipfs admission permits into a \
shared Arc<WalkAdmission> whose clones ride the blocking work, so admission is \
released only once that work completes (the /ipfs half of #174 P1-a)"
);
// U2 / KTD2 (#173 round-10): the probe/read children on the /ipfs path must be the
// duration-bounded twins (process-group teardown via run_bounded_git), not the bare
// `store::object_type` / `read_object_content` (or an unbounded `cat-file -s`) a tokio
// timeout cannot cancel — otherwise a wedged cat-file lingers and pins the held
// admission past the deadline. Reverting any twin call site back to a bare read trips
// this.
assert!(
ipfs.contains("object_type_bounded(")
&& ipfs.contains("object_size_bounded(")
&& ipfs.contains("read_object_content_bounded("),
"U2/KTD2 gate missing: the /ipfs probe+read must call the run_bounded_git-backed \
*_bounded twins so a wedged cat-file is reaped at the deadline, not left to pin \
the held /ipfs walk admission"
);
// P1-e non-bypass tripwire: the bounded recipients walk is spawn_blocking'd nowhere
// but inside withheld_recipients_gated. A second call site (count > 1) is a new
// detached git walk that skips the admission gate — exactly the class U5 closed.
assert_eq!(
repos.matches("withheld_blob_recipients_bounded").count(),
1,
"P1-e bypass: the bounded recipients walk must be invoked only inside \
withheld_recipients_gated; a new call site bypasses the encrypt-walk admission cap"
);
// U4 / P2-2: the detached post-push encryption task must be gated by the per-repo
// coalescing set (`encrypt_inflight.try_begin`) so the OUTSTANDING parked-task set is
// bounded to <=1 per repo. Removing the gate lets N rapid pushes spawn N parked
// waiters (the unbounded set U4 closed); the semaphore only caps active walks.
assert!(
repos.contains("state.encrypt_inflight.try_begin"),
"U4/P2-2 gate missing: the detached post-push encryption spawn must consult \
encrypt_inflight.try_begin to coalesce per repo (bound the outstanding-task set)"
);
// F5: coalescing must REQUEUE, not shed. The in-flight task pins only its own
// pre-spawn snapshot, so a coalesced push's tip pairs are recorded on its key and
// the task must loop-drain them (`finish_or_take_pending`) before releasing it.
// Removing the drain reverts to the silent loss: a coalesced push's pins and
// recovery copies are absent until an unrelated later push. Scan only the
// production half of the file — the u5 tests in its `mod tests` also name the
// drain call, and matching them would make this check vacuous.
// Split at the TEST MODULE, not at the first `#[cfg(test)]`. api/repos.rs carries
// test-only items (the drain fault seam, the task entry point) ABOVE the production
// code these gates scan for, so splitting on the attribute truncated the production
// half above every line being checked and the gate went vacuously blind.
let repos_production = repos
.split("\nmod tests {")
.next()
.expect("split always yields a first chunk");
assert!(
repos_production.contains("finish_or_take_pending"),
"F5 gate missing: the post-push encryption task must loop-drain coalesced \
pushes via finish_or_take_pending before releasing its repo key"
);
// F4: every post-receive scan helper admits itself to the shared scan pool via
// `crate::state::acquire_scan_permit` BEFORE its spawn_blocking git walk, so a
// push burst cannot accumulate unbounded concurrent scans once the write permit
// is released. Two halves, both load-bearing: the helper body must actually
// acquire the pool (state.rs sits the helper at the end of the file, so the
// definition tail contains no other `acquire_owned` to match vacuously), and
// within each scan helper the first qualified call precedes the first
// `spawn_blocking`. Severing a call site pushes the next occurrence past the
// helper's own `spawn_blocking` (or off the end of the file), turning the
// assertion red; comments name the helper without the `crate::state::` prefix,
// so this targets the real call sites.
let state_rs = src("state.rs");
let helper_def = state_rs
.find("fn acquire_scan_permit")
.expect("F4 gate missing: state.rs no longer defines acquire_scan_permit");
assert!(
state_rs[helper_def..].contains("acquire_owned"),
"F4 gate gutted: acquire_scan_permit must acquire the scan pool via acquire_owned"
);
let push_delta = src("git/push_delta.rs");
for (file_src, file, helper) in [
(&repos, "api/repos.rs", "fn replication_withheld_set"),
(&repos, "api/repos.rs", "fn fail_closed_full_scan_objects"),
(
&push_delta,
"git/push_delta.rs",
"fn resolve_candidates_for_push",
),
] {
let start = file_src
.find(helper)
.unwrap_or_else(|| panic!("{file}: `{helper}` not found"));
let tail = &file_src[start..];
let acquire = tail
.find("crate::state::acquire_scan_permit(")
.unwrap_or_else(|| {
panic!("F4 gate missing: {file} `{helper}` no longer acquires a scan permit")
});
let spawn = tail.find("spawn_blocking").unwrap_or_else(|| {
panic!("{file}: `{helper}` lost its spawn_blocking walk — update this guard")
});
assert!(
acquire < spawn,
"F4 gate bypassed: {file} `{helper}` must acquire its git_encrypt_semaphore \
permit BEFORE dispatching the blocking git scan"
);
}
}
/// F4 (repo_store advisory-unlock cancellation safety): `RepoWriteGuard::release`
/// must await `pg_advisory_unlock` while `self` still owns the pooled connection,
/// and must not mark itself `released` until that await resolves. Either shape,
/// reintroduced, re-opens the mid-unlock cancellation leak: taking the connection
/// early leaves `Drop` with `conn == None`, and setting `released = true` early
/// leaves the `Drop` backstop inert — both strand the session lock on cancellation.
///
/// Scoped to the `release` fn body: the `Drop` impl legitimately takes the
/// connection and unlocks, so a whole-file scan would match it and read as a false
/// pass. Reverting either ordering turns this red (proven load-bearing).
#[test]
fn f4_release_keeps_conn_owned_until_unlock_resolves() {
let repo_store = src("git/repo_store.rs");
let rel_start = repo_store
.find("pub async fn release(self")
.or_else(|| repo_store.find("pub async fn release(mut self"))
.expect("F4 gate: repo_store.rs no longer defines RepoWriteGuard::release");
let rel_end = repo_store[rel_start..]
.find("impl Drop for RepoWriteGuard")
.map(|off| rel_start + off)
.expect("F4 gate: release fn / Drop impl markers moved — update this guard");
let release_body = &repo_store[rel_start..rel_end];
let unlock = release_body
.find("pg_advisory_unlock")
.expect("F4 gate: release must still issue pg_advisory_unlock");
let before_unlock = &release_body[..unlock];
// (a) the connection must still be owned by `self` at the unlock await.
assert!(
!before_unlock.contains("self.conn.take()"),
"F4 regression: RepoWriteGuard::release takes self.conn BEFORE awaiting \
pg_advisory_unlock. A cancellation during the unlock await then strands the \
session advisory lock (Drop sees conn == None and skips its backstop). \
Unlock through the still-owned connection instead."
);
// (b) `released` must not be set before the unlock await — the other
// reintroduction shape a single-reorder check on (a) alone is blind to.
assert!(
!before_unlock.contains("released = true"),
"F4 regression: RepoWriteGuard::release sets `released = true` BEFORE awaiting \
pg_advisory_unlock. A cancellation during the await then leaves the Drop \
backstop inert (it early-returns on released). Set released only AFTER the \
unlock await resolves."
);
}
/// F6/KTD-5 (IPFS metadata queries deadline-wrapped): `get_by_cid` acquires
/// the scarce walk permits (RAII, held for the whole request) BEFORE its metadata
/// queries, and the per-repo loop's first budget gate runs only later. So
/// both `list_repos_page_for_scan` and `list_visibility_rules_for_repos` must be clamped to the
/// remaining request budget — otherwise a query blocked in Postgres pins the walk slot
/// for the whole stall, past the budget. This scans the PRODUCTION half of `api/ipfs.rs`
/// (the `mod tests` half names the same calls in its own harness and would make the
/// check vacuous) and asserts each query call is immediately preceded by a
/// `tokio::time::timeout(` wrapper. Removing either wrapper turns this red (proven
/// load-bearing).
#[test]
fn f6_ipfs_metadata_queries_are_deadline_wrapped() {
let ipfs = src("api/ipfs.rs");
// Split at the TEST MODULE, not at the first `#[cfg(test)]`: the handler carries
// in-line `#[cfg(test)]` query counters, so splitting on the attribute cut the
// production half off above the calls this guard exists to check and the scan went
// vacuously quiet (found 0 of each rather than failing on a missing wrapper).
let production = ipfs
.split("\nmod tests {")
.next()
.expect("split always yields a first chunk");
// EVERY occurrence must be wrapped, not just the first. The handler now reaches
// these queries from two places (the provenance fast path, one repo at a time, and
// the legacy scan's preload), and an exact-count check would have to be relaxed
// every time a call site is added, which is how a guard quietly stops covering the
// site that matters. Requiring all of them scales with the code instead.
for call in [
".list_repos_page_for_scan(",
".list_visibility_rules_for_repos(",
".get_repo_by_id(",
".is_repo_quarantined(",
] {
let occurrences = production.matches(call).count();
assert!(
occurrences >= 1,
"F6 guard stale: api/ipfs.rs no longer calls `{call}` — update this guard"
);
for (n, (idx, _)) in production.match_indices(call).enumerate() {
// The wrapper opens a few lines above the call (match tokio::time::timeout(
// ... remaining budget ..., <query> )); a 240-char lookback covers it
// without reaching the previous statement.
let window = &production[idx.saturating_sub(240)..idx];
assert!(
window.contains("tokio::time::timeout("),
"F6 gate missing: occurrence {n} of `{call}` (of {occurrences}) is not \
wrapped in tokio::time::timeout(...) clamped to the remaining request \
budget. An unwrapped await pins the held walk permit for the whole DB \
stall, past GITLAWB_IPFS_REQUEST_BUDGET_SECS."
);
}
}
}
/// #174 F2 / KTD-3: the detached post-receive Pinata replication task must enqueue
/// only the push's `(ref, old, new)` tuples and RE-DERIVE its object set inside the
/// worker once a pin slot frees — it must NOT move the full per-push object list into
/// the closure and hold it across the `pin_semaphore` acquire. Retaining the list makes
/// every parked task (under a slow Pinata backend) hold an MB-scale OID list, so
/// outstanding memory grows O(pushes x object-list). Coalescing/shedding the task is
/// forbidden (its per-ref effects are non-idempotent), so the fix bounds the retained
/// data, not the task count.
///
/// Two load-bearing checks, both against the PRODUCTION half of `api/repos.rs` (the
/// `mod tests` half names the same identifiers in its own harness and would make the
/// scan vacuous): (a) the closure-local `object_list_pinata` binding — the retain form —
/// must be gone; reintroducing `let object_list_pinata = object_list;` turns this red.
/// (b) the re-derivation (`pinata_object_list_for_refs`) must appear AFTER the Pinata
/// pin permit is acquired, so the object list is materialized only inside the pin-bounded
/// section and a parked task holds O(ref tuples).
#[test]
fn f2_pinata_enqueues_refs_not_retained_object_lists() {
let repos = src("api/repos.rs");
// Split at the TEST MODULE, not at the first `#[cfg(test)]`. api/repos.rs carries
// test-only items (the drain fault seam, the task entry point) ABOVE the production
// code these gates scan for, so splitting on the attribute truncated the production
// half above every line being checked and the gate went vacuously blind.
let production = repos
.split("\nmod tests {")
.next()
.expect("split always yields a first chunk");
// (a) the retained-list form must be gone from production.
assert!(
!production.contains("object_list_pinata"),
"F2/KTD-3 regression: the Pinata task retains a full per-push object list \
(`object_list_pinata`) across the pin-permit acquire. Enqueue only the ref \
tuples and re-derive the object set inside the worker (pinata_object_list_for_refs)."
);
// (b) the re-derivation runs AFTER the pin permit is acquired. Anchor on the
// Pinata pin-admission clone so the window is the Pinata task, not the sibling
// IPFS/encrypt spawn (which shares `pin_semaphore` but never re-derives).
let anchor = production
.find("let pin_sem_pinata")
.expect("F2 gate stale: the Pinata pin-admission clone (pin_sem_pinata) moved");
let tail = &production[anchor..];
let acquire = tail
.find(".acquire_owned()")
.expect("F2 gate: the Pinata task no longer acquires a pin permit (acquire_owned)");
let rederive = tail.find("pinata_object_list_for_refs(").expect(
"F2 gate missing: the Pinata task must re-derive its object set via \
pinata_object_list_for_refs; it can no longer move a pre-resolved list into the closure",
);
assert!(
acquire < rederive,
"F2 gate bypassed: the Pinata object-set re-derivation must run AFTER the pin \
permit is acquired, so a parked task never holds the MB-scale object list"
);
// The re-derivation is driven by the push's ref tuples (the enqueued unit), not a
// retained object list — tie "enqueue ref_updates" to the call explicitly.
assert!(
tail[rederive..].starts_with("pinata_object_list_for_refs(")
&& tail[rederive..]
.get(..400)
.map(|w| w.contains("ref_updates_clone"))
.unwrap_or(false),
"F2 gate: pinata_object_list_for_refs must re-derive from the push's ref tuples \
(ref_updates_clone), the small unit the parked task retains"
);
}
/// #174 U2 / F3 (second same-repo writer serialized until a disconnected first push's
/// git group is reaped): `git_receive_pack` must take the per-repo in-process write
/// lease and CARRY it on the write-path `AdmissionGuard` (via `.with_lease`), so the
/// lease rides `KillGroupOnDrop`'s detached reaper and frees only after the group is
/// reaped — supplementing the pg advisory lock, which `RepoWriteGuard::Drop` releases at
/// the disconnect instant. Tying the lease to `RepoWriteGuard` instead (or dropping the
/// `.with_lease` carry) reopens the race; the behavioral RED/GREEN test
/// (`f3_second_push_serialized_until_disconnected_group_reaped`) is the real bar, this
/// is the completeness tripwire. Scanned against the PRODUCTION half of `api/repos.rs`
/// (the `mod tests` half names these identifiers in its own harness).
#[test]
fn f3_second_writer_leased_until_reap() {
let repos = src("api/repos.rs");
let smart_http = src("git/smart_http.rs");
// Split at the TEST MODULE, not at the first `#[cfg(test)]`. api/repos.rs carries
// test-only items (the drain fault seam, the task entry point) ABOVE the production
// code these gates scan for, so splitting on the attribute truncated the production
// half above every line being checked and the gate went vacuously blind.
let repos_production = repos
.split("\nmod tests {")
.next()
.expect("split always yields a first chunk");
// The lease is acquired, then carried by the AdmissionGuard (.with_lease), before
// receive_pack runs the write. Severing any of the three turns this red.
let lease_acquire = repos_production.find("repo_write_leases").expect(
"F3 gate missing: git_receive_pack no longer takes the per-repo write lease \
(state.repo_write_leases)",
);
let with_lease = repos_production.find(".with_lease(").expect(
"F3 gate missing: the write-path AdmissionGuard no longer carries the lease \
(.with_lease). The lease must ride the disconnect reaper via the AdmissionGuard, \
NOT RepoWriteGuard (which drops at the disconnect instant, reopening F3).",
);
let receive = repos_production
.find("smart_http::receive_pack(")
.expect("F3 gate stale: git_receive_pack no longer calls smart_http::receive_pack");
assert!(
lease_acquire < with_lease && with_lease < receive,
"F3 gate bypassed: the write lease must be acquired, then carried by the \
AdmissionGuard (.with_lease), BEFORE receive_pack runs the write"
);
// The AdmissionGuard must actually hold the lease (Option<RepoWriteLease>) via a
// with_lease setter, so it travels into the detached reaper. Removing the field or
// method turns this red.
assert!(
smart_http.contains("_lease: Option<crate::state::RepoWriteLease>")
&& smart_http.contains("pub fn with_lease("),
"F3 gate missing: AdmissionGuard must hold an Option<RepoWriteLease> set via \
with_lease, so the lease rides the guard into KillGroupOnDrop's reaper"
);
}
/// #174 U1 — every blocking walk in the `/ipfs` scan carries the request's walk
/// admission.
///
/// The admission is an `Arc<WalkAdmission>` cloned into each `spawn_blocking`
/// closure, so the global + per-source permits release only when the last holder
/// drops: a client disconnect leaves the abandoned closure holding the slot, and a
/// panicking closure leaves the handler holding it.
///
/// Only the walk site runs under `state.git_bin`, so only it can be pinned by the
/// fake-git harness and mutation-verified dynamically (see
/// `get_by_cid_walk_permit_held_through_blocking_walk`). The probe and the content
/// read deliberately shell to the real `git`, which is why this structural check
/// exists: it binds all three sites, and any blocking site added to this loop
/// later, without reversing that deliberate independence.
///
/// MUTATION (RED): delete any one `Arc::clone(ctx.admission)` binding, or drop the
/// clone from inside its closure, and the count falls below three.
#[test]
fn inv22_ipfs_walk_admission_reaches_every_blocking_site() {
let ipfs = src("api/ipfs.rs");
// The shared owner must exist and be built once per request.
assert!(
ipfs.contains("struct WalkAdmission") && ipfs.contains("Arc::new(WalkAdmission {"),
"U1 gate missing: the /ipfs walk admission must be a shared WalkAdmission, \
not a handler-local permit pair"
);
// Every blocking site in the scan takes its own clone...
// `Arc::clone(ctx.admission)` in the gate, `Arc::clone(&admission)` if a site is
// ever added back in the handler body itself; count both spellings.
let clones = ipfs.matches("Arc::clone(ctx.admission)").count()
+ ipfs.matches("Arc::clone(&admission)").count();
assert!(
clones >= 3,
"U1 gate bypassed: expected an admission clone for each of the three \
/ipfs spawn_blocking sites (probe, walk, read); found {clones}. A blocking \
walk that does not hold the admission lets a disconnect or a panic free the \
slot while its git child is still running."
);
// ...and each clone is actually moved INTO the blocking closure, not merely
// created in the async frame (which would hold nothing across the join).
let held = ipfs.matches("let _admission = ").count();
assert!(
held >= 3,
"U1 gate bypassed: each admission clone must be bound inside its \
spawn_blocking closure so the blocking work owns it; found {held} of 3."
);
// The count of blocking sites is itself the thing being covered: if a fourth
// appears, it needs an admission clone too and this gate must be revisited.
let sites = ipfs.matches("spawn_blocking(move ||").count();
assert_eq!(
sites, 3,
"the /ipfs scan grew or lost a spawn_blocking site ({sites} found, expected 3); \
give any new blocking walk its own Arc::clone(&admission) and update this gate"
);
}
/// #174 U5: the post-receive replication tail is spawned at the DURABILITY BOUNDARY,
/// which is the moment receive-pack returns success, not the end of the handler and
/// not after `guard.release()`.
///
/// The tail owes this push its pins, recovery copy, and announcements. Everything
/// below the spawn stays in the cancellable request future, so anything the tail is
/// spawned after is a window where a client disconnect drops that work while the pack
/// is already durable on disk. `guard.release()` is such a window: on success it
/// awaits the Tigris upload and then the advisory unlock.
///
/// The lower bound matters just as much as the upper one: `release` runs on failure
/// too, so an ungated spawn would fire for a push git rejected, pinning and announcing
/// a half-applied repo. Above `release` the `?` on `receive_result` can no longer be
/// what gates it, so the success check is explicit and this gate binds it: the spawn
/// must sit inside `if push_succeeded`, and `release` must consume the same flag so
/// the two cannot drift apart.
///
/// This is an ordering check rather than a cancellation-race test on purpose: it is
/// the companion to `receive_pack_tail_survives_a_disconnect_during_release`, which
/// drives the actual disconnect through a parked `release`. Same instrument the F3
/// gate above uses.
///
/// MUTATION (RED): move the `tokio::spawn(post_receive_replication_tail` call below
/// `guard.release(` and the ordering assertion fails; take it out of the
/// `if push_succeeded` block and the failed-push assertion fails.
#[test]
fn inv22_replication_tail_spawns_at_the_durability_boundary() {
let repos = src("api/repos.rs");
// Production half only — the tests below name these identifiers too.
// Split at the TEST MODULE, not at the first `#[cfg(test)]`. api/repos.rs carries
// test-only items (the drain fault seam, the task entry point) ABOVE the production
// code these gates scan for, so splitting on the attribute truncated the production
// half above every line being checked and the gate went vacuously blind.
let production = repos
.split("\nmod tests {")
.next()
.expect("split always yields a first chunk");
let success_flag = production
.find("let push_succeeded = receive_result.is_ok();")
.expect(
"U5 gate missing: the tail's success gate must be bound from receive_result.is_ok()",
);
let gate_open = production
.find("if push_succeeded {")
.expect("U5 gate missing: the tail spawn must be gated on the push having succeeded");
let spawn = production
.find("tokio::spawn(post_receive_replication_tail(")
.expect("U5 gate missing: the replication tail must be spawned by git_receive_pack");
let release = production
.find(".release(push_succeeded)")
.expect("U5 gate stale: release must consume the same success flag as the tail gate");
let touch = production
.find("state.db.touch_repo(")
.expect("U5 gate stale: git_receive_pack no longer calls touch_repo");
let webhook = production
.find("webhooks::fire_event(")
.expect("U5 gate stale: git_receive_pack no longer fires push webhooks");
assert!(
success_flag < gate_open && gate_open < spawn,
"U5 gate bypassed: the tail must be spawned inside `if push_succeeded`, or a \
rejected push spawns a tail that pins and announces a half-applied repo"
);
// Still inside that block: no `}` may close it between the gate and the spawn.
assert!(
!production[gate_open + "if push_succeeded {".len()..spawn].contains('}'),
"U5 gate bypassed: the tail spawn left the `if push_succeeded` block, so a \
rejected push now spawns a tail"
);
assert!(
spawn < release && spawn < touch && spawn < webhook,
"U5 gate bypassed: the tail must be spawned BEFORE guard.release, touch_repo \
and the webhook fan-out, so a disconnect in any of those windows cannot drop \
this push's pins, recovery copy, and announcements"
);
}
+2 -2
View File
@@ -1,7 +1,7 @@
[package]
name = "gl"
description = "The gitlawb CLI — identity management, node control, MCP server"
version = "0.7.1" # x-release-please-version
version = "0.6.0" # x-release-please-version
edition.workspace = true
rust-version.workspace = true
license.workspace = true
@@ -11,7 +11,7 @@ name = "gl"
path = "src/main.rs"
[dependencies]
gitlawb-core = { path = "../gitlawb-core", features = ["redirect"] }
gitlawb-core = { path = "../gitlawb-core" }
icaptcha-client = { path = "../icaptcha-client" }
base64 = { workspace = true }
tokio = { workspace = true }
+8 -259
View File
@@ -12,58 +12,9 @@ use serde::Deserialize;
use std::path::Path;
use std::process::Command;
use crate::http::{sanitize_node_msg, NodeClient};
use crate::http::NodeClient;
use crate::identity::load_keypair_from_dir;
/// Report why one candidate blob could not be recovered, in the register of the
/// recovery loop's other warnings.
///
/// Both halves are gateway-supplied text on their way to a terminal: the oid comes
/// out of a manifest fetched from a caller-chosen Arweave gateway, and a transport
/// error carries the URL the same manifest chose. So the whole line is defanged and
/// length-capped, like every other node/gateway string this crate prints.
///
/// Under `cfg(test)` the line is also mirrored into a per-thread buffer, so a test
/// can assert the loop actually says what it skipped rather than only that it
/// returned nothing.
fn warn_skip(oid: &str, why: &str) {
emit_warning(&format!(
"warning: could not fetch encrypted blob {oid}: {why}; skipping"
));
}
/// The same report for a manifest rather than a blob. The manifest is one step
/// earlier in the same recovery: losing it silently means every blob it would have
/// named is missing with no reason given anywhere.
fn warn_skip_manifest(id: &str, why: &str) {
emit_warning(&format!(
"warning: could not fetch blob manifest {id}: {why}; skipping"
));
}
/// Sanitize a warning and write it, once.
///
/// The write goes to [`warn_sink`], which is stderr in a normal build and the tests'
/// per-thread mirror under `cfg(test)`. That indirection is the point: the mirror
/// used to sit BESIDE an `eprintln!`, so deleting the user-visible write left every
/// assertion on the mirror green and the shipped behaviour could be removed with the
/// tests unchanged. There is one write now, and the tests observe it.
fn emit_warning(line: &str) {
use std::io::Write;
let line = sanitize_node_msg(line);
let _ = writeln!(warn_sink(), "{line}");
}
#[cfg(not(test))]
fn warn_sink() -> impl std::io::Write {
std::io::stderr()
}
#[cfg(test)]
fn warn_sink() -> impl std::io::Write {
tests::WarnMirror
}
#[derive(Args)]
pub struct CloneArgs {
/// Repo to clone: gitlawb://<owner_did>/<name> or <owner>/<name>.
@@ -361,25 +312,10 @@ async fn recover_encrypted_blobs(
.await
{
Ok(r) if r.status().is_success() => r,
// The node path has the same silent exit the gateway path had: a 403, a
// 404, or a dead connection all reached the caller as "blob not
// recoverable" with no reason attached. One unreachable blob still must
// not end the recovery of the rest, so it warns and moves on.
Ok(r) => {
warn_skip(oid, &format!("node returned {}", r.status()));
continue;
}
Err(e) => {
warn_skip(oid, &format!("node request failed: {e}"));
continue;
}
_ => continue,
};
let envelope = match env_resp.bytes().await {
Ok(b) => b,
Err(e) => {
warn_skip(oid, &format!("reading the envelope failed: {e}"));
continue;
}
let Ok(envelope) = env_resp.bytes().await else {
continue;
};
let plaintext = match open_blob(&envelope, keypair) {
Ok(p) => p,
@@ -706,14 +642,7 @@ async fn recover_from_arweave(
for r in refs {
let m = match client.get(format!("{ag}/{}", r.id)).send().await {
Ok(resp) if resp.status().is_success() => resp,
Ok(resp) => {
warn_skip_manifest(&r.id, &format!("gateway returned {}", resp.status()));
continue;
}
Err(e) => {
warn_skip_manifest(&r.id, &format!("gateway request failed: {e}"));
continue;
}
_ => continue,
};
if let Ok(parsed) = m.json::<Manifest>().await {
manifests.push((parsed, r.height));
@@ -755,31 +684,10 @@ async fn recover_from_arweave(
}
let env_resp = match client.get(format!("{ig}/ipfs/{cid}")).send().await {
Ok(r) if r.status().is_success() => r,
// Every other outcome used to leave through a bare `continue`, so a
// gateway that answered 503, 404, or nothing at all was reported to the
// caller as "blob not recoverable" with no reason attached. This is the
// second in-repo client of GET /ipfs/{cid} and it has no resume ladder,
// so saying what happened is all the recourse there is. It still skips to
// the next candidate either way: one unreachable blob must not end the
// recovery of the rest.
Ok(r) => {
warn_skip(&oid, &format!("gateway returned {}", r.status()));
continue;
}
Err(e) => {
warn_skip(&oid, &format!("gateway request failed: {e}"));
continue;
}
_ => continue,
};
// A body that dies part-way through is the same mid-read failure the capped
// read exists to stop rendering as silence, and it sits INSIDE the loop whose
// status arms were already fixed.
let envelope = match env_resp.bytes().await {
Ok(b) => b,
Err(e) => {
warn_skip(&oid, &format!("reading the envelope failed: {e}"));
continue;
}
let Ok(envelope) = env_resp.bytes().await else {
continue;
};
// open_blob succeeds only if this caller is a recipient: this is the
// authorization gate (no node, no DID check needed).
@@ -893,42 +801,9 @@ pub async fn run(args: CloneArgs) -> Result<()> {
#[cfg(test)]
mod tests {
use super::*;
use std::cell::RefCell;
use std::process::Command;
use tempfile::TempDir;
thread_local! {
/// Test-visible copy of the stderr warnings `warn_skip` emits.
/// `#[tokio::test]` runs the future on the test's own thread, so a
/// thread-local is enough.
static WARNINGS: RefCell<String> = const { RefCell::new(String::new()) };
}
/// The `cfg(test)` warning sink. `emit_warning` writes here instead of to
/// stderr, so the assertions below observe the shipped write rather than a copy
/// made beside it: delete that write and they go red.
pub(super) struct WarnMirror;
impl std::io::Write for WarnMirror {
fn write(&mut self, buf: &[u8]) -> std::io::Result<usize> {
let text = String::from_utf8_lossy(buf).into_owned();
WARNINGS.with(|w| w.borrow_mut().push_str(&text));
Ok(buf.len())
}
fn flush(&mut self) -> std::io::Result<()> {
Ok(())
}
}
fn reset_warnings() {
WARNINGS.with(|w| w.borrow_mut().clear());
}
fn warnings() -> String {
WARNINGS.with(|w| w.borrow().clone())
}
fn g(args: &[&str], dir: &Path) {
assert!(Command::new("git")
.args(args)
@@ -1380,132 +1255,6 @@ mod tests {
assert_eq!(a.get("o1").map(String::as_str), Some("cidTS"));
}
/// A gateway that refuses one blob must SAY so, and must not take the rest of the
/// recovery down with it.
///
/// The IPFS fetch used to leave through a bare `_ => continue` on every
/// non-success, so a 503 (the truncation status a tuned node makes more likely),
/// a 404, and a dead connection all reached the caller as the same silent "blob
/// not recoverable". Two withheld blobs here: the gateway answers 503 for the
/// first and serves the second. The recovered path proves the loop continued; the
/// warning proves the skip was reported and names both the object and the status.
#[tokio::test]
async fn recover_from_arweave_reports_a_refusing_gateway_and_continues() {
use gitlawb_core::encrypt::seal_blob;
use gitlawb_core::identity::Keypair;
reset_warnings();
let (td, url) = bare_remote(&[
("public/a.txt", b"pub\n"),
("secret/b.txt", b"SECRET B\n"),
("secret/c.txt", b"SECRET C\n"),
]);
let dest = td.path().join("dest");
let bare = url.strip_prefix("file://").unwrap();
assert!(Command::new("git")
.args(["-C", bare, "config", "uploadpack.allowFilter", "true"])
.status()
.unwrap()
.success());
setup_partial_clone(&dest, &url, &["/secret/**".to_string()], &[], None).unwrap();
let oid_of = |path: &str| {
let out = Command::new("git")
.args([
"-C",
dest.to_str().unwrap(),
"rev-parse",
&format!("HEAD:{path}"),
])
.output()
.unwrap();
String::from_utf8_lossy(&out.stdout).trim().to_string()
};
let refused_oid = oid_of("secret/b.txt");
let served_oid = oid_of("secret/c.txt");
// Origin death, so recovery has to go through the gateways.
std::fs::remove_dir_all(bare).unwrap();
let reader = Keypair::generate();
let envelope = seal_blob(b"SECRET C\n", &[reader.verifying_key()]).unwrap();
let refused_cid = "cidrefused";
let served_cid = "cidserved";
let mut server = mockito::Server::new_async().await;
let _gql = server
.mock("POST", "/graphql")
.with_status(200)
.with_header("content-type", "application/json")
.with_body(r#"{"data":{"transactions":{"edges":[{"node":{"id":"TX1"}}]}}}"#)
.create_async()
.await;
let manifest_body = serde_json::json!({
"timestamp": "2026-06-11T00:00:00Z",
"blobs": [
{ "oid": refused_oid, "cid": refused_cid },
{ "oid": served_oid, "cid": served_cid },
],
})
.to_string();
let _tx = server
.mock("GET", "/TX1")
.with_status(200)
.with_header("content-type", "application/json")
.with_body(manifest_body)
.create_async()
.await;
let refusal = server
.mock("GET", format!("/ipfs/{refused_cid}").as_str())
.with_status(503)
.with_header("content-type", "application/json")
.with_body(r#"{"error":"search_incomplete","message":"scan truncated"}"#)
.expect(1)
.create_async()
.await;
let ok = server
.mock("GET", format!("/ipfs/{served_cid}").as_str())
.with_status(200)
.with_body(envelope)
.expect(1)
.create_async()
.await;
let paths = recover_from_arweave(
&server.url(),
&server.url(),
"alice",
"myrepo",
&dest,
&reader,
)
.await
.unwrap();
refusal.assert_async().await;
ok.assert_async().await;
assert_eq!(
paths,
vec!["secret/c.txt".to_string()],
"a refused candidate must not stop the loop reaching the next one"
);
let warnings = warnings();
assert!(
warnings.contains(&refused_oid),
"the skip must name the object it could not fetch, got: {warnings}"
);
assert!(
warnings.contains("503"),
"the skip must name the gateway's status rather than failing silently, \
got: {warnings}"
);
assert!(
!warnings.contains(&served_oid),
"the blob that was served must not be reported as skipped, got: {warnings}"
);
}
/// Read-path end-to-end over a mocked Arweave + IPFS gateway: discover the
/// manifest via GraphQL, fetch it, fetch the envelope, decrypt with the
/// caller's key, and install the previously-withheld blob.
+1 -682
View File
@@ -15,47 +15,6 @@ use icaptcha_client::IcaptchaCfg;
/// (absorbs proof expiry / first-seen replay).
const MAX_ICAPTCHA_RETRIES: usize = 2;
/// Total request timeout: from the start of connecting through the end of the
/// response body, so it bounds a slow download and not just a slow handshake.
const TOTAL_REQUEST_TIMEOUT: std::time::Duration = std::time::Duration::from_secs(30);
/// Follow a redirect only when it stays on the origin that issued it AND re-issues the
/// identical request-target, and only for as long as the chain bound allows.
///
/// The decision itself is [`gitlawb_core::redirect::may_follow`], shared with
/// `git-remote-gitlawb` so the two signing clients cannot drift apart on it. Refusal
/// is `stop`, not `error`: the 3xx comes back as an ordinary response and each caller
/// reports it through the status path it already has.
///
/// `Policy::custom` replaces reqwest's built-in limit, so the chain bound is restated
/// here. Same-origin redirects can cycle, and this is what stops a node answering 302
/// to itself from being followed indefinitely. It is not what makes the request
/// finite: `.timeout(...)` on the same builder is a TOTAL request timeout covering the
/// whole chain, so without this bound the worst case is a 30 second spin, not an
/// endless one. The bound is what keeps that spin from costing the node a request per
/// round trip for the full 30 seconds.
///
/// `>` and not `>=`: reqwest pushes the redirecting URL onto `previous` before
/// consulting the policy, so on the first redirect `previous.len()` is already 1, and
/// `>=` would permit `MAX_REDIRECTS - 1` follows. `Policy::limited(max)` refuses at
/// `previous.len() > max`, and matching it is the point of reusing its value.
fn same_origin_redirect(attempt: reqwest::redirect::Attempt<'_>) -> reqwest::redirect::Action {
let Some(previous) = attempt.previous().last() else {
// No previous URL to compare against. Unreachable through reqwest, which
// pushes the redirecting URL before consulting the policy, but the safe
// reading of "cannot prove same-origin" is to refuse.
return attempt.stop();
};
if attempt.previous().len() > gitlawb_core::redirect::MAX_REDIRECTS {
return attempt.stop();
}
if gitlawb_core::redirect::may_follow(previous, attempt.url()) {
attempt.follow()
} else {
attempt.stop()
}
}
pub struct NodeClient {
inner: reqwest::Client,
pub node_url: String,
@@ -64,20 +23,8 @@ pub struct NodeClient {
impl NodeClient {
pub fn new(node_url: impl Into<String>, keypair: Option<Keypair>) -> Self {
Self::with_timeout(node_url, keypair, TOTAL_REQUEST_TIMEOUT)
}
/// `new` with the total request timeout as a parameter, so a test can drive the
/// timeout's behaviour without waiting out the shipped value. `new` supplies the
/// shipped one, which is what makes the scaled-down test cover the real client.
fn with_timeout(
node_url: impl Into<String>,
keypair: Option<Keypair>,
timeout: std::time::Duration,
) -> Self {
let inner = reqwest::Client::builder()
.timeout(timeout)
.redirect(reqwest::redirect::Policy::custom(same_origin_redirect))
.timeout(std::time::Duration::from_secs(30))
.user_agent(format!("gl/{} gitlawb-cli", env!("CARGO_PKG_VERSION")))
.build()
.expect("failed to build HTTP client");
@@ -256,79 +203,6 @@ async fn obtain_proof(cfg: IcaptchaCfg) -> Result<String> {
.context("iCaptcha solver task panicked")?
}
/// What a capped body read produced. `text` is the bytes that arrived; the two flags
/// say why the read stopped where it did, which a caller that CLASSIFIES on the body
/// cannot work out from the text alone.
pub(crate) struct CappedBody {
/// The bytes read, lossily decoded.
pub(crate) text: String,
/// The cap cut the body short.
pub(crate) truncated: bool,
/// A chunk read FAILED part-way through, so the body is not merely short, it is
/// unfinished and the node may have had more to say.
pub(crate) read_failed: bool,
}
/// Read at most `cap` bytes of a response body. Bounds the allocation from a
/// hostile or broken node returning a huge error body — the display is capped
/// separately, but the read itself must not be unbounded (INV-6, read half).
///
/// `truncated` reports whether the cap cut the body short. A caller that CLASSIFIES
/// on the body needs it: a cut body fails JSON parse, and a parse failure is
/// indistinguishable from a node that sent no code at all, so without this flag an
/// oversized body silently picks a different arm.
///
/// `read_failed` reports the other way a body can end early. A mid-body read error
/// used to leave through the same exit as a clean end of stream, so a 500 whose body
/// died in transit surfaced as an empty message and the caller was told
/// `node returned 500: ` with nothing after the colon. That is a report of what the
/// node said, and the node never got to say it.
pub(crate) async fn read_body_capped(mut resp: reqwest::Response, cap: usize) -> CappedBody {
let mut buf: Vec<u8> = Vec::new();
let mut truncated = false;
let mut read_failed = false;
while buf.len() < cap {
match resp.chunk().await {
Ok(Some(chunk)) => {
let take = (cap - buf.len()).min(chunk.len());
buf.extend_from_slice(&chunk[..take]);
if take < chunk.len() {
truncated = true;
break; // hit the cap mid-chunk
}
}
Ok(None) => break, // clean end of body
Err(_) => {
read_failed = true;
break;
}
}
}
// A body that lands exactly on the cap may or may not have more behind it;
// report it as cut, since the classification that follows cannot tell either.
if buf.len() >= cap {
truncated = true;
}
CappedBody {
text: String::from_utf8_lossy(&buf).into_owned(),
truncated,
read_failed,
}
}
/// Strip terminal-dangerous characters from (and cap the length of) a
/// node-supplied error string before surfacing it. The node a caller talks to
/// could be hostile and embed escape sequences in its error body; those must not
/// reach the terminal verbatim (INV-6). We drop the C0/C1 control bytes (which
/// defangs ANSI/OSC escapes) AND the Unicode bidi/format controls (which
/// `char::is_control` does not cover — they can reorder the displayed line).
pub(crate) fn sanitize_node_msg(s: &str) -> String {
s.chars()
.filter(|c| !c.is_control() && !gitlawb_core::sanitize::is_bidi_format(*c))
.take(200)
.collect()
}
#[cfg(test)]
mod tests {
use super::*;
@@ -713,559 +587,4 @@ mod tests {
ic.challenge.assert();
ic.answer.assert();
}
// ── redirect policy ─────────────────────────────────────────────────
/// The signed headers must not survive a redirect off the node's origin.
///
/// reqwest strips only `Authorization`, `Cookie`, `Proxy-Authorization` and
/// `WWW-Authenticate` across hosts, so `Signature` and `Signature-Input` used to
/// ride a 302 straight to whatever origin the node named. The signature binds
/// `@method`, `@path` and `content-digest` and nothing about the authority, so the
/// receiving host holds a credential that reads path-scoped objects as the caller
/// at any node until the clock-skew window closes.
///
/// Two mockito servers are two ports on one host, which is exactly the boundary
/// this policy draws (and the one reqwest's own header stripping draws). The
/// second server answers everything and expects nothing: a followed redirect
/// fails the expectation whether or not the signature came with it. MUTATION
/// (RED): drop the `.redirect(...)` line and the second server is hit.
#[tokio::test]
async fn cross_origin_redirect_is_not_followed() {
let mut elsewhere = Server::new_async().await;
let never = elsewhere
.mock("GET", mockito::Matcher::Any)
.with_status(200)
.with_body("bytes from the redirect target")
.expect(0)
.create_async()
.await;
let signature_seen = elsewhere
.mock("GET", mockito::Matcher::Any)
.match_header("signature", mockito::Matcher::Any)
.with_status(200)
.expect(0)
.create_async()
.await;
let mut node = Server::new_async().await;
let bounce = node
.mock("GET", "/api/v1/thing")
.with_status(302)
.with_header("location", &format!("{}/api/v1/thing", elsewhere.url()))
.expect(1)
.create_async()
.await;
let client = NodeClient::new(node.url(), Some(test_keypair()));
let resp = client.get_signed("/api/v1/thing").await.unwrap();
assert_eq!(
resp.status(),
302,
"a refused redirect stops rather than errors, so the caller sees the 3xx \
and reports it through the status path it already has"
);
let body = resp.text().await.unwrap();
assert!(
!body.contains("bytes from the redirect target"),
"the redirect target's bytes must never reach the caller, got: {body}"
);
bounce.assert_async().await;
never.assert_async().await;
signature_seen.assert_async().await;
}
/// A node redirecting to itself is same-origin, so the origin predicate follows it
/// every time and only the chain bound ends the loop. Deleting the bound left the
/// whole suite green, because nothing here had ever built a cycle.
///
/// It has a second job now. A redirect back to the identical path and query is the
/// only same-origin hop the predicate still follows, so the eleven hits below are
/// also this crate's executed proof that such a hop IS followed. The old positive
/// fixture drove a trailing-slash rewrite, which the request-target rule refuses,
/// and an http-to-https upgrade cannot be mocked over mockito's plain http.
///
/// The route answers 301 pointing back at itself. Bounded, the handler is hit
/// once for the original request plus `MAX_REDIRECTS` follows and the call returns
/// the 301 (a refused redirect stops rather than errors). Unbounded, it runs until
/// the client's total request timeout cuts it off, which is a 30 second spin at the
/// shipped value and a request per round trip for the node.
///
/// MUTATION (RED): delete the `previous().len()` check.
#[tokio::test]
async fn a_self_redirect_stops_at_the_chain_bound() {
let mut node = Server::new_async().await;
let hits = std::sync::Arc::new(std::sync::atomic::AtomicUsize::new(0));
let h = hits.clone();
let loop_route = node
.mock("GET", "/api/v1/loop")
.with_status(301)
.with_header("location", "/api/v1/loop")
.with_body_from_request(move |_req| {
h.fetch_add(1, std::sync::atomic::Ordering::SeqCst);
Vec::new()
})
.expect_at_least(1)
.create_async()
.await;
let client = NodeClient::with_timeout(
node.url(),
Some(test_keypair()),
std::time::Duration::from_secs(5),
);
let resp = client
.get_signed("/api/v1/loop")
.await
.expect("the bound must end the chain, not the timeout");
assert_eq!(
resp.status(),
301,
"the chain ends by refusing the next hop, so the last 3xx is what comes back"
);
assert_eq!(
hits.load(std::sync::atomic::Ordering::SeqCst),
gitlawb_core::redirect::MAX_REDIRECTS + 1,
"one original request plus MAX_REDIRECTS follows, matching what \
Policy::limited(MAX_REDIRECTS) would have permitted"
);
loop_route.assert_async().await;
}
/// A same-origin hop that REWRITES the request-target is refused, even though the
/// origin never changes.
///
/// The signature binds `@path` as the literal path-and-query the client signed, and
/// the node rebuilds it from the URI it received. A `/api/v1/thing` to
/// `/api/v1/thing/` bounce therefore presents a signature over a target the node
/// never saw, and every signed read behind such a proxy 401s. Refusing the hop
/// hands the caller the 3xx that names what happened instead.
///
/// The target mock expects zero hits and is asserted: mockito only checks
/// `.expect(N)` when `.assert()` runs, so an unbound or unasserted mock passes
/// vacuously.
#[tokio::test]
async fn same_origin_path_changing_redirect_is_refused() {
let mut node = Server::new_async().await;
let bounce = node
.mock("GET", "/api/v1/thing")
.with_status(301)
.with_header("location", "/api/v1/thing/")
.expect(1)
.create_async()
.await;
let target = node
.mock("GET", "/api/v1/thing/")
.with_status(200)
.with_body("bytes from the rewritten target")
.expect(0)
.create_async()
.await;
let client = NodeClient::new(node.url(), Some(test_keypair()));
let resp = client.get_signed("/api/v1/thing").await.unwrap();
assert_eq!(
resp.status(),
301,
"a refused redirect stops rather than errors, so the caller sees the 3xx \
and reports it through the status path it already has"
);
let body = resp.text().await.unwrap();
assert!(
!body.contains("bytes from the rewritten target"),
"the rewritten target's bytes must never reach the caller, got: {body}"
);
bounce.assert_async().await;
target.assert_async().await;
}
// ── the node's own verification, run over what the client actually sent ──
/// What the verifying mock made of a request that reached it.
///
/// The empty slot (`None`) is its own state and means the mock was never hit at
/// all, which is what the refusal test asserts. It must stay distinguishable from
/// [`Verdict::WrongIdentity`], because "nobody verified anything" and "something
/// verified against the wrong key" are opposite findings.
///
/// The payloads are read through `Debug` in the assertion messages and nowhere
/// else, which the dead-code pass does not count; they carry the detail that makes
/// a failure legible, so they stay.
#[derive(Debug)]
#[allow(dead_code)]
enum Verdict {
/// The chain accepted the signature AND the key it resolved is the test's DID.
Accepted,
/// The chain refused it. Carries the error so a failure reads as the actual
/// rejection rather than a bare hit count.
Rejected(String),
/// The chain accepted a signature made by somebody else. A key resolved from
/// the parsed `key_id` is read out of the artifact under verification, so an
/// accept on it alone proves consistency, never authenticity.
WrongIdentity { expected: String, got: String },
}
/// The node's `require_signature` verification, over a request this crate did not
/// build: parse the headers, recompute the content-digest from the body, rebuild
/// the signing string over `@method`/`@path`/`content-digest`, Ed25519-verify.
/// Returns the DID the signature resolved to, so a caller can pin the identity.
///
/// A hand-copy of its twin in `crates/git-remote-gitlawb/src/main.rs`, which gl
/// cannot import (`git-remote-gitlawb` is a binary crate and this is its test
/// module). Keep the two textually identical apart from the mockito seam around
/// them, so an edit to one is visibly an edit to both.
///
/// The production verifier both copies mirror is `crate::auth::require_signature` in
/// `crates/gitlawb-node/src/auth/mod.rs`. This is a re-implementation, not a call, so
/// an edit to that middleware has to land here too: otherwise the copies drift and
/// this test keeps passing against a rule the node has stopped applying.
///
/// It asserts internally, which is deliberate but constrains its callers: inside
/// `with_body_from_request` those assertions fire on the server thread and reach
/// the client as a transport error, not as a recorded verdict. So the identity
/// check lives in the caller as a [`Verdict`] variant, never as an assert in here.
fn node_verifies(
method: &str,
path_and_query: &str,
body: &[u8],
sig_input: &str,
sig_header: &str,
content_digest: &str,
) -> anyhow::Result<String> {
use gitlawb_core::http_sig::{build_signing_string, compute_content_digest, HttpSignature};
use gitlawb_core::identity::verify;
use std::collections::HashMap;
let sig = HttpSignature::parse(sig_input, sig_header)?;
sig.check_created()?;
assert!(
sig.missing_components().is_empty(),
"signature must cover all required components"
);
assert_eq!(sig.alg, "ed25519");
assert_eq!(
content_digest,
compute_content_digest(body),
"content-digest must match the body"
);
let vk = sig.key_id.to_verifying_key()?;
let mut values = HashMap::new();
values.insert("@method".to_string(), method.to_uppercase());
values.insert("@path".to_string(), path_and_query.to_string());
values.insert("content-digest".to_string(), content_digest.to_string());
let sig_params_value = sig_input.strip_prefix("sig1=").unwrap_or(sig_input);
let components: Vec<&str> = sig.components.iter().map(String::as_str).collect();
let signing_string = build_signing_string(&components, sig_params_value, &values)?;
let sig_array: [u8; 64] = sig.signature_bytes.as_slice().try_into()?;
verify(&vk, signing_string.as_bytes(), &sig_array)?;
Ok(sig.key_id.to_string())
}
/// Pull a header value off a received mockito request, or explain which one the
/// client failed to send.
fn received_header(req: &mockito::Request, name: &str) -> String {
req.header(name)
.first()
.unwrap_or_else(|| panic!("the client sent no {name} header"))
.to_str()
.unwrap()
.to_string()
}
/// Run [`node_verifies`] over a GET that arrived at the mock and record what the
/// node would have made of it, pinned to `expected_did`.
fn record_get_verdict(
req: &mockito::Request,
expected_did: &str,
slot: &std::sync::Arc<std::sync::Mutex<Option<Verdict>>>,
) {
let verdict = match node_verifies(
"GET",
req.path_and_query(),
b"",
&received_header(req, "signature-input"),
&received_header(req, "signature"),
&received_header(req, "content-digest"),
) {
Ok(did) if did == expected_did => Verdict::Accepted,
Ok(did) => Verdict::WrongIdentity {
expected: expected_did.to_string(),
got: did,
},
Err(e) => Verdict::Rejected(e.to_string()),
};
*slot.lock().unwrap() = Some(verdict);
}
/// The finding's repro, now a guard: a rewritten same-origin target must never
/// receive the signature, and the proof is the node's own verification, not a hit
/// count.
///
/// The target mock runs the full `require_signature` chain over the request it
/// receives. Post-fix the hop is refused, so the slot stays empty. Pre-fix the hop
/// is followed and the slot records the Ed25519 rejection of a signature made over
/// `/api/v1/thing` and presented at `/api/v1/thing/`, which is the 401 an operator
/// behind such a proxy actually sees. The verdict is asserted first, so a failure
/// speaks about verification rather than about reachability.
///
/// Its paired positive control is
/// `a_direct_signed_get_verifies_under_the_node_verifier`: without it, an empty
/// slot would be satisfied just as well by a harness that can never record
/// anything.
#[tokio::test]
async fn a_rewritten_target_never_receives_the_signature() {
let kp = test_keypair();
let expected_did = kp.did().to_string();
let slot = std::sync::Arc::new(std::sync::Mutex::new(None::<Verdict>));
let mut node = Server::new_async().await;
let bounce = node
.mock("GET", "/api/v1/thing")
.with_status(301)
.with_header("location", "/api/v1/thing/")
.expect(1)
.create_async()
.await;
let recorder = slot.clone();
let did_for_target = expected_did.clone();
let target = node
.mock("GET", "/api/v1/thing/")
.with_status(200)
.with_body_from_request(move |req| {
record_get_verdict(req, &did_for_target, &recorder);
Vec::new()
})
.expect(0)
.create_async()
.await;
let client = NodeClient::new(node.url(), Some(kp));
let resp = client.get_signed("/api/v1/thing").await.unwrap();
let verdict = slot.lock().unwrap().take();
assert!(
verdict.is_none(),
"the node's own verifier must never see this request: the signature covers \
/api/v1/thing and the rewritten target is /api/v1/thing/, so what arrives \
there is a stale request-target; recorded verdict: {verdict:?}"
);
assert_eq!(
resp.status(),
301,
"the caller sees the 3xx, not the rewritten target's answer"
);
bounce.assert_async().await;
target.assert_async().await;
}
/// The positive control for the test above, and the proof that the client signs
/// the query it sends.
///
/// A direct signed GET, no redirect anywhere, through the same verifying mock. The
/// verdict must be `Accepted`, which is what makes the refusal test's empty slot
/// attributable to the refusal rather than to a harness that cannot record. The
/// path carries a query, so a client that signed the bare path would land here as
/// `Rejected`.
#[tokio::test]
async fn a_direct_signed_get_verifies_under_the_node_verifier() {
let kp = test_keypair();
let expected_did = kp.did().to_string();
let slot = std::sync::Arc::new(std::sync::Mutex::new(None::<Verdict>));
let mut node = Server::new_async().await;
let recorder = slot.clone();
let did_for_target = expected_did.clone();
let route = node
.mock("GET", "/api/v1/thing?x=1")
.with_status(200)
.with_body_from_request(move |req| {
record_get_verdict(req, &did_for_target, &recorder);
Vec::new()
})
.expect(1)
.create_async()
.await;
let client = NodeClient::new(node.url(), Some(kp));
let resp = client.get_signed("/api/v1/thing?x=1").await.unwrap();
assert_eq!(resp.status(), 200);
let verdict = slot.lock().unwrap().take();
assert!(
matches!(verdict, Some(Verdict::Accepted)),
"a direct signed GET must verify under the node's own chain and resolve to \
{expected_did}, or the refusal test's empty slot proves nothing; recorded \
verdict: {verdict:?}"
);
route.assert_async().await;
}
// ── read_body_capped ────────────────────────────────────────────────
/// A body whose read FAILS mid-stream must be distinguishable from a body that
/// ended. Both used to leave through the same `_ => break`, so a 500 whose body
/// died in transit produced an empty message and the caller was told
/// `node returned 500: ` with nothing after the colon.
///
/// The fixture is a raw listener that promises 64 bytes in `Content-Length`,
/// writes 5, and closes. mockito cannot express that: it always completes the
/// response it advertises. MUTATION (RED): restore the single `_ => break` arm
/// (or hard-code `read_failed: false`) and the flag reads false.
#[tokio::test]
async fn read_body_capped_flags_a_mid_body_read_failure() {
let addr = spawn_short_body_listener().await;
let resp = reqwest::get(format!("http://{addr}/truncated"))
.await
.expect("headers arrive before the body is cut");
let body = read_body_capped(resp, 8192).await;
assert!(
body.read_failed,
"a body cut off mid-stream must be reported as a failed read, not as a \
body that ended: got {:?}",
body.text
);
assert!(
!body.truncated,
"the cap did not cut this one; 5 bytes are nowhere near 8 KiB"
);
}
/// The must-not half: a body that ends cleanly must NOT be flagged, or the flag
/// means nothing and every terminal starts claiming the node went quiet.
#[tokio::test]
async fn read_body_capped_does_not_flag_a_clean_body() {
let mut server = Server::new_async().await;
let _m = server
.mock("GET", "/ok")
.with_status(500)
.with_body("node said this")
.create_async()
.await;
let resp = reqwest::get(format!("{}/ok", server.url())).await.unwrap();
let body = read_body_capped(resp, 8192).await;
assert_eq!(body.text, "node said this");
assert!(!body.read_failed, "a complete body is not a failed read");
assert!(!body.truncated, "a complete body is not a truncated one");
}
/// Answer one request with headers promising more body than gets written, then
/// close the connection. Returns the listener's address.
async fn spawn_short_body_listener() -> std::net::SocketAddr {
use tokio::io::{AsyncReadExt, AsyncWriteExt};
let listener = tokio::net::TcpListener::bind("127.0.0.1:0").await.unwrap();
let addr = listener.local_addr().unwrap();
tokio::spawn(async move {
let (mut sock, _) = listener.accept().await.unwrap();
let mut scratch = [0u8; 1024];
let _ = sock.read(&mut scratch).await;
let _ = sock
.write_all(b"HTTP/1.1 500 Internal Server Error\r\nContent-Length: 64\r\n\r\nshort")
.await;
let _ = sock.flush().await;
// Drop closes the socket with 59 of the promised bytes never sent.
});
addr
}
// ── total request timeout ───────────────────────────────────────────
/// The client's timeout is a TOTAL request timeout, so it bounds a download and
/// not just the handshake. `gl ipfs get`'s documentation leans on exactly that:
/// the wall-clock deadline covers the search and deliberately stops at the
/// response headers, and this timeout is the only thing left bounding the body.
///
/// Driven at 250ms through `with_timeout`, the seam `new` itself calls with
/// `TOTAL_REQUEST_TIMEOUT`, because a test at the shipped 30s has no place in this
/// suite. What that costs is the value; what it proves is the SHAPE, which is the
/// part in doubt: that the deadline keeps running once the headers have landed. A
/// timeout that covered only the handshake would let this request hang until the
/// listener gives up.
#[tokio::test]
async fn total_timeout_cuts_off_a_body_that_outruns_it() {
use tokio::io::{AsyncReadExt, AsyncWriteExt};
let listener = tokio::net::TcpListener::bind("127.0.0.1:0").await.unwrap();
let addr = listener.local_addr().unwrap();
tokio::spawn(async move {
let (mut sock, _) = listener.accept().await.unwrap();
let mut scratch = [0u8; 1024];
let _ = sock.read(&mut scratch).await;
// Headers land immediately, then the body stalls indefinitely.
let _ = sock
.write_all(b"HTTP/1.1 200 OK\r\nContent-Length: 1024\r\n\r\nfirst")
.await;
let _ = sock.flush().await;
tokio::time::sleep(std::time::Duration::from_secs(30)).await;
});
let client = NodeClient::with_timeout(
format!("http://{addr}"),
None,
std::time::Duration::from_millis(250),
);
let started = std::time::Instant::now();
let resp = client.get("/slow").await.expect("headers arrive promptly");
assert_eq!(
resp.status(),
200,
"the stall is in the body, not the status"
);
let err = resp
.bytes()
.await
.expect_err("a body still arriving past the total timeout must be cut off");
let elapsed = started.elapsed();
assert!(
err.is_timeout(),
"the body read must end in a timeout, got: {err}"
);
assert!(
elapsed < std::time::Duration::from_secs(5),
"the timeout must fire on its own schedule, not wait out the listener; \
took {elapsed:?}"
);
}
#[test]
fn shipped_client_uses_the_documented_total_timeout() {
// The scaled-down test above proves the shape at 250ms. This pins the value
// `new` actually ships, so the two together cover the documented behaviour.
assert_eq!(TOTAL_REQUEST_TIMEOUT, std::time::Duration::from_secs(30));
}
#[test]
fn sanitize_strips_controls_bidi_and_caps_length() {
// C0 (ESC/BEL) and the Cf bidi override (U+202E) are both removed; the
// printable text survives. (Note: a stripped ESC leaves any following
// "[31m" as inert literal text — that is the point, so the input here
// avoids that residue to keep the expectation unambiguous.)
let out = sanitize_node_msg("a\u{1b}\u{07}b\u{202e}c");
assert!(
!out.chars().any(|c| c.is_control()),
"control char leaked: {out:?}"
);
assert!(
!out.contains('\u{202e}'),
"RLO bidi override leaked: {out:?}"
);
assert_eq!(out, "abc");
// Length is capped at 200 chars regardless of input size.
let long = "x".repeat(250);
assert_eq!(sanitize_node_msg(&long).chars().count(), 200);
}
#[test]
fn sanitize_preserves_legitimate_and_rtl_text() {
// Must not over-strip: a plain word, a genuine RTL SCRIPT letter (Arabic
// U+0627, category Lo — NOT a format char), and ZWJ (U+200D, a legitimate
// Cf char, e.g. emoji sequences) all survive. Guards the shared predicate
// against being widened into a blanket Cf stripper.
let out = sanitize_node_msg("ok \u{0627}\u{200D}b");
assert_eq!(out, "ok \u{0627}\u{200D}b");
}
}
+22 -2111
View File
File diff suppressed because it is too large. Load diff
+12 -406
View File
@@ -7,7 +7,7 @@ use clap::{Args, Subcommand};
use serde_json::Value;
use std::path::PathBuf;
use crate::http::{read_body_capped, sanitize_node_msg, NodeClient};
use crate::http::NodeClient;
use crate::identity::load_keypair_from_dir;
#[derive(Args)]
@@ -97,74 +97,6 @@ async fn cmd_list(node: String) -> Result<()> {
Ok(())
}
/// The warning text for a local peer-add reply, or `None` when the node
/// accepted it.
///
/// Split out so the refusal path is assertable. The node now answers 403 when
/// an unproven caller tries to repoint an existing peer's `http_url`, so a
/// reply that arrived and refused must not reach the success line.
fn local_add_refusal(status: reqwest::StatusCode, body: &Value) -> Option<String> {
if status.is_success() {
return None;
}
// The message is the node's, not ours, so it is defanged before it reaches
// the terminal. Only the message: the status and the prose are ours.
let msg = sanitize_node_msg(body["message"].as_str().unwrap_or("unknown error"));
Some(format!(
"warning: local peer list not updated ({status}): {msg}"
))
}
/// The error text for a remote announce reply, or `None` when the peer accepted
/// it.
///
/// Split out so the ordering is assertable: the status has to reach the caller
/// even when the body does not parse. That case is reachable rather than
/// theoretical, because the read above is capped, so a peer answering with a
/// body past the cap leaves truncated JSON behind, and `peer_url` is whatever
/// the caller passed, so a non-JSON error page is just as easy. Parsing first
/// and failing on the parse would swallow the status the user needs.
fn remote_announce_failure(status: reqwest::StatusCode, raw: &str) -> Option<String> {
if status.is_success() {
return None;
}
// Falls back to the body itself when there is no `message` field, which is
// what a truncated or non-JSON reply leaves. Sanitized either way: it is the
// peer's text and it is headed for the terminal.
let parsed: Value = serde_json::from_str(raw).unwrap_or(Value::Null);
let msg = sanitize_node_msg(parsed["message"].as_str().unwrap_or(raw));
Some(format!("announce failed ({status}): {msg}"))
}
/// Which line the local peer-add prints, and whether it is a warning.
///
/// Split out because the SELECTION is what the fix is: `local_add_refusal` was
/// already unit-tested six ways, and replacing this whole match with an
/// unconditional "Added to local peer list." still left 312 tests green. The
/// helper was pinned; the wiring was not, which is the shape
/// `unit-test-on-helper-does-not-prove-handler-wiring` names. With the choice
/// extracted, the call site is a two-line dispatch carrying no logic.
fn local_add_report(status: reqwest::StatusCode, body: &Value) -> (bool, String) {
match local_add_refusal(status, body) {
Some(warning) => (true, warning),
None => (false, " Added to local peer list.".to_string()),
}
}
/// The block printed after a peer accepts our announce. Split out so the
/// defanging is assertable: the DID and URL are the remote peer's own strings,
/// and `peer_url` is caller-chosen, so this is the least trusted body the
/// command handles. Only the display is sanitized; the values sent on to our
/// local node stay verbatim, since that is data rather than terminal output and
/// the node runs its own validation on them.
fn announced_peer_summary(their_did: &str, their_url: &str, peer_count: u64) -> String {
format!(
"Announced to peer node:\n DID: {}\n URL: {}\n Their peers: {peer_count}\n",
sanitize_node_msg(their_did),
sanitize_node_msg(their_url),
)
}
async fn cmd_add(peer_url: String, node: String, dir: Option<PathBuf>) -> Result<()> {
let keypair = load_keypair_from_dir(dir.as_deref())?;
let my_did = keypair.did().to_string();
@@ -195,26 +127,21 @@ async fn cmd_add(peer_url: String, node: String, dir: Option<PathBuf>) -> Result
.await
.context("failed to connect to peer")?;
let status = resp.status();
// `peer_url` is fully caller-chosen, so this body is the least trusted one
// in the command: bound the read, and defang the message before it reaches
// the terminal through the error return. An announce reply is a DID, a URL
// and a count, so 8 KiB is well past what the shape needs.
let raw = read_body_capped(resp, 8 * 1024).await.text;
let result: Value = resp.json().await.context("invalid JSON response")?;
if let Some(failure) = remote_announce_failure(status, &raw) {
anyhow::bail!("{failure}");
if !status.is_success() {
let msg = result["message"].as_str().unwrap_or("unknown error");
anyhow::bail!("announce failed ({status}): {msg}");
}
let result: Value = serde_json::from_str(&raw).context("invalid JSON response")?;
let their_did = result["node_did"].as_str().unwrap_or("?");
let their_url = result["node_url"].as_str().unwrap_or("?");
let peer_count = result["peer_count"].as_u64().unwrap_or(0);
print!(
"{}",
announced_peer_summary(their_did, their_url, peer_count)
);
println!("Announced to peer node:");
println!(" DID: {their_did}");
println!(" URL: {their_url}");
println!(" Their peers: {peer_count}");
// Also add their info to our local node's peer list
// (the peer's /announce response includes their did + url)
@@ -223,31 +150,9 @@ async fn cmd_add(peer_url: String, node: String, dir: Option<PathBuf>) -> Result
"did": their_did,
"http_url": their_url,
}))?;
// This requires the local node to be running, so a transport failure
// stays a warning rather than failing the command. A reply that
// ARRIVED and refused is different: the announce route answers 403
// when an unproven caller tries to repoint an existing peer, and
// printing the success line over that reports a completed add that
// never happened. Same status handling as the remote announce above,
// except best effort.
match local_client.post("/api/v1/peers/announce", &add_body).await {
Ok(resp) => {
let status = resp.status();
// Same bound as the remote announce above: a compromised local
// node (or a MITM on plain http) must not force an unbounded
// read. A body that does not parse stays `Null`, which still
// routes a non-success status to the warning.
let raw = read_body_capped(resp, 8 * 1024).await.text;
let result: Value = serde_json::from_str(&raw).unwrap_or(Value::Null);
let (is_warning, line) = local_add_report(status, &result);
if is_warning {
eprintln!("{line}");
} else {
println!("{line}");
}
}
Err(e) => eprintln!("warning: local peer list not updated: {e}"),
}
// This requires the local node to be running; ignore errors here
let _ = local_client.post("/api/v1/peers/announce", &add_body).await;
println!(" Added to local peer list.");
}
Ok(())
@@ -305,302 +210,3 @@ async fn cmd_resolve(did: String, node: String) -> Result<()> {
}
Ok(())
}
#[cfg(test)]
mod tests {
use super::{
announced_peer_summary, cmd_add, local_add_refusal, local_add_report,
remote_announce_failure,
};
use reqwest::StatusCode;
use serde_json::json;
/// The node's 403 for repointing an existing peer must surface, not print
/// as a completed add. Without the status check the command prints "Added
/// to local peer list." over a refusal.
///
/// The fixture body is the real wire shape, not an invented one: the
/// message is the Display of `PeerWriteDenied::UnprovenRepoint`, declared
/// in `crates/gitlawb-node/src/db/mod.rs`. The executable detector for a
/// wording change is the node-side pin
/// `announce_unsigned_repoint_is_a_403_denial` in
/// `crates/gitlawb-node/src/api/peers.rs`, which asserts this same
/// substring against a response driven through the real router, so
/// rewording the Display turns that test red and the grep for the old
/// phrase lands here.
///
/// The assertion bites rather than echoing its own input because the
/// substring is reachable only through the helper's `body["message"]`
/// extraction; the fixture's `error` field carries a different string, so
/// reading `body["error"]` instead fails this test.
#[test]
fn a_refused_local_add_warns_with_the_node_reason() {
let warning = local_add_refusal(
StatusCode::FORBIDDEN,
&json!({
"error": "forbidden",
"message": "unproven announce cannot change an existing peer's http_url: did:key:z6MkuMqUm4i228K9qXidJ57zqSWAcQLgrcbMxB8RKVLuqitj"
}),
)
.expect("a refusal must not render as success");
assert!(warning.contains("403"), "must name the status: {warning}");
assert!(
warning.contains("unproven announce cannot change an existing peer"),
"must carry the node's own reason: {warning}"
);
}
/// A refusal with no message body still warns rather than passing silently.
#[test]
fn a_refusal_without_a_message_still_warns() {
let warning = local_add_refusal(StatusCode::BAD_REQUEST, &serde_json::Value::Null)
.expect("a refusal must not render as success");
assert!(warning.contains("400"), "must name the status: {warning}");
}
/// The accepted case stays quiet, so the success line still prints.
#[test]
fn an_accepted_local_add_produces_no_warning() {
assert!(local_add_refusal(StatusCode::OK, &json!({ "peer_count": 3 })).is_none());
}
/// The warning is printed straight to a terminal, so a node-supplied message
/// carrying an OSC title-rewrite plus a C0 escape must reach it defanged.
#[test]
fn a_refusal_message_cannot_carry_terminal_escapes() {
let warning = local_add_refusal(
StatusCode::FORBIDDEN,
&json!({ "message": "\u{1b}]2;pwned\u{7}legit-looking text" }),
)
.expect("a refusal must not render as success");
assert!(
!warning.chars().any(|c| c.is_control()),
"control char leaked to the terminal: {warning:?}"
);
assert!(
warning.contains("legit-looking text"),
"printable text dropped: {warning:?}"
);
}
/// Bidi format chars are not `char::is_control`, yet they reorder the
/// displayed line, so a refusal must not carry them either.
#[test]
fn a_refusal_message_cannot_carry_bidi_overrides() {
let warning = local_add_refusal(
StatusCode::FORBIDDEN,
&json!({ "message": "peer \u{202e}refused\u{202c} here" }),
)
.expect("a refusal must not render as success");
assert!(
!warning.chars().any(gitlawb_core::sanitize::is_bidi_format),
"bidi format char leaked to the terminal: {warning:?}"
);
}
/// A hostile node must not be able to dump an arbitrary amount of text into
/// the user's scrollback through the warning line.
#[test]
fn a_refusal_message_is_length_bounded() {
let warning = local_add_refusal(
StatusCode::FORBIDDEN,
&json!({ "message": "x".repeat(5000) }),
)
.expect("a refusal must not render as success");
assert!(
warning.chars().count() < 500,
"warning not bounded: {} chars",
warning.chars().count()
);
}
/// The status is the part the user can act on, so it must survive a body
/// that does not parse. The read is capped, so a peer answering past the cap
/// leaves exactly this: a valid status and truncated JSON.
#[test]
fn a_remote_announce_failure_keeps_the_status_when_the_body_does_not_parse() {
let truncated = r#"{"error":"service_unavailable","message":"backend is do"#;
let failure = remote_announce_failure(StatusCode::SERVICE_UNAVAILABLE, truncated)
.expect("a non-success status must produce a failure");
assert!(
failure.contains("503"),
"the status must survive an unparseable body: {failure:?}"
);
}
/// A parseable error body still reports the peer's own message.
#[test]
fn a_remote_announce_failure_carries_the_peers_message() {
let failure = remote_announce_failure(
StatusCode::FORBIDDEN,
r#"{"error":"forbidden","message":"unproven announce cannot change an existing peer"}"#,
)
.expect("a non-success status must produce a failure");
assert!(
failure.contains("403")
&& failure.contains("unproven announce cannot change an existing peer"),
"status and reason must both survive: {failure:?}"
);
}
/// The fallback path prints the raw body, so it is defanged like every other
/// peer-supplied string headed for the terminal.
#[test]
fn a_remote_announce_failure_defangs_an_unparseable_body() {
let failure = remote_announce_failure(
StatusCode::BAD_GATEWAY,
"\u{1b}]2;pwned\u{7}<html>gateway error</html>",
)
.expect("a non-success status must produce a failure");
assert!(
!failure.chars().any(|c| c.is_control()),
"control char leaked to the terminal: {failure:?}"
);
assert!(failure.contains("502"), "status dropped: {failure:?}");
assert!(
failure.contains("gateway error"),
"the body is all the user gets when there is no message field: {failure:?}"
);
}
/// The unit tests above bind the helper. This one binds the CALL SITE: the
/// failure check has to run before the parse, or a body past the read cap
/// turns a 503 into "invalid JSON response" and the status the user needs is
/// gone. Reordering the two statements in `cmd_add` compiles and passes every
/// other test in this file, so only driving the command catches it.
#[tokio::test]
async fn a_peer_failure_past_the_read_cap_still_reports_its_status() {
let mut peer = mockito::Server::new_async().await;
let mut local = mockito::Server::new_async().await;
let dir = tempfile::TempDir::new().unwrap();
let kp = gitlawb_core::identity::Keypair::generate();
std::fs::write(
dir.path().join("identity.pem"),
kp.to_pem().unwrap().as_bytes(),
)
.unwrap();
let _local_info = local
.mock("GET", "/")
.with_status(200)
.with_header("content-type", "application/json")
.with_body(r#"{"public_url":"https://me.example"}"#)
.create_async()
.await;
// Valid JSON, but past the 8 KiB read cap, so what survives the read is
// truncated and unparseable. This is the shape the cap itself creates.
let oversized = format!(r#"{{"message":"{}"}}"#, "x".repeat(16 * 1024));
let _peer_refusal = peer
.mock("POST", "/api/v1/peers/announce")
.with_status(503)
.with_header("content-type", "application/json")
.with_body(oversized)
.create_async()
.await;
let err = cmd_add(peer.url(), local.url(), Some(dir.path().to_path_buf()))
.await
.expect_err("a 503 from the peer must fail the command");
let rendered = format!("{err:#}");
assert!(
rendered.contains("503"),
"the status must survive a body past the read cap: {rendered:?}"
);
assert!(
!rendered.contains("invalid JSON response"),
"the parse failure must not stand in for the status: {rendered:?}"
);
}
/// An accepted announce produces no error, so the caller proceeds to parse.
#[test]
fn an_accepted_remote_announce_produces_no_failure() {
assert!(remote_announce_failure(StatusCode::OK, r#"{"peer_count":3}"#).is_none());
}
/// The selection, not the helper. A refusal must never yield the success
/// line, and the success line must never carry the warning prefix. This is
/// what stayed green when the entire match was replaced with an
/// unconditional "Added to local peer list."
#[test]
fn a_local_add_refusal_never_selects_the_success_line() {
let (is_warning, line) = local_add_report(
StatusCode::FORBIDDEN,
&json!({ "message": "unproven announce cannot change an existing peer's http_url: did:key:zAbc" }),
);
assert!(is_warning, "a 403 must select the warning stream: {line:?}");
assert!(
!line.contains("Added to local peer list"),
"a refusal rendered as the success line: {line:?}"
);
assert!(
line.contains("unproven announce cannot change an existing peer"),
"the node's reason must reach the user: {line:?}"
);
}
/// The other direction, so the fix cannot be satisfied by always warning.
#[test]
fn an_accepted_local_add_selects_the_success_line() {
let (is_warning, line) = local_add_report(StatusCode::OK, &json!({ "peer_count": 3 }));
assert!(!is_warning, "a 200 must not select the warning stream");
assert!(
line.contains("Added to local peer list"),
"the success line must still print: {line:?}"
);
}
/// The accepted-announce block prints the remote peer's own DID and URL, so
/// a hostile peer answering 200 must not get escapes onto the terminal that
/// way either. The refusal path is not the only sink in this command.
#[test]
fn an_accepted_announce_summary_cannot_carry_terminal_escapes() {
let summary = announced_peer_summary(
"did:key:z6Mk\u{1b}]2;pwned\u{7}abc",
"https://peer.example\u{202e}moc.live//:sptth",
3,
);
assert!(
!summary.chars().any(|c| c.is_control() && c != '\n'),
"control char leaked to the terminal: {summary:?}"
);
assert!(
!summary.chars().any(gitlawb_core::sanitize::is_bidi_format),
"bidi format char leaked to the terminal: {summary:?}"
);
assert!(
summary.contains("did:key:z6Mk") && summary.contains("Their peers: 3"),
"printable content dropped: {summary:?}"
);
}
/// The counterweight to the stripping tests: a legitimate message with RTL
/// script letters and a ZWJ must survive intact, so the sanitizer is not
/// proven by over-stripping.
#[test]
fn a_legitimate_refusal_message_survives_intact() {
let warning = local_add_refusal(
StatusCode::FORBIDDEN,
&json!({ "message": "peer refused \u{0627}\u{200D}b" }),
)
.expect("a refusal must not render as success");
assert!(
warning.contains("peer refused \u{0627}\u{200D}b"),
"legitimate text mangled: {warning:?}"
);
}
}
+67 -3
View File
@@ -2,7 +2,7 @@ use anyhow::{Context, Result};
use clap::{Args, Subcommand};
use std::path::PathBuf;
use crate::http::{read_body_capped, sanitize_node_msg, NodeClient};
use crate::http::NodeClient;
use crate::identity::load_keypair_from_dir;
#[derive(Args)]
@@ -46,7 +46,7 @@ pub async fn run(args: SyncArgs) -> Result<()> {
if !status.is_success() {
// Bound the read: a hostile or broken node must not force an
// unbounded allocation just to surface a denial (INV-6, read half).
let raw = read_body_capped(resp, 8 * 1024).await.text;
let raw = read_body_capped(resp, 8 * 1024).await;
let msg = serde_json::from_str::<serde_json::Value>(&raw)
.ok()
.and_then(|v| {
@@ -104,6 +104,39 @@ fn trigger_counts(resp: &serde_json::Value) -> (u64, u64) {
)
}
/// Read at most `cap` bytes of a response body. Bounds the allocation from a
/// hostile or broken node returning a huge error body — the display is capped
/// separately, but the read itself must not be unbounded (INV-6, read half).
async fn read_body_capped(mut resp: reqwest::Response, cap: usize) -> String {
let mut buf: Vec<u8> = Vec::new();
while buf.len() < cap {
match resp.chunk().await {
Ok(Some(chunk)) => {
let take = (cap - buf.len()).min(chunk.len());
buf.extend_from_slice(&chunk[..take]);
if take < chunk.len() {
break; // hit the cap mid-chunk
}
}
_ => break, // end of body or read error — return what we have
}
}
String::from_utf8_lossy(&buf).into_owned()
}
/// Strip terminal-dangerous characters from (and cap the length of) a
/// node-supplied error string before surfacing it. The node a caller talks to
/// could be hostile and embed escape sequences in its error body; those must not
/// reach the terminal verbatim (INV-6). We drop the C0/C1 control bytes (which
/// defangs ANSI/OSC escapes) AND the Unicode bidi/format controls (which
/// `char::is_control` does not cover — they can reorder the displayed line).
fn sanitize_node_msg(s: &str) -> String {
s.chars()
.filter(|c| !c.is_control() && !gitlawb_core::sanitize::is_bidi_format(*c))
.take(200)
.collect()
}
#[cfg(test)]
mod tests {
use super::*;
@@ -227,6 +260,37 @@ mod tests {
run(args).await.unwrap();
}
#[test]
fn sanitize_strips_controls_bidi_and_caps_length() {
// C0 (ESC/BEL) and the Cf bidi override (U+202E) are both removed; the
// printable text survives. (Note: a stripped ESC leaves any following
// "[31m" as inert literal text — that is the point, so the input here
// avoids that residue to keep the expectation unambiguous.)
let out = sanitize_node_msg("a\u{1b}\u{07}b\u{202e}c");
assert!(
!out.chars().any(|c| c.is_control()),
"control char leaked: {out:?}"
);
assert!(
!out.contains('\u{202e}'),
"RLO bidi override leaked: {out:?}"
);
assert_eq!(out, "abc");
// Length is capped at 200 chars regardless of input size.
let long = "x".repeat(250);
assert_eq!(sanitize_node_msg(&long).chars().count(), 200);
}
#[test]
fn sanitize_preserves_legitimate_and_rtl_text() {
// Must not over-strip: a plain word, a genuine RTL SCRIPT letter (Arabic
// U+0627, category Lo — NOT a format char), and ZWJ (U+200D, a legitimate
// Cf char, e.g. emoji sequences) all survive. Guards the shared predicate
// against being widened into a blanket Cf stripper.
let out = sanitize_node_msg("ok \u{0627}\u{200D}b");
assert_eq!(out, "ok \u{0627}\u{200D}b");
}
#[tokio::test]
async fn trigger_handles_oversized_error_body_without_unbounded_output() {
// A hostile/broken node returns a 2 MB error body. The command must still
@@ -261,7 +325,7 @@ mod tests {
.create_async()
.await;
let resp = reqwest::get(format!("{}/big", server.url())).await.unwrap();
let out = read_body_capped(resp, 8192).await.text;
let out = read_body_capped(resp, 8192).await;
assert!(out.len() <= 8192, "read not bounded: {} bytes", out.len());
assert!(!out.is_empty(), "expected some body");
}
+14 -334
View File
@@ -1,11 +1,11 @@
//! `gl whoami` — print current identity and optional node registration info.
use anyhow::{bail, Result};
use anyhow::Result;
use clap::Args;
use serde_json::{json, Value};
use std::path::PathBuf;
use crate::http::{read_body_capped, sanitize_node_msg, NodeClient};
use crate::http::NodeClient;
use crate::identity::load_keypair_from_dir;
#[derive(Args)]
@@ -22,10 +22,6 @@ pub struct WhoamiArgs {
}
pub async fn run(args: WhoamiArgs) -> Result<()> {
run_to_writer(args, &mut std::io::stdout()).await
}
pub(crate) async fn run_to_writer(args: WhoamiArgs, w: &mut impl std::io::Write) -> Result<()> {
let keypair = load_keypair_from_dir(args.dir.as_deref())?;
let did = keypair.did().to_string();
let short = did.split(':').next_back().unwrap_or(&did).to_string();
@@ -45,7 +41,7 @@ pub(crate) async fn run_to_writer(args: WhoamiArgs, w: &mut impl std::io::Write)
if let Some(caps) = info["capabilities"].as_array() {
capabilities = caps
.iter()
.filter_map(|c| c.as_str().map(sanitize_node_msg))
.filter_map(|c| c.as_str().map(String::from))
.collect();
}
// Try to get repo count
@@ -55,35 +51,11 @@ pub(crate) async fn run_to_writer(args: WhoamiArgs, w: &mut impl std::io::Write)
}
}
}
Ok(resp) if resp.status().as_u16() == 404 => {
Ok(_) => {
registered = Some(false);
}
Ok(resp) => {
let status = resp.status();
let raw = read_body_capped(resp, 8 * 1024).await.text;
let msg = serde_json::from_str::<Value>(&raw)
.ok()
.and_then(|v| {
let non_empty = |m: Option<&Value>| {
m.and_then(|m| m.as_str())
.map(String::from)
.filter(|s| !s.is_empty())
};
non_empty(v.get("message")).or_else(|| non_empty(v.get("error")))
})
.unwrap_or(raw);
bail!(
"agent lookup failed ({status}): {}",
sanitize_node_msg(&msg)
);
}
Err(e) => {
let detail: String = e
.chain()
.map(|e| sanitize_node_msg(&e.to_string()))
.collect::<Vec<_>>()
.join(": ");
bail!("agent lookup failed: {detail}");
Err(_) => {
registered = Some(false);
}
}
}
@@ -105,21 +77,21 @@ pub(crate) async fn run_to_writer(args: WhoamiArgs, w: &mut impl std::io::Write)
if let Some(rc) = repo_count {
out["repos"] = json!(rc);
}
writeln!(w, "{}", serde_json::to_string_pretty(&out)?)?;
println!("{}", serde_json::to_string_pretty(&out)?);
} else {
writeln!(w, "DID: {did}")?;
writeln!(w, "Short: {short}")?;
println!("DID: {did}");
println!("Short: {short}");
if let Some(reg) = registered {
writeln!(w, "Registered: {}", if reg { "yes" } else { "no" })?;
println!("Registered: {}", if reg { "yes" } else { "no" });
}
if let Some(ts) = trust_score {
writeln!(w, "Trust: {ts:.2}")?;
println!("Trust: {ts:.2}");
}
if !capabilities.is_empty() {
writeln!(w, "Caps: {}", capabilities.join(", "))?;
println!("Caps: {}", capabilities.join(", "));
}
if let Some(rc) = repo_count {
writeln!(w, "Repos: {rc}")?;
println!("Repos: {rc}");
}
}
Ok(())
@@ -220,236 +192,7 @@ mod tests {
node: Some(server.url()),
json: false,
};
let mut out = Vec::new();
run_to_writer(args, &mut out).await.unwrap();
let out = String::from_utf8(out).unwrap();
assert!(out.contains("Registered: no"), "unexpected output: {out}");
let args = WhoamiArgs {
dir: Some(dir.path().to_path_buf()),
node: Some(server.url()),
json: true,
};
let mut out = Vec::new();
run_to_writer(args, &mut out).await.unwrap();
let out = String::from_utf8(out).unwrap();
assert!(
out.contains("\"registered\": false"),
"unexpected output: {out}"
);
}
#[tokio::test]
async fn test_whoami_with_node_forbidden() {
let dir = TempDir::new().unwrap();
let kp = gitlawb_core::identity::Keypair::generate();
let pem = kp.to_pem().unwrap();
std::fs::write(dir.path().join("identity.pem"), pem.as_bytes()).unwrap();
let did = kp.did().to_string();
let mut server = mockito::Server::new_async().await;
let _agent = server
.mock("GET", format!("/api/v1/agents/{did}").as_str())
.with_status(403)
.with_header("content-type", "application/json")
.with_body(r#"{"message":"forbidden"}"#)
.create_async()
.await;
let args = WhoamiArgs {
dir: Some(dir.path().to_path_buf()),
node: Some(server.url()),
json: false,
};
let err = run(args).await.unwrap_err();
let msg = format!("{err:?}");
assert!(msg.contains("403"), "expected 403 error, got: {msg}");
assert!(
msg.contains("forbidden"),
"expected 'forbidden' in error, got: {msg}"
);
}
#[tokio::test]
async fn test_whoami_with_node_server_error() {
let dir = TempDir::new().unwrap();
let kp = gitlawb_core::identity::Keypair::generate();
let pem = kp.to_pem().unwrap();
std::fs::write(dir.path().join("identity.pem"), pem.as_bytes()).unwrap();
let did = kp.did().to_string();
let mut server = mockito::Server::new_async().await;
let _agent = server
.mock("GET", format!("/api/v1/agents/{did}").as_str())
.with_status(500)
.with_header("content-type", "application/json")
.with_body(r#"{"message":"internal error"}"#)
.create_async()
.await;
let args = WhoamiArgs {
dir: Some(dir.path().to_path_buf()),
node: Some(server.url()),
json: false,
};
let err = run(args).await.unwrap_err();
let msg = format!("{err:?}");
assert!(msg.contains("500"), "expected 500 error, got: {msg}");
assert!(
msg.contains("internal error"),
"expected 'internal error' in error, got: {msg}"
);
}
#[tokio::test]
async fn test_whoami_server_error_unusable_message_uses_error() {
let dir = TempDir::new().unwrap();
let kp = gitlawb_core::identity::Keypair::generate();
let pem = kp.to_pem().unwrap();
std::fs::write(dir.path().join("identity.pem"), pem.as_bytes()).unwrap();
let did = kp.did().to_string();
let mut server = mockito::Server::new_async().await;
let _agent = server
.mock("GET", format!("/api/v1/agents/{did}").as_str())
.with_status(500)
.with_header("content-type", "application/json")
.with_body(r#"{"message":"","error":"boom"}"#)
.create_async()
.await;
let args = WhoamiArgs {
dir: Some(dir.path().to_path_buf()),
node: Some(server.url()),
json: false,
};
let err = run(args).await.unwrap_err();
let msg = format!("{err:?}");
assert!(
msg.contains("boom"),
"expected 'boom' from error field, got: {msg}"
);
}
#[tokio::test]
async fn test_whoami_with_node_transport_error() {
let dir = TempDir::new().unwrap();
let kp = gitlawb_core::identity::Keypair::generate();
let pem = kp.to_pem().unwrap();
std::fs::write(dir.path().join("identity.pem"), pem.as_bytes()).unwrap();
let listener = std::net::TcpListener::bind("127.0.0.1:0").unwrap();
let node = format!("http://{}", listener.local_addr().unwrap());
drop(listener);
let args = WhoamiArgs {
dir: Some(dir.path().to_path_buf()),
node: Some(node),
json: false,
};
let err = run(args).await.unwrap_err();
let msg = format!("{err:?}");
assert!(
msg.contains("agent lookup failed"),
"expected transport error, got: {msg}"
);
}
#[tokio::test]
async fn test_whoami_transport_sanitizes_control_chars_in_node_url() {
let dir = TempDir::new().unwrap();
let kp = gitlawb_core::identity::Keypair::generate();
let pem = kp.to_pem().unwrap();
std::fs::write(dir.path().join("identity.pem"), pem.as_bytes()).unwrap();
let listener = std::net::TcpListener::bind("127.0.0.1:0").unwrap();
let port = listener.local_addr().unwrap().port();
drop(listener);
let node = format!("http://127.0.0.1:{port}/\u{1b}]0;PWNED\u{7} \u{202e}gnitirw");
let args = WhoamiArgs {
dir: Some(dir.path().to_path_buf()),
node: Some(node),
json: false,
};
let err = run(args).await.unwrap_err();
let msg = format!("{err:?}");
assert!(
msg.contains("agent lookup failed"),
"expected transport error, got: {msg}"
);
assert!(!msg.contains('\u{1b}'), "ESC leaked into output: {msg:?}");
assert!(!msg.contains('\u{7}'), "BEL leaked into output: {msg:?}");
assert!(!msg.contains('\u{202e}'), "RLO leaked into output: {msg:?}");
}
#[tokio::test]
async fn test_whoami_server_error_body_display_bounded() {
let dir = TempDir::new().unwrap();
let kp = gitlawb_core::identity::Keypair::generate();
let pem = kp.to_pem().unwrap();
std::fs::write(dir.path().join("identity.pem"), pem.as_bytes()).unwrap();
let did = kp.did().to_string();
let mut server = mockito::Server::new_async().await;
let _agent = server
.mock("GET", format!("/api/v1/agents/{did}").as_str())
.with_status(502)
.with_header("content-type", "application/json")
.with_body("x".repeat(100_000))
.create_async()
.await;
let args = WhoamiArgs {
dir: Some(dir.path().to_path_buf()),
node: Some(server.url()),
json: false,
};
let err = run(args).await.unwrap_err();
let msg = format!("{err:?}");
assert!(
msg.contains("502"),
"expected 502 error with bounded body, got: {msg}"
);
let display = format!("{err}");
assert!(
display.len() < 1000,
"error message too long ({} bytes) — body was not capped",
display.len()
);
}
#[tokio::test]
async fn test_whoami_server_error_sanitizes_controls() {
let dir = TempDir::new().unwrap();
let kp = gitlawb_core::identity::Keypair::generate();
let pem = kp.to_pem().unwrap();
std::fs::write(dir.path().join("identity.pem"), pem.as_bytes()).unwrap();
let did = kp.did().to_string();
let mut server = mockito::Server::new_async().await;
let _agent = server
.mock("GET", format!("/api/v1/agents/{did}").as_str())
.with_status(500)
.with_header("content-type", "application/json")
.with_body("{\"message\":\"\\u001b[31mowned\\u0007\\u202eevil\"}")
.create_async()
.await;
let args = WhoamiArgs {
dir: Some(dir.path().to_path_buf()),
node: Some(server.url()),
json: false,
};
let err = run(args).await.unwrap_err();
let msg = format!("{err:?}");
assert!(
msg.contains("owned"),
"expected sanitized error body, got: {msg}"
);
assert!(!msg.contains('\u{1b}'), "ESC control char leaked: {msg}");
assert!(!msg.contains('\u{07}'), "BEL control char leaked: {msg}");
assert!(!msg.contains('\u{202e}'), "RTL override leaked: {msg}");
run(args).await.unwrap();
}
#[tokio::test]
@@ -487,67 +230,4 @@ mod tests {
};
run(args).await.unwrap();
}
#[tokio::test]
async fn test_whoami_sanitizes_hostile_capabilities() {
let dir = TempDir::new().unwrap();
let kp = gitlawb_core::identity::Keypair::generate();
let pem = kp.to_pem().unwrap();
std::fs::write(dir.path().join("identity.pem"), pem.as_bytes()).unwrap();
let did = kp.did().to_string();
let short = did.split(':').next_back().unwrap().to_string();
let mut server = mockito::Server::new_async().await;
let _agent = server
.mock("GET", format!("/api/v1/agents/{did}").as_str())
.with_status(200)
.with_header("content-type", "application/json")
.with_body(
"{\"trust_score\":0.5,\"capabilities\":[\"\\u001b]0;PWNED\\u0007repo:write\",\"\\u202egnitirw-tfel\"]}",
)
.create_async()
.await;
let _repos = server
.mock(
"GET",
mockito::Matcher::Regex(format!(r"^/api/v1/repos\?owner={short}")),
)
.with_status(200)
.with_header("content-type", "application/json")
.with_body("[]")
.create_async()
.await;
// Human mode: no control bytes or bidi overrides reach the terminal
let mut buf = Vec::new();
let args = WhoamiArgs {
dir: Some(dir.path().to_path_buf()),
node: Some(server.url()),
json: false,
};
run_to_writer(args, &mut buf).await.unwrap();
let out = String::from_utf8(buf).unwrap();
assert!(!out.contains('\u{1b}'), "ESC leaked in human mode: {out:?}");
assert!(!out.contains('\u{07}'), "BEL leaked in human mode: {out:?}");
assert!(
!out.contains('\u{202e}'),
"RLO leaked in human mode: {out:?}"
);
assert!(out.contains("repo:write"), "benign text missing: {out:?}");
assert!(out.contains("tfel"), "reversed text missing: {out:?}");
// JSON mode: serde escapes C0 but passes bidi — ensure no U+202E
let mut buf = Vec::new();
let args = WhoamiArgs {
dir: Some(dir.path().to_path_buf()),
node: Some(server.url()),
json: true,
};
run_to_writer(args, &mut buf).await.unwrap();
let out = String::from_utf8(buf).unwrap();
assert!(
!out.contains('\u{202e}'),
"RLO leaked in JSON mode: {out:?}"
);
}
}
-211
View File
@@ -550,215 +550,4 @@ mod tests {
);
assert!(key_trusted, "key stays trusted with operator origin");
}
#[test]
fn new_withholds_key_from_untrusted_origin_but_attaches_for_operator() {
let guard = ICAPTCHA_ENV_LOCK.lock().unwrap();
let prev_key = std::env::var_os("GITLAWB_ICAPTCHA_API_KEY");
let prev_op = std::env::var_os("GITLAWB_ICAPTCHA_URL");
std::env::set_var("GITLAWB_ICAPTCHA_API_KEY", "secret-bearer");
// Arm 1: No operator configured, node advertises an attacker origin.
// The key has no trusted destination; the advert is rejected.
std::env::remove_var("GITLAWB_ICAPTCHA_URL");
let untrusted =
IcaptchaCfg::new("did:key:zTEST", Some("https://evil.example".into()), None);
// Arm 1b: No operator configured, no URL advertised. The fallback goes
// to the default public origin, and with no operator to trust, the key
// must not ride.
let no_operator_no_advert = IcaptchaCfg::new("did:key:zTEST", None, None);
// Arm 2: Operator origin configured and advertised — key attached.
std::env::set_var("GITLAWB_ICAPTCHA_URL", "https://icap.mynode.example");
let operator_match = IcaptchaCfg::new(
"did:key:zTEST",
Some("https://icap.mynode.example/v1".into()),
None,
);
// Arm 3: Operator configured, default host advertised. The default host
// is allowlisted and selected, but its origin differs from the
// operator's, so key must not ride.
let default_advert = IcaptchaCfg::new(
"did:key:zTEST",
Some("https://icaptcha.gitlawb.com/v2".into()),
None,
);
// Arm 4: Operator configured, no URL advertised (the node sent only
// x-icaptcha-level, not x-icaptcha-url). resolve_solver_url falls back
// to the operator origin and marks the key trusted. This is the common
// API-key deployment path — a regression that additionally required
// url.is_some() for key attachment would leave all current tests green
// but make these fallback challenge/answer requests omit Authorization
// and fail against the protected operator service.
let operator_fallback = IcaptchaCfg::new("did:key:zTEST", None, Some(3));
// Arm 5: Operator configured, node advertises an attacker origin.
// The advert is rejected and the solver falls back to the operator's
// own origin, which keeps the key trusted — this is the exfiltration
// scenario the test name describes (withhold key from *untrusted*
// origin, not from the operator's fallback).
let operator_hostile_advert =
IcaptchaCfg::new("did:key:zTEST", Some("https://evil.example".into()), None);
// Arm 6: Operator configured, same host but different port.
// origin_key includes the port, so :8443 must not match the
// operator origin at the default :443 — the advert is rejected
// and the solver falls back to the operator origin.
let operator_diff_port = IcaptchaCfg::new(
"did:key:zTEST",
Some("https://icap.mynode.example:8443/evil".into()),
None,
);
// Restore env and release the lock BEFORE asserting, so a failing
// assertion can't poison the shared lock or cascade into a sibling.
match prev_key {
Some(v) => std::env::set_var("GITLAWB_ICAPTCHA_API_KEY", v),
None => std::env::remove_var("GITLAWB_ICAPTCHA_API_KEY"),
}
match prev_op {
Some(v) => std::env::set_var("GITLAWB_ICAPTCHA_URL", v),
None => std::env::remove_var("GITLAWB_ICAPTCHA_URL"),
}
drop(guard);
let mut failures: Vec<String> = Vec::new();
if untrusted.api_key.is_some() {
failures.push(format!(
"arm 1 (no operator, attacker advert): expected api_key=None, got {:?}",
untrusted.api_key
));
}
if untrusted.url != DEFAULT_URL {
failures.push(format!(
"arm 1 (no operator, attacker advert): expected url={DEFAULT_URL}, got {}",
untrusted.url
));
}
if no_operator_no_advert.api_key.is_some() {
failures.push(format!(
"arm 1b (no operator, no advert): expected api_key=None, got {:?}",
no_operator_no_advert.api_key
));
}
if no_operator_no_advert.url != DEFAULT_URL {
failures.push(format!(
"arm 1b (no operator, no advert): expected url={DEFAULT_URL}, got {}",
no_operator_no_advert.url
));
}
if operator_match.api_key.as_deref() != Some("secret-bearer") {
failures.push(format!(
"arm 2 (operator match): expected api_key=Some(\"secret-bearer\"), got {:?}",
operator_match.api_key
));
}
if operator_match.url != "https://icap.mynode.example/v1" {
failures.push(format!(
"arm 2 (operator match): expected url=https://icap.mynode.example/v1, got {}",
operator_match.url
));
}
if default_advert.api_key.is_some() {
failures.push(format!(
"arm 3 (operator + default advert): expected api_key=None, got {:?}",
default_advert.api_key
));
}
if default_advert.url != "https://icaptcha.gitlawb.com/v2" {
failures.push(format!(
"arm 3 (operator + default advert): expected url=https://icaptcha.gitlawb.com/v2, got {}",
default_advert.url
));
}
if operator_fallback.api_key.as_deref() != Some("secret-bearer") {
failures.push(format!(
"arm 4 (operator fallback, no advert): expected api_key=Some(\"secret-bearer\"), got {:?}",
operator_fallback.api_key
));
}
if operator_fallback.url != "https://icap.mynode.example" {
failures.push(format!(
"arm 4 (operator fallback, no advert): expected url=https://icap.mynode.example, got {}",
operator_fallback.url
));
}
if operator_hostile_advert.api_key.as_deref() != Some("secret-bearer") {
failures.push(format!(
"arm 5 (operator + hostile advert): expected api_key=Some(\"secret-bearer\"), got {:?}",
operator_hostile_advert.api_key
));
}
if operator_hostile_advert.url != "https://icap.mynode.example" {
failures.push(format!(
"arm 5 (operator + hostile advert): expected url=https://icap.mynode.example, got {}",
operator_hostile_advert.url
));
}
if operator_diff_port.api_key.as_deref() != Some("secret-bearer") {
failures.push(format!(
"arm 6 (operator + different port advert): expected api_key=Some(\"secret-bearer\"), got {:?}",
operator_diff_port.api_key
));
}
if operator_diff_port.url != "https://icap.mynode.example" {
failures.push(format!(
"arm 6 (operator + different port advert): expected url=https://icap.mynode.example, got {}",
operator_diff_port.url
));
}
assert!(failures.is_empty(), "\n{}", failures.join("\n"));
}
#[test]
fn new_normalizes_empty_api_key_to_none_on_trusted_origin() {
let guard = ICAPTCHA_ENV_LOCK.lock().unwrap();
let prev_key = std::env::var_os("GITLAWB_ICAPTCHA_API_KEY");
let prev_op = std::env::var_os("GITLAWB_ICAPTCHA_URL");
// Operator origin configured and resolved as trusted (no advert), so
// api_key reaches the env read and the emptiness filter is the only
// thing standing between a blank var and an empty bearer token.
std::env::set_var("GITLAWB_ICAPTCHA_URL", "https://icap.mynode.example");
// "Operator set the var but left it blank" must stay keyless: a
// Some("") would attach `Authorization: Bearer ` (empty token) to the
// challenge and answer requests instead of omitting the header.
std::env::set_var("GITLAWB_ICAPTCHA_API_KEY", "");
let empty = IcaptchaCfg::new("did:key:zTEST", None, None);
// Contrast arm: a real key on the same trusted path must survive the
// filter, so the filter can't regress to dropping every key.
std::env::set_var("GITLAWB_ICAPTCHA_API_KEY", "secret-bearer");
let non_empty = IcaptchaCfg::new("did:key:zTEST", None, None);
// Restore env and release the lock BEFORE asserting, so a failing
// assertion can't poison the shared lock or cascade into a sibling.
match prev_key {
Some(v) => std::env::set_var("GITLAWB_ICAPTCHA_API_KEY", v),
None => std::env::remove_var("GITLAWB_ICAPTCHA_API_KEY"),
}
match prev_op {
Some(v) => std::env::set_var("GITLAWB_ICAPTCHA_URL", v),
None => std::env::remove_var("GITLAWB_ICAPTCHA_URL"),
}
drop(guard);
assert_eq!(
empty.api_key, None,
"empty GITLAWB_ICAPTCHA_API_KEY must normalize to None"
);
assert_eq!(non_empty.api_key.as_deref(), Some("secret-bearer"));
}
}
@@ -1,146 +0,0 @@
//! Tripwire: a committed negative case that locks the blackhole mechanism.
//!
//! The guard against non-loopback egress works because `obtain_proof` builds a
//! stock `reqwest::blocking::Client` that inherits the process's proxy env vars.
//! If the builder later gains `.no_proxy()` or a custom connector, the guard
//! disarms silently and every existing test stays green (no live call is made
//! during the normal mock-consumed run because the mock is on loopback and
//! `NO_PROXY` covers it — the blackhole isn't even exercised).
//!
//! This test asserts that a destination the blackhole's `NO_PROXY` does NOT
//! cover fails *while the blackhole is armed*. The destination is a local HTTP
//! server on `[::1]` (IPv6 loopback), which `NO_PROXY` (`127.0.0.1, localhost`)
//! does not list. With the blackhole active the proxy intercepts the
//! connection and blocks it; without the blackhole the request would reach the
//! server directly and succeed, turning this test RED.
//!
//! A positive control (disarmed blackhole) runs first, proving the fixture is
//! valid and `[::1]` is reachable.
//!
//! Design constraints (see issue #211):
//! - An unresolvable host would keep the request failing even when the guard
//! is disarmed (DNS failure masquerading as the blackhole), so we must use
//! a reachable address.
//! - A real external host would make a live network call the moment the guard
//! disarms, and on any runner that cannot reach that host the connect error
//! again masquerades as the blackhole. A local loopback address avoids both
//! problems — it is always reachable when unblocked and never reaches the
//! real network.
//! - The address must NOT be covered by `NO_PROXY`, yet must exist on every
//! platform. An arbitrary `127/8` alias is not portable: macOS only binds
//! `127.0.0.1` by default. `[::1]` (IPv6 loopback) is present by default
//! on Linux, macOS, and Windows, and `NO_PROXY` only lists `127.0.0.1` and
//! `localhost`, so the blackhole still intercepts it.
//! - This depends on hyper-util's proxy matcher having no implicit loopback
//! bypass — a host is only proxied-around when literally listed in
//! `NO_PROXY` (verified in the pinned hyper-util; recheck on upgrade).
use std::io::{Read, Write};
use std::net::TcpListener;
use std::thread;
use icaptcha_client::{obtain_proof, Challenge, IcaptchaCfg};
mod support;
/// A minimal HTTP server that responds to iCaptcha challenge and answer
/// requests on the IPv6 loopback address NOT covered by NO_PROXY.
///
/// When the blackhole is working, the proxy should intercept these requests
/// and `obtain_proof` should fail. When the blackhole is disarmed, the
/// requests reach this server and the flow succeeds.
fn serve_icaptcha(listener: TcpListener) {
thread::spawn(move || {
for stream in listener.incoming() {
let Ok(mut stream) = stream else { continue };
// Read until the first \n (end of request line) so TCP
// segmentation cannot split the path out of a single read.
let mut buf = [0u8; 4096];
let mut n = 0usize;
while n < buf.len() {
match stream.read(&mut buf[n..]) {
Ok(0) | Err(_) => break,
Ok(read) => {
n += read;
if buf[..n].contains(&b'\n') {
break;
}
}
}
}
if n == 0 {
continue;
}
let request = String::from_utf8_lossy(&buf[..n]);
let body = if request.contains("/v1/answer") {
r#"{"status":"passed","proof":"PROOF-TRIP"}"#
} else {
r#"{"challengeId":"c1","type":"anagram","difficulty":1,"prompt":"listen","token":"tok-1"}"#
};
let response = format!(
"HTTP/1.1 200 OK\r\nContent-Type: application/json\r\nContent-Length: {}\r\nConnection: close\r\n\r\n{body}",
body.len(),
);
let _ = stream.write_all(response.as_bytes());
}
});
}
fn make_cfg(url: &str) -> IcaptchaCfg {
IcaptchaCfg {
url: url.to_string(),
did: "did:key:zTEST".to_string(),
level: 1,
api_key: None,
}
}
#[test]
fn obtain_proof_blackhole_tripwire() {
// Bind to [::1], IPv6 loopback, which NO_PROXY (127.0.0.1, localhost)
// does not cover. Unlike an arbitrary 127/8 alias, IPv6 loopback is
// present by default on Linux, macOS, and Windows, so the fixture is
// portable across platforms. Fail loudly if it cannot be bound: a
// silent skip would report this tripwire as green while exercising
// none of the negative control, recreating the very gap it exists to
// close.
let listener = TcpListener::bind(("::1", 0)).expect(
"cannot bind [::1] for the blackhole tripwire. IPv6 loopback (::1) \
is present by default on Linux, macOS, and Windows. This test fails \
rather than silently skipping so a platform without it cannot count \
the negative control as passed.",
);
let port = listener.local_addr().unwrap().port();
let url = format!("http://[::1]:{port}");
serve_icaptcha(listener);
// ── Positive control: disarmed blackhole → obtain_proof succeeds ──────
let _disarmed_guard = support::disarm_proxy_env();
let solve: &dyn Fn(&Challenge) -> Option<String> = &|_c| Some("silent".to_string());
let cfg = make_cfg(&url);
let proof = obtain_proof(&cfg, Some(solve))
.expect("positive control: obtain_proof should succeed when the proxy is disarmed");
assert_eq!(
proof, "PROOF-TRIP",
"positive control: unexpected proof value"
);
drop(_disarmed_guard);
// ── Negative control: armed blackhole → obtain_proof fails with connect error ──
let _armed_guard = support::arm_blackhole("http://127.0.0.1:1");
let result = obtain_proof(&cfg, Some(solve));
let err = result.expect_err(
"blackhole tripwire: obtain_proof succeeded against [::1], \
meaning the proxy blackhole did not intercept the request; \
the no-live-call guard is disarmed",
);
assert!(
err.chain().any(|c| c
.downcast_ref::<reqwest::Error>()
.is_some_and(|e| e.is_connect())),
"expected a connect/proxy error, got: {err:#}",
);
}
@@ -1,4 +1,4 @@
//! INV-19 guard: the icaptcha flow must actually call the service (consuming the
//! INV-19 guard: the icaptcha flow must actually call the service (consume the
//! mock), not short-circuit or make a live call. If a future change let
//! `obtain_proof` skip the network (e.g. a stray `cfg(test)` relaxation, which
//! INV-19/INV-20 warn is inert across crates), the `.assert()` calls below go
@@ -8,144 +8,50 @@
//! see a request that ALSO leaks the DID / answer / API key to `DEFAULT_URL` or
//! any other origin (a different host the mock never observes). To make the
//! "no live call" half of the guard load-bearing, the test blackholes every
//! non-loopback destination via a proxy observer: `NO_PROXY` lets the loopback
//! mock through while every proxy variable routes any other host through a TCP
//! listener that counts connections. Zero observed connections proves no leak
//! reached the network on the exercised path.
//!
//! The observer design counts every accepted connection regardless of whether
//! the caller propagates the transport error, so it catches leaks that a
//! closed-port blackhole (which only surfaces propagating errors) would miss.
//! To close the timing race inherent in a detached accept loop, the count is
//! read only after a bounded drain: the test opens a flush connection and
//! blocks until the accept thread has accepted *that specific connection*
//! (tagged by its source port), not merely until any next counter tick.
//! Because `accept()` hands out connections FIFO, every leak connection that
//! completed its TCP handshake before the flush is counted before the flush —
//! the assertion therefore covers all egress in flight when `obtain_proof`
//! returned. A leak that starts its handshake after the drain is outside the
//! window; that is the irreducible limit of observing a detached accept loop,
//! and it is the remaining, well-defined gap this guard accepts.
//!
//! SCOPE: The observer only witnesses leaks issued on the path this test drives
//! (a single passing round, 200 on both endpoints, no PoW, no Continue/Failed,
//! no non-2xx response). A leak living on an error or retry branch makes no
//! connection during the run and the zero-count assertion stays green. The
//! guarantee covers only the exercised happy path and proxy-honoring transports;
//! a future `.no_proxy()` or raw `TcpStream` bypasses the observer entirely.
//! non-loopback destination: `NO_PROXY` lets the loopback mock through while a
//! closed proxy port routes any other host nowhere, so the client contacting
//! anything but the injected mock fails the request instead of silently
//! succeeding. Point `cfg.url` at `DEFAULT_URL` and this test goes RED.
//!
//! `ALL_PROXY` alone is NOT enough: reqwest honors the scheme-specific
//! `HTTPS_PROXY` / `https_proxy` (and the http equivalents) OVER `ALL_PROXY`, so
//! a runner with a working `HTTPS_PROXY` in its environment would route an https
//! leak to `DEFAULT_URL` (which is https) through that proxy while the loopback
//! mock is still consumed and this test stayed green. So we blackhole the
//! scheme-specific variables too.
use std::net::{TcpListener, TcpStream};
use std::sync::{Arc, Condvar, Mutex};
//! mock is still consumed and this test stayed green. So we blackhole the
//! scheme-specific variables too. No restore is needed: this file has a single
//! `#[test]`, so it runs in its own process and the override never races a
//! sibling test.
use icaptcha_client::{obtain_proof, Challenge, IcaptchaCfg};
mod support;
/// A TCP listener that counts every accepted connection.
///
/// Acts as the proxy target: any leaked connection to a non-loopback
/// destination will be routed through this listener and counted. Every
/// accepted connection is recorded by its source port so the test's own
/// flush connection ([`ProxyObserver::drain`]) can be recognised and
/// excluded from the leak count.
struct ProxyObserver {
_listener: Arc<TcpListener>,
port: u16,
/// Source ports of every accepted connection, in accept order.
accepted: Arc<Mutex<Vec<u16>>>,
/// Source ports of the test's own flush connections (never counted).
flush_ports: Arc<Mutex<Vec<u16>>>,
/// Signalled after each accept so `drain` can wait on the flush.
ack: Arc<Condvar>,
}
impl ProxyObserver {
fn bind() -> Self {
let listener = Arc::new(TcpListener::bind("127.0.0.1:0").expect("bind proxy observer"));
let port = listener.local_addr().unwrap().port();
let accepted = Arc::new(Mutex::new(Vec::new()));
let flush_ports = Arc::new(Mutex::new(Vec::new()));
let ack = Arc::new(Condvar::new());
let a = Arc::clone(&accepted);
let c = Arc::clone(&ack);
let l = Arc::clone(&listener);
std::thread::spawn(move || {
for stream in l.incoming() {
let Ok(stream) = stream else { continue };
a.lock()
.unwrap()
.push(stream.peer_addr().map(|a| a.port()).unwrap_or(0));
c.notify_all();
}
});
ProxyObserver {
_listener: listener,
port,
accepted,
flush_ports,
ack,
}
}
fn port(&self) -> u16 {
self.port
}
/// Number of accepted connections that are not the test's own flush
/// connections — i.e. the leak count.
fn count(&self) -> usize {
let flush = self.flush_ports.lock().unwrap();
let accepted = self.accepted.lock().unwrap();
accepted.iter().filter(|p| !flush.contains(p)).count()
}
/// Bounded acknowledgement barrier for the accept loop.
///
/// Opens a flush connection and blocks until the accept thread has
/// accepted *that connection* (identified by its source port), bounded by
/// a timeout. Because `accept()` hands out connections FIFO, once the
/// flush is accepted every connection that completed its handshake before
/// it has been accepted and recorded too — so counting on afterwards
/// cannot miss a leak that was already in flight. The flush connection
/// itself is never counted.
///
/// A counter-tick barrier would be unsound here: sampling "current + 1"
/// and waiting for any increment lets a leak accepted after the sample
/// satisfy the wait, and the flush is then miscounted as that leak.
fn drain(&self) {
let flush =
TcpStream::connect(("127.0.0.1", self.port)).expect("observer flush connection");
let flush_port = flush.local_addr().unwrap().port();
self.flush_ports.lock().unwrap().push(flush_port);
let mut accepted = self.accepted.lock().unwrap();
while !accepted.contains(&flush_port) {
let (guard, timed_out) = self
.ack
.wait_timeout(accepted, std::time::Duration::from_secs(5))
.unwrap();
accepted = guard;
assert!(
!timed_out.timed_out(),
"proxy observer drain timed out; accept loop stalled"
);
}
}
}
#[test]
fn obtain_proof_consumes_the_mocked_service_and_makes_no_live_call() {
let observer = ProxyObserver::bind();
let _guard = support::arm_blackhole(&format!("http://127.0.0.1:{}", observer.port()));
// Blackhole every non-loopback origin: the loopback mock bypasses the proxy
// (NO_PROXY); any other host is forced through a closed port and fails.
// reqwest reads these at client-build time, and obtain_proof builds its
// client fresh, so this bounds the transport for the whole flow.
//
// Set the scheme-specific vars too (both cases), not just ALL_PROXY: reqwest
// gives HTTPS_PROXY/https_proxy precedence over ALL_PROXY for https URLs, so
// an ALL_PROXY-only blackhole leaves an https leak to DEFAULT_URL open on a
// runner whose environment already has a working HTTPS_PROXY.
let blackhole = "http://127.0.0.1:1";
std::env::set_var("NO_PROXY", "127.0.0.1,localhost");
std::env::set_var("no_proxy", "127.0.0.1,localhost");
for var in [
"ALL_PROXY",
"all_proxy",
"HTTPS_PROXY",
"https_proxy",
"HTTP_PROXY",
"http_proxy",
] {
std::env::set_var(var, blackhole);
}
let mut server = mockito::Server::new();
// The two endpoints the flow hits, each expected exactly once.
let challenge = server
.mock("POST", "/v1/challenge")
.with_status(200)
@@ -171,29 +77,18 @@ fn obtain_proof_consumes_the_mocked_service_and_makes_no_live_call() {
api_key: None,
};
// "anagram" is not a built-in solvable type, so the flow consults this
// solver callback. The answer body itself does not matter: the mocked
// /v1/answer returns "passed" regardless of what is submitted.
let solve = |_c: &Challenge| Some("silent".to_string());
let solver: &dyn Fn(&Challenge) -> Option<String> = &solve;
let count_before = observer.count();
let proof = obtain_proof(&cfg, Some(solver))
.expect("obtain_proof should complete against the mocked service");
assert_eq!(proof, "PROOF-XYZ");
// INV-19: prove the client actually called the mocked endpoints. Had it made
// a live call or skipped the network, these would fail (mock never hit).
challenge.assert();
answer.assert();
// Drain the accept loop before asserting: the flush connection is not
// counted, and because accept() is FIFO every leak connection that
// completed its handshake before the flush is recorded before the flush
// is — so the count after drain reflects all egress in flight during the
// call.
observer.drain();
let leak_count = observer.count() - count_before;
assert_eq!(
leak_count, 0,
"proxy observer caught leaked connections; \
the blackhole failed to contain egress"
);
}
@@ -1,93 +0,0 @@
//! Shared helpers for the icaptcha-client integration tests.
//!
//! CONCURRENCY CONSTRAINT: [`arm_blackhole`] and [`disarm_proxy_env`] mutate
//! process-wide proxy env vars and restore them on drop. Each test binary
//! that consumes this module currently has exactly one `#[test]`, so there is
//! no concurrent use. `cargo test` runs tests within a binary in parallel by
//! default, so the NEXT test added to either file MUST be serialized against
//! these helpers (e.g. a shared `Mutex` in this module, or the `serial_test`
//! crate) or it will race on the proxy vars.
use std::ffi::OsString;
const PROXY_VARS: &[&str] = &[
"NO_PROXY",
"no_proxy",
"ALL_PROXY",
"all_proxy",
"HTTPS_PROXY",
"https_proxy",
"HTTP_PROXY",
"http_proxy",
"REQUEST_METHOD",
];
/// Captures the current values of a set of env vars and restores them on drop.
pub struct EnvSnapshot(Vec<(String, Option<OsString>)>);
impl EnvSnapshot {
pub fn capture(vars: &[&str]) -> Self {
EnvSnapshot(
vars.iter()
.map(|v| (v.to_string(), std::env::var_os(v)))
.collect(),
)
}
}
impl Drop for EnvSnapshot {
fn drop(&mut self) {
for (var, val) in &self.0 {
match val {
Some(v) => std::env::set_var(var, v),
None => std::env::remove_var(var),
}
}
}
}
/// Guard that restores env vars captured at construction time.
pub struct EnvGuard {
_prev: EnvSnapshot,
}
/// Override every proxy-related env var so that non-loopback destinations are
/// forced through `proxy_url`, with NO_PROXY covering loopback. Also clears
/// REQUEST_METHOD, which causes hyper-util to ignore all proxy variables when
/// set. Restores prior values on drop.
///
/// Not safe to call from concurrent tests (see module docs).
pub fn arm_blackhole(proxy_url: &str) -> EnvGuard {
let prev = EnvSnapshot::capture(PROXY_VARS);
std::env::set_var("NO_PROXY", "127.0.0.1,localhost");
std::env::set_var("no_proxy", "127.0.0.1,localhost");
for var in [
"ALL_PROXY",
"all_proxy",
"HTTPS_PROXY",
"https_proxy",
"HTTP_PROXY",
"http_proxy",
] {
std::env::set_var(var, proxy_url);
}
std::env::remove_var("REQUEST_METHOD");
EnvGuard { _prev: prev }
}
/// Clear every proxy-related env var so connections go direct. Restores
/// prior values on drop.
///
/// Not safe to call from concurrent tests (see module docs).
#[allow(dead_code)]
pub fn disarm_proxy_env() -> EnvGuard {
let prev = EnvSnapshot::capture(PROXY_VARS);
for var in PROXY_VARS {
std::env::remove_var(var);
}
EnvGuard { _prev: prev }
}
-4
View File
@@ -44,10 +44,6 @@ services:
GITLAWB_BOOTSTRAP_PEERS: ${GITLAWB_BOOTSTRAP_PEERS:-}
GITLAWB_BOOTSTRAP_DISABLE_SEEDS: ${GITLAWB_BOOTSTRAP_DISABLE_SEEDS:-false}
GITLAWB_REQUIRE_SIGNED_PEER_WRITES: ${GITLAWB_REQUIRE_SIGNED_PEER_WRITES:-false}
# Owner-only push. The default matches the node's own, so the container path
# behaves like a source build; forwarding it is what makes the documented
# rolling-upgrade opt-out reachable for an operator running compose.
GITLAWB_ENFORCE_OWNER_PUSH: ${GITLAWB_ENFORCE_OWNER_PUSH:-true}
GITLAWB_P2P_BOOTSTRAP: ${GITLAWB_P2P_BOOTSTRAP:-}
GITLAWB_PUBLIC_READ: ${GITLAWB_PUBLIC_READ:-true}
GITLAWB_MAX_PACK_BYTES: ${GITLAWB_MAX_PACK_BYTES:-2147483648}
+2 -2
View File
@@ -38,8 +38,8 @@ Owner focus: node reliability.
Owner focus: protocol/auth.
- Close default-open repo write authorization and wire trusted UCAN delegation into repository permissions.
- Close the remaining visibility gaps: restrict or document task, pin, and anchor metadata routes, and publish an irreversible-publication policy.
- Implement repo write authorization: repo owner checks, protected branches, UCAN capability checks, and clear delegation semantics.
- Implement private-read enforcement or remove private-repo affordances until it exists.
- Add UCAN revocation or blocklisting, with an emergency compromised-key runbook.
- Add mutation-aware GraphQL auth before GraphQL becomes a public write API surface.
- Harden peer registration and outbound peer calls against SSRF and peer-list poisoning.
+1 -6
View File
@@ -3,11 +3,6 @@
Date: 2026-05-28
Repo state inspected: `main` tracking `origin/main`, starting at `b12c6bc feat: per-DID rate limiting on creation endpoints (10/hour) (#13)`.
> **Historical snapshot.** This audit describes the repository at the commit above. Its
> statements about incomplete UCAN chain validation and absent per-repository read
> enforcement have been superseded; see [`SECURITY.md`](../SECURITY.md) for the current
> security posture.
## Commands run
```sh
@@ -93,7 +88,7 @@ Fixed or staged in this pass:
Live-network blockers to prioritize:
- GraphQL POST is still open for compatibility; GraphQL mutations should get mutation-aware auth before it becomes a public write API surface.
- Push authorization is still not capability-complete. A valid DID signature is authentication, not authorization. Owner checks are now enforced on every branch, protected or not (`GITLAWB_ENFORCE_OWNER_PUSH`, on by default); what remains is that a UCAN `git/push` capability is not yet honored, so a delegated or CI key cannot push.
- Push authorization is still not capability-complete. A valid DID signature is authentication, not authorization; unprotected repo branches do not yet enforce owner/UCAN capability checks.
- UCAN chain validation is incomplete and UCAN revocation/blocklisting is not implemented as an operator feature.
- Private repository reads are not enforced. `is_public` and `GITLAWB_PUBLIC_READ` exist, but per-repository private-read behavior is not wired.
- Peer URLs are self-asserted by DIDs. Signatures prove control of the DID key when present, not ownership/safety of the announced URL.
+20 -35
View File
@@ -141,43 +141,31 @@ During the cooldown your node still earns rewards if it keeps heartbeating.
---
## Owner-only push
## Hardening: owner-only push
The node requires the authenticated pusher to be the repo owner on **every**
branch. A push whose authenticated DID is not the repo owner is rejected with
HTTP 403 before any ref update is applied. The owner is matched in both the full
`did:key:z6Mk…` form and its bare `z6Mk…` suffix.
By default the node authenticates every `git-receive-pack` push (a valid RFC 9421
`did:key` signature) but does **not** check that the pusher owns the repo, except
on branches that are explicitly protected. Because `did:key` is self-certifying,
any party can generate a key, derive its DID, sign, and push to an unprotected
branch — authentication is not authorization.
This is on by default, and the default is the point. The node authenticates every
`git-receive-pack` push with a valid RFC 9421 `did:key` signature, but `did:key`
is self-certifying: any party can generate a key, derive its DID and sign.
Authentication is not authorization, so without this gate every signed caller can
push to every repository — private ones included — on any branch that is not
explicitly protected.
### Turning it off
To require the authenticated pusher to be the repo owner on **every** branch, set:
```bash
GITLAWB_ENFORCE_OWNER_PUSH=false
GITLAWB_ENFORCE_OWNER_PUSH=true
```
Both the environment variable and `--enforce-owner-push false` disable it; the
bare `--enforce-owner-push` flag still means `true`.
Do this only for a rolling upgrade whose pushers are not yet the repo owner, and
treat it as temporary: while it is off, your node accepts a push to any repository
from anyone who can generate a keypair.
### Caution: delegated and CI keys are non-owners
Push authorization is owner-only today. A UCAN `git/push` capability is verified
but **not yet honored for authorization**, so a delegated key cannot push while
this gate is on, even when it holds a valid capability for the repo.
If your automation pushes under its own DID rather than the owner's, it will start
getting 403s. Either have it push as the owner, or set
`GITLAWB_ENFORCE_OWNER_PUSH=false` until scoped collaborator / UCAN-delegated push
rights land — that work is what removes this trade-off.
- **Default `false`** — preserves current behavior so live nodes are unaffected by
an upgrade. Turn it on once you're ready for owner-only writes.
- **When `true`** — a push whose authenticated DID is not the repo owner is
rejected (HTTP 403) before any ref update is applied. The owner is matched in
both the full `did:key:z6Mk…` form and its bare `z6Mk…` suffix.
- **Caution: this blocks every non-owner pusher, including your own delegated and
CI agents.** Push authorization is owner-only today — a UCAN `git/push`
capability is verified but not yet honored for authorization, so delegated keys
cannot push while this is on. Don't enable it until every identity that pushes
to your repos is the owner, or you'll lock out your own automation. Scoped
collaborator / UCAN-delegated push rights are a planned follow-up.
---
@@ -189,8 +177,7 @@ rights land — that work is what removes this trade-off.
| Missed heartbeats | After 3 days without one, you're excluded from rewards until you beat again |
| Operator key | Dedicated wallet, small ETH balance, not your main treasury |
| Monitoring | Watch `lastHeartbeat` on-chain; alert if > 22h since last beat |
| Public URL | Must resolve and serve `/health` for liveness and DB-aware `/ready` for peer readiness |
| Rolling upgrade (advisory-lock algorithm) | The SHA-256 advisory-lock key swap (replacing `DefaultHasher`) re-keys every repo in the first release that carries it. During a shared-Postgres rolling upgrade, old nodes compute a different `i64` key than new nodes for the same repo, so PostgreSQL treats them as independent locks and cross-machine write-exclusion is lost. **Drain in-flight writes** or cut over through a **single node** before bringing new-binary nodes online. |
| Public URL | Must resolve and serve `/health` — peers will ping it |
---
@@ -200,8 +187,6 @@ rights land — that work is what removes this trade-off.
**Node refuses to start with "strict-mode operator check failed"** — either `gl node register` first, or unset `GITLAWB_OPERATOR_STRICT_MODE`.
**Node refuses to start with "GITLAWB_DB_MAX_CONNECTIONS (20) must be at least max_concurrent_git_pushes (32) + 8 headroom"**: raise `GITLAWB_DB_MAX_CONNECTIONS` to at least the push cap plus 8 (40 with the default cap of 32; 48 is the shipped default and the recommended value), or lower `GITLAWB_MAX_CONCURRENT_GIT_PUSHES`. This bites a node upgraded in place that still sets the old pool size of 20. The check is deliberate: each concurrent push pins one pooled connection for its whole receive-pack, so a pool that does not clear the push cap lets a burst of slow pushes starve every other database path.
**Rewards are 0 after a week** — run `gl node onchain-status`. If `currentlyActive: false`, check your heartbeat loop (node logs for `operator heartbeat sent`).
**Want to rotate operator wallet** — requires unstake → re-register with new wallet. No in-place rotation in v1.
+11 -60
View File
@@ -91,39 +91,13 @@ User-data only runs at first boot, so upgrades go through SSM:
$(terraform output -raw upgrade_command)
```
This reinstalls the rendered `/opt/gitlawb/compose.yaml`, then runs
`docker compose pull && docker compose up -d` on the instance.
**Run `terraform apply` before the upgrade command.** The command is an SSM
document managed by Terraform, and it embeds the compose file rendered from your
current variables. An instance that has not had a fresh `apply` still holds the
previous document and will reinstall the previous compose file.
This runs `docker compose pull && docker compose up -d` on the instance.
- With `image_tag = "latest"` (default) that picks up the newest release.
- With a **pinned tag**, set `image_tag` in `terraform.tfvars` and `apply` — the
rendered compose carries it, so the upgrade installs the right version.
- **`compose.yaml` is Terraform-owned and is overwritten on every upgrade.**
Per-instance settings belong in `/opt/gitlawb/.env`, which is never touched.
- **The upgrade reconciles the service set, not just the image.** The rendering is
conditional — `postgres` exists only when `db_host` is empty, `caddy` only when
`domain_name` is set — and the command runs `--remove-orphans`. So setting
`use_rds`, or clearing `domain_name`, and then upgrading will **stop and remove**
the local postgres or the TLS terminator. That is the intended outcome of those
variables, but it happens on the upgrade rather than on `apply`. Data survives
either way: both bind-mount under `/mnt/data`.
### Why the upgrade rewrites the compose file
`aws_instance` deliberately ignores `user_data` drift, so an instance keeps the
compose file it was created with. Compose passes only the variables named in a
service's `environment:` block into the container, so **a node setting added to
the template never reaches an existing instance** — writing it to
`/opt/gitlawb/.env` and restarting leaves the node on its built-in default, with
no error to indicate it. `pull && up -d` alone cannot fix that, because the file
on disk is the authoritative one.
Reinstalling the rendering first makes the upgrade a real migration. It is
idempotent: on an already-current instance it writes identical bytes.
- With a **pinned tag**, first edit the tag in `/opt/gitlawb/compose.yaml` on
the instance (via SSM session), then run the upgrade command — and keep
`image_tag` in terraform.tfvars in sync so a future instance replacement
boots the same version.
Replace the instance itself (OS/AMI/instance-type changes) with
`terraform apply -replace=aws_instance.node` — the data volume reattaches and
@@ -131,24 +105,12 @@ Replace the instance itself (OS/AMI/instance-type changes) with
## Changing configuration
`terraform apply` alone never changes a running instance: user-data runs once and
the instance ignores `user_data` drift. How a change rolls out depends on which
of the two files carries it.
**Rendered into `compose.yaml`** — `image_tag`, `gitlawb_port`, `metrics_port`,
`domain_name`, `db_host`, the `icaptcha_*` values. `apply`, then run
`upgrade_command`: it installs the new rendering and restarts.
```sh
terraform apply
$(terraform output -raw upgrade_command)
```
**Written into `/opt/gitlawb/.env` at first boot** — `public_url`,
`bootstrap_peers`, `auto_sync`, `max_pack_bytes`, and the integration secrets.
Editing these in `terraform.tfvars` does **not** reach a running instance, because
nothing rewrites `.env` after the first boot. Either edit `/opt/gitlawb/.env` on
the instance and `docker compose up -d`, or replace it:
User-data only runs at first boot, and the instance ignores `user_data` drift
(`ignore_changes`), so editing terraform.tfvars values that feed the bootstrap
(`bootstrap_peers`, `public_url`, integrations, `image_tag`) does **not**
affect a running instance on `terraform apply`. To roll out such changes,
either edit `/opt/gitlawb/.env` on the instance (SSM session, then
`docker compose up -d`), or replace the instance:
```sh
terraform apply -replace=aws_instance.node
@@ -156,17 +118,6 @@ terraform apply -replace=aws_instance.node
The data volume reattaches; repos, postgres data, and the identity key survive.
**A node setting the installed `compose.yaml` does not already name** —
`GITLAWB_ENFORCE_OWNER_PUSH` on an instance created before it was added, for
example. Editing `.env` is **not enough**: compose only passes through the
variables its `environment:` block lists, so the value is read from `.env` and
then dropped. Run `apply` plus `upgrade_command` first, which installs a compose
file that names the key; after that, `.env` governs it like any other.
That last case is the one worth remembering: `.env` reaches the container only for
keys the *installed* compose interpolates, and the upgrade is what makes a newly
added key one of them.
## Remote state (optional)
Local state is the default. To move state to S3: create a versioned bucket,
-3
View File
@@ -72,9 +72,6 @@ services:
GITLAWB_BOOTSTRAP_PEERS: $${GITLAWB_BOOTSTRAP_PEERS}
GITLAWB_AUTO_SYNC: $${GITLAWB_AUTO_SYNC}
GITLAWB_MAX_PACK_BYTES: $${GITLAWB_MAX_PACK_BYTES}
# Owner-only push. Matches the node's own default; forwarding it is what makes
# the documented rolling-upgrade opt-out reachable on the container path.
GITLAWB_ENFORCE_OWNER_PUSH: $${GITLAWB_ENFORCE_OWNER_PUSH:-true}
# On-chain PoS (optional — empty unless set in terraform.tfvars)
GITLAWB_CHAIN_RPC_URL: $${GITLAWB_CHAIN_RPC_URL}
GITLAWB_CONTRACT_NODE_STAKING: $${GITLAWB_CONTRACT_NODE_STAKING}
+7 -62
View File
@@ -350,16 +350,10 @@ resource "aws_instance" "node" {
# ami: new AL2023 releases shouldn't churn the instance; replace
# deliberately for OS upgrades.
# user_data: only runs at first boot, so re-rendering it on a live
# instance is a pointless stop/start. How a config change reaches a running
# instance depends on which file carries it:
# - rendered into compose.yaml (image_tag, ports, domain_name, db_host,
# icaptcha_*): `terraform apply` then `upgrade_command`, which
# reinstalls the rendering and restarts.
# - written into /opt/gitlawb/.env at first boot (public_url,
# bootstrap_peers, integrations): nothing rewrites .env afterwards, so
# these need an edit on the instance or
# `terraform apply -replace=aws_instance.node`.
# See README "Changing configuration".
# instance is a pointless stop/start. Config changes that feed user-data
# (bootstrap peers, integrations, image tag) require a deliberate
# `terraform apply -replace=aws_instance.node` — see README "Changing
# configuration".
ignore_changes = [ami, user_data]
}
}
@@ -443,62 +437,13 @@ resource "aws_ssm_document" "upgrade" {
content = jsonencode({
schemaVersion = "2.2"
description = "Reinstall the rendered compose file, pull the latest image, and restart"
description = "Pull the latest gitlawb node image and restart the compose stack"
mainSteps = [{
action = "aws:runShellScript"
name = "upgrade"
inputs = {
# Reinstall the CURRENT rendering before restarting.
#
# `aws_instance` ignores user_data drift, so an instance created before a
# template change keeps the compose file it was born with. Compose passes
# only the variables named in its `environment:` block into the container,
# so a node setting added to the template never reaches an existing
# instance — the operator writes it to /opt/gitlawb/.env, restarts, and the
# node silently keeps the built-in default. `docker compose pull && up -d`
# cannot fix that: the authoritative file is the one on disk.
#
# Idempotent: writing the same bytes and restarting is a no-op. This does
# overwrite hand-edits to compose.yaml, which is intended — the file is
# Terraform-owned; per-instance settings belong in /opt/gitlawb/.env.
#
# One shell step, under flock, writing through mktemp. Two overlapping
# executions — a double run, a retry while the first is still pulling, two
# operators — otherwise share one fixed temp path: the second truncates the
# file the first is about to install, and once the first renames, the
# second's open descriptor keeps writing into the live compose.yaml inode.
# Both outcomes stay valid YAML, so the `up -d --remove-orphans` on the next
# line accepts a blended or truncated service set and deletes whatever fell
# off — on a live node, the local postgres or the TLS terminator. The lock
# also covers pull and up, so a second run cannot restart against a file the
# first is mid-install.
runCommand = [<<-UPGRADE
set -euo pipefail
install -d -m 0755 /opt/gitlawb
exec 9>/opt/gitlawb/.upgrade.lock
if ! flock -n 9; then
echo "another upgrade is already running on this instance" >&2
exit 1
fi
tmp="$(mktemp /opt/gitlawb/compose.yaml.XXXXXX)"
trap 'rm -f "$tmp"' EXIT
# base64 rather than a nested heredoc: the rendered compose contains
# `$${VAR}` passthroughs that must reach the file unexpanded, and a
# heredoc inside an indented Terraform heredoc depends on the dedent
# landing the delimiter at column 0. Decoding a single line has neither
# failure mode, and no delimiter can collide with the content.
echo '${base64encode(local.compose_yaml)}' | base64 -d > "$tmp"
chmod 0644 "$tmp"
mv "$tmp" /opt/gitlawb/compose.yaml
trap - EXIT
cd /opt/gitlawb
docker compose pull
docker compose up -d --remove-orphans
docker image prune -f
UPGRADE
runCommand = [
"cd /opt/gitlawb && docker compose pull && docker compose up -d --remove-orphans && docker image prune -f"
]
}
}]
-65
View File
@@ -1,65 +0,0 @@
# fly.toml app configuration file generated for gitlawb-node-2 on 2026-07-22T09:47:18+08:00
#
# See https://fly.io/docs/reference/configuration/ for information about how to use this file.
#
app = 'gitlawb-node-2'
primary_region = 'sjc'
[build]
dockerfile = '../../Dockerfile'
[env]
GITLAWB_AUTO_SYNC = 'true'
GITLAWB_BOOTSTRAP_PEERS = 'https://node.gitlawb.com,https://node3.gitlawb.com'
GITLAWB_HOST = '0.0.0.0'
GITLAWB_KEY = '/data/keys/identity.pem'
GITLAWB_MAX_PACK_BYTES = '524288000'
GITLAWB_P2P_PORT = '7546'
GITLAWB_PORT = '7545'
GITLAWB_PUBLIC_URL = 'https://node2.gitlawb.com'
GITLAWB_REPOS_DIR = '/data/repos'
ICAPTCHA_MODE = 'enforce'
ICAPTCHA_PUBKEY = 'xjyPNqIbvc9U-kwXW6u9mDqRJ7E2UUMOaJdUWhpEXq8'
ICAPTCHA_REQUIRED_LEVEL = '3'
[[mounts]]
source = 'gitlawb_data'
destination = '/data'
[http_service]
internal_port = 7545
force_https = true
auto_stop_machines = 'off'
auto_start_machines = true
min_machines_running = 1
[http_service.concurrency]
type = 'connections'
hard_limit = 500
soft_limit = 400
[http_service.http_options]
idle_timeout = 120
[[http_service.checks]]
interval = '15s'
timeout = '5s'
grace_period = '30s'
method = 'GET'
path = '/ready'
[[services]]
protocol = 'udp'
internal_port = 7546
[[services.ports]]
port = 7546
[[vm]]
size = 'shared-cpu-1x'
memory = '1024'
[[metrics]]
port = 9091
path = '/metrics'
-65
View File
@@ -1,65 +0,0 @@
# fly.toml app configuration file generated for gitlawb-node-3 on 2026-07-22T09:47:19+08:00
#
# See https://fly.io/docs/reference/configuration/ for information about how to use this file.
#
app = 'gitlawb-node-3'
primary_region = 'nrt'
[build]
dockerfile = '../../Dockerfile'
[env]
GITLAWB_AUTO_SYNC = 'true'
GITLAWB_BOOTSTRAP_PEERS = 'https://node.gitlawb.com,https://node2.gitlawb.com'
GITLAWB_HOST = '0.0.0.0'
GITLAWB_KEY = '/data/keys/identity.pem'
GITLAWB_MAX_PACK_BYTES = '524288000'
GITLAWB_P2P_PORT = '7546'
GITLAWB_PORT = '7545'
GITLAWB_PUBLIC_URL = 'https://node3.gitlawb.com'
GITLAWB_REPOS_DIR = '/data/repos'
ICAPTCHA_MODE = 'enforce'
ICAPTCHA_PUBKEY = 'xjyPNqIbvc9U-kwXW6u9mDqRJ7E2UUMOaJdUWhpEXq8'
ICAPTCHA_REQUIRED_LEVEL = '3'
[[mounts]]
source = 'gitlawb_data'
destination = '/data'
[http_service]
internal_port = 7545
force_https = true
auto_stop_machines = 'off'
auto_start_machines = true
min_machines_running = 1
[http_service.concurrency]
type = 'connections'
hard_limit = 500
soft_limit = 400
[http_service.http_options]
idle_timeout = 120
[[http_service.checks]]
interval = '15s'
timeout = '5s'
grace_period = '30s'
method = 'GET'
path = '/ready'
[[services]]
protocol = 'udp'
internal_port = 7546
[[services.ports]]
port = 7546
[[vm]]
size = 'shared-cpu-1x'
memory = '1024'
[[metrics]]
port = 9091
path = '/metrics'
@@ -19,11 +19,6 @@ class Config {
var operatorPrivateKey: String = ""
var tigrisBucket: String = ""
var autoSync: Bool = false
/// Owner-only push. Persisted as a setting rather than read from a hand-edited
/// `.env`, because `writeEnvFile()` regenerates that file on every start and
/// would discard the edit. Matches the node's own default; set false only for a
/// rolling upgrade whose pushers are not yet the repo owner.
var enforceOwnerPush: Bool = true
var repoPathString: String = ""
// MARK: - Paths
@@ -81,7 +76,6 @@ class Config {
"operatorPrivateKey": operatorPrivateKey,
"tigrisBucket": tigrisBucket,
"autoSync": autoSync,
"enforceOwnerPush": enforceOwnerPush,
"repoPath": repoPathString,
]
@@ -111,7 +105,6 @@ class Config {
operatorPrivateKey = dict["operatorPrivateKey"] as? String ?? operatorPrivateKey
tigrisBucket = dict["tigrisBucket"] as? String ?? tigrisBucket
autoSync = dict["autoSync"] as? Bool ?? autoSync
enforceOwnerPush = dict["enforceOwnerPush"] as? Bool ?? enforceOwnerPush
repoPathString = dict["repoPath"] as? String ?? repoPathString
}
@@ -128,7 +121,6 @@ class Config {
lines.append("GITLAWB_PORT=\(httpPort)")
lines.append("GITLAWB_P2P_PORT=\(p2pPort)")
lines.append("GITLAWB_AUTO_SYNC=\(autoSync)")
lines.append("GITLAWB_ENFORCE_OWNER_PUSH=\(enforceOwnerPush)")
if !chainRpcURL.isEmpty {
lines.append("GITLAWB_CHAIN_RPC_URL=\(chainRpcURL)")
@@ -294,10 +294,6 @@ services:
GITLAWB_HOST: 0.0.0.0
GITLAWB_PUBLIC_URL: ${GITLAWB_PUBLIC_URL:-http://localhost:7545}
GITLAWB_P2P_PORT: 7546
# Owner-only push. Must match the bundled Resources/docker-compose.yml: this
# generated file — not the bundled resource — is what a packaged app runs, so
# omitting it here leaves the documented opt-out unreachable for those users.
GITLAWB_ENFORCE_OWNER_PUSH: ${GITLAWB_ENFORCE_OWNER_PUSH:-true}
GITLAWB_CHAIN_RPC_URL: ${GITLAWB_CHAIN_RPC_URL:-}
GITLAWB_CONTRACT_NODE_STAKING: ${GITLAWB_CONTRACT_NODE_STAKING:-}
GITLAWB_OPERATOR_PRIVATE_KEY: ${GITLAWB_OPERATOR_PRIVATE_KEY:-}
@@ -33,9 +33,6 @@ services:
GITLAWB_PORT: ${GITLAWB_PORT:-7545}
GITLAWB_PUBLIC_URL: ${GITLAWB_PUBLIC_URL:-http://localhost:7545}
GITLAWB_P2P_PORT: ${GITLAWB_P2P_PORT:-7546}
# Owner-only push. Matches the node's own default; forwarding it is what makes
# the documented rolling-upgrade opt-out reachable on the container path.
GITLAWB_ENFORCE_OWNER_PUSH: ${GITLAWB_ENFORCE_OWNER_PUSH:-true}
# Sync
GITLAWB_AUTO_SYNC: ${GITLAWB_AUTO_SYNC:-false}
# On-chain PoS (optional — leave unset for local/dev)
@@ -28,7 +28,6 @@ struct SettingsView: View {
@State private var operatorPrivateKey: String = Config.shared.operatorPrivateKey
@State private var tigrisBucket: String = Config.shared.tigrisBucket
@State private var autoSync: Bool = Config.shared.autoSync
@State private var enforceOwnerPush: Bool = Config.shared.enforceOwnerPush
var body: some View {
VStack(alignment: .leading, spacing: 12) {
@@ -54,15 +53,6 @@ struct SettingsView: View {
.font(.caption)
.foregroundColor(.secondary)
}
VStack(alignment: .leading, spacing: 2) {
Toggle("Require repo owner to push", isOn: $enforceOwnerPush)
Text("A signature proves who is pushing, not that they own the repo. "
+ "Turn this off only during a rolling upgrade whose pushers are "
+ "not yet the owner — while off, anyone who can sign may push to "
+ "any repo on this node.")
.font(.caption)
.foregroundColor(.secondary)
}
Divider()
@@ -106,7 +96,6 @@ struct SettingsView: View {
Config.shared.operatorPrivateKey = operatorPrivateKey
Config.shared.tigrisBucket = tigrisBucket
Config.shared.autoSync = autoSync
Config.shared.enforceOwnerPush = enforceOwnerPush
Config.shared.persist()
Config.shared.writeEnvFile()
+1 -2
View File
@@ -12,8 +12,7 @@
{ "type": "generic", "path": "crates/gitlawb-node/Cargo.toml" },
{ "type": "generic", "path": "crates/gl/Cargo.toml" },
{ "type": "generic", "path": "crates/git-remote-gitlawb/Cargo.toml" },
{ "type": "generic", "path": "crates/gitlawb-attest/Cargo.toml" },
{ "type": "generic", "path": "crates/icaptcha-client/Cargo.toml" }
{ "type": "generic", "path": "crates/gitlawb-attest/Cargo.toml" }
]
}
}
+1 -1
View File
@@ -46,7 +46,7 @@ build_darwin_arm64() {
echo "==> Building darwin-arm64 (native cargo)..."
cd "$ROOT"
export PATH="$HOME/.cargo/bin:$PATH"
cargo build --release --locked -p gl -p git-remote-gitlawb
cargo build --release -p gl -p git-remote-gitlawb
cp target/release/gl "$OUT/gl-darwin-arm64"
cp target/release/git-remote-gitlawb "$OUT/git-remote-gitlawb-darwin-arm64"
chmod +x "$OUT/gl-darwin-arm64" "$OUT/git-remote-gitlawb-darwin-arm64"
-75
View File
@@ -1,75 +0,0 @@
#!/usr/bin/env bash
#
# Dependency-purity gate for the `gitlawb-core` crate.
#
# gitlawb-core is embedded by every consumer (the gl CLI, git-remote-gitlawb,
# the node daemon), so it must stay lean. This script recomputes gitlawb-core's
# NORMAL (non-dev, non-build) transitive dependency set and fails if it contains
# any crate not present in ci/gitlawb-core-allowed-deps.txt.
#
# Hard-fail direction: a crate present now but NOT allowlisted. That is the case
# the gate exists to catch (core silently gaining a heavy dependency).
# Informational only: an allowlisted crate no longer present (stale entry) — a
# legitimate dependency removal should not red CI, so it is reported, not failed.
#
# Runnable from anywhere in the repo.
set -euo pipefail
ROOT="$(git rev-parse --show-toplevel)"
ALLOW="$ROOT/ci/gitlawb-core-allowed-deps.txt"
if [ ! -f "$ALLOW" ]; then
echo "ERROR: allowlist not found at $ALLOW" >&2
exit 1
fi
# `cargo tree` resolves WITHOUT --locked on purpose, unlike the blocking cargo
# jobs in pr-checks. This gate answers one question, whether gitlawb-core's
# dependency closure gained something unallowlisted, and a stale Cargo.lock
# would red it for a reason that has nothing to do with that closure. Lock
# staleness is already caught by the --locked cargo jobs, so it does not need a
# second gate here.
#
# Resolving can refresh Cargo.lock as a side effect, so snapshot and restore
# it: this check must never leave the working tree dirty when run locally.
lock_backup="$(mktemp)"
cp "$ROOT/Cargo.lock" "$lock_backup"
restore_lock() { cp "$lock_backup" "$ROOT/Cargo.lock"; rm -f "$lock_backup"; }
trap restore_lock EXIT
# Current normal-dependency closure of gitlawb-core, one crate name per line.
# Must match the regen command documented in the allowlist header exactly.
current="$(
cargo tree -p gitlawb-core --edges normal --prefix none --manifest-path "$ROOT/Cargo.toml" \
| sed -E 's/ v[0-9].*$//' \
| grep -v '^gitlawb-core$' \
| sort -u
)"
# Allowlist with comments and blank lines stripped.
allowed="$(grep -vE '^[[:space:]]*(#|$)' "$ALLOW" | sort -u)"
# comm needs sorted input; both sides are sorted above.
offenders="$(comm -23 <(printf '%s\n' "$current") <(printf '%s\n' "$allowed"))"
stale="$(comm -13 <(printf '%s\n' "$current") <(printf '%s\n' "$allowed"))"
if [ -n "$stale" ]; then
echo "NOTE: allowlisted crates no longer in gitlawb-core's dependency tree"
echo " (safe to prune from ci/gitlawb-core-allowed-deps.txt):"
printf ' %s\n' $stale
echo
fi
if [ -n "$offenders" ]; then
{
echo "ERROR: gitlawb-core gained dependencies not on the allowlist:"
printf ' %s\n' $offenders
echo
echo "gitlawb-core must stay embeddable and lean. If a new dependency is"
echo "intentional, add it to ci/gitlawb-core-allowed-deps.txt (regen command"
echo "is in that file's header). Otherwise, drop the dependency."
} >&2
exit 1
fi
echo "gitlawb-core dependency purity: OK ($(printf '%s\n' "$current" | wc -l | tr -d ' ') normal deps, all allowlisted)."
-117
View File
@@ -1,117 +0,0 @@
#!/usr/bin/env bash
#
# Release-versioning coverage gate for the workspace crates.
#
# release-please owns every crate's version. It rewrites a version line only
# when BOTH halves hold: the line carries an `# x-release-please-version`
# annotation, and the crate's manifest is listed under `extra-files` in
# release-please-config.json. A crate missing either half is silently skipped
# at release time and freezes at whatever version it was created with, which
# is exactly what happened to icaptcha-client (added as a workspace member
# after the config was last written, so it sat at 0.4.0 while the rest of the
# workspace walked to 0.7.1 across four releases).
#
# Exhaustive-by-construction: the required set is DERIVED from the workspace
# member list via `cargo metadata`, never from a hand-maintained list here. A
# newly added crate reds CI whether or not anyone remembered this gate exists,
# which is the property a hand list cannot have.
#
# Hard-fail directions, all three release-silent:
# 1. a workspace member absent from extra-files
# 2. a workspace member whose version line lacks the annotation
# 3. an extra-files entry pointing at a manifest that no longer exists
#
# Runnable from anywhere in the repo.
set -euo pipefail
ROOT="$(git rev-parse --show-toplevel)"
CONFIG="$ROOT/release-please-config.json"
if [ ! -f "$CONFIG" ]; then
echo "ERROR: release-please config not found at $CONFIG" >&2
exit 1
fi
# Resolving can refresh Cargo.lock as a side effect, so snapshot and restore it:
# this check must never leave the working tree dirty when run locally. Same
# reasoning as scripts/check-gitlawb-core-deps.sh.
lock_backup="$(mktemp)"
cp "$ROOT/Cargo.lock" "$lock_backup"
restore_lock() { cp "$lock_backup" "$ROOT/Cargo.lock"; rm -f "$lock_backup"; }
trap restore_lock EXIT
# Workspace members as repo-relative manifest paths, one per line. This is the
# derived required set: whatever cargo considers a member must be release-managed.
members="$(
cargo metadata --no-deps --format-version 1 --manifest-path "$ROOT/Cargo.toml" \
| jq -r '.packages[].manifest_path' \
| sed "s#^$ROOT/##" \
| sort -u
)"
# Manifest paths release-please is configured to rewrite.
listed="$(jq -r '.packages["."]["extra-files"][] | .path' "$CONFIG" | sort -u)"
unlisted="$(comm -23 <(printf '%s\n' "$members") <(printf '%s\n' "$listed"))"
orphaned="$(comm -13 <(printf '%s\n' "$members") <(printf '%s\n' "$listed"))"
# Second half of the invariant: being listed does nothing without the marker,
# because the generic updater keys on the annotation to find the line.
#
# Scoped to the [package] table on purpose. A manifest-wide match is satisfied by
# an annotated `version` under any other table, so a crate could leave the version
# cargo actually reads unannotated, pass this gate, and still freeze at release
# time while release-please rewrote the decoy line instead.
unmarked=""
while IFS= read -r manifest; do
[ -n "$manifest" ] || continue
if ! awk '
/^[[:space:]]*\[/ { in_package = ($0 ~ /^[[:space:]]*\[package\][[:space:]]*$/); next }
in_package && /^version = ".*" # x-release-please-version/ { found = 1 }
END { exit(found ? 0 : 1) }
' "$ROOT/$manifest"; then
unmarked="$unmarked$manifest"$'\n'
fi
done <<< "$members"
failed=0
if [ -n "$unlisted" ]; then
{
echo "ERROR: workspace members missing from extra-files in release-please-config.json:"
printf ' %s\n' $unlisted
echo
echo "release-please will never bump these crates, so they freeze at their"
echo "current version while the rest of the workspace moves. Add each one as"
echo ' { "type": "generic", "path": "<manifest path>" }'
} >&2
failed=1
fi
if [ -n "$unmarked" ]; then
{
echo "ERROR: workspace members whose version line lacks the release-please annotation:"
printf ' %s\n' $unmarked
echo
echo 'The generic updater finds the line by its trailing `# x-release-please-version`'
echo "comment. Without it the crate is listed but never rewritten."
} >&2
failed=1
fi
if [ -n "$orphaned" ]; then
{
echo "ERROR: extra-files entries pointing at manifests that are not workspace members:"
printf ' %s\n' $orphaned
echo
echo "release-please fails the release when an extra-file path does not exist."
echo "Drop the stale entry, or restore the crate to the workspace."
} >&2
failed=1
fi
if [ "$failed" -ne 0 ]; then
exit 1
fi
echo "release versioning coverage: OK ($(printf '%s\n' "$members" | wc -l | tr -d ' ') workspace members, all annotated and listed)."
-23
View File
@@ -1,23 +0,0 @@
#!/usr/bin/env bash
set -euo pipefail
tag="${1-}"
: "${GITHUB_OUTPUT:?GITHUB_OUTPUT must be set}"
case "$tag" in
""|*[!A-Za-z0-9._-]*)
printf '%s\n' "::error::release tag is empty or contains invalid characters"
exit 1
;;
esac
case "$tag" in
v[0-9]*) ;;
*)
printf '%s\n' "::error::'$tag' does not look like a release tag (vX.Y.Z)"
exit 1
;;
esac
printf '%s\n' "tag=$tag" >> "$GITHUB_OUTPUT"
printf '%s\n' "version=${tag#v}" >> "$GITHUB_OUTPUT"
-163
View File
@@ -1,163 +0,0 @@
#!/usr/bin/env bash
set -euo pipefail
repo_root="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
resolver="$repo_root/scripts/resolve-release-tag.sh"
test_tmp="$(mktemp -d)"
trap 'rm -r -- "$test_tmp"' EXIT
valid_output="$test_tmp/valid-output"
GITHUB_OUTPUT="$valid_output" "$resolver" "v1.2.3"
expected_output="$test_tmp/expected-output"
printf '%s\n' "tag=v1.2.3" "version=1.2.3" > "$expected_output"
cmp "$expected_output" "$valid_output"
newline_output="$test_tmp/newline-output"
newline_stdout="$test_tmp/newline-stdout"
newline_stderr="$test_tmp/newline-stderr"
if GITHUB_OUTPUT="$newline_output" "$resolver" $'v1.2.3\nname=owned' \
> "$newline_stdout" 2> "$newline_stderr"
then
printf '%s\n' "newline-containing release tag unexpectedly passed" >&2
exit 1
fi
test ! -s "$newline_output"
grep -qxF "::error::release tag is empty or contains invalid characters" "$newline_stdout"
test ! -s "$newline_stderr"
empty_output="$test_tmp/empty-output"
if GITHUB_OUTPUT="$empty_output" "$resolver" ""; then
printf '%s\n' "empty release tag unexpectedly passed" >&2
exit 1
fi
test ! -s "$empty_output"
invalid_output="$test_tmp/invalid-output"
for invalid_tag in \
"v1.2.3 tag" \
"v1.2.3;name=owned" \
"v1.2.3+build.5" \
"release-1.2.3" \
"v-1" \
"vlatest" \
"v" \
"v1.2.3/../../x" \
"V1.2.3"
do
: > "$invalid_output"
if GITHUB_OUTPUT="$invalid_output" "$resolver" "$invalid_tag"; then
printf '%s\n' "invalid release tag unexpectedly passed: $invalid_tag" >&2
exit 1
fi
test ! -s "$invalid_output"
done
release_workflow="$repo_root/.github/workflows/release.yml"
actual_resolver_steps="$test_tmp/actual-resolver-steps"
expected_resolver_steps="$test_tmp/expected-resolver-steps"
# Pin each resolver step and the checkout step that supplies its script. Any
# change to one of these reviewed blocks must be reflected here deliberately.
awk '
function emit_step() {
if (in_step && (is_rel || is_workflow_scripts_checkout)) {
printf "job=%s\n%s", job, step
}
in_step = 0
is_rel = 0
is_workflow_scripts_checkout = 0
step = ""
}
/^ [A-Za-z0-9_-]+:[[:space:]]*$/ {
emit_step()
job = $0
sub(/^ /, "", job)
sub(/:[[:space:]]*$/, "", job)
next
}
/^ - / {
emit_step()
in_step = 1
is_workflow_scripts_checkout = ($0 ~ /^ - name:[[:space:]]*Check out workflow scripts[[:space:]]*$/)
step = $0 ORS
next
}
in_step {
step = step $0 ORS
if ($0 ~ /^ id:[[:space:]]*rel[[:space:]]*$/) {
is_rel = 1
}
}
END {
emit_step()
}
' "$release_workflow" > "$actual_resolver_steps"
cat > "$expected_resolver_steps" <<'EOF'
job=docker
- name: Check out workflow scripts
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
persist-credentials: false
job=docker
- name: Resolve release tag
id: rel
env:
DISPATCH_TAG: ${{ inputs.docker_backfill_tag }}
RELEASE_TAG: ${{ needs.release-please.outputs.tag_name }}
run: |
set -euo pipefail
scripts/resolve-release-tag.sh "${DISPATCH_TAG:-$RELEASE_TAG}"
# ghcr requires a lowercase repository path, and unlike metadata-action,
# buildx's `--output name=` does no lowercasing — a mixed-case owner
# makes the digest push fail with "invalid reference format".
echo "image=ghcr.io/${GITHUB_REPOSITORY,,}" >> "$GITHUB_OUTPUT"
job=docker-manifest
- name: Check out workflow scripts
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
persist-credentials: false
job=docker-manifest
- name: Resolve release tag
id: rel
env:
DISPATCH_TAG: ${{ inputs.docker_backfill_tag }}
RELEASE_TAG: ${{ needs.release-please.outputs.tag_name }}
run: |
set -euo pipefail
scripts/resolve-release-tag.sh "${DISPATCH_TAG:-$RELEASE_TAG}"
job=npm-publish
- name: Check out workflow scripts
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
persist-credentials: false
job=npm-publish
- name: Resolve release tag
id: rel
env:
DISPATCH_TAG: ${{ inputs.npm_backfill_tag }}
RELEASE_TAG: ${{ needs.release-please.outputs.tag_name }}
run: |
set -euo pipefail
scripts/resolve-release-tag.sh "${DISPATCH_TAG:-$RELEASE_TAG}"
EOF
if ! cmp "$expected_resolver_steps" "$actual_resolver_steps"; then
printf '%s\n' \
"release workflow resolver steps differ from the three reviewed blocks" >&2
diff -u "$expected_resolver_steps" "$actual_resolver_steps" >&2 || true
exit 1
fi
printf '%s\n' "release tag validation tests passed"