Compare commits

..
Author SHA1 Message Date
beardthelion 370a364b0f test(node): cover the sweep's boot wiring and prove an interrupted push never reaches the upload
Two claims from earlier commits were argued rather than executed. Both are now
observed.

The legacy-CID sweep's spawn sat inline in main() where no test could reach it,
so the batch size, the inter-batch delay, the detachment and the shutdown select
were all reasoned. The block moves verbatim into `spawn_legacy_cid_sweep`, which
a test calls directly. It seeds rows whose keys are already canonical, so the
cost gate skips each and the persisted cursor reads out walk progress cleanly,
then asserts the cursor stops at the configured batch boundary, the call returns
without blocking, the task is still alive inside its delay, and shutdown ends it.
Never spawning, reading the batch or delay from the wrong knob, or removing the
shutdown branch each turns it red. What remains reasoned is the single call line
inside main(), down from a twenty-six line block.

The claim that a disconnect-interrupted push never uploads to Tigris was
structural: the tests build the store with no Tigris client, so a counter inside
the client arm could never move and would have proved nothing. The counter now
sits at the upload decision, before the client is consulted, and the disconnect
test asserts zero while the success test asserts exactly one. That positive
control is what makes the zero meaningful, and it is itself load-bearing:
releasing the success path with failure semantics drops it to zero, and adding a
Drop impl that uploads raises the disconnect count to one.

The counter observes reaching the upload site, not an S3 request. TigrisClient
wraps a concrete SDK client with no injectable endpoint, and plumbing one through
would be out of proportion to the risk here.
2026-07-27 21:44:31 -05:00
beardthelion 03aee267e6 fix(node): shed 503 from the issue and PR write paths when the lock pool is exhausted
The lock-pool work gave `acquire_write` a distinct exhaustion error and mapped it
to 503 with Retry-After, but only on the push handler. The three other callers,
both issue writes and the PR merge, still turned it into a generic git error and
returned 500. Those are exactly the callers that hold no concurrency permit, so
they are the likeliest to meet an exhausted pool in the first place.

All three now go through the push handler's helper rather than repeating the
mapping, so the log level and the message cannot drift between the four sites.
Each has a test that occupies the single connection of a one-slot pool with a
guard on a different repo, so the failure is pool capacity rather than
advisory-lock contention, and asserts status 503 plus retry-after through
IntoResponse. Reverting the mapping at one site turns only that site's test red.

Worth knowing for review, and left alone deliberately: close_issue takes the
write guard before its author check, so this shed is reachable by any
authenticated caller whether or not they could close the issue. That ordering
predates this change and reordering the handler is not this commit's business,
but it does mean the 503 discloses that the repo's write pool is busy.
2026-07-27 21:14:55 -05:00
beardthelion c519d54fc1 fix(node): keep the incompleteness marker until a source is really recorded, and let the sweep retry
Four follow-ups from an adversarial pass over the previous commits.

The marker clear ran unconditionally inside `record_pin_source`'s transaction,
but its INSERT is `ON CONFLICT DO NOTHING` guarded by the cap, so it records
nothing when the repo is already a source. That is the common case, not a rare
one: the pin path calls `record_pin_source` for every already-pinned object, and
on a requeue pass that is the whole-repo enumeration. So a re-push from the repo
that was already recorded cleared a marker set by a different repo's failure and
the public copy went back to 404ing. The clear is now gated on the insert
actually adding a row. A genuine record from a third repo still clears the
marker, which needs a per-(oid, repo) table to close and is documented rather
than papered over.

The sweep advanced its cursor before every skip, which stops a wedge but made a
transient skip permanent: once the walk reached the end of the table the cursor
parked there and no later boot read another row. On a Tigris-backed node the
common skip is "repo is not on local disk", so the sweep would repair whatever
happened to be warm at boot and then never run again, and the new advertise
filter turned those rows from "advertised but 404" into "never advertised, never
repaired". Repairs now report whether a skip was retryable or terminal, and a run
that skipped anything retryable rewinds the cursor once, after the walk, so the
next run re-walks. A permanently unrepairable row is terminal and does not
trigger a rewind, so this cannot become a hot loop.

The sweep also reached its repo paths through the raw join helper, skipping the
three-layer traversal barrier `RepoStore::local_path` applies. A test with a repo
row named `../../escapee` showed it reading and repairing bytes from outside
`repos_dir`. That barrier is now extracted and shared, so the two callers cannot
drift. And the repair's synchronous `git cat-file` moved under `spawn_blocking`,
matching the existing treatment of the withheld walk, so a wedged read no longer
parks a worker thread for the full service timeout.

Finally the resolver's two source-set queries ran ahead of the work-budget peek
that exists to shed a spent caller before doing work. The peek now runs first. A
throttled caller whose set turns out complete gets 429 where it used to get 404,
which is the honest answer since its search never ran, and it removes an oracle
that let a throttled caller tell a complete source set from an incomplete one.
2026-07-27 18:29:01 -05:00
beardthelion 1c82060c2b fix(node): stop the lock pool from being occupied by spinners and racing the reaper
Three regressions from the previous four commits, each demonstrated before being
fixed.

The advisory-lock retry loop checked its connection out before spinning, so a
caller that lost the lock held a pool connection through all sixty one-second
attempts. The old pool-per-query shape returned it between iterations, so a
spinner occupied nothing. That matters more than the pool sizing suggested:
`acquire_write` has three non-push callers (two in issues, one in pulls) that
hold no concurrency permit, and the issue routes carry no rate limit, so any
self-minted DID could fire enough concurrent closes at one repo to strand the
pool and fail authenticated pushes on every repo. The checkout now happens per
attempt and a losing attempt returns the connection before sleeping, so only the
winner retains one. Lock-pool exhaustion also carries its own error now and sheds
503 with Retry-After instead of surfacing as a 500.

Holding the lock on one connection then meant a client disconnect released it
early: the handler future dies mid-receive-pack, the guard drops without
`release`, and `after_release` frees the lock while the detached reaper is still
giving the process group its SIGTERM grace. A second writer was admitted 90ms
after the drop, well inside that window, which is the invariant smart_http
already documents. The write guard now rides the AdmissionGuard into the reaper,
the same seam the admission permits already use, so the lock outlives the group.
The success path still reclaims the guard and releases it synchronously, and a
dropped guard performs no Tigris upload, so an interrupted push cannot publish a
half-applied repo.

The requeue loop's exit on an exhausted re-read discarded a push that coalesced
during the retry window: `requeue_or_release` had cleared the dirty bit without
marking the guard released, so `Drop` removed the key and no pass was ever
attempted. It now falls through with nothing to replicate and lets the tail's
atomic check-and-clear decide. A lap only happens when a push actually coalesced
and each pays a full bounded re-read, so a sustained outage still terminates,
which is tested with ten thousand injected faults under a watchdog rather than
argued.

Also clamps the lock pool's derived size, which previously followed
max_concurrent_git_pushes up to its million-connection ceiling, and documents the
node's total connection budget on that knob.
2026-07-27 17:47:56 -05:00
beardthelion 73429c7a56 fix(node): sweep legacy provider-CID pins instead of waiting for a re-push
`repair_legacy_provider_cid` had exactly one trigger: the already-pinned skip
branch inside `pin_new_objects`, which a push only reaches when it re-carries the
object. Normal git negotiation omits objects the node already has, so on an
upgraded node that push generally never arrives. Meanwhile `list_pinned_cids`
kept advertising the stored provider CID that this branch's `/ipfs/{cid}`
deliberately withholds, so the node handed out CIDs it would then refuse to
serve, and previously pinned data stayed unresolvable.

Two halves, both needed. A detached background sweep walks `pinned_cids` in
bounded batches with an inter-batch delay, resolves each row's repo from its
recorded provenance, and reuses the existing repair (and its cost gate, so a row
already keyed on a raw CIDv1 reads no bytes). Migration v16 adds a single-row
cursor table so a restart mid-walk resumes rather than rewinding; the cursor
advances on every row read, before any skip, so an unrepairable row cannot wedge
the walk. A row whose bytes are gone, whose provenance is NULL, or whose repo is
not on local disk is left exactly as it is. And `list_pinned_cids` now omits rows
whose key is not a raw CIDv1, so the window before the sweep catches up
advertises nothing the resolver withholds.

The sweep resolves paths directly rather than through `repo_store.acquire`, which
would pull cold repos back from Tigris and turn a repair pass into a bulk
restore. Defaults are deliberately quiet: 64 rows per batch, 60s between batches.

Ten tests cover the repair, the bytes-gone and unrepairable arms, the cost gate,
the batch bound, cursor resumption, both states of one row through the advertise
filter, and the degenerate empty and no-legacy-rows tables. Removing the filter,
the batch bound, the cursor advance, the bytes-gone early return, or the
inter-batch sleep each turns a test red.

One pre-existing test used placeholder resolver keys that the new filter drops;
its values are now real raw CIDv1s with every assertion unchanged.
2026-07-27 15:53:26 -05:00
beardthelion fccdfc1820 fix(node): record a failed pin-source write so the resolver keeps its fallback
The resolver treated a non-empty, below-cap pin-source set as proof that every
source had been recorded, and skipped the bounded scan on that basis. But
`record_pin_source` is best effort at every call site: the ipfs_pin sites were
warn-only once their retries exhausted, and both pinata sites had no retry at
all. An object first pinned from a private repo and later pushed from a public
one whose source write failed keeps a set naming only the private source, so
`GET /ipfs/{cid}` returned 404 for an object the public repo would have served.

Migration v15 adds `pinned_cids.pin_sources_incomplete`, NOT NULL DEFAULT FALSE
so every existing row reads as complete. A record that fails outright sets it; a
later successful record clears it, inside the same transaction as the insert so
the two paths cannot drift. `needs_scan` now ORs the marker in, short-circuiting
after the existing empty and at-cap checks so it costs nothing on the serve path.
Pinata's two bare calls now share `retry_db_record` with ipfs_pin. The ipfs_pin
first-pin path additionally records the CID and its source in one transaction;
Pinata's remains two retried calls.

INV-10 is preserved: a complete set still tail-404s with no preload, asserted on
the existing preload counter. The fallback is not an authorization bypass, and
the test for that asserts the scan actually ran before checking the denial, so
the denial cannot pass vacuously. Dropping the marker term from the gate, forcing
it true, or removing the clear each turns a test red.

The marker is per-object rather than per-(object, repo): a successful record from
one repo clears a marker set by another's failure. That is the conservative
direction, since the marker only ever adds the fallback back.
2026-07-27 14:40:03 -05:00
beardthelion 8e6e2ad0fe fix(node): retry the requeue re-read so a coalesced push is not silently dropped
`requeue_or_release` clears the dirty flag atomically with the decision to loop,
which is correct and stays as it is. The problem was what the pass did next: a
failed `get_repo_by_id` fell into an Err arm that yielded no rules, and
`list_visibility_rules(...).ok()` collapsed its own error the same way. With no
rules, `replication_withheld_set` returns `(false, None)`, the pass skipped its
work entirely, and the loop exited having consumed the dirty bit that stood for
the coalesced push. There is no reconciliation sweep to re-derive it, so that
push's objects were never pinned and its withheld blobs never sealed.

`requeue_refresh_state` now separates the three cases the old code collapsed.
`Ok(Some)` proceeds. `Ok(None)` means the repo really is gone and releases
without touching the retry budget. `Err` from either read is transient and is
retried with doubling backoff up to three attempts. A rules-read error is no
longer indistinguishable from a repo that has no rules.

Exhausting the retries still drops the pass, which is the pre-existing residual,
but it now logs at error level with the repo id and attempt count instead of
disappearing behind a warning. Six tests cover the retried-then-lands path, the
bounded-exhaustion path, repo-gone, the rules-read arm, freshness of the applied
rules, and the coalescing property that had to survive. Reverting the Err arm,
restoring the `.ok()`, raising the bound, or downgrading the error log each turns
a test red.
2026-07-27 13:52:44 -05:00
beardthelion 9a70d75d13 fix(node): own the repo write lock on one connection so cancellation cannot leak it
`acquire_write` took `pg_try_advisory_lock` through the shared pool, then awaited
the Tigris exists/download path before constructing `RepoWriteGuard`. Two bugs
fell out of that. The `tokio::time::timeout` this branch added around the call
can drop the future inside that window, after the lock is held and before the
only unlock (`RepoWriteGuard::release`) exists, wedging every later push to the
repo. And because a session advisory lock belongs to the connection that took
it, running the lock and the unlock through the pool lets them land on different
connections: the unlock returns false and leaks, while a competing acquire that
draws the holding connection re-enters the lock and two writers run against the
same repo concurrently.

`RepoStore` now owns a dedicated lock pool built with an `after_release` hook
that runs `pg_advisory_unlock_all()`. `acquire_write` checks out one connection,
takes the lock on it, and moves it into the guard, which unlocks on that same
connection. sqlx's `PoolConnection::drop` spawns `return_to_pool`, which runs the
hook, so a connection dropped by cancellation still clears its locks; the unlock
is asynchronous with respect to the drop, which the tests account for. The pool
is separate from the main query pool because a push holds its connection for the
whole receive-pack and `db_max_connections` (20) sits below
`max_concurrent_git_pushes` (32).

Seven tests cover the cancellation window, the same-session release, and the
serialization the pool-mismatch bug was breaking. Both the cancellation
regression and the concurrent-writer case were observed red before the fix, and
removing the `after_release` hook or moving the unlock back to the pool turns
them red again.
2026-07-27 13:22:45 -05:00
t 62a99270a9 fix(node): bound the post-push pin and repair reads so a wedge cannot pin the coalescing key
A cross-model adversarial pass found that run_post_push_replication holds the
per-repo coalescing key until requeue_or_release, but pin_new_objects reached it
only after two unbounded git reads: the U7 legacy-repair read and the
pre-existing pin read, both plain Command::output with no teardown. A wedged
cat-file on a stuck backend hung the task forever, so the key was held until
process death and later pushes only marked it dirty without spawning a
replacement (the same class this PR closed on the /ipfs serve path).

Add read_object_bounded (the bounded twins under one shared deadline) and use it
at both sites; a timeout skips that object and lets the task proceed to
requeue_or_release. Adds a wedge-reap regression proven load-bearing two ways.
2026-07-20 21:06:10 -05:00
t fba3c6fda3 fix(review): share one deadline across the /ipfs size+content reads
Code review found each candidate's size and content reads were each granted a
full git_service_timeout, so a single served candidate could hold the /ipfs
walk permit for 2x the timeout. Share one deadline across both stages (mirrors
build_filtered_pack) so the read holds admission for one timeout total, and
correct the admission-bound comment: the worst case is O(candidates x timeout)
bounded by the walk concurrency cap, not a single deadline.
2026-07-20 20:27:35 -05:00
t 45e269b2ae fix(node): opportunistically repair legacy provider-CID pins
Before this PR the pin path stored the provider CID (Kubo dag-pb / Pinata) in
pinned_cids.cid, which the new resolver recomputes-and-404s, while
/api/v1/ipfs/pins still advertises the stale value. Repair a legacy row in the
already-pinned skip branch: gate on a cheap stored-CID codec check (only a
non-raw-codec CID reads bytes, so raw rows keep the DB-only skip cost),
recompute the raw CID, and rewrite it while stashing the old value in a new v14
legacy_provider_cid column (distinct from pinata_cid, which gates the Pinata
pin-skip). Rows whose bytes are gone stay withheld. Documents that the deferred
one-shot sweep, not this opportunistic path, fully retires the window.
2026-07-20 20:11:42 -05:00
t b774c435a6 fix(node): retry pin-source and pinned-CID recording on transient errors
The three warn-only record sites in the detached post-push task dropped a
transient DB failure silently, leaving a permanently incomplete pin-source set
that makes the resolver 404 a valid public copy. Wrap them in a bounded retry
(inserts are idempotent via ON CONFLICT DO NOTHING); on exhaustion the warn
still fires so behavior degrades to today's. Documents the residual crash/outage
window that only a reconciliation sweep retires.
2026-07-20 19:50:23 -05:00
t 0a1618f7fb fix(node): split /ipfs work accounting off the route rate limiter
The route middleware charged ipfs_rate_limiter once per request while the
legacy scan and provenance walk charged the same bucket per probe/walk, so a
one-probe request cost two tokens and GITLAWB_IPFS_RATE_LIMIT=1 admitted the
request at the route then 429'd it at the pre-scan peek.

Add a bounded ipfs_work_rate_limiter, move the in-scan, per-walk, and pre-scan
charges onto it, and leave the route limiter as the pure once-per-request
brake. The work-budget capacity derives from the route limit with a floor of
one full legacy-probe budget per window (no new operator knob), so a
default-config deep search never self-throttles. Adds it to the periodic sweep,
updates all three AppState construction sites, and rewrites the contradictory
limiter comments.
2026-07-20 19:41:57 -05:00
t c2fce67bae fix(node): wire GITLAWB_IPFS_MAX_REPOS_WALKED to the legacy-probe budget
The knob was parsed and documented as the per-request /ipfs fan-out cap but
never read in production: AppState seeded the legacy-probe budget from a fixed
256 constant, so setting the knob did nothing and an operator could still incur
256 acquire/probe operations per request.

Seed ipfs_max_legacy_probes from the knob (default raised to 256 to preserve
shipped behavior); leave the history-walk ceiling on its MAX_PIN_SOURCES+1
constant so a provenanced request is never truncated into a false 503. Updates
the README and .env.example to the shipped behavior and adds cap-honored and
plumbing tests.
2026-07-20 19:17:12 -05:00
t 3351e09c08 fix(node): requeue coalesced pushes instead of dropping them
Per-repo coalescing dropped every push that arrived while a repo's encryption
task was in flight, and no reconciliation ever processed it, so a withheld
blob added by the coalesced push had its recovery copy permanently absent. The
drop also silently skipped the coalesced push's local-IPFS pin work.

EncryptInflight becomes a dirty-flag map; the detached task loops, and the
atomic check-and-clear sits at the task tail, unconditional on the
has_path_scoped_rule gate and walk success (a public or rules-free repo would
otherwise exit before it ran). Each requeue re-reads repo state fresh and
re-enumerates the pin half through the fail-closed full scan, never bare
list_all_objects, so a coalesced rule change is honored and no withheld or
dangling object leaks. At-most-one-task-per-repo is preserved.
2026-07-20 19:04:55 -05:00
t 757edf560c fix(node): own /ipfs admission for the life of the walk
get_by_cid held the global walk permit and the per-source permit as handler
locals, so dropping the request future released both while the spawn_blocking
probe, visibility walk, and object read kept running; a cancel-spam or timeout
client could exceed MAX_CONCURRENT_IPFS_WALKS and the per-source cap, and the
bare cat-file probe/read children had no teardown at all.

Move the gated serve pipeline into a detached task that owns an AdmissionGuard,
awaited by the handler, so admission releases only after the bounded work is
gone; back the /ipfs probe and read with run_bounded_git twins for
process-group teardown. Adds cancel-mid-walk, timeout-reap, and cancel-spam
regressions plus two load-bearing INV-22 rows. Also reflows one U1 test line to
satisfy cargo fmt.
2026-07-20 18:34:09 -05:00
t 86828046ea fix(node): hold read admission through the filtered-pack reaper
The path-scoped upload-pack branch kept the read and per-caller permits as
handler locals and called upload_pack_excluding with no AdmissionGuard, so a
cancelled filtered clone released admission while KillGroupOnDrop was still
reaping the git group, bypassing the read and per-caller concurrency caps.

drive_git_child now returns the disarmed guard on success so one guard rides
both build_filtered_pack stages (rev-list then pack-objects); the handler builds
the guard from the permits as the plain path does. Adds a disconnect regression
and an INV-22 gate row, both proven load-bearing.
2026-07-20 17:54:49 -05:00
t d26cf6dd1d fix(node): close the /ipfs pin-source griefing hole via a bounded scan fallback
An attacker who pins a public object from MAX_PIN_SOURCES repos before the
legitimate public source registers fills pin_repo_sources (record_pin_source is
first-N-wins and drops later sources silently); the buried public source is then
never reached because a non-empty provenance set suppressed the legacy scan, so
anon GET /ipfs/{cid} 404s forever for a public object (re-breaks F1).

U1: db::pin_sources_at_cap reports whether pin_repo_sources is at MAX_PIN_SOURCES
for an oid (the only observable signal that a servable source may have been
dropped, since the write cap never overshoots).

U2: get_by_cid falls back to the bounded legacy scan on a provenance miss when the
set is empty OR at_cap. The scan gates every repo through the real per-caller gate,
so it finds the buried public copy. A complete (non-full) set still fast-404s, so
ordinary denials never fan out to O(repos) (INV-10/F3); the fallback honors the
is_throttled peek.

Regression ipfs_cid_buried_public_source_still_serves_via_scan_fallback: 404 before,
200 after; pin_sources_at_cap_flips_at_max covers the boundary.
2026-07-17 10:15:56 -05:00
t 94ca744269 fix(node): align ipfs_pin return with pinata; add Retry-After to SearchIncomplete
- ipfs_pin::pin_new_objects returns the provider Hash (not the raw resolver key),
  matching the pinata twin's contract; the DB cid stays the raw content CID. The
  return is logging-only here, so this is drift-avoidance, not a functional fix.
- AppError::SearchIncomplete now carries Retry-After: 1 like Overloaded, so both
  retryable 503s from the /ipfs handler advertise the retry hint consistently.
2026-07-17 10:15:56 -05:00
t 2eea354188 fix(node): tie the /ipfs walk ceiling to MAX_PIN_SOURCES + 1
A provenanced object has a bounded source set (first-pinner + up to
MAX_PIN_SOURCES additional). With the walk ceiling at 16 == MAX_PIN_SOURCES, an
authorizing public source sorting after 16 path-scoped denials was never reached:
the ceiling truncated the search and returned a false 503 for a readable object.
Raise the ceiling to MAX_PIN_SOURCES + 1 so the whole bounded provenance set is
always tried; the legacy scan stays bounded by MAX_LEGACY_PROBES_PER_REQUEST.
2026-07-17 10:15:56 -05:00
t c767658b7b fix(node): bound the /ipfs cat-file probes under the walk permit; backfill pinata provenance
P2-E: the object-type probe and the F6 size+read ran bare git cat-file inside
spawn_blocking while the /ipfs walk permit was held, so a wedged cat-file could
pin the global walk slot for the request's life. Bound both under
git_service_timeout_secs; on timeout free the slot (truncated -> retryable 503)
and skip the repo.

P2-D: the pinata already-pinned branch now backfills NULL first-pinner
provenance in lockstep with the ipfs_pin skip branch (consistency; the object
was already resolvable via the pin_repo_sources union).
2026-07-17 10:15:56 -05:00
t 8afc9eaf97 fix(node): close grok P1 findings on the #173/#174 IPFS integration
P1-A: gate_and_serve held the /ipfs walk permit across a bare repo_store.acquire;
wrap it in git_acquire_timeout_secs (mirroring #174's own handler) so a cold/hung
Tigris acquire cannot pin the global walk slot. On expiry skip the repo and mark
the search truncated (retryable 503, not a false 404).

P1-B: the local-IPFS pin path recorded Kubo's provider Hash as pinned_cids.cid;
for objects above the block size that is a dag-pb root that does not hash the raw
content, so GET /ipfs/{cid} listed then 404'd them under the F2 integrity check.
Record the locally-computed raw-content CID, mirroring the pinata twin.
2026-07-17 10:15:56 -05:00
t 433c8915bf feat(node): integrate the #173 IPFS CID tree-gate onto the #174 walk-concurrency base
Rebased onto #174 (fix/served-git-concurrency-cap). #173's incremental history
cannot replay commit-by-commit because #174 rewrote the same handler files in
parallel; this single commit carries the fully-integrated tree (identical to the
verified merge result), with the follow-up fixes as separate commits on top.

Brings in #173: GET /ipfs/{cid} CID->oid resolution via pinned_cids, per-caller
path-scoped blob/tree/commit-tag gating (#135/#173 F1-F6), and the pin-source
provenance table. Adapts #173's tree/commit-tag walks to #174's run_bounded_git
so the held /ipfs walk-concurrency permit is duration-safe (F5).
2026-07-17 10:15:56 -05:00
t 413d6cf0bc fix(node): reject unsupported git service before the read slot; document the new admission knobs (#174)
P2-1: git_info_refs treated every service other than git-receive-pack as a read op
and did the DB/visibility/Tigris work under a read permit, validating the service
string only downstream in smart_http. So an unauthenticated `?service=anything` to
a public repo consumed a read slot + storage work before rejection. Validate the
service is exactly git-upload-pack or git-receive-pack immediately after parsing,
returning 400 before the pre-DB shed and the permit. Handler-layer regression:
with the read pool exhausted, `?service=git-explode` returns 400 (validated first),
not 503 — removing the validation makes it 503 (RED).

U5/INV-24 docs: document the new knobs from U2/U3 in .env.example and the README
env-var table — GITLAWB_GIT_ACQUIRE_TIMEOUT_SECS, GITLAWB_MAX_CONCURRENT_IPFS_WALKS,
GITLAWB_IPFS_WALK_PER_SOURCE, GITLAWB_IPFS_MAX_REPOS_WALKED, GITLAWB_IPFS_RATE_LIMIT.
Each is already enforced on the path it names (U2/U3 tests), so the docs match the
implemented routing.
2026-07-15 13:16:40 -05:00
t 394768711e fix(node): bound the post-push encryption task set by per-repo coalescing without dropping recovery copies (#174)
After a successful path-scoped push the handler spawns a DETACHED task that runs
pin_new_objects then withheld_recipients_gated, which acquire_owned().await's on
git_encrypt_semaphore — it DEFERS (blocks) when the pool is full. The semaphore
caps active walks but nothing capped how many detached tasks were spawned and
PARKED on that await: N rapid path-scoped pushes spawned N tasks, each holding
cloned object lists/rules/paths/keys — an unbounded parked-waiter set (P2-2).

2a54c15 deliberately chose defer-not-shed because dropping the walk loses the
withheld-blob recovery copy and there is no reconciliation sweep to rebuild it.
So bound the OUTSTANDING-TASK set by per-repo coalescing rather than shedding:
EncryptInflight (a bounded Arc<Mutex<HashSet<repo_id>>>) + try_begin — before the
spawn, if a task for the repo is already in-flight skip the duplicate (the pending
walk covers the newer objects); otherwise spawn and move an RAII
EncryptInflightGuard into the task that removes the key on drop (completion, error,
or panic-unwind). This bounds the set to <=1 pending task per repo and never drops
work; withheld_recipients_gated's defer is unchanged.

The second detached post-push spawn (Pinata replication, also runs pin_new_objects)
is scoped out with rationale: it parks on no semaphore (no unbounded waiter set)
and does per-push per-ref work (branch->CID, gossip, subscription, Arweave,
peer-notify) keyed to this push's ref_updates — coalescing it would DROP a later
push's announcements, a correctness regression.

Regressions (mutation-verified RED->GREEN): the outstanding set is bounded to 1
per repo under saturation (never-coalesce -> 32 tasks RED); a coalesced repo's key
is released when its task ends so it is reprocessed, never permanently skipped
(guard-drop no-op -> repo locked out, recovery copy lost forever, RED — the exact
durability regression the fix prevents); distinct repos each admit; cold-set first
push admits; plus an inv22_gates structural tripwire that fails if the handler gate
is removed.
2026-07-15 13:10:04 -05:00
t f424ccbbcd fix(node): gate the /ipfs walk with bounded concurrency admission + per-source cap + route rate limit (#174)
GET /ipfs/{cid} (get_by_cid, publicly reachable via optional_signature) ran
allowed_blob_set_for_caller_bounded in spawn_blocking inside a per-repo loop with
NO global/per-source admission and NO route rate limit — the surface had zero
concurrency or rate control. A permissionless caller could fan out concurrent
full-history git walks (each up to the walk timeout) exhausting blocking-pool
threads and PIDs outside every served-git pool (P1-3).

Acquire a global git_ipfs_walk permit (try_acquire_owned, shed 503) plus a
per-source sub-cap (with_default_max_keys, reject-before-insert, keyed via
client_key/TrustedProxy so XFF can't spoof it) ONCE after CID validation, held as
handler locals across the whole request — including every spawn_blocking walk, so
the slot reflects real blocking-thread occupancy (a tokio timeout cannot cancel
spawn_blocking). None key -> global pool only. Cap repos walked per request so one
request can't serialize N full-history walks. Attach rate_limit_by_ip +
IpRateLimiter to the /ipfs route (mirrors the git/create/peer routers; the
extension is required or the middleware is a silent no-op). Visibility/authz
semantics unchanged.

New knobs: GITLAWB_MAX_CONCURRENT_IPFS_WALKS (32), GITLAWB_IPFS_WALK_PER_SOURCE
(4), GITLAWB_IPFS_MAX_REPOS_WALKED (64), GITLAWB_IPFS_RATE_LIMIT (600/hr, 0
disables) — all range-validated and enforced on-path.

Handler-layer regressions (all mutation-verified RED->GREEN): shed-at-capacity
503 (delete acquire -> falls through), per-source cap sheds same source / admits
another, None-key arm sheds on the global pool, map self-bounds reject-before-insert,
the walk permit is held while the spawn_blocking walk runs (available_permits()==0;
dropping it before the loop -> RED), repos-walked cap (remove break -> 2 walks),
and the route IP rate limit actually fires 429 (drop the Extension -> RED no-op).
2026-07-15 12:42:55 -05:00
t 8d17038e5b fix(node): bound the served-git storage-acquisition phase with a release-on-expiry deadline (#174)
The served-git admission permit is acquired, then RepoStore::acquire*/advisory-lock
runs before the bounded git runner starts — with no local deadline. acquire/acquire_fresh
await Tigris HEAD/GET, and acquire_write's advisory-lock loop awaits a per-iteration
pg_try_advisory_lock that can block indefinitely on a hung Postgres pool. Since
GITLAWB_GIT_SERVICE_TIMEOUT_SECS only starts once git spawns, a stalled backend pins
the permit and drains the pool until every later request 503s (P1-2).

Wrap each git-path acquire (git_info_refs read/advert, git_upload_pack, git_receive_pack)
and the /ipfs per-repo acquire loop in tokio::time::timeout(git_acquire_timeout_secs, ..).
On expiry the git paths return AppError::Overloaded (503 + Retry-After); the permit is a
handler-local at that point (moved into the AdmissionGuard only after acquire) so the early
return frees the slot. The /ipfs per-repo acquire keeps its fail-closed `continue` on
timeout (never serves an un-acquired repo). No repo_store.rs signature change: the outer
timeout cancels a mid-sleep/mid-fetch_one future, so no in-loop deadline is needed.

New knob GITLAWB_GIT_ACQUIRE_TIMEOUT_SECS (default 30, range 1..), modeled on
git_service_timeout_secs, kept separate because acquisition and git execution are distinct
cost centers.

Handler-layer regression receive_pack_acquire_deadline_sheds_and_releases_permit: holds the
same pg advisory lock acquire_write derives on a second connection so the real loop must
retry, drives git-receive-pack with the deadline at 2s and write pool size 1, and asserts
503 at ~2s + the permit recovers + a follow-up admits once the lock frees. Removing the
timeout wrapper turns it RED (blocks ~31s to the test ceiling on the ~59s advisory loop).
2026-07-15 12:00:02 -05:00
t 72e899dae0 fix(node): hold served-git admission until the process group is reaped on the plain spawn paths (#174)
On the plain (non-path-scoped) info_refs / upload-pack / receive-pack paths the
global + per-source admission permits were handler-locals that dropped the
instant the handler future dropped on a client disconnect, while KillGroupOnDrop
reaps the git process group in a DETACHED task (SIGTERM -> grace -> SIGKILL ->
reap). So a disconnect-spammer admitted replacements while prior groups were
still alive, defeating the process cap and the per-source cap during teardown.
be0cdd6 already fixed this for the path-scoped upload-pack walk; this closes the
residual plain paths (P1-a).

Introduce AdmissionGuard, a move-only holder of the permits, threaded through
info_refs/upload_pack/receive_pack/run_git_service into drive_git_child's
KillGroupOnDrop. On disconnect the guard moves into the detached reaper and drops
only after the group is ESRCH-confirmed reaped; on success/timeout disarm()
returns it for the earliest provably-free drop. The rev-list/pack-objects
visibility-walk callers hold no admission and pass None.

Thread git_bin through upload_pack/receive_pack (they hardcoded "git") so the
plain POST path is drivable by an injected fake git, matching the existing
injectable seam on info_refs/run_git_service/upload_pack_excluding.

Handler-layer regression upload_pack_plain_permit_held_through_group_reap_after_disconnect:
isolates the global read pool (size 1, per-source + rate limiter permissive), a
SIGTERM-ignoring descendant keeps the group alive across the reap window, and
asserts available_permits()==0 held on disconnect + a cross-source replacement
sheds 503, then frees after ESRCH. Reverting the guard-into-reaper move (release
immediately) turns it RED at the held-through-reap assertion. Plus a None-key arm.
2026-07-15 12:00:02 -05:00
t 5749df6e67 test(node): INV-22 completeness guard for the served-git concurrency surface (#174)
A source-scan tripwire that fails when a new site reintroduces the concurrency-cap
class PR #174 fixed, or when one of the five gates (U1-U5) is removed. INV-22: a
permit held per op recovers only if every path is duration-bounded and reaps the
group before releasing admission, and every detached git task carries admission.

Lives in tests/ (a separate crate) on purpose — a guard scanning the file it lives
in would match its own identifier literals and pass vacuously; scanning src/ from
here keeps every check load-bearing. Checks: run_bounded_git confirms child exit via
child_terminated_without_reaping (U1); KillGroupOnDrop launches the reaper via
Handle::try_current (U2); git_receive_pack acquires state.git_write_per_caller (U4);
the encryption walk runs through withheld_recipients_gated over git_encrypt_semaphore
(U5); and the bounded recipients walk is invoked exactly once — only inside the gated
helper — so a new spawn_blocking bypass (count > 1) fails (P1-e non-bypass).

Proven load-bearing: a synthetic second call site turns the count tripwire RED, and
each named gate maps to a fix whose own regression already goes RED on revert.
2026-07-14 20:33:01 -05:00
t 2a54c159e6 fix(node): bound concurrent post-push encryption walks with an admission pool (#174)
After a successful path-scoped push the handler released its write permit and
then ran a DETACHED tokio task whose spawn_blocking(withheld_blob_recipients_bounded)
performed another full-history git walk under no admission. N fast completed
pushes spawned N concurrent full-history walks past GITLAWB_MAX_CONCURRENT_GIT_PUSHES
(which bounds only the in-handler phase), and the per-child bounding from 71389e6
caps each walk's duration but not their count (P1-e).

Add git_encrypt_semaphore, a pool of its own (sized from max_concurrent_git_pushes,
no new knob — Q1) so a long background walk never holds a foreground write slot and
a handler holding a write permit can't self-deadlock. Route the walk through a new
withheld_recipients_gated helper that acquires the pool before the walk and DEFERS
(blocks) when full rather than shedding — dropping the walk would lose the
withheld-blob recovery copy, so durability stays fail-closed.

Regression: encrypt_walk_defers_when_pool_exhausted drives the gated helper with the
pool exhausted and asserts the walk blocks and does not run its rev-list, then runs
once a permit frees. RED before (the walk runs regardless of the pool), GREEN after.
The detached push task calls this exact helper.
2026-07-14 20:28:52 -05:00
t 12643573f6 fix(node): cap concurrent receive-pack pushes per source IP (#174)
The authenticated git-receive-pack POST acquired only the global write semaphore
— no per-source sub-cap, unlike the read and anon-advert pools. Owner enforcement
defaults off, so one host minting disposable did:key identities could open
max_concurrent_git_pushes slow POSTs from a single source IP and 503 every other
source's push; the 600/hour push limiter bounds arrival rate, not in-flight
concurrency (P1-d).

Add git_write_per_caller (a PerCallerConcurrency sized like the advert cap,
max_concurrent_git_pushes/8), and acquire it before the global write permit,
keyed on the resolved source IP via read_caller_key — never the signed DID (a DID
farm defeats a DID key). This required adding the PeerAddr + HeaderMap extractors
to git_receive_pack, which it lacked (without them the key is None and the cap is
inert). Neutralized the shared acquire helper's log wording now that it serves
both read and write paths.

Regression: receive_pack_per_source_write_cap_sheds_capped_source_not_others — a
source at its write sub-cap sheds Overloaded/503 (proving the extractors resolve a
key), while a different source is not shed. RED before (the capped source proceeds
past admission to a git error instead of shedding), GREEN after; the existing
receive-pack advert cap test still passes.
2026-07-14 20:20:49 -05:00
t be0cdd6a9e fix(node): hold upload-pack admission through the walk on disconnect (#174)
On the path-scoped upload-pack path the per-source and global read permits were
handler locals, so a client disconnect dropped the handler future and released
both permits while the awaited spawn_blocking withheld-blob walk kept running
(spawn_blocking can't be cancelled). A permitless caller could disconnect-spam to
exceed the read cap while real git work continued (P1-b).

Move both permits into the blocking task and return them out: on success they
flow back so the serve phase keeps them; on a dropped future the returned tuple
(with the permits) is discarded only when the blocking task completes, so
admission tracks the walk's real duration.

Regression: upload_pack_permit_held_through_walk_after_disconnect drives the
handler to mid-walk (fake git hangs on rev-list), disconnects, and asserts the
global read slot stays held (available_permits == 0). RED before (the slot frees
to 1 the instant the future drops), GREEN after. The test roots the repo store at
its own TempDir so it is isolated per run.
2026-07-14 20:09:40 -05:00
t ac59bd7e01 fix(node): reap the served-git process group on disconnect, not just SIGTERM (#174)
On a client disconnect KillGroupOnDrop sent a lone SIGTERM with no escalation or
reap, so a group member that ignores SIGTERM (a wedged pack-objects, a malicious
helper) survived while the handler released its concurrency permit — letting
disconnect-spam accumulate live git processes and defeat the load shed (P1-c).
The timeout path already ran the full teardown.

Make the disconnect teardown as strong as the timeout path: the guard now owns
the child and, on drop, launches a detached reaper that runs the same
TERM -> grace -> SIGKILL -> reap-the-group sequence (reap_group_on_timeout).
Owning the tokio Child in the reaper keeps it the sole reaper of the leader, so
it never races tokio's own orphan reaper (a raw waitpid(-pgid) would).

Regression: run_git_service_sigkills_a_sigterm_ignoring_child_on_disconnect, RED
before (the descendant survives the lone SIGTERM), GREEN after; all existing
teardown/timeout/disarm tests still pass (the three hand-built-guard tests were
updated to the owned-child shape).

Residual: capacity (the handler's semaphore permit) still releases at disconnect
rather than strictly after the reaper confirms the group dead, so accumulation is
now bounded to ~one teardown window (<=4s) rather than eliminated; gating the
permit on the reap needs threading it through the serve handlers and is left for
a follow-up (it also touches the receive-pack permit lifetime U4 changes).
2026-07-14 19:50:41 -05:00
t 3cc01890fb fix(node): keep the visibility-walk watchdog armed until the child is reaped (#174)
A group member (or the leader itself) that closes stdout before the deadline made
the stdout drain return EOF early, which stood the watchdog down via done_tx
before it could ever fire; child.wait() then blocked on the still-live child,
pinning the walk thread and its read/write permit past
GITLAWB_GIT_SERVICE_TIMEOUT_SECS. This is distinct from the descendant case
already covered: there the leader sleeps until the deadline so the watchdog does
time out; here the drain-EOF races ahead of it.

Stand the watchdog down only once the child has actually terminated, detected
without reaping (waitid + WNOWAIT) so the leader's pid stays unreaped and its
pgid un-recycled until the watchdog finishes its teardown and is joined. Past the
deadline the watchdog still owns the full SIGTERM -> grace -> SIGKILL sequence.

Regression: run_bounded_git_reaps_a_leader_that_closes_stdout_then_hangs, RED
before (blocks on child.wait past the recv budget), GREEN after; the three
existing teardown tests still pass (no recycle-hazard regression).
2026-07-14 19:27:12 -05:00
t a030eefb70 test(node): retry the ETXTBSY exec race in build_filtered_pack timeout tests (#174)
build_filtered_pack_times_out_a_hung_rev_list and _a_hung_pack_objects
spawn a freshly-written fake git directly, bypassing the fake_git_run_with_pids
retry the rest of the module uses. Under cargo test fork-storm load a concurrent
worker forks while the fake's write fd is still open, so execve returns ETXTBSY
and the raw io::Error surfaces where the test asserts GitServiceTimeout, failing
test (stable) intermittently.

Route both tests through build_filtered_pack_or_exec_race_retry, which retries
ONLY errno 26 (ETXTBSY) and keeps the outer 10s watchdog per attempt, so a
genuinely-missing async timeout bound still trips the watchdog and a wrong error
type still fails at the assertion. Production paths unchanged.
2026-07-13 16:01:49 -05:00
t cf97405bc1 test(node): execution-cover the delta-path push-scan timeout wiring (#174)
The finding-2 wiring test covered only list_all_objects (the full-scan fallback).
Add delta_path_exec_fns_time_out_on_a_hung_git covering the common push path:
peeled_object_type (cat-file), rev_list_delta (rev-list), and list_all_objects_with_type
(cat-file, which all_blob_oids delegates to). Each must return within the watchdog
budget on a hung git; reverting any to a bare Command::output() blocks past the recv
budget (RED-verified).
2026-07-13 10:55:57 -05:00
t 4410b834df docs(node): correct concurrency + timeout docs to match the final git routing (#174)
jatmn P2 (INV-24): the config help, .env.example, and README described routing the
handler no longer does.

- The read pool serves upload-pack and the UPLOAD-PACK info/refs advertisement only;
  the anon receive-pack info/refs advertisement draws from its own dedicated pool
  (sized like the write pool but disjoint), not the read pool and not the write pool.
- The write pool is the authenticated push POST only.
- The per-caller read cap is keyed on the resolved source IP, never the DID: a
  signature does not move a caller off it (the docs claimed a per-DID budget, which
  would tell a NATed authenticated client it escapes the shared cap when it does not).
- git_service_timeout_secs now also bounds the push-side pin-candidate discovery
  (rev-list / cat-file), reaped via process-group teardown, alongside the info/refs
  advertisements and the withheld-blob walk.
2026-07-13 10:46:24 -05:00
t 9412aa9c5b fix(node): bound and reap push-candidate git children under the write permit (#174)
Route push_delta's candidate-discovery git children (cat-file / rev-list) through the
bounded run_bounded_git (deadline + process-group SIGKILL teardown) instead of bare
Command::output(). A hung scan no longer pins the git-receive-pack write permit past
the deadline or a client disconnect. Threads git_bin + a shared per-scan deadline
(git_service_timeout_secs) through resolve_candidates_for_push and all_blob_oids.
2026-07-13 10:41:31 -05:00
t 7a6bd359eb fix(node): SIGKILL a SIGTERM-ignoring member of the visibility-walk process group (#174)
jatmn/CodeRabbit P1: the walk watchdog stood down as soon as the group LEADER was
reaped. A leader that exits cleanly on SIGTERM while a background member ignores
SIGTERM and closes its inherited pipes let the main drain unblock, the leader be
reaped, and the watchdog skip its SIGKILL escalation — so the handler returned while
the descendant kept running. jatmn's suggested "continue until ESRCH" could not work:
the main thread reaping the leader is what frees the pgid, so continuing to kill(-pgid)
after that races a recycled group.

Restructure the coordination instead: the main thread now defers reaping the leader
until the watchdog thread returns. The watchdog always escalates through an
unconditional group SIGKILL (no leader-reap short-circuit), and because the leader is
still unreaped while every kill(-pgid) fires, the pgid cannot have been recycled. The
grace before SIGKILL is a fixed budget (only paid on the exceptional timeout path).

Regression run_bounded_git_sigkills_a_sigterm_ignoring_descendant_after_leader_exits:
a leader that exits 0 on SIGTERM spawns an sh -c descendant (its own $$, not a ( )
subshell's parent pid) that ignores SIGTERM and closes its pipes; the descendant must
be dead when the runner returns. RED on the old teardown (survives), GREEN now. The
existing leader-ignores-SIGTERM and hung-walk teardown tests still pass.
2026-07-13 10:11:41 -05:00
t 473924dcfe fix(node): compile the bounded visibility git runner on non-Unix targets (#174)
jatmn P1: run_bounded_git was gated #[cfg(unix)], but its callers
(assert_all_refs_are_commits, blob_paths) and the module are compiled on all
targets, so a non-Unix build (the continue-on-error Windows release target) could
not resolve the function.

Add a #[cfg(not(unix))] twin with the same signature and result semantics. It bounds
a single child (no process-group teardown, which is Unix-only): threads feed stdin
and drain stderr, the main thread drains stdout, and a watchdog thread kills the
child at the deadline. The child is shared with the watchdog behind a mutex the main
thread does not hold while draining, so the watchdog can always acquire it to kill,
and killing closes stdout to unblock the drain. Best-effort (reaps only the direct
child), which is why the hardened group-aware path stays Unix-only.

Verified the twin type-checks by compiling its (portable-std) body on the native
target; the full non-Unix link is CI's Windows job.
2026-07-13 09:49:33 -05:00
t f19f2813a2 fix(node): give the receive-pack advert its own pool so it can't shed authed pushes (#174)
jatmn P1: the anon-reachable GET info/refs?service=git-receive-pack drew from the
same git_write_semaphore as the authenticated git-receive-pack POST. The per-source
advert cap reserves no global slots, so ~8 sources (each within the per-IP push rate
limit) could hold all 32 write permits across acquire_fresh + the advertisement, and
the next authenticated push then 503s at admission. That is an anonymous caller
shedding an authenticated push across the auth boundary (INV-10).

Add a dedicated git_push_advert_semaphore, disjoint from the write pool. Both the
pre-DB early-shed peek and the held acquire for the receive-pack advertisement now
draw from it; the write pool is left exclusively for the POST. An advert flood can at
worst exhaust the advert pool (each source still capped by git_push_advert_per_caller
and the per-IP push rate limiter); the authenticated push pool is untouched.

Test receive_pack_advertisement_draws_from_dedicated_advert_pool proves at the handler
layer that the advert sheds when its own pool is saturated AND survives when the write
pool is saturated (the reservation). Both flip under revert, verified independently for
the pre-DB peek and the held acquire.
2026-07-13 09:44:53 -05:00
t 79b9ee5c89 fix(node): shed the served-git pool before the DB and cap per-source before it (#174)
CodeRabbit review: the held git_permit was acquired at the top of git_info_refs
and git_upload_pack, before the per-source cap, so one source could occupy the
global pool during the DB/visibility window before its sub-cap rejected the
excess. Move the held git_permit to after acquire_read_caller_permit (still
before acquire_fresh/git, so INV-10's bound on the fresh Tigris acquire holds),
and add a cheap pre-DB early shed (available_permits() == 0 -> 503, holds no
permit) so the #62 shed-before-DB property is preserved without holding a permit
across the DB work.

The receive-pack POST is unchanged (owner-only; its top-of-handler git_permit is
deliberate for acquire_write bounding).

Tests: the two #62 shed-before-DB tests now exercise the explicit early check
(load-bearing: disable it and they fall through to the DB); three new
*_per_source_cap_sheds_with_global_capacity tests prove the per-source sub-cap
sheds a capped source even with global capacity free (load-bearing on the caller
cap). Full workspace green.
2026-07-12 14:40:59 -05:00
t 454e59ba53 docs(node): the withheld-walk is bounded by the service timeout, not a fixed const (#174)
Follow-up to threading the configured timeout into the walk: the walk now honors
GITLAWB_GIT_SERVICE_TIMEOUT_SECS on both the serve and replication paths, so the
README no longer says a fixed internal deadline.
2026-07-12 02:12:37 -05:00
t c35452ed41 test(node): vet the withheld-walk bound end to end; use the configured timeout (#174)
Close the reasoned-not-run gaps from the code review by making the walk's git
binary and timeout injectable, then driving the missing branches with a real
handler and a fake git instead of reasoning about them.

- Add state.git_bin and *_bounded variants of the walk entry points taking
  (git_bin, timeout); the served handlers (upload-pack serve, receive-pack
  replication and full-scan and encrypt-pin, and the ipfs gate) now pass the
  operator-configured GITLAWB_GIT_SERVICE_TIMEOUT_SECS, so the whole walk is bounded
  by the same budget as the other served-git ops rather than a fixed constant. The
  git_bin-less wrappers stay for the real-git classification tests.

Newly vetted by execution (not reasoning):
- receive-pack replication path is bounded: replication_withheld_set with an injected
  hung git returns within the budget and fails closed, so it cannot pin the write
  permit git_receive_pack holds across it.
- a hung withheld-blob walk on the upload-pack POST returns 504 (real handler, real
  repo on disk, injected hung git), proving the GitServiceTimeout -> git_service_app_error
  wiring end to end.
- the watchdog status-gate: a child that exits successfully is not reported as a
  timeout even when the watchdog fired (mutation-checked: drop the guard -> RED).
- SIGKILL escalation: a SIGTERM-ignoring child is still reaped via SIGKILL and the
  group is gone; a truly uninterruptible D-state child (unreapable by any signal) is
  the documented residual, matching the async teardown.
- the advertisement per-source cap sizing never derives 0.

Full gitlawb-node suite 515 green.
2026-07-12 02:12:08 -05:00
t e750c3042d fix(review): cap the receive-pack advertisement per source so anon cannot starve the write pool (#174)
U2 moved the receive-pack info/refs advertisement onto git_write_semaphore to keep
an anonymous read flood from starving the push handshake. But the advertisement is
anon-reachable on public repos and holds its write permit across the slow
acquire_fresh Tigris download, and the only per-source brake on it was the push
RATE limiter, not a concurrency cap. So a multi-source flood of receive-pack
advertisements could hold the write pool's slots across those downloads and shed
authenticated pushes (both the advertisement and the owner-gated git-receive-pack
POST draw from the same pool). U2 thus introduced the first anonymous consumer of
the write pool the state doc promised anon could never reach; the plan's residual
note (no worse than the POST) was wrong, because the POST is owner-gated and the
advertisement is not.

Add git_push_advert_per_caller, a per-source concurrency sub-cap on the receive-pack
advertisement keyed on the resolved source IP (the same PerCallerConcurrency
mechanism U1 uses for reads), sized to an eighth of the write pool so a single
source holds at most that share and saturating the pool takes many distinct source
IPs, each also braked by the per-IP push rate limiter. The upload-pack advertisement
keeps its read-pool per-caller cap; the owner-gated POST is unchanged. Correct the
state doc for git_write_semaphore accordingly.

Handler-layer proof: a source at its receive-pack advertisement cap sheds 503 (RED
before the acquisition, 500-not-503), while a different source and the upload-pack
advertisement are unaffected. Full suite 510 green.
2026-07-12 00:56:24 -05:00
t 87a7e40513 fix(review): close the withheld-walk watchdog reused-pgid race (#174)
Code review found run_bounded_git's watchdog could return a spurious 504 and
signal a recycled process group. The watchdog runs off a wall clock on its own
thread; done_tx.send() only fires after child.wait() reaps the leader, so a walk
that finished within microseconds of the deadline took the watchdog's Timeout
branch, discarded a fully-captured successful result, and returned GitServiceTimeout
(a 504 for a walk that actually completed). Worse, the Timeout branch SIGTERMed
-pgid unconditionally after the leader was reaped, so a recycled pgid could be
signalled, the exact hazard smart_http guards via disarm-after-wait.

Set a reaped AtomicBool the instant the main thread reaps the child; the watchdog
checks it before every kill and stands down if the leader is already reaped. Gate
the timeout verdict on !status.success(), so a child that exited on its own is never
reported as a timeout even if the watchdog fired late. Add the survived-SIGKILL warn
smart_http's reap already emits, for operator visibility on a wedged (D-state) git.

The hung-walk test stays green (a killed child exits by signal, not success, so it
still surfaces GitServiceTimeout and reaps the group) and the 32 real-git
classification tests stay green (a fast walk is never spuriously killed).
2026-07-12 00:36:42 -05:00
t ab0ea240fc docs(node): correct the git-service-timeout coverage note (#174)
GITLAWB_GIT_SERVICE_TIMEOUT_SECS bounds the info/refs advertisement too:
smart_http::info_refs drives it through drive_git_child under this timeout, with a
passing test proving the 504. The old note claimed it does not. It also claimed the
withheld-blob path is unbounded; after the blob_paths seam bound (this PR) the walk
is bounded and reaped, by a fixed internal deadline rather than this env var, so the
line now states that precisely instead of overclaiming this setting covers it.
2026-07-12 00:20:07 -05:00
t 71389e621a fix(node): bound the withheld-blob walk at the shared blob_paths seam (#174)
The withheld-blob classification walk (blob_paths) fanned out blocking git children
with no deadline and no process-group teardown: git for-each-ref, git cat-file, git
rev-list, a git ls-tree per commit, and an uncounted git rev-parse (via
store::head_commit). A hung or pathologically slow child pinned the caller's
served-git permit for the whole hang, and on client disconnect the spawn_blocking
task and its git children ran on, orphaned. blob_paths is the shared core of five
callers: the upload-pack serve path (holds a read permit) AND, inside
git_receive_pack, the post-push replication and encrypt-then-pin walks (hold the
write permit U2 reserves for pushes). So the same unbounded walk could pin either
pool, and leaving the write-side twin unbounded would have made U2's reservation a
claim that does not match behavior.

Bound every git child at the blob_paths spawn seam on the blocking side: each child
runs in its own process group with a watchdog thread that SIGTERMs (then SIGKILLs)
the group on one shared deadline spanning the whole walk, and retains admission
until the group is reaped. This is the blocking-side counterpart of
smart_http::drive_git_child (spawn_blocking cannot be cancelled by an async
timeout). blob_paths stays sync, so all five callers keep their signatures and the
32 classification tests are unchanged; because every caller funnels through
blob_paths, one seam bounds both the serve and replication paths. The previously
unbounded store::head_commit child becomes a bounded git rev-parse inside the walk.
A walk that hits its deadline carries GitServiceTimeout, which the serve handler now
maps to 504 rather than a generic 500.

Proof: a fake git that hangs on rev-list makes blob_paths return GitServiceTimeout
within the watchdog budget (not block on the child) and the recorded process-group
leader is reaped, not orphaned; neutralizing the watchdog kill makes it hang past
the budget (RED). The 32 real-git classification tests stay green through the
refactor, including detached-HEAD, non-standard-ref, and deleted-in-history cases.
2026-07-12 00:20:07 -05:00
t e444bc0ae5 fix(node): draw the receive-pack advertisement from the write pool (#174)
git_info_refs acquired git_read_semaphore for BOTH services, so the push handshake
(GET /info/refs?service=git-receive-pack) competed in the global read pool. An
anonymous clone flood could exhaust that pool and shed a legitimate push with 503
during its required advertisement phase, before it ever reached git_write_semaphore
on the POST. The write pool exists precisely so anonymous reads cannot shed an
authenticated push, but only the POST drew from it.

Select the pool by service: the receive-pack advertisement (phase one of a push)
now draws from git_write_semaphore, like the git-receive-pack POST, so a saturated
read pool cannot starve it. The per-IP push_rate_limiter that already brakes the
advertisement stays as the anti-flood control, and the advertisement stays
reader-visible with no new auth requirement. Because the receive-pack branch is now
a write-path op, it no longer consumes a read per-caller slot.

Handler-layer proofs: with the read pool at zero the receive-pack advertisement
survives while the upload-pack advertisement sheds; with the write pool at zero the
receive-pack advertisement sheds while upload-pack is unaffected; and a receive-pack
advertisement from an IP whose read per-caller budget is full still gets through
(mutation-checked, RED when the skip is neutralized).
2026-07-11 23:54:21 -05:00
t 4983b854d1 fix(node): key the per-caller read cap on source IP, not the signed DID (#174)
read_caller_key returned the authenticated DID when a caller signed, dropping the
source-IP key. Public read routes accept any valid did:key via optional_signature
with no admission step, so one host could mint N disposable DIDs and hold
max_concurrent_reads_per_caller slots under each, multiplying its budget N-fold and
filling the global read pool, while the same host unsigned was capped on its IP.

Key the read sub-cap on the resolved source IP for every caller, signed or not,
mirroring the push path's IpRateLimiter which already throttles on source IP for
this exact DID-farm reason. Drops the now-unused caller_did parameter at both call
sites (git_info_refs, git_upload_pack).

Inverts info_refs_per_caller_cap_keys_on_did_not_ip into
info_refs_per_caller_cap_keys_on_ip_not_did: fill one source IP's slot, then two
requests signed under different DIDs from that same IP both shed 503 (farm
defeated), while a signed request from a different IP keeps its own budget. RED on
the DID-keyed tree, GREEN after.
2026-07-11 23:44:13 -05:00
t 362ee3462e fix(node,git): bound the rev-list stage under drive_git_child so a hung enumeration can't pin a git permit (#174) 2026-07-11 13:02:24 -05:00
t 3b1fa54b73 docs(config): add GITLAWB_MAX_CONCURRENT_GIT_OPS to .env.example (#174)
The read-pool knob was referenced by the push and per-caller entries'
comments but had no example line of its own, so operators couldn't
discover it from the template. Add it with the config default (128).
2026-07-10 16:05:29 -05:00
t 5069cd13ca fix(node,concurrency): resolve #174 code-review findings
Review of the served-git concurrency cap found no P0/P1; these are the
verified P2 follow-ups.

config: the max_concurrent_git_ops doc overclaimed that "every capped path
is duration-bounded." The rev-list object enumeration in the withheld-blob
path still runs in an uncancellable spawn_blocking, so a stuck rev-list can
hold its slot until git exits. Scope the guarantee to the streaming stages
and name the residual. Also tighten the fairness claim: the receive-pack
advertisement shares the read pool (a shed advertisement is a cheap retryable
GET); only the push POST is on the isolated write pool.

api/repos: add info_refs_per_caller_cap_keys_on_did_not_ip, the missing
handler proof that a signed caller is keyed by its DID, not its source IP.
Filling the DID slot sheds a request from a free IP; collapsing read_caller_key
to its IP arm turns the assertion green-not-503 (mutation-verified RED).

api/repos: extract acquire_read_caller_permit so both read handlers share one
shed path instead of a duplicated match block.

rate_limit: recover from a poisoned PerCallerConcurrency mutex instead of
panicking. The critical section is pure counter arithmetic and cannot poison
the lock, but a panic there would brick the limiter for every caller.

505 tests pass; clippy -D warnings and fmt clean.

Part of #174.
2026-07-10 13:45:38 -05:00
t 6336657efc docs(node,config): reconcile the concurrency-cap comments with the fix (#62)
The max_concurrent_git_ops and git_service_timeout_secs doc-comments (and .env.example) described the info/refs and withheld-blob paths as unbounded follow-up gaps. Both are now closed (#174): the comments reflect the read/write pool split, the per-caller sub-cap, and that every capped path is duration-bounded with process-group teardown. Verified the pattern-doc pre-ship checklist: every git_permit / write-permit / per-caller site holds only a timeout+teardown git path, and all three size knobs are range(1..=1_048_576).

Closes the #174 work. No behavior change.
2026-07-10 13:20:18 -05:00
t 57c6e01c26 fix(node,git): bound and reap the withheld-blob pack build (#62)
The filtered-pack path ran the whole rev-list + pack-objects build inside a spawn_blocking, so an outer tokio timeout could not cancel the blocking thread and a client disconnect orphaned the git child while the permit freed (jatmn P1b, the second gap path). Split it: rev-list enumeration stays blocking off the runtime (rev_list_keep), but the streaming pack-objects stage now runs under the shared drive_git_child on the async side, so it is duration-bounded (GitServiceTimeout -> 504) and its process group is reaped on disconnect. build_filtered_pack becomes async and takes a git_bin seam + timeout; upload_pack_excluding threads the git_service_timeout through.

A watchdog-bounded test proves a hung pack-objects times out (rev-list fast, pack-objects hangs). The refactor's happy path is covered by the existing filtered-pack correctness and real-git partial-clone/fetch tests, all still green; disconnect/group-teardown is the shared drive_git_child code proven by the run_git_service tests. 504 tests pass.

Part of #174.
2026-07-10 13:16:41 -05:00
t 2c264e08e3 fix(node,git): bound and reap the info/refs advertisement (#62)
info_refs ran a bare Command::output() with no timeout and no process-group teardown, so a hung git pinned its concurrency slot indefinitely and a client disconnect orphaned the child (jatmn P1b). Extract the timeout + process_group(0) + KillGroupOnDrop core from run_git_service into a shared drive_git_child, and route info_refs through it with an injectable git_bin. A hung advertisement now aborts with GitServiceTimeout (mapped to 504); disconnect reaps the group.

run_git_service's teardown tests all pass through the shared core (proving the group teardown info_refs inherits), the real-git filter tests cover the advertisement happy path, and a new watchdog-bounded test proves a hung advertisement times out. 503 tests pass.

Part of #174.
2026-07-10 13:09:34 -05:00
t fc4c206c3a fix(node,git): per-caller concurrency sub-cap on the read pool (#62)
Adds PerCallerConcurrency, a bounded-keyed in-flight limiter (distinct from the request-rate RateLimiter) so no single caller monopolizes the served-git read pool. Each caller (per-DID when signed via optional_signature, else per-source-IP via client_key) may hold at most max_concurrent_reads_per_caller concurrent reads; over that it sheds 503. The key map is self-bounding (a key is dropped when its in-flight count hits zero) with a reject-before-insert max_keys backstop so a key farm can't grow it (INV-15). Applied in git_upload_pack and both info/refs advertisements, acquired after the visibility gate so a denied request never consumes a slot (KTD7).

Primitive unit-tested (cap + self-bounding + reject-before-insert) and mutation-checked. Handler-layer SC2: same-caller sheds while a different caller passes, proven on BOTH git_info_refs and git_upload_pack with independent mutation probes; plus a None-key bypass test. Per-source-IP keying is trust-config dependent, documented on the config knob. 502 tests pass.

Part of #174.
2026-07-10 12:50:46 -05:00
t 2434dde42f fix(node,git): isolate authed pushes in a dedicated write pool (#62)
git-receive-pack now draws from a separate git_write_semaphore (max_concurrent_git_pushes) instead of the shared pool, so a flood of anonymous reads can no longer shed an authenticated push at admission (jatmn P1a). The shared field is renamed git_read_semaphore and continues to gate upload-pack and both info/refs advertisements. The write permit stays above acquire_write so it precedes the Tigris fresh-acquire (INV-10).

Handler-layer tests: write-pool shed (503), and a cross-boundary proof that an exhausted read pool does NOT shed a push; both mutation-checked (routing receive-pack back to the read pool flips each RED). 497 tests pass.

Part of #174.
2026-07-10 12:29:30 -05:00
t db4de844ad feat(node,config): add write-pool and per-caller read-cap knobs (#62)
Add max_concurrent_git_pushes (default 32) and max_concurrent_reads_per_caller (default 16), both clap range(1..=1_048_576) so an oversized value is a clean CLI error, not a Semaphore::new boot panic. The per-caller knob documents that per-source-IP keying is only as granular as GITLAWB_TRUSTED_PROXY. Wiring lands in the following commits; these are the config surface for the #174 concurrency-fairness fix.

Resolves jatmn P1a/P1b groundwork on #174.
2026-07-10 12:18:07 -05:00
t 65ba1374af feat(node,git): cap concurrent served git ops with a 503 load-shed (#62)
PR3 of the #62 served-git hardening stack (timeout #165 and teardown
wiring #150 are merged). A bounded semaphore caps how many upload-pack /
receive-pack / info-refs operations run at once; past the cap a request
is shed with a clean 503 + Retry-After before spawning another git
subprocess, instead of exhausting the PID/thread table. A permit is
acquired at the top of each of the three handlers and held for the whole
op, releasing on return.

The cap is a portable backstop: the compose pids_limit is absent on Fly,
whose 500-connection cap is a different axis. Size --max-concurrent-git-ops
(GITLAWB_MAX_CONCURRENT_GIT_OPS, default 128) below the process budget.
Range 1..=1_048_576 so 0 (shed everything) and an oversized value that
would panic tokio's Semaphore at boot are clean CLI errors.

Known gap, tracked separately: info/refs and the withheld-blob
(upload_pack_excluding) path are not duration-bounded and do not reap
their git child on client disconnect, so a hung git on those two paths
holds its slot until it exits and live git can briefly exceed the cap.
The main pack path (run_git_service) tears its group down on drop.

Tests: Overloaded maps to 503 + Retry-After; the config knob defaults
and rejects out-of-range; git_permit sheds at capacity and releases; and
each of the three endpoints sheds with 503 when the semaphore is
exhausted (load-bearing: drop the permit line and the endpoint test goes
red).
2026-07-10 12:10:30 -05:00
109 changed files with 2743 additions and 48693 deletions

No files matched your search

+7 -103
View File
@@ -24,11 +24,8 @@ DATABASE_URL=postgresql://gitlawb:changeme@localhost:5432/gitlawb
# ── Database pool & startup resilience ────────────────────────────────────
# Maximum connections in the PostgreSQL pool. A cap, not a floor —
# connections open lazily. Size against the DB server's max_connections,
# remembering admin tooling opens its own pool. Each concurrent write pins one
# connection for its whole duration (the connection-affine advisory lock), so the
# node REJECTS at boot any value below GITLAWB_MAX_CONCURRENT_GIT_PUSHES + 8
# headroom — keep this comfortably above that (default 48 for pushes 32).
GITLAWB_DB_MAX_CONNECTIONS=48
# remembering admin tooling opens its own pool.
GITLAWB_DB_MAX_CONNECTIONS=20
# Seconds a request waits for a pool connection before failing with 503.
GITLAWB_DB_ACQUIRE_TIMEOUT_SECS=5
# Upper bound on each startup connect+migrate attempt, in seconds. Keep it
@@ -94,14 +91,9 @@ GITLAWB_REQUIRE_SIGNED_PEER_WRITES=false
# Require the authenticated pusher to be the repo owner on git-receive-pack.
# A valid did:key signature is authentication, not authorization: anyone can
# mint a key, derive its DID and sign, so with this off every signed caller may
# push to every repository, private ones included. On by default.
#
# Set to false only for a rolling upgrade whose pushers are not yet the repo
# owner. Note that delegated and CI keys count as non-owners: UCAN git/push is
# verified but not yet honored for authorization, so they cannot push while this
# is on. See docs/RUN-A-NODE.md.
GITLAWB_ENFORCE_OWNER_PUSH=true
# sign as their own DID. When true, pushes from a non-owner DID are rejected.
# Keep false until the repo owner is ready for owner-only writes.
GITLAWB_ENFORCE_OWNER_PUSH=false
# Comma-separated libp2p multiaddrs.
# Example: /ip4/1.2.3.4/udp/7546/quic-v1/p2p/12D3KooW...
@@ -120,13 +112,7 @@ GITLAWB_MAX_PACK_BYTES=2147483648
# worker and, on push, the repo write lock. Also bounds both info/refs
# advertisements, the withheld-blob pack build, and the push-side candidate
# discovery (rev-list / cat-file), all reaped via process-group teardown (#174).
# On the path-scoped upload-pack path the withheld-blob classification walk and
# the pack serve share ONE deadline, so this value bounds their COMBINED duration
# rather than giving each stage a full budget: a walk that consumes it leaves the
# serve nothing and the clone gets a 504. Serving large path-scoped repos may
# need a higher value here than when each stage was budgeted separately.
# Must be 1..=3153600000 (100 years): the node derives deadlines from this value,
# and a larger one cannot be represented. Default 600.
# Default 600.
GITLAWB_GIT_SERVICE_TIMEOUT_SECS=600
# Max seconds the storage-ACQUISITION phase of a served git op may run before the
@@ -155,25 +141,6 @@ GITLAWB_MAX_CONCURRENT_GIT_OPS=128
# Retry-After. Default 32.
GITLAWB_MAX_CONCURRENT_GIT_PUSHES=32
# Max pushes PARKED at once waiting for the per-repo write lease of the SAME repo.
# A parked push holds its already-buffered pack body in memory, so this bounds that
# memory for a hot repo. Past the cap the newest push sheds a clean 503 +
# Retry-After ("repo is busy with another push") instead of joining the queue; a
# push to a DIFFERENT repo is never affected. The lease holder is not counted.
# Raising GITLAWB_MAX_CONCURRENT_GIT_PUSHES does not raise this; set it explicitly.
# Default 8.
GITLAWB_REPO_LEASE_MAX_WAITERS=8
# Max concurrent post-push pin loops (IPFS + Pinata pin_new_objects) across all
# repos. The per-repo encrypt-task set caps the pin-task COUNT; this caps how many
# of those loops RUN CONCURRENTLY, and so how much MB-scale object-id list memory is
# held WHILE BEING PINNED. It does not bound retained memory in general: on the local
# IPFS path the caller builds its list before this pool is entered, so a loop parked
# waiting for a permit still holds its full list, and how many park is capped only
# per repo. Do not size memory from this knob alone. A loop DEFERS (waits) when the
# pool is full, never drops a pin. Default 8.
GITLAWB_MAX_CONCURRENT_PIN_TASKS=8
# Max concurrent read ops (upload-pack + the upload-pack info/refs advertisement)
# a single caller may hold, so one caller cannot monopolize the read pool. Keyed
# on the resolved SOURCE IP, never the DID: a signature does not move a caller off
@@ -184,24 +151,6 @@ GITLAWB_MAX_CONCURRENT_PIN_TASKS=8
# Default 16.
GITLAWB_MAX_CONCURRENT_READS_PER_CALLER=16
# Two further per-source concurrency caps exist on the PUSH side but have NO
# environment variable of their own: both are DERIVED as
# GITLAWB_MAX_CONCURRENT_GIT_PUSHES / 8, with a floor of 1. They are not settable
# independently, so raising the push pool raises both proportionally, and the
# default push pool of 32 gives each a cap of 4.
# * The anon receive-pack info/refs advertisement cap: one source IP may hold at
# most this many slots in the dedicated advert pool, so saturating that pool
# takes ~8 distinct source IPs.
# * The authenticated receive-pack POST cap: one source IP may hold at most this
# many WRITE-pool slots. This one is load-bearing for push availability, since
# it is acquired before the global write permit: without it, one host minting
# disposable did:key identities could open enough slow pushes to monopolize the
# write pool and 503 every other source (the push rate limiter caps arrival
# rate, not in-flight concurrency, and owner enforcement can be turned off for
# a rolling upgrade).
# Keyed on the resolved source IP, never the DID, so a DID farm does not defeat
# them; keying granularity follows GITLAWB_TRUSTED_PROXY like the read cap above.
# ── Push rate limiting (git-receive-pack flood brake) ─────────────────────
# Max receive-pack requests (info/refs advertisement + push POST) per client
# IP per hour. 0 disables. Default 600.
@@ -222,52 +171,7 @@ GITLAWB_IPFS_WALK_PER_SOURCE=4
# Max legacy (NULL-provenance) repos probed per single /ipfs request, bounding the
# scan-fallback fan-out (git cat-file per candidate repo) for an anonymous caller. A
# truncated scan sheds a retryable 503, never a false 404. Default 256.
GITLAWB_IPFS_MAX_LEGACY_PROBES=256
# Max repo ROWS one /ipfs request's legacy scan may read from the database. The probe
# ceiling above only counts once a probe runs, and quarantined or private repos are
# denied before that, so without this an all-denying inventory paged the whole repo
# table for one anonymous request. A truncated scan sheds a retryable 503 carrying a
# sealed `continuation` token; echo it as ?scan= to resume, so a holder buried past the
# ceiling is reachable in ceil(repos / ceiling) + 1 requests. Raising this also raises
# the per-caller /ipfs work allowance, since each page is charged to it. Lowering it
# sharpens a coarse oracle: laddering to the end reveals the node's total repo count to
# within one ceiling. Default 2048.
GITLAWB_IPFS_MAX_LEGACY_SCAN_ROWS=2048
# Max EXPENSIVE path-scope visibility walks per single /ipfs request (only a
# blob in a path-scoped repo costs a full-history walk). Over-cap repos are
# skipped without a verdict and the scan continues; if the object is then found
# nowhere the request sheds a retryable 503 instead of a false 404. The effective
# cap is the tighter of this value and the node's internal per-request ceiling of
# 17, so values above 17 have no effect; lower values do tighten it. Default 64.
GITLAWB_IPFS_MAX_REPOS_WALKED=64
# Ceiling on repos one /ipfs request may VISIT past the visibility gate. Each
# visit costs a repo acquire — on a Tigris cache miss a full archive download,
# so this is also the worst-case object-store fetch count per request — plus a
# cat-file probe. On exhaustion the scan stops and sheds a retryable 503.
# Default 1024.
GITLAWB_IPFS_MAX_REPO_VISITS=1024
# Absolute wall-clock budget for one admitted /ipfs request's acquire+walk
# lifetime (all stages of the whole scan). Per-stage clamps bound the acquire
# and walk stages to the remaining budget, and no stage starts once it is
# exhausted (the scan then sheds a retryable 503). The object-type probe and
# content-read cat-file subprocesses are budget-checked before starting AND each
# run under their own deadline (the lesser of GITLAWB_GIT_SERVICE_TIMEOUT_SECS and
# the remaining budget), reaped via process-group teardown, so a hung cat-file
# cannot hold the request's walk slot past it. One hang path is still unbounded:
# the probe's object-store readability check is a plain filesystem sweep with
# nothing to reap, so a wedged filesystem can hold the slot past the deadline.
# Must be 1..=3153600000 (100 years): the node derives an Instant deadline from
# this value, and a larger one cannot be represented. Default 600.
GITLAWB_IPFS_REQUEST_BUDGET_SECS=600
# Shorter budget (seconds) for the pre-walk CID resolve: the lookup that maps a
# requested CID to its git oid(s), which runs while the scarce walk admission is
# already held. A well-formed CID with no pin row does no probe and no walk work,
# so without this a stalled lookup could hold a walk slot for the whole request
# budget while nothing walked. The effective deadline is the lesser of this and
# the remaining request budget; walk and probe work stay on the request budget,
# so a slow but progressing scan is never shed by it.
# Must be 1..=3153600000 (100 years). Default 10.
GITLAWB_IPFS_RESOLVE_BUDGET_SECS=10
GITLAWB_IPFS_MAX_REPOS_WALKED=256
# Max /ipfs/{cid} requests per client IP per hour (route flood brake, distinct
# from the concurrency caps above). 0 disables. Default 600.
GITLAWB_IPFS_RATE_LIMIT=600
-121
View File
@@ -1,121 +0,0 @@
# Tracks main's CI state as a labeled issue: opens (or updates) a ci-main-red
# issue when a push-to-main run of PR Checks ends red, and closes it when a
# subsequent run on main's tip goes green.
#
# Why: two individually-green PRs can break main in combination when the
# second merges on checks that ran against a stale base (semantic merge
# race — #113 + #145 broke test compilation, found only via #229). A red main
# is otherwise silent, and release-please PRs inherit the breakage.
#
# The alert reflects main's CURRENT STATE, not raw CI events: runs whose
# commit is no longer main's tip are ignored (a re-run of an old red run
# after a fix has landed must not cry wolf), and a green run on the tip
# closes any open alert.
name: Main red alert
on:
workflow_run:
workflows: ["PR Checks"]
types: [completed]
branches: [main]
permissions:
issues: write
contents: read # getBranch tip check
# Serialize runs: concurrent completions would race the label/issue lookups
# into duplicates (or 422 on createLabel). queue: max keeps every pending
# completion instead of letting a later (possibly green-skipped) arrival
# displace a pending red alert under burst merges; it is only incompatible
# with cancel-in-progress: true, which we don't use.
concurrency:
group: main-red-alert
cancel-in-progress: false
queue: max
jobs:
track:
name: Track main CI state
# timed_out (the test job runs 45 min against Postgres) and
# startup_failure (malformed workflow) are red too — only cancelled is
# excluded as a deliberate human abort. success runs are needed to close
# the alert once main recovers.
if: >-
github.event.workflow_run.event == 'push' &&
contains(fromJSON('["failure", "timed_out", "startup_failure", "success"]'), github.event.workflow_run.conclusion)
runs-on: ubuntu-latest
timeout-minutes: 5
steps:
- name: Reconcile the broken-main issue with main's current state
uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8.0.0
with:
script: |
const run = context.payload.workflow_run;
const label = 'ci-main-red';
// Freshness gate: only the run for main's current tip may open or
// close anything. Re-runs of superseded commits are events about
// the past, not statements about main's state.
const { data: branch } = await github.rest.repos.getBranch({
...context.repo, branch: 'main',
});
if (branch.commit.sha !== run.head_sha) {
core.info(`run ${run.id} is for ${run.head_sha}, main is at ${branch.commit.sha} — ignoring stale run`);
return;
}
// listForRepo returns PRs carrying the label too — those must
// never be mistaken for the tracking issue.
const open = (await github.rest.issues.listForRepo({
...context.repo, state: 'open', labels: label, per_page: 100,
})).data.filter(item => !item.pull_request);
if (run.conclusion === 'success') {
const body = `Recovered: [PR Checks](${run.html_url}) is green on main at \`${run.head_sha}\`.`;
for (const issue of open) {
await github.rest.issues.createComment({
...context.repo, issue_number: issue.number, body,
});
await github.rest.issues.update({
...context.repo, issue_number: issue.number, state: 'closed', state_reason: 'completed',
});
}
return;
}
const headline = run.head_commit?.message?.split('\n')[0] ?? '';
const body = [
`[PR Checks run](${run.html_url}) concluded \`${run.conclusion}\` on main at \`${run.head_sha}\`.`,
'',
`> ${headline}`,
'',
'Likely a semantic merge race: each merged PR was green on a stale base.',
'Fix main before merging anything else — release-please PRs inherit this breakage.',
'',
"This issue closes automatically when a run on main's tip goes green.",
].join('\n');
try {
await github.rest.issues.getLabel({ ...context.repo, name: label });
} catch (e) {
if (e.status !== 404) throw e;
await github.rest.issues.createLabel({
...context.repo,
name: label,
color: 'b60205',
description: 'CI is failing on main',
});
}
if (open.length > 0) {
await github.rest.issues.createComment({
...context.repo, issue_number: open[0].number, body,
});
} else {
await github.rest.issues.create({
...context.repo,
title: 'CI is red on main',
body,
labels: [label],
});
}
+4 -120
View File
@@ -2,17 +2,6 @@ name: PR Checks
on:
pull_request:
# Run in the merge queue so a PR is re-validated against the true merged
# result before it lands. Guards against semantic merge races where two
# individually-green PRs break main in combination (#113 + #145 → #229).
merge_group:
# Release-branch heads are created and pushed by GITHUB_TOKEN (release-please
# and the lock-sync job), and GitHub's recursion guard means those pushes
# trigger no workflow runs. workflow_dispatch is the documented carve-out that
# GITHUB_TOKEN CAN trigger, so the release flow dispatches this workflow at
# the release branch to attach check runs to its head. Read-only workflow
# (contents: read), so the dispatch surface adds no privilege.
workflow_dispatch:
push:
branches:
- main
@@ -53,20 +42,7 @@ jobs:
run: cargo fmt --all -- --check
- name: cargo clippy
run: cargo clippy --locked --workspace --all-targets -- -D warnings
release-tag-validation:
name: release tag validation
runs-on: ubuntu-latest
timeout-minutes: 5
steps:
- name: Check out repository
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
persist-credentials: false
- name: Test release tag resolver
run: scripts/test-resolve-release-tag.sh
run: cargo clippy --workspace --all-targets -- -D warnings
test:
name: test (${{ matrix.toolchain }})
@@ -114,7 +90,7 @@ jobs:
- name: cargo test
env:
DATABASE_URL: postgres://postgres:postgres@localhost:5432/gitlawb_test
run: cargo test --locked --workspace
run: cargo test --workspace
build-release:
name: build --release
@@ -137,7 +113,7 @@ jobs:
key: release
- name: cargo build --release
run: cargo build --locked --release --workspace
run: cargo build --release --workspace
audit:
name: cargo audit
@@ -219,7 +195,7 @@ jobs:
key: msrv
- name: cargo check on MSRV
run: cargo check --locked --workspace --all-targets
run: cargo check --workspace --all-targets
docker-build:
name: Docker build smoke test
@@ -236,95 +212,3 @@ jobs:
- name: Smoke test --version
run: docker run --rm gitlawb-node:ci-test --version
test-windows:
name: test (windows, non-blocking)
runs-on: windows-latest
timeout-minutes: 45
# Non-blocking, mirroring the informational `beta` lane above. We ship `gl`
# and `git-remote-gitlawb` to Windows users (release.yml) with zero Windows
# test coverage, so run their suites here to make Windows-only regressions
# VISIBLE on the PR without gating merge (#228). These two are the client
# CLIs and depend on no database, so no Postgres service is needed (service
# containers do not run on windows-latest regardless). `git` is preinstalled
# on windows-latest, which the real_git_fetch harness relies on.
continue-on-error: true
steps:
- name: Check out repository
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
persist-credentials: false
- name: Set up Rust toolchain
uses: dtolnay/rust-toolchain@b3b07ba8b418998c39fb20f53e8b695cdcc8de1b # stable
with:
toolchain: stable
- name: Cache cargo
uses: Swatinem/rust-cache@98c8021b550208e191a6a3145459bfc9fb29c4c0 # v2.8.0
with:
key: windows
- name: cargo test (shipped Windows crates)
run: cargo test --locked -p gl -p git-remote-gitlawb
# gitlawb-core is embedded by every consumer (gl, git-remote-gitlawb, the node
# daemon), so it must stay lean. This gate fails if gitlawb-core's normal
# (non-dev, non-build) dependency tree gains any crate not on the allowlist in
# ci/gitlawb-core-allowed-deps.txt. Exhaustive-by-construction: a new heavy
# dependency reds CI whether or not anyone thought to ban it, which a denylist
# cannot do. Regen instructions live in the allowlist header.
core-deps-purity:
name: gitlawb-core dependency purity
runs-on: ubuntu-latest
timeout-minutes: 15
steps:
- name: Check out repository
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
persist-credentials: false
- name: Set up Rust toolchain
uses: dtolnay/rust-toolchain@b3b07ba8b418998c39fb20f53e8b695cdcc8de1b # stable
with:
toolchain: stable
- name: Cache cargo
uses: Swatinem/rust-cache@98c8021b550208e191a6a3145459bfc9fb29c4c0 # v2.8.0
with:
key: core-deps-purity
- name: Check gitlawb-core dependency allowlist
run: bash scripts/check-gitlawb-core-deps.sh
# release-please only rewrites a crate's version when the line carries the
# `# x-release-please-version` annotation AND the manifest is listed under
# extra-files in release-please-config.json. Miss either half and the crate
# is skipped silently at release time, freezing at its creation version:
# icaptcha-client was added as a workspace member after the config was last
# written and sat at 0.4.0 through four releases before anyone noticed.
# Exhaustive-by-construction, like the allowlist gate above: the required set
# is derived from `cargo metadata`, so a newly added crate reds CI whether or
# not its author knew this gate existed.
release-versioning:
name: release versioning coverage
runs-on: ubuntu-latest
timeout-minutes: 15
steps:
- name: Check out repository
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
persist-credentials: false
- name: Set up Rust toolchain
uses: dtolnay/rust-toolchain@b3b07ba8b418998c39fb20f53e8b695cdcc8de1b # stable
with:
toolchain: stable
- name: Cache cargo
uses: Swatinem/rust-cache@98c8021b550208e191a6a3145459bfc9fb29c4c0 # v2.8.0
with:
key: release-versioning
- name: Check every workspace crate is release-managed
run: bash scripts/check-release-versioning.sh
+2 -8
View File
@@ -24,7 +24,7 @@ jobs:
timeout-minutes: 10
steps:
- name: Label and guide
uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8.0.0
uses: actions/github-script@f28e40c7f34bde8b3046d885e986cb6290c5673b # v7
with:
script: |
const pr = context.payload.pull_request;
@@ -70,15 +70,9 @@ jobs:
const changedRust = names.some(n => n.startsWith("crates/") && n.endsWith(".rs"));
// A test can live in a tests/ dir, a *_test.rs file, or (the common Rust
// pattern) an inline #[test] / #[cfg(test)] block added inside the source file.
// Async tests use a namespaced attribute (#[tokio::test], #[async_std::test],
// possibly with args like #[tokio::test(flavor = ...)]) rather than the bare
// #[test]. Match only on added lines, with the attribute at the start of the
// line after horizontal whitespace, covering bare, cfg(test), namespaced,
// optional leading ::, and raw-identifier (r#test) forms. No head-file fetch
// or lexer: needs-tests is advisory, not a merge gate.
const addsInlineTest = files.some(f =>
f.filename.endsWith(".rs") && f.patch &&
/^\+[ \t]*#\[[ \t]*(?:cfg[ \t]*\([ \t]*test[ \t]*\)|(?:::[ \t]*)?(?:[\w-]+[ \t]*::[ \t]*)*(?:r#)?test\b)/m.test(f.patch));
/^\+.*#\[(test\]|cfg\(test\))/m.test(f.patch));
const touchedTests =
names.some(n => n.includes("/tests/") || n.endsWith("_test.rs")) || addsInlineTest;
if (changedRust && !touchedTests) want.add("needs-tests");
+53 -360
View File
@@ -4,20 +4,6 @@ on:
push:
branches:
- main
# Manual backfill: republish an already-cut release to a channel that
# missed it. Fill in only the channel(s) you want — an empty input leaves
# that channel's jobs skipped. npm reuses the existing release's binary
# assets; docker rebuilds the image from the tag and re-tags the manifest.
workflow_dispatch:
inputs:
npm_backfill_tag:
description: "Existing release tag to publish to npm (e.g. v0.6.0)"
required: false
type: string
docker_backfill_tag:
description: "Existing release tag to publish to ghcr (e.g. v0.6.0)"
required: false
type: string
concurrency:
group: release-${{ github.ref }}
@@ -28,10 +14,7 @@ permissions: {}
jobs:
release-please:
# Skipped on manual dispatch: a backfill must never create or advance a
# release PR as a side effect (npm-publish tolerates the skipped need via
# !cancelled(), and every other job gates on release_created == 'true').
if: ${{ github.repository == 'Gitlawb/node' && github.event_name != 'workflow_dispatch' }}
if: ${{ github.repository == 'Gitlawb/node' }}
name: Release Please
runs-on: ubuntu-latest
permissions:
@@ -41,7 +24,6 @@ jobs:
release_created: ${{ steps.release.outputs.release_created }}
tag_name: ${{ steps.release.outputs.tag_name }}
version: ${{ steps.release.outputs.version }}
pr: ${{ steps.release.outputs.pr }}
steps:
- name: Run release-please
id: release
@@ -49,197 +31,24 @@ jobs:
with:
token: ${{ secrets.GITHUB_TOKEN }}
# The release PR bumps the workspace crate versions in the Cargo.tomls but
# release-please cannot update Cargo.lock, so without this job every release
# merge ships a tag whose lockfile disagrees with its manifests (a --locked
# build from the tag fails, and pr-checks builds --locked). Sync the lock on
# the release branch so the tagged tree is internally consistent.
#
# Runs on every push-triggered release run, not only when release-please
# reports the PR: the `pr` output is set only on runs that CREATE or UPDATE
# the release PR, so gating on it would leave an already-open release PR
# (opened before this job existed, or untouched by a non-releasable push)
# permanently without the sync and without check runs. The resolve step
# discovers any open release-please PR itself and no-ops when there is none.
sync-release-lock:
name: Sync Cargo.lock on the release PR
needs: release-please
if: ${{ !cancelled() && github.repository == 'Gitlawb/node' && github.event_name != 'workflow_dispatch' }}
runs-on: ubuntu-latest
permissions:
contents: write
pull-requests: read
# actions: write lets the final step dispatch pr-checks on the release
# branch. Note this grant is repo-wide dispatch rights (GitHub cannot
# scope it to one workflow); acceptable here because reaching this job's
# steps already requires write to main. Needed because every push to the
# release branch (release-please's and the lock sync below) is made with
# GITHUB_TOKEN, which triggers no workflow runs, so without an explicit
# dispatch the release PR's head carries no check runs and the --locked
# gate never validates it.
actions: write
steps:
# Prefer the fresh `pr` output when this run just created/updated the
# release PR; otherwise fall back to querying for an open release-please
# branch (prefix match: the branch carries `--components--` suffixes).
- name: Resolve the open release PR branch
id: resolve
env:
GH_TOKEN: ${{ github.token }}
OWNER: ${{ github.repository_owner }}
BRANCH_FROM_OUTPUT: ${{ needs.release-please.outputs.pr && fromJSON(needs.release-please.outputs.pr).headBranchName || '' }}
run: |
branch="$BRANCH_FROM_OUTPUT"
if [ -z "$branch" ]; then
# The branch name alone is not a trust signal. `gh pr list` includes
# fork PRs, and a fork's headRefName is bare (no owner prefix), so a
# prefix match by itself lets any fork nominate the branch this job
# checks out, commits to, and dispatches checks on. Require the head
# repo to be THIS repository and the author to be the release-please
# app, so neither a fork nor a same-repo human branch named
# `release-please--*` can be selected. Then take the highest PR
# number: `[0]` depended on gh's default ordering, which is not a
# documented guarantee, and picking a stable one keeps a second
# (component) release PR from making the choice flap between runs.
branch=$(gh pr list --repo "$GITHUB_REPOSITORY" --state open \
--json headRefName,headRepositoryOwner,number,author \
--jq '[ .[]
| select(.headRepositoryOwner.login == env.OWNER)
| select(.author.login == "app/github-actions")
| select(.headRefName | startswith("release-please--"))
] | max_by(.number).headRefName // empty')
fi
if [ -z "$branch" ]; then
echo "no open release PR; nothing to sync"
fi
echo "branch=$branch" >> "$GITHUB_OUTPUT"
- name: Checkout release branch
if: ${{ steps.resolve.outputs.branch != '' }}
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
ref: ${{ steps.resolve.outputs.branch }}
# Resolve on a declared toolchain instead of whatever Rust the runner
# image happens to ship, so the lock this job writes comes off the same
# stable channel the other cargo jobs use. Not a claim of identical
# resolution: `stable` still moves between runs, and the MSRV gate
# checks this same lock on 1.91 (pr-checks.yml). It only removes the
# runner image as an undeclared input.
- name: Install Rust toolchain
if: ${{ steps.resolve.outputs.branch != '' }}
uses: dtolnay/rust-toolchain@b3b07ba8b418998c39fb20f53e8b695cdcc8de1b # stable
with:
toolchain: stable
- name: Sync workspace crate versions into Cargo.lock
if: ${{ steps.resolve.outputs.branch != '' }}
run: cargo update --workspace
- name: Commit and push when the lock changed
if: ${{ steps.resolve.outputs.branch != '' }}
run: |
if git diff --quiet Cargo.lock; then
echo "Cargo.lock already in sync"
exit 0
fi
git config user.name "github-actions[bot]"
git config user.email "41898282+github-actions[bot]@users.noreply.github.com"
git add Cargo.lock
git commit -m "chore: sync Cargo.lock with the release version bump"
git push
# Every push to the release branch is made with GITHUB_TOKEN (release-
# please's own pushes and the lock sync above), and events created by
# GITHUB_TOKEN start no workflow runs. workflow_dispatch is the documented
# exception GITHUB_TOKEN may trigger, so dispatch pr-checks at the release
# branch to attach check runs to its current head. Unconditional (also on
# the already-in-sync path) because the branch's ORIGINAL head has the
# same no-runs problem. Loud on failure by design: a silent fallback
# would recreate the checkless-head problem, and a job rerun is safe
# (sync no-ops, dispatch retries). Loop-safe: pr-checks is read-only and
# dispatches nothing. Runs after the push ack, so the dispatched ref tip
# is the sync commit. Merge-queue note: this validates the branch tip;
# the enforced merge queue's merge_group run still re-validates the true
# merged result before landing.
- name: Dispatch PR checks on the release branch
if: ${{ steps.resolve.outputs.branch != '' }}
env:
GH_TOKEN: ${{ github.token }}
BRANCH: ${{ steps.resolve.outputs.branch }}
run: |
# Every dispatch failure fails the job, including the 422 a release
# branch cut before pr-checks gained its workflow_dispatch trigger
# returns ("does not have a workflow_dispatch trigger", reported
# against the dispatched ref's copy of the file). An earlier revision
# tolerated exactly that 422 on the theory that release-please would
# rewrite the branch and heal it, but a rewrite only happens when a
# later releasable commit lands: an open release PR nobody touches is
# never rewritten. Tolerating it therefore left the lock-sync commit
# this job had just pushed sitting on the release head with no
# --locked validation, which is the precise state this workflow exists
# to prevent. The tolerance is also no longer needed: once this lands,
# every release branch is cut from a main that already carries the
# trigger, so the only branch that could 422 is one open at merge time
# (there is none), and a job rerun after a manual rebase is safe (the
# sync no-ops, the dispatch retries).
gh api "repos/${GITHUB_REPOSITORY}/actions/workflows/pr-checks.yml/dispatches" \
-f "ref=$BRANCH"
# Each architecture builds NATIVELY (amd64 on ubuntu-latest, arm64 on
# ubuntu-24.04-arm) and pushes by digest; docker-manifest below stitches the
# digests into the tagged multi-arch image. No QEMU: emulating the arm64
# Rust release build wedged the v0.6.0 run for 3+ hours and, via the
# workflow concurrency group, blocked every queued release run behind it.
docker:
name: Build & Push Docker Image (${{ matrix.arch }})
name: Build & Push Docker Image
needs: release-please
# Runs on a fresh release, or on manual dispatch for an existing tag
# (release-please is skipped on dispatch, hence !cancelled()).
if: >-
${{ !cancelled() && (
needs.release-please.outputs.release_created == 'true' ||
(github.event_name == 'workflow_dispatch' && inputs.docker_backfill_tag != '')
) }}
strategy:
fail-fast: false
matrix:
include:
- arch: amd64
os: ubuntu-latest
platform: linux/amd64
- arch: arm64
os: ubuntu-24.04-arm
platform: linux/arm64
runs-on: ${{ matrix.os }}
if: ${{ needs.release-please.outputs.release_created == 'true' }}
runs-on: ubuntu-latest
permissions:
contents: read
packages: write
steps:
- name: Check out workflow scripts
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
persist-credentials: false
- name: Resolve release tag
id: rel
env:
DISPATCH_TAG: ${{ inputs.docker_backfill_tag }}
RELEASE_TAG: ${{ needs.release-please.outputs.tag_name }}
run: |
set -euo pipefail
scripts/resolve-release-tag.sh "${DISPATCH_TAG:-$RELEASE_TAG}"
# ghcr requires a lowercase repository path, and unlike metadata-action,
# buildx's `--output name=` does no lowercasing — a mixed-case owner
# makes the digest push fail with "invalid reference format".
echo "image=ghcr.io/${GITHUB_REPOSITORY,,}" >> "$GITHUB_OUTPUT"
- name: Checkout release tag
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
ref: ${{ steps.rel.outputs.tag }}
ref: ${{ needs.release-please.outputs.tag_name }}
persist-credentials: false
- name: Set up QEMU
uses: docker/setup-qemu-action@29109295f81e9208d7d86ff1c6c12d2833863392 # v3.6.0
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@b5ca514318bd6ebac0fb2aedd5d36ec1b5c232a2 # v3.10.0
@@ -250,130 +59,45 @@ jobs:
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Extract metadata
id: meta
uses: docker/metadata-action@902fa8ec7d6ecbf8d84d538b9b233a880e428804 # v5.7.0
with:
images: ghcr.io/${{ github.repository }}
tags: |
type=semver,pattern={{version}},value=${{ needs.release-please.outputs.version }}
type=semver,pattern={{major}}.{{minor}},value=${{ needs.release-please.outputs.version }}
type=raw,value=latest
- name: Build and load locally (smoke)
uses: docker/build-push-action@14487ce63c7a62a4a324b0bfb37086795e31c6c1 # v6.16.0
with:
context: .
platforms: ${{ matrix.platform }}
load: true
tags: gitlawb-node:smoke
cache-from: type=gha,scope=docker-${{ matrix.arch }}
cache-from: type=gha
# Native runner on both arches, so the built image can always execute.
# Assert the released version so a stale artifact fails here, not in prod.
- name: Smoke test
env:
VERSION: ${{ steps.rel.outputs.version }}
run: |
set -euo pipefail
out="$(docker run --rm gitlawb-node:smoke --version)"
echo "$out"
grep -qF "$VERSION" <<<"$out" || {
echo "::error::image --version did not report expected version $VERSION (got: $out)"
exit 1
}
run: docker run --rm gitlawb-node:smoke --version
# Push by digest only — tags are applied once by docker-manifest so a
# half-finished matrix can never publish a partially-tagged image.
# provenance:false keeps each push a plain single-arch manifest, which is
# what imagetools create expects to merge.
- name: Build and push by digest
id: push
- name: Build and push (multi-arch)
uses: docker/build-push-action@14487ce63c7a62a4a324b0bfb37086795e31c6c1 # v6.16.0
with:
context: .
platforms: ${{ matrix.platform }}
provenance: false
outputs: type=image,name=${{ steps.rel.outputs.image }},push-by-digest=true,name-canonical=true,push=true
cache-from: type=gha,scope=docker-${{ matrix.arch }}
cache-to: type=gha,mode=max,scope=docker-${{ matrix.arch }}
- name: Export digest
env:
DIGEST: ${{ steps.push.outputs.digest }}
run: |
set -euo pipefail
mkdir -p /tmp/digests
touch "/tmp/digests/${DIGEST#sha256:}"
- name: Upload digest
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
with:
name: docker-digest-${{ matrix.arch }}
path: /tmp/digests/*
if-no-files-found: error
retention-days: 1
docker-manifest:
name: Publish Docker manifest
needs: [release-please, docker]
if: ${{ !cancelled() && needs.docker.result == 'success' }}
runs-on: ubuntu-latest
permissions:
contents: read
packages: write
steps:
- name: Check out workflow scripts
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
persist-credentials: false
- name: Resolve release tag
id: rel
env:
DISPATCH_TAG: ${{ inputs.docker_backfill_tag }}
RELEASE_TAG: ${{ needs.release-please.outputs.tag_name }}
run: |
set -euo pipefail
scripts/resolve-release-tag.sh "${DISPATCH_TAG:-$RELEASE_TAG}"
- name: Download digests
uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4.3.0
with:
pattern: docker-digest-*
path: /tmp/digests
merge-multiple: true
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@b5ca514318bd6ebac0fb2aedd5d36ec1b5c232a2 # v3.10.0
- name: Log in to GitHub Container Registry
uses: docker/login-action@74a5d142397b4f367a81961eba4e8cd7edddf772 # v3.4.0
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Create and push multi-arch manifest
env:
VERSION: ${{ steps.rel.outputs.version }}
run: |
set -euo pipefail
# ghcr requires a lowercase repository path.
IMAGE="ghcr.io/${GITHUB_REPOSITORY,,}"
MAJOR_MINOR="${VERSION%.*}"
digests=""
for f in /tmp/digests/*; do
digests="$digests $IMAGE@sha256:$(basename "$f")"
done
# shellcheck disable=SC2086
docker buildx imagetools create \
-t "$IMAGE:$VERSION" \
-t "$IMAGE:$MAJOR_MINOR" \
-t "$IMAGE:latest" \
$digests
docker buildx imagetools inspect "$IMAGE:$VERSION"
platforms: linux/amd64,linux/arm64
push: true
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
cache-from: type=gha
cache-to: type=gha,mode=max
- name: Release summary
env:
VERSION: ${{ steps.rel.outputs.version }}
TAG: ${{ steps.rel.outputs.tag }}
run: |
{
echo "## Released $TAG"
echo "## Released ${{ needs.release-please.outputs.tag_name }}"
echo
echo "- Image: \`ghcr.io/${GITHUB_REPOSITORY,,}:$VERSION\` (linux/amd64 + linux/arm64)"
echo "- GitHub: https://github.com/$GITHUB_REPOSITORY/releases/tag/$TAG"
echo "- Image: \`ghcr.io/${{ github.repository }}:${{ needs.release-please.outputs.version }}\`"
echo "- GitHub: https://github.com/${{ github.repository }}/releases/tag/${{ needs.release-please.outputs.tag_name }}"
} >> "$GITHUB_STEP_SUMMARY"
release-binaries:
@@ -445,7 +169,7 @@ jobs:
set -euo pipefail
args=""
for b in $BINS; do args="$args -p $b"; done
cargo build --release --locked --target ${{ matrix.target }} $args
cargo build --release --target ${{ matrix.target }} $args
# Run each packaged binary's --version so a broken release artifact fails the
# build instead of shipping. Every target builds on a matching-arch runner,
@@ -524,61 +248,37 @@ jobs:
npm-publish:
name: Publish to npm
needs: [release-please, release-binaries]
# Runs on a fresh release, or on manual dispatch for an existing tag
# (release-binaries is skipped on dispatch, hence !cancelled()). Publishing
# auth is npm Trusted Publishing (GitHub OIDC) — no NPM_TOKEN. Each
# @gitlawb package must have this repo + workflow (release.yml) configured
# as its trusted publisher on npmjs.com, or publish fails loudly here —
# deliberately loud: the silent secret-guard skip is how 0.4.x–0.6.0
# never reached npm.
if: >-
${{ !cancelled() && (
(needs.release-please.outputs.release_created == 'true' && needs.release-binaries.result == 'success') ||
(github.event_name == 'workflow_dispatch' && inputs.npm_backfill_tag != '')
) }}
if: ${{ needs.release-please.outputs.release_created == 'true' }}
runs-on: ubuntu-latest
permissions:
contents: read
id-token: write # npm trusted publishing (OIDC) + provenance
id-token: write # npm provenance
steps:
- name: Check out workflow scripts
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
persist-credentials: false
- name: Resolve release tag
id: rel
- name: Guard on secret
id: guard
env:
DISPATCH_TAG: ${{ inputs.npm_backfill_tag }}
RELEASE_TAG: ${{ needs.release-please.outputs.tag_name }}
NPM_TOKEN: ${{ secrets.NPM_TOKEN }}
run: |
set -euo pipefail
scripts/resolve-release-tag.sh "${DISPATCH_TAG:-$RELEASE_TAG}"
if [ -z "${NPM_TOKEN:-}" ]; then
echo "::warning::NPM_TOKEN is not set — skipping npm publish."
echo "enabled=false" >> "$GITHUB_OUTPUT"
else
echo "enabled=true" >> "$GITHUB_OUTPUT"
fi
- name: Checkout release tag
if: ${{ steps.guard.outputs.enabled == 'true' }}
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
ref: ${{ steps.rel.outputs.tag }}
ref: ${{ needs.release-please.outputs.tag_name }}
persist-credentials: false
# npm >= 11.5.1 performs the OIDC token exchange automatically when the
# package has a trusted publisher configured; older npm silently falls
# back to (absent) token auth and fails confusingly.
- name: Set up Node 24 + OIDC-capable npm
uses: actions/setup-node@a0853c24544627f65ddf259abe73b1d18a591444 # v5.0.0
with:
node-version: "24"
- name: Ensure npm supports trusted publishing
run: |
set -euo pipefail
npm install -g npm@^11.5.1
npm --version
- name: Lay in release binaries
if: ${{ steps.guard.outputs.enabled == 'true' }}
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
VERSION: ${{ steps.rel.outputs.version }}
TAG: ${{ steps.rel.outputs.tag }}
VERSION: ${{ needs.release-please.outputs.version }}
TAG: ${{ needs.release-please.outputs.tag_name }}
run: |
set -euo pipefail
# npm platform package -> Rust target triple (unix only; Windows is not
@@ -605,8 +305,9 @@ jobs:
done
- name: Set versions
if: ${{ steps.guard.outputs.enabled == 'true' }}
env:
VERSION: ${{ steps.rel.outputs.version }}
VERSION: ${{ needs.release-please.outputs.version }}
run: |
set -euo pipefail
node -e '
@@ -626,12 +327,13 @@ jobs:
'
- name: Publish
if: ${{ steps.guard.outputs.enabled == 'true' }}
env:
VERSION: ${{ steps.rel.outputs.version }}
NPM_TOKEN: ${{ secrets.NPM_TOKEN }}
VERSION: ${{ needs.release-please.outputs.version }}
run: |
set -euo pipefail
# No token: npm exchanges this job's GitHub OIDC identity with the
# registry (trusted publishing); provenance is attested automatically.
echo "//registry.npmjs.org/:_authToken=${NPM_TOKEN}" > ~/.npmrc
# Platform packages first, then the wrapper (so its optionalDependencies resolve).
# Skip versions already on the registry so a rerun after a partial publish
# is idempotent instead of erroring on the first existing package.
@@ -727,15 +429,6 @@ jobs:
bin.install "git-remote-gitlawb"
end
def caveats
<<~CAVEATS
oh-my-zsh's git plugin aliases gl='git pull', which shadows this
binary in interactive shells. If \`gl\` prints "fatal: not a git
repository", run:
echo 'unalias gl 2>/dev/null' >> ~/.zshrc && source ~/.zshrc
CAVEATS
end
test do
assert_match version.to_s, shell_output("#{bin}/gl --version")
end
+1 -1
View File
@@ -1,3 +1,3 @@
{
".": "0.7.1"
".": "0.5.1"
}
-44
View File
@@ -1,44 +0,0 @@
# Contributor guide for coding agents
This file carries the rules a patch author cannot recover from the code alone, for humans and coding agents alike. For build, test, and PR basics, read `CONTRIBUTING.md` first.
## Authorization invariants (the rules PRs most often break)
Every repo-scoped endpoint must bind the caller to an authorization decision before serving or mutating anything. The `known_ungated` allowlist in `authz_guard::every_repo_scoped_handler_is_gated` is empty on main: `list_webhooks`, `list_replicas`, and `list_protected_branches` are gated (`list_webhooks` with read-visibility then owner; the other two with `authorize_repo_read`). Do not re-open those surfaces as exceptions.
- Owner-only mutations (visibility, webhooks, protected branches, merges) are gated against the repo owner. Three gate forms are in use and all are recognized: webhooks and merges call `require_repo_owner` (`crates/gitlawb-node/src/api/mod.rs`), visibility uses its module-local `require_owner`, and protected branches compare inline with `did_matches`. Use the form the surrounding module already uses.
- Not every write is owner-only, and that is by design: `star_repo`/`unstar_repo` bind to the signer (they still require repo read access), `register_replica`/`unregister_replica` bind to the replica's own DID, the bounty actions (claim, submit, approve, cancel, dispute) have their own multi-party rules, and closing a PR or issue is owner-or-author. Do not add owner gates to these.
- Content-serving reads (blobs, trees, raw files) gate through `authorize_repo_read` with the specific path being served, so a withheld subtree is denied even on an otherwise-public repo; repo-level reads (listings) pass `"/"` by the helper's own contract. The source scans below cannot check this argument's value, so review will.
- A route that reaches repo data through a global id, with no repo in its path, or through an extractor shape the scans do not recognize still needs the same gate; the scans cannot see those.
- Read-surface denials must not reveal existence: a caller who may not read a repo (or a withheld subtree) gets the same 404 as a missing repo, never a 403. (Owner-gated mutations currently return 403 after the repo lookup; do not widen that shape to read paths.)
- Two tests in module `authz_guard` (`crates/gitlawb-node/src/api/mod.rs`) scan handler sources and fail on a repo-scoped handler without a recognized gate: `every_in_scope_mutation_has_its_gate` and `every_repo_scoped_handler_is_gated`. If they fail on your handler, add the gate. The second test's `known_ungated` allowlist is empty; never add an entry to it without an open issue and review.
## Adding or changing routes
- A new gated handler needs tests proving its route-specific gate fires: exercise an unauthorized authenticated caller and an anonymous caller where applicable, assert the exact denial status each way, and assert the body leaks nothing. The non-owner tests in `crates/gitlawb-node/src/test_support.rs` show the shape for owner-gated routes.
- A new mutation handler also gets a row in `every_in_scope_mutation_has_its_gate` naming its gate type; the per-handler row is the point of that test.
- A table-driven deny suite covering every deny-bearing route is in review (#194, #195). If it is present in your checkout, add a probe row there for any new deny-bearing route.
## Verifying signatures and certificates
- Derive the verifying key from something outside the artifact you are checking: the node's own DID, a pinned or configured value, a peers lookup. A key read from a field of the object being verified (`Did::from_str(&cert.node_did)`) only proves the artifact is self-consistent, and identities here are permissionless, so anyone can mint a keypair, name it in the artifact, self-sign, and be told it is valid. A mismatch against that anchor has to fail the result, not log a warning.
- A handler whose correct answer is "valid" needs a positive case driven by a trusted, independently anchored artifact, and a rejection case for a well-formed, fully populated forged artifact. Invalid-only suites miss an implementation that rejects everything; a forged artifact that still returns valid is the regression that matters.
- The fields a signature covers are a versioned format. Adding one invalidates every artifact already signed, so give the payload a version, keep a path that verifies the older form, and add a test that verifies an artifact signed before your change.
## Removing a serving path
When a fix removes a path that used to serve content, the same PR must add a test asserting the removed path now denies. The deletion is what turns that test green; without it the path can quietly come back.
## Database migrations
Schema changes are code-defined in `MIGRATIONS` (`crates/gitlawb-node/src/db/mod.rs`). Always append a new versioned entry; never edit an entry that has merged (operators deploy from main). Test runs build the schema from scratch, so editing an applied version stays green in CI while breaking every existing deployment.
## Client behavior (`gl`, `git-remote-gitlawb`)
When a node denies a request, surface the denial to the user. Never render a denial as an empty list or a silent success; that turns an authorization boundary into invisible data loss.
## Toolchain and CI reality
- MSRV is Rust 1.91 (`rust-version` in the workspace `Cargo.toml`). The MSRV CI job runs `cargo check --workspace --all-targets` on exactly 1.91, so check that an API you rely on is stable there; the full test suite runs on stable.
- The PR workflow runs these jobs on every PR: `cargo fmt --check`, `cargo clippy --workspace --all-targets -- -D warnings`, `cargo test --workspace` (against Postgres), a release build, `cargo audit`, the MSRV check, and a Docker build. The `main` branch ruleset currently requires one approving review but does not require these checks to pass before merge; a failing job should block review approval in practice, but nothing enforces that mechanically today.
- Conventional commit titles (`feat:`, `fix:`, `docs:`, ...) are required by convention and drive releases, but no CI job checks them; a bad title fails review, not the pipeline.
-60
View File
@@ -1,65 +1,5 @@
# Changelog
## [0.7.1](https://github.com/Gitlawb/node/compare/v0.7.0...v0.7.1) (2026-08-03)
### Bug Fixes
* **ci:** make the docker dependency cache work and unbreak Dockerfile.bins ([99b2b9a](https://github.com/Gitlawb/node/commit/99b2b9a16662761a3959a2941798e6823621bda5))
* **ci:** make the docker dependency cache work and unbreak Dockerfile.bins ([a49e04d](https://github.com/Gitlawb/node/commit/a49e04ddcd23f3b666421486ae60c3c4329e2242)), closes [#265](https://github.com/Gitlawb/node/issues/265) [#266](https://github.com/Gitlawb/node/issues/266)
* **db:** pin profile DID matching with lone non-key fixture and boundary matrix ([b4016dc](https://github.com/Gitlawb/node/commit/b4016dce9d1fe450603173f9d92bd880127620fa)), closes [#298](https://github.com/Gitlawb/node/issues/298)
* **node:** add a shared owner/name slug validator ([#272](https://github.com/Gitlawb/node/issues/272)) ([51d82de](https://github.com/Gitlawb/node/commit/51d82ded57a776d9c7e2fe65895efdbf9e227dc8))
* **node:** bound the slug owner by NAME_MAX, not the DID limit ([#272](https://github.com/Gitlawb/node/issues/272)) ([27b0495](https://github.com/Gitlawb/node/commit/27b0495a6f7edeecddbe62b0f075a7e3b1ef5053))
* **node:** fail a sync row terminally when a non-directory holds the owner path ([2e664b7](https://github.com/Gitlawb/node/commit/2e664b7885036ee72ce175af95bb567b18222583))
* **node:** make a stalled sync queue visible, and fix two test claims ([9aa4c17](https://github.com/Gitlawb/node/commit/9aa4c17aefd1885d134b1334b54551366cdd8ef0))
* **node:** move the attempted_at migration to v17, clear of two open branches ([fbc5868](https://github.com/Gitlawb/node/commit/fbc5868bd16946ea184dad26abcdb0e89d68e99a))
* **node:** prove the mirror path is inside repos_dir before running git ([#272](https://github.com/Gitlawb/node/issues/272)) ([3ca9473](https://github.com/Gitlawb/node/commit/3ca94739ed7b90e4c265a0947025dc3b50e7583b))
* **node:** reject a malformed slug at the notify boundary ([#272](https://github.com/Gitlawb/node/issues/272)) ([c587a40](https://github.com/Gitlawb/node/commit/c587a40a508acea9f74c3468d42ff037397c75d0))
* **node:** reset a peer's reachability gate when an announce changes its http_url ([#270](https://github.com/Gitlawb/node/issues/270)) ([1964f88](https://github.com/Gitlawb/node/commit/1964f88378fe6942ffca5836de3a092b5b8f67b4))
* **node:** reset a peer's reachability gate when an announce changes its http_url ([#270](https://github.com/Gitlawb/node/issues/270)) ([e2560ee](https://github.com/Gitlawb/node/commit/e2560ee46c28e4e966d87867a86dc4bad4ca7a3a))
* **node:** stamp every dequeued sync row so a stuck one yields the batch ([6e64d0d](https://github.com/Gitlawb/node/commit/6e64d0d0a7fdd38ef3a56820c1808dd7c9bbd658))
* **node:** stop a peer-supplied repo slug from escaping repos_dir ([#272](https://github.com/Gitlawb/node/issues/272)) ([c70b1df](https://github.com/Gitlawb/node/commit/c70b1df42d5c3f70ec4c272886c85fcc16ed92d1))
* **node:** validate the queued slug before the worker derives a path ([#272](https://github.com/Gitlawb/node/issues/272)) ([9f00e24](https://github.com/Gitlawb/node/commit/9f00e24aec217e4b8256dd437cff4741b1478b96))
* **review:** correct two comment claims and guard two more wrong shapes ([#270](https://github.com/Gitlawb/node/issues/270)) ([811ad50](https://github.com/Gitlawb/node/commit/811ad504bc268c8e9cddac93822adcd00bfe9ff8))
* **review:** stop the dequeue overclaiming, and pin the starvation premise ([457be11](https://github.com/Gitlawb/node/commit/457be11d728a2a34775cf96be2bc5af9a58cb250))
* strict DID matching in get_profile ([acb4714](https://github.com/Gitlawb/node/commit/acb47145beec8ce7b20b7a2479f7a55ed68e2a89))
* strict DID matching in get_profile ([13b565b](https://github.com/Gitlawb/node/commit/13b565bcac09f075a7fd555a6e9acdd27de6c7bb))
* target stored did on profile update aliases ([9836c3a](https://github.com/Gitlawb/node/commit/9836c3a21bc96a2ce95cb1fd12bd84b22142787d))
## [0.7.0](https://github.com/Gitlawb/node/compare/v0.6.0...v0.7.0) (2026-07-22)
### Features
* **gl:** actually verify certificate signatures in gl cert show ([b0a2ed8](https://github.com/Gitlawb/node/commit/b0a2ed830821b5a77b0dafdcf5ce4b1ad904ec25))
* **gl:** doctor warns when a shell alias shadows the gl binary ([0765c42](https://github.com/Gitlawb/node/commit/0765c42feff47b29054b9a277881a7cd5bfd9135))
### Bug Fixes
* **gl:** address review findings — non-fatal DID check, exact-host loopback, detached HEAD, multi-URL remotes, command-aware unalias ([3fa38dc](https://github.com/Gitlawb/node/commit/3fa38dce4e435ef017f607d355266c2892161f8d))
* **gl:** anchor cert --verify to a trusted issuer; pin canonical payload form ([c681af7](https://github.com/Gitlawb/node/commit/c681af70dba3a76b11795d324f2ca0fecf7e42fd))
* **gl:** detect gitlawb remotes beyond origin in gl status ([2d88937](https://github.com/Gitlawb/node/commit/2d88937fbabf21013f28debb4d4f93270a4bb8cd))
* **gl:** doctor treats a reachable local node as configuration, not failure ([94c61f7](https://github.com/Gitlawb/node/commit/94c61f79421fa26ecc1bdf7c255e0c48e66a2048))
* **gl:** make gl init's push guidance match the repo's actual state ([cb42a48](https://github.com/Gitlawb/node/commit/cb42a4830d0d070165e2212f316d009ec716dbee))
* warn about the oh-my-zsh gl alias at install time ([e3df51d](https://github.com/Gitlawb/node/commit/e3df51d76c0275fde33954ff0af0aab95fd0cb7d))
## [0.6.0](https://github.com/Gitlawb/node/compare/v0.5.1...v0.6.0) (2026-07-22)
### Features
* **icaptcha-client:** solve the iCaptcha proof-of-work on answer ([#181](https://github.com/Gitlawb/node/issues/181)) ([c98b503](https://github.com/Gitlawb/node/commit/c98b503f90cade54ce9c588bd2373a4e3486f2fc))
### Bug Fixes
* **core:** honor wildcard capabilities in Ucan::can, matching is_attenuated_by ([#200](https://github.com/Gitlawb/node/issues/200)) ([b6877a6](https://github.com/Gitlawb/node/commit/b6877a65981dbf41c4a7838971eb5c826c7fcaf2))
* **gl:** point doctor's version check at Gitlawb/node, not the frozen Gitlawb/releases repo ([#198](https://github.com/Gitlawb/node/issues/198)) ([98d728c](https://github.com/Gitlawb/node/commit/98d728c325f68f0ee4a62e473dc484b7dd627504))
* **node:** carry full owner DID on ref-update wire event ([#144](https://github.com/Gitlawb/node/issues/144)) ([#145](https://github.com/Gitlawb/node/issues/145)) ([69a62e5](https://github.com/Gitlawb/node/commit/69a62e51c2d2f090f9cae88f4d8ce3b9017b7149))
* **node:** gate /hooks and sibling read surfaces leaking private repo metadata ([#94](https://github.com/Gitlawb/node/issues/94)) ([#113](https://github.com/Gitlawb/node/issues/113)) ([ee8d7c0](https://github.com/Gitlawb/node/commit/ee8d7c03b26a0b678cc90b8801aa340fe81da700))
* strip Cf bidi/format controls in terminal-bound error sanitizers ([#183](https://github.com/Gitlawb/node/issues/183)) ([#188](https://github.com/Gitlawb/node/issues/188)) ([b07d41e](https://github.com/Gitlawb/node/commit/b07d41ea97569d79d3f545ae5d43c373cd6f0f33))
## [0.5.1](https://github.com/Gitlawb/node/compare/v0.5.0...v0.5.1) (2026-07-10)
+2 -5
View File
@@ -31,9 +31,7 @@ crates/
├── gitlawb-core/ crypto primitives (DID, CID, HTTP sigs, UCAN, ref certs)
├── gitlawb-node/ axum HTTP server, git smart HTTP, P2P, GraphQL
├── gl/ CLI — identity, repos, MCP server, Base L2 names
├── git-remote-gitlawb/ git remote helper for gitlawb:// URLs
├── gitlawb-attest/ provenance attestations for ref-update certificates
└── icaptcha-client/ client for the iCaptcha proof-of-intelligence service
└── git-remote-gitlawb/ git remote helper for gitlawb:// URLs
docs/ Operator guides
scripts/ Build helpers
```
@@ -56,7 +54,6 @@ Smart contracts live in a separate repo: [github.com/Gitlawb/contracts](https://
3. **No breaking changes without discussion.** Open an issue first for protocol-level changes.
4. **Conventional commits.** Use `feat:`, `fix:`, `docs:`, `refactor:`, `test:`, `chore:`. Releases are automated by [release-please](https://github.com/googleapis/release-please) — your commit prefixes drive the next version bump.
5. **Format and lint.** Run `cargo fmt --all` and `cargo clippy --workspace --all-targets -- -D warnings` before submitting. CI will reject anything that fails these.
6. **Security-surface rules.** [`AGENTS.md`](AGENTS.md) carries the authorization invariants PRs most often break; it is written for coding agents and humans alike, so read it before touching endpoints, visibility, or migrations.
## What gets merged, what gets closed
@@ -108,7 +105,7 @@ cargo test -p gitlawb-node
- **TypeScript SDK** (`@gitlawb/sdk`) — client library for the HTTP API
- **Python SDK** (`gitlawb`) — for ML/agent pipeline integration
- **UCAN authorization** — add trusted issuer anchoring, capability checks, and authorization-aware revocation
- **UCAN chain validation** — complete the auth middleware
- **Filecoin storage tier** — wire up cold storage deals
- **Documentation** — guides, tutorials, API examples
- **Node operators** — run a public node and report issues
Generated
+12 -93
View File
@@ -770,23 +770,6 @@ dependencies = [
"zeroize",
]
[[package]]
name = "ark-ff"
version = "0.6.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f7a806ac6c8307b929df4645776290a50ee2aac754ad09d8bdf73391309e43af"
dependencies = [
"ark-ff-asm 0.6.0",
"ark-ff-macros 0.6.0",
"ark-serialize 0.6.0",
"ark-std 0.6.0",
"digest 0.10.7",
"educe",
"num-bigint",
"num-traits",
"zeroize",
]
[[package]]
name = "ark-ff-asm"
version = "0.3.0"
@@ -817,16 +800,6 @@ dependencies = [
"syn 2.0.117",
]
[[package]]
name = "ark-ff-asm"
version = "0.6.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1479009684adc073dff49a1025d3a7065b317a9ead25aaaca38cdc70058ba8a2"
dependencies = [
"quote",
"syn 2.0.117",
]
[[package]]
name = "ark-ff-macros"
version = "0.3.0"
@@ -865,19 +838,6 @@ dependencies = [
"syn 2.0.117",
]
[[package]]
name = "ark-ff-macros"
version = "0.6.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "4a0691ed21ef00ef89c1e9bda832eba493dda3ec2f8d892fb25b705f73f06bb8"
dependencies = [
"num-bigint",
"num-traits",
"proc-macro2",
"quote",
"syn 2.0.117",
]
[[package]]
name = "ark-serialize"
version = "0.3.0"
@@ -911,30 +871,6 @@ dependencies = [
"num-bigint",
]
[[package]]
name = "ark-serialize"
version = "0.6.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a74dd304fd536fb95d0a328e72be759209cc496a9da094c5bc56e5fea4f9e86b"
dependencies = [
"ark-serialize-derive",
"ark-std 0.6.0",
"digest 0.10.7",
"num-bigint",
"serde_with",
]
[[package]]
name = "ark-serialize-derive"
version = "0.6.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "4f153690697a2b91e5e1251ff98411ee5371500a111a0fd317a70e588eb300f9"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.117",
]
[[package]]
name = "ark-std"
version = "0.3.0"
@@ -965,16 +901,6 @@ dependencies = [
"rand 0.8.6",
]
[[package]]
name = "ark-std"
version = "0.6.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "367c9c827ed431bff6868b7aa926e05b16eb46603cc8b6e768e4a5553fa1d155"
dependencies = [
"num-traits",
"rand 0.8.6",
]
[[package]]
name = "arrayref"
version = "0.3.9"
@@ -3374,22 +3300,19 @@ dependencies = [
[[package]]
name = "git-remote-gitlawb"
version = "0.7.1"
version = "0.5.1"
dependencies = [
"anyhow",
"gitlawb-core",
"libc",
"mockito",
"reqwest",
"tempfile",
"tracing",
"tracing-subscriber",
"windows-sys 0.61.2",
]
[[package]]
name = "gitlawb-attest"
version = "0.7.1"
version = "0.5.1"
dependencies = [
"base64",
"ed25519-dalek",
@@ -3406,7 +3329,7 @@ dependencies = [
[[package]]
name = "gitlawb-core"
version = "0.7.1"
version = "0.5.1"
dependencies = [
"anyhow",
"base64",
@@ -3427,14 +3350,13 @@ dependencies = [
"sha2",
"thiserror 2.0.18",
"tokio",
"url",
"uuid",
"zeroize",
]
[[package]]
name = "gitlawb-node"
version = "0.7.1"
version = "0.5.1"
dependencies = [
"alloy",
"anyhow",
@@ -3490,11 +3412,10 @@ dependencies = [
[[package]]
name = "gl"
version = "0.7.1"
version = "0.5.1"
dependencies = [
"alloy",
"anyhow",
"base64",
"chrono",
"clap",
"dirs",
@@ -3543,9 +3464,9 @@ dependencies = [
[[package]]
name = "h2"
version = "0.4.18"
version = "0.4.13"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "839c0e8a181239723652be9062bb56ca5bf5f64011f73b623f6f4fc59086a228"
checksum = "2f44da3a8150a6703ed5d34e164b875fd14c2cdab9af1252a9a1020bde2bdc54"
dependencies = [
"atomic-waker",
"bytes",
@@ -3864,7 +3785,7 @@ dependencies = [
"libc",
"percent-encoding",
"pin-project-lite",
"socket2 0.5.10",
"socket2 0.6.3",
"tokio",
"tower-service",
"tracing",
@@ -3900,7 +3821,6 @@ version = "0.4.0"
dependencies = [
"anyhow",
"gitlawb-core",
"mockito",
"reqwest",
"serde",
"serde_json",
@@ -5432,7 +5352,7 @@ dependencies = [
"quinn-udp",
"rustc-hash",
"rustls",
"socket2 0.5.10",
"socket2 0.6.3",
"thiserror 2.0.18",
"tokio",
"tracing",
@@ -5469,7 +5389,7 @@ dependencies = [
"cfg_aliases",
"libc",
"once_cell",
"socket2 0.5.10",
"socket2 0.6.3",
"tracing",
"windows-sys 0.52.0",
]
@@ -5819,15 +5739,14 @@ dependencies = [
[[package]]
name = "ruint"
version = "1.20.0"
version = "1.17.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f5e99bff0393163bb25029a6af25d3d8d202ba5b5438a74d1bd8789f5c822970"
checksum = "c141e807189ad38a07276942c6623032d3753c8859c146104ac2e4d68865945a"
dependencies = [
"alloy-rlp",
"ark-ff 0.3.0",
"ark-ff 0.4.2",
"ark-ff 0.5.0",
"ark-ff 0.6.0",
"bytes",
"fastrlp 0.3.1",
"fastrlp 0.4.0",
-3
View File
@@ -53,9 +53,6 @@ chrono = { version = "0.4", features = ["serde"] }
uuid = { version = "1", features = ["v4"] }
# http client
reqwest = { version = "0.12", features = ["blocking", "json", "multipart", "rustls-tls"], default-features = false }
# URL parsing (what reqwest::Url re-exports, so the shared redirect predicate can
# take a parsed URL without pulling reqwest into gitlawb-core)
url = "2"
# HMAC
hmac = "0.12"
+5 -18
View File
@@ -8,33 +8,20 @@ FROM rust:1.91-bookworm AS builder
WORKDIR /build
# Cache dependencies first for faster rebuilds.
# EVERY workspace member's manifest must be copied here, not just the ones this
# image ships: cargo loads the whole workspace before it resolves anything, so a
# missing member's Cargo.toml (or a path dependency pointing at one) aborts the
# layer before a single crate is fetched.
# Cache dependencies first for faster rebuilds
COPY Cargo.toml Cargo.lock ./
COPY crates/gitlawb-core/Cargo.toml crates/gitlawb-core/
COPY crates/gitlawb-node/Cargo.toml crates/gitlawb-node/
COPY crates/gl/Cargo.toml crates/gl/
COPY crates/git-remote-gitlawb/Cargo.toml crates/git-remote-gitlawb/
COPY crates/gitlawb-attest/Cargo.toml crates/gitlawb-attest/
COPY crates/icaptcha-client/Cargo.toml crates/icaptcha-client/
# Fetch deps (this layer caches until Cargo.{toml,lock} change).
# No `|| true`: this layer is expected to succeed, and swallowing its exit code
# is what let it sit dead from the moment gl gained a path dependency on
# icaptcha-client. A failure here should stop the build, not silently turn every
# image build into a cold one.
RUN mkdir -p crates/gitlawb-core/src crates/gitlawb-node/src crates/gl/src \
crates/git-remote-gitlawb/src crates/gitlawb-attest/src crates/icaptcha-client/src && \
# Fetch deps (this layer caches until Cargo.{toml,lock} change)
RUN mkdir -p crates/gitlawb-core/src crates/gitlawb-node/src crates/gl/src crates/git-remote-gitlawb/src && \
echo 'fn main() {}' > crates/gitlawb-node/src/main.rs && \
echo 'fn main() {}' > crates/gl/src/main.rs && \
echo 'fn main() {}' > crates/git-remote-gitlawb/src/main.rs && \
echo '' > crates/gitlawb-core/src/lib.rs && \
echo '' > crates/gitlawb-attest/src/lib.rs && \
echo '' > crates/icaptcha-client/src/lib.rs && \
cargo build --release --locked -p gitlawb-node -p gl -p git-remote-gitlawb
cargo build --release -p gitlawb-node -p gl -p git-remote-gitlawb || true
# Now copy real sources and build for real.
# Force-bump mtimes so cargo's fingerprint check rebuilds — without this,
@@ -47,7 +34,7 @@ RUN find crates -name "*.rs" -exec touch {} + && \
rm -rf target/release/.fingerprint/gitlawb-node-* \
target/release/.fingerprint/gl-* \
target/release/.fingerprint/git-remote-gitlawb-* && \
cargo build --release --locked -p gitlawb-node -p gl -p git-remote-gitlawb && \
cargo build --release -p gitlawb-node -p gl -p git-remote-gitlawb && \
strip target/release/gitlawb-node target/release/gl target/release/git-remote-gitlawb
# ── Runtime stage ───────────────────────────────────────────────────────────
+2 -5
View File
@@ -3,10 +3,7 @@
# Used by scripts/build-bins.sh — not used for node deployment.
ARG TARGET=x86_64-unknown-linux-musl
# Must stay at or above the workspace `rust-version` in Cargo.toml. An unmet
# MSRV is a hard cargo error, and nothing in CI builds this file, so a stale
# base image here fails only when someone runs scripts/build-bins.sh.
FROM rust:1.91-bookworm AS builder
FROM rust:1.85-bookworm AS builder
ARG TARGET
WORKDIR /build
@@ -26,7 +23,7 @@ COPY crates/ crates/
# Configure cross-linker for aarch64
ENV CARGO_TARGET_AARCH64_UNKNOWN_LINUX_MUSL_LINKER=aarch64-linux-gnu-gcc
RUN cargo build --release --locked --target "$TARGET" -p gl -p git-remote-gitlawb
RUN cargo build --release --target "$TARGET" -p gl -p git-remote-gitlawb
FROM debian:bookworm-slim
ARG TARGET
+23 -86
View File
@@ -4,7 +4,7 @@
Gitlawb Node is the open-source node software behind the Gitlawb network. It lets anyone run a self-hosted node, publish repositories under a DID, sign writes with Ed25519 HTTP signatures, replicate git activity across peers, and move toward a resilient app-delivery network where code and build assets can be served closer to users.
Gitlawb is not trying to be only "another git host." The long-term direction is:
Gitlawb is not trying to be only “another git host.” The long-term direction is:
```txt
Decentralized GitHub
@@ -19,7 +19,7 @@ The mission is simple: once code is pushed to the network, it should not disappe
## What is in this repository?
This is a Rust workspace with six crates:
This is a Rust workspace with four crates:
| Crate | Purpose |
|---|---|
@@ -27,8 +27,6 @@ This is a Rust workspace with six crates:
| `gl` | The Gitlawb CLI for identity, repos, issues, PRs, bounties, tasks, peers, node status, MCP, and setup flows. |
| `git-remote-gitlawb` | Git remote helper for `gitlawb://` URLs, so normal `git clone`, `git fetch`, and `git push` can talk to Gitlawb nodes. |
| `gitlawb-core` | Shared primitives: Ed25519 identities, `did:key`, CIDs, RFC 9421 HTTP signatures, certificates, and UCAN tokens. |
| `gitlawb-attest` | Pluggable external provenance attestations for ref-update certificates. |
| `icaptcha-client` | Client for the iCaptcha proof-of-intelligence flow used to protect spam-prone writes. |
---
@@ -36,12 +34,12 @@ This is a Rust workspace with six crates:
Most git hosting today depends on a small number of centralized platforms. Gitlawb Nodes are designed for a different model:
- **Own your identity**: every user, agent, and node is an Ed25519 keypair represented as `did:key:z6Mk...`.
- **Signed writes by default**: write requests use RFC 9421 HTTP Signatures instead of passwords.
- **Git-native transport**: repositories are still real git repositories served over smart HTTP.
- **Agent-native workflows**: the `gl` CLI and MCP server expose repo, issue, task, PR, and UCAN flows to AI agents.
- **Peer-aware delivery**: nodes can announce, discover, gossip, and sync with each other.
- **App CDN direction**: the network can evolve from decentralized code storage into code + asset + app delivery.
- **Own your identity** — every user, agent, and node is an Ed25519 keypair represented as `did:key:z6Mk...`.
- **Signed writes by default** — write requests use RFC 9421 HTTP Signatures instead of passwords.
- **Git-native transport** — repositories are still real git repositories served over smart HTTP.
- **Agent-native workflows** — the `gl` CLI and MCP server expose repo, issue, task, PR, and UCAN flows to AI agents.
- **Peer-aware delivery** — nodes can announce, discover, gossip, and sync with each other.
- **App CDN direction** — the network can evolve from decentralized code storage into code + asset + app delivery.
---
@@ -56,7 +54,6 @@ Good today:
- Bare git repository storage.
- Git smart-HTTP clone/fetch/push.
- RFC 9421-signed writes.
- Repository and path-scoped visibility enforcement for repository and Git content reads, with 404-shaped repository denials.
- DID identities.
- `gl` CLI workflows.
- libp2p peer discovery/gossip foundation.
@@ -66,13 +63,11 @@ Good today:
Known limitations:
- Repository write authorization is not secure by default: `GITLAWB_ENFORCE_OWNER_PUSH` defaults to `false` for compatibility, so a valid HTTP Signature identifies a pusher but does not enforce owner-only pushes.
- UCAN proof chains are validated when supplied, but UCAN capabilities are not consulted by write authorization and the root issuer is not independently trust-anchored. UCANs therefore do not yet grant scoped collaborator access.
- Agent lifecycle revocation is not enforced by HTTP Signature authorization; do not rely on removing or revoking an agent record to block a compromised signer.
- Read visibility is not a blanket data-classification boundary: task, IPFS-pin, and Arweave-anchor listings are not repository-gated; withheld path names can be visible to a root reader; and later visibility changes cannot retract content already announced or externally anchored.
- Private repository read enforcement is not wired yet. Treat public nodes as public infrastructure unless you restrict access at your proxy/firewall.
- UCAN chain validation and revocation are not complete.
- Repository write authorization is not capability-complete yet; HTTP signatures prove identity, not full authorization policy.
- Peer writes are signed by upgraded nodes, but strict signed-peer enforcement is opt-in during rolling upgrades.
- Current GraphQL mutations require an authenticated signer, but there is no mutation-specific guardrail that prevents a future mutation from omitting that check.
- Pull-request review comments do not yet have threaded line-level anchors, and merges do not enforce approval requirements.
- GraphQL mutations need mutation-aware auth before becoming a public write surface.
See:
@@ -194,14 +189,14 @@ For public-network use, make sure `GITLAWB_NODE` points to the node you want. Th
Public nodes (e.g. `node.gitlawb.com`) require two things on writes:
1. **RFC 9421 HTTP Signatures**: every write is signed by your identity key. `gl`
1. **RFC 9421 HTTP Signatures** — every write is signed by your identity key. `gl`
and the `git-remote-gitlawb` helper do this automatically. An old/unsigned CLI
fails with `401 not_an_agent`; `gl` will tell you to upgrade and register.
2. **An iCaptcha proof** on the spam-gated writes (**repo create, fork, register**).
`gl` solves this for you: on the node's `403 icaptcha_proof_required` it reads the
`x-icaptcha-url` / `x-icaptcha-level` hints, requests a challenge, solves it
locally (arithmetic / algebra / sequence), and **retries the same signed request**
with the `x-icaptcha-proof` header. No manual steps, no env vars.
with the `x-icaptcha-proof` header — no manual steps, no env vars.
```bash
gl identity new # create did:key identity
@@ -220,61 +215,14 @@ Notes:
- **Proofs are short-lived (~5 min TTL) and single-use.** If one expires between
solving and use, the client transparently solves a fresh one and retries.
- **What needs what:** create / fork / register are signed **and** iCaptcha-gated;
`git push` is **signed-only** (owner signature is the gate, no per-push challenge);
`git push` is **signed-only** (owner signature is the gate — no per-push challenge);
reads (clone / fetch / `repo info`) need no proof. A non-existent repo returns a
clear `404`, never a placeholder.
- **API-key iCaptcha deployments:** set `GITLAWB_ICAPTCHA_URL` to your iCaptcha
origin and `GITLAWB_ICAPTCHA_API_KEY` to its key. The client only talks to an
`https` origin whose host is allowlisted (that URL or the public default), and
sends the bearer token **only** to your configured origin, never to a URL a
node advertises, so a hostile node can't capture the key or redirect the solve.
### Fetching an object by CID
```bash
gl ipfs list # CIDs this node has pinned
gl ipfs get bafkrei... > object.bin # object bytes on stdout
```
Objects that were pinned before the node started recording which repo they came
from are found by scanning its repo inventory, and that scan stops at the
per-request ceilings in the [Configuration](#configuration) table. A stopped scan
answers 503 with a resume token instead of a false "not found", and `gl ipfs get`
follows the token automatically: up to 8 resumes after the first request, so at
most 9 calls to the node, waiting between attempts for as long as the node's
`Retry-After` asks and never longer than 5 seconds.
The whole ladder runs under a 60 second wall-clock deadline. The deadline bounds
the search, not the download: each attempt gets the time left on it to produce
response headers, and once an object is found its bytes stream outside that
deadline. They are not unbounded, though. The client's own 30 second HTTP timeout
is a total request timeout, running from the start of a request until its body
has finished, so a transfer still going 30 seconds after its request began is cut
off. Waits between attempts never run past the deadline either, so a single run
spends at most around 90 seconds on the network: the deadline plus the 30 second
timeout covering the last attempt. Writing the object out sits outside both
bounds, so piping into a reader that stops reading can hold the command open
longer than that.
Two node-side brakes end a ladder early and are reported rather than retried
around. A 429 is terminal, because the node's rate-limit window is an hour and
that wait cannot be honored inside one invocation; a transient overload (a 503
carrying no incomplete-scan code) is retried on the token already held, under the
same cap, clamp and deadline. The per-IP fanout brake can also stop a ladder well
short of the 9 calls, so automatic resumption is not a guarantee of reaching the
object.
When a bound stops the ladder with a usable token in hand, the command prints the
token and the invocation that continues from it before exiting nonzero:
```txt
resume from where this stopped: gl ipfs get bafkrei... --scan <token>
```
Run that to carry on from where the scan stopped. Re-running without `--scan`
restarts at the first row, reproduces the same truncation and spends the node's
per-IP budget again, so the token is the only thing that makes progress. Tokens
are valid for an hour.
sends the bearer token **only** to your configured origin — never to a URL a
node advertises — so a hostile node can't capture the key or redirect the solve.
---
@@ -361,7 +309,7 @@ POST /api/v1/peers/announce
POST /api/v1/sync/notify
```
When `GITLAWB_REQUIRE_SIGNED_PEER_WRITES=false`, unsigned legacy peers are accepted on those two routes, but signed requests are verified when signature headers are present. Staged rollout relaxes who may announce, not who owns a peer row. An unsigned announce may register a previously unseen peer, and it may refresh an existing row whose `http_url` is unchanged, but changing an existing peer's `http_url` requires an RFC 9421 signature from that peer's own DID and is refused with 403 otherwise. An unsigned announce can only register a `did:key`, since that is the only method whose verifying key can be resolved and therefore the only kind of row anyone could ever correct through the signed path; any other DID is refused with 400. Once all live peers upgrade, operators can set:
When `GITLAWB_REQUIRE_SIGNED_PEER_WRITES=false`, unsigned legacy peers are accepted on those two routes, but signed requests are verified when signature headers are present. Once all live peers upgrade, operators can set:
```bash
GITLAWB_REQUIRE_SIGNED_PEER_WRITES=true
@@ -392,25 +340,14 @@ Important node settings:
| `GITLAWB_BOOTSTRAP_PEERS` | Comma-separated HTTP peer URLs. |
| `GITLAWB_P2P_BOOTSTRAP` | Comma-separated libp2p multiaddrs. |
| `GITLAWB_BOOTSTRAP_DISABLE_SEEDS` | Disable embedded seed peers for isolated dev/test networks. |
| `GITLAWB_REQUIRE_SIGNED_PEER_WRITES` | Require signed peer announce/sync writes. Defaults to `false` during the staged rollout below. |
| `GITLAWB_ENFORCE_OWNER_PUSH` | Require the authenticated pusher to be the repo owner on `git-receive-pack`. **Defaults to `true`.** A `did:key` signature is authentication, not authorization — anyone can mint a key and sign — so with this off every signed caller may push to every repository, private ones included. Delegated and CI keys count as non-owners: a UCAN `git/push` capability is verified but not yet honored for authorization, so they cannot push while this is on. Set `false` only for a rolling upgrade; see [`docs/RUN-A-NODE.md`](docs/RUN-A-NODE.md). |
| `GITLAWB_REQUIRE_SIGNED_PEER_WRITES` | Require signed peer announce/sync writes. |
| `GITLAWB_AUTO_SYNC` | Enable automatic sync from known peers. |
| `GITLAWB_MAX_PACK_BYTES` | Max git pack body size for smart-HTTP routes. |
| `GITLAWB_GIT_SERVICE_TIMEOUT_SECS` | Max seconds a served git upload-pack, receive-pack, or `info/refs` advertisement may run before it is aborted (504). Default 600. Also bounds the withheld-blob classification walk (on both the upload-pack serve and receive-pack replication paths) and the push-side pin-candidate discovery (`rev-list` / `cat-file`), each reaped via process-group teardown at the deadline. On the path-scoped upload-pack path the classification walk and the pack serve share ONE deadline, so this value bounds their combined duration rather than granting each stage a full budget: a walk that consumes it leaves the serve nothing and the clone gets a 504. Serving large path-scoped repos may therefore need a higher value than they did when each stage was budgeted separately. Accepted range is 1 to 3153600000 (100 years), since the node derives deadlines from this value and a larger one cannot be represented. |
| `GITLAWB_GIT_SERVICE_TIMEOUT_SECS` | Max seconds a served git upload-pack, receive-pack, or `info/refs` advertisement may run before it is aborted (504). Default 600. Also bounds the withheld-blob classification walk (on both the upload-pack serve and receive-pack replication paths) and the push-side pin-candidate discovery (`rev-list` / `cat-file`), each reaped via process-group teardown at the deadline. |
| `GITLAWB_GIT_ACQUIRE_TIMEOUT_SECS` | Max seconds the storage-acquisition phase (Tigris HEAD/GET, push advisory-lock) of a served git op may run before the request is shed with a 503, separate from the git-run timeout. The concurrency permit is released on expiry so a stalled backend cannot pin the pool. Default 30. |
| `GITLAWB_MAX_CONCURRENT_GIT_OPS` | Max concurrent served git READ ops (upload-pack and its `info/refs` advertisement) across all callers; over-cap sheds a 503 + Retry-After. Anonymous reads draw from this pool, so pair it with `GITLAWB_MAX_CONCURRENT_READS_PER_CALLER`. Pushes and the receive-pack advertisement have their own pools, so a read flood cannot shed an authenticated push. Default 128. |
| `GITLAWB_MAX_CONCURRENT_GIT_PUSHES` | Max concurrent `git-receive-pack` POST operations, in a pool separate from the read pool. The anon receive-pack `info/refs` advertisement runs in a third pool of the same size, disjoint from both, so an advertisement flood cannot shed a push either. Two per-source push caps are derived from this value (`/8`, floor 1) and have no env var of their own. Over-cap sheds a 503 + Retry-After. Default 32. |
| `GITLAWB_MAX_CONCURRENT_READS_PER_CALLER` | Max concurrent read ops a single caller may hold, so one caller cannot monopolize the read pool. Keyed on the resolved source IP, never the DID, and only as granular as `GITLAWB_TRUSTED_PROXY`: left unset, a node behind an edge or NAT keys every caller on the edge IP and this collapses to one global cap. Default 16. |
| `GITLAWB_MAX_CONCURRENT_PIN_TASKS` | Max post-push pin loops (IPFS + Pinata) running concurrently across all repos. This caps how many loops RUN at once, not how much object-id list memory the node retains: on the local IPFS path a loop parked waiting for a permit still holds its full list. Do not size memory from this knob alone. A loop over cap waits, never drops a pin. Default 8. |
| `GITLAWB_REPO_LEASE_MAX_WAITERS` | Max pushes parked at once waiting for the same repo's write lease. Each waiter pins its buffered pack body, so this bounds that memory for a hot repo; past the cap the newest push sheds a 503 + Retry-After instead of queueing. Pushes to other repos are unaffected, and the lease holder is not counted. Default 8. |
| `GITLAWB_MAX_CONCURRENT_IPFS_WALKS` | Max concurrent `GET /ipfs/{cid}` visibility walks across all callers (own pool, disjoint from the served-git pools); over-cap sheds 503. Default 32. |
| `GITLAWB_IPFS_WALK_PER_SOURCE` | Max concurrent `/ipfs` walks a single source IP may hold. Default 4. |
| `GITLAWB_IPFS_MAX_LEGACY_PROBES` | Max legacy (NULL-provenance) repos probed per `/ipfs/{cid}` request, bounding the scan-fallback fan-out. A truncated scan returns a retryable 503, not a false 404. Default 256. |
| `GITLAWB_IPFS_MAX_LEGACY_SCAN_ROWS` | Max repo rows one `/ipfs/{cid}` request's legacy scan may read from the database. The probe ceiling above only starts counting once a probe runs, and quarantined or private repos are denied before that, so this is what bounds a scan over an inventory that denies the caller everywhere. A truncated scan sheds a retryable 503 carrying an opaque `continuation` token; echoing it as `?scan=` resumes the scan where it stopped. Every per-request ceiling on this path (rows, probes, visits, retained rule bytes) mints one, so each request advances the ladder by at least the rows it read and a holder buried past a ceiling is reached in a bounded number of requests, `ceil(repos / ceiling) + 1` when this row ceiling is the one that binds. No ceiling ever produces a 404. Every page is charged to the caller's `/ipfs` work allowance, so raising this raises that allowance too. Lowering it sharpens an oracle: because a truncation emits a token and a completed wrap does not, laddering to the end reveals the node's total repo count (private and quarantined included) to within one ceiling. Default 2048. |
| `GITLAWB_IPFS_MAX_REPOS_WALKED` | Max expensive path-scope visibility walks a `/ipfs/{cid}` request may run per phase; over-cap repos are skipped and the scan continues, shedding a retryable 503 (not a false 404) if the object is then found nowhere. The effective cap is the tighter of this knob and the node's internal history-walk ceiling of 17 (`MAX_PIN_SOURCES + 1`), so a value above 17 has no effect while a value below it does tighten the cap. It is charged per phase: the provenance lookup and the legacy-scan fallback get separate equal budgets, so one request can run up to twice the cap in total. Default 64. |
| `GITLAWB_IPFS_MAX_REPO_VISITS` | Ceiling on repos one `/ipfs/{cid}` request may visit (acquire + probe) past the visibility gate. Also the worst-case per-request Tigris fetch count. On exhaustion the scan stops with a retryable 503. Default 1024. |
| `GITLAWB_IPFS_REQUEST_BUDGET_SECS` | Absolute wall-clock budget for one admitted `/ipfs/{cid}` request's acquire+walk lifetime. Per-stage clamps bound the acquire and walk stages to the remaining budget, and no stage starts once it is exhausted; the scan then stops with a retryable 503. The object-type probe and content-read `cat-file` subprocesses are budget-checked before starting and each also run under their own deadline (the lesser of `GITLAWB_GIT_SERVICE_TIMEOUT_SECS` and the remaining budget), reaped via process-group teardown, so a hung `cat-file` cannot hold the request's walk slot past it. One hang path is still unbounded: the probe's object-store readability check is a plain filesystem sweep with nothing to reap, so a wedged filesystem can hold the slot past the deadline. Default 600. Accepted range is 1 to 3153600000 (100 years), since the node derives a deadline from this value and a larger one cannot be represented. |
| `GITLAWB_IPFS_RESOLVE_BUDGET_SECS` | Shorter budget for the pre-walk CID resolve inside an admitted `/ipfs/{cid}` request: the lookup that maps the requested CID to its git oid(s), which runs while the scarce walk admission is already held. A well-formed CID with no pin row does no probe and no walk work, so without this it could hold a walk slot for the whole request budget while nothing walked, and enough such requests shed every real retrieval at admission. The effective deadline is the lesser of this and the remaining request budget, so a value above `GITLAWB_IPFS_REQUEST_BUDGET_SECS` degrades to the request budget. Only the resolve is on this clock; walk and probe work stay on the request budget, so a slow but progressing scan is never shed by it. Default 10. Accepted range is 1 to 3153600000 (100 years). |
| `GITLAWB_IPFS_MAX_REPOS_WALKED` | Max legacy (NULL-provenance) repos probed per `/ipfs/{cid}` request, bounding the scan-fallback fan-out. A truncated scan returns a retryable 503, not a false 404. Default 256. |
| `GITLAWB_IPFS_RATE_LIMIT` | Max `/ipfs/{cid}` requests per client IP per hour (route flood brake). 0 disables. Default 600. |
| `GITLAWB_TIGRIS_BUCKET` | Optional S3/Tigris shared repo storage bucket. |
| `GITLAWB_PINATA_JWT` | Optional Pinata/IPFS warm-storage pinning. |
@@ -524,8 +461,8 @@ Short-term priorities:
2. Add Docker and installer smoke tests.
3. Improve operator docs and `gl doctor` checks.
4. Harden peer writes and publish the signed-peer rollout plan.
5. Close default-open write authorization and wire trusted UCAN delegation into repository permissions.
6. Add threaded, line-level pull-request discussions and enforce approval requirements on merges.
5. Implement repo write authorization: owner checks, protected branches, and UCAN capability checks.
6. Implement private-read enforcement or remove private repo affordances until it exists.
7. Add metrics for pushes, fetches, pack sizes, peer sync, failed auth, and webhooks.
Product direction:
+22 -32
View File
@@ -28,11 +28,10 @@ We will acknowledge receipt within 48 hours and aim to release a fix within 14 d
- Every git object is content-addressed via CIDv1 (SHA-256)
- Tamper-evident by construction — a modified object changes its CID
**UCAN token validation**
- Bootstrap UCAN tokens are issued at registration.
- A supplied token's signature, audience, expiry, and proof-chain attenuation are validated.
- Tokens use a signed JSON wire format with expiry.
- Capability grants are not yet consulted by repository write authorization; see the limitations below.
**UCAN capability tokens**
- Bootstrap UCAN tokens issued at registration
- Capability-scoped: `git:push`, `git:fetch`, `issue:create`, `pr:open`
- JWT-format tokens with expiry
**Smart contracts (Base Sepolia testnet)**
- `GitlawbDIDRegistry` — on-chain DID → document registry
@@ -49,20 +48,21 @@ We will acknowledge receipt within 48 hours and aim to release a fix within 14 d
---
## Known Limitations
## Known Limitations (Planned for v0.2)
These are documented limitations of the current live release. They should be prioritized without breaking existing nodes during rolling upgrades.
These are **documented, accepted limitations** for the current live release and should be prioritized without breaking existing nodes during rolling upgrades.
### Repository write authorization defaults
- `git-receive-pack` verifies HTTP Signatures, but `GITLAWB_ENFORCE_OWNER_PUSH` defaults to `false` for compatibility during rollout.
- **Impact:** With the default setting, a valid signature authenticates the pusher but does not require that DID to be the repository owner.
- **Mitigation:** Set `GITLAWB_ENFORCE_OWNER_PUSH=true` on nodes where owner-only pushes are required. Confirm that every legitimate pusher uses the owner DID before enabling it.
### UCAN chain validation
- The auth middleware verifies HTTP Signatures and token structure, but does not yet walk the full UCAN delegation chain.
- **Impact:** A node cannot yet enforce fine-grained capability delegation. Currently, any registered agent with a valid HTTP Signature can push.
- **Mitigation:** Keep write endpoints signed, treat public nodes as public infrastructure, and treat trust scores as soft rate-limiting signals rather than authorization.
- **Fix target:** v0.2
### UCAN delegation and revocation
- The middleware validates a supplied UCAN's complete proof chain, but a root token is accepted without an independently trusted issuer anchor. `Ucan::can` is not yet used by write handlers, so a UCAN does not grant scoped repository access.
- Agent lifecycle revocation is not checked by HTTP Signature authorization. Removing or revoking an agent record does not itself block a compromised DID from authenticating.
- **Impact:** Do not use UCANs for collaborator permissions or rely on agent-record revocation as a key-compromise response.
- **Mitigation:** Keep sensitive deployments behind operational network controls and enable owner-only push enforcement where it fits the deployment until trusted delegation and authorization-aware revocation are implemented.
### UCAN revocation
- Issued UCAN tokens cannot be revoked before expiry.
- **Impact:** If a keypair is compromised, the attacker retains access until the UCAN expires (default: 30 days).
- **Mitigation:** Regenerate your identity (`gl identity new --force`) and re-register to issue a new UCAN. Until revocation/blocklisting is implemented, operators should remove compromised DIDs directly from their local database.
- **Fix target:** v0.2
### git-receive-pack authentication
- The `git-receive-pack` endpoint enforces HTTP Signature auth. Plain Git smart-HTTP clients do not generate those headers, so the `git-remote-gitlawb` helper is required for pushes.
@@ -71,20 +71,10 @@ These are documented limitations of the current live release. They should be pri
- **Fix target:** v0.2
### Private repository reads
- Repository and path-scoped visibility checks are enforced for repository API and Git content reads. A denied whole-repository or root read returns the same 404 shape as a missing repository, so the denial does not reveal private-repository existence.
- Sparse-clone support exposes withheld path globs to callers who may read the repository root. Do not put sensitive information in withheld path names.
- `GET /api/v1/tasks`, `/api/v1/ipfs/pins`, and `/api/v1/arweave/anchors` are not repository-gated. Task records include a UCAN token; pin and anchor listings expose object and ref metadata.
- Changing a repository's visibility controls future serving, but cannot retract ref metadata or configured external pins and anchors already announced while the repository was public. Do not push secrets to an announceable repository.
- **Impact:** Visibility policies protect the repository and Git content routes they gate, not every metadata endpoint or previously published content.
- **Remaining boundary:** This read control does not address the independent write-authorization and UCAN-delegation limitations described above.
### Pull-request review enforcement
- Pull-request review comments are not yet threaded or line-anchored, and merges do not enforce required approvals.
- **Impact:** Teams must use their own review policy or external controls for merge approval requirements.
### GraphQL mutation coverage
- Existing GraphQL mutations require an authenticated signer, but a mutation-specific source-level guardrail has not yet been added for future mutations.
- **Impact:** A new mutation could accidentally omit its signer check without an explicit test fence.
- Repository records have an `is_public` field and the node exposes `GITLAWB_PUBLIC_READ`, but per-repository private-read enforcement is not wired in the current live release.
- **Impact:** Do not store private repositories or secrets on public nodes.
- **Mitigation:** Run isolated nodes for non-public data and restrict network access at the reverse proxy or firewall layer.
- **Fix target:** v0.2
### Peer route hardening rollout
- Peer announce and sync notification routes accept signed requests and verify DID matches when a signature is present.
@@ -111,7 +101,7 @@ These are documented limitations of the current live release. They should be pri
| Key storage | PKCS#8 PEM, 0600 permissions |
| Content hashing | SHA-256 via CIDv1 |
| HTTP Signatures | RFC 9421 (Ed25519 + SHA-256 Content-Digest) |
| UCAN tokens | Signed JSON object (Ed25519 signature) |
| UCAN tokens | JWT (Ed25519 signatures) |
| On-chain | ECDSA secp256k1 (Base L2 / Ethereum) |
---
@@ -119,7 +109,7 @@ These are documented limitations of the current live release. They should be pri
## Threat Model
gitlawb is designed to be secure against:
- **Unauthenticated writes** — HTTP Signature auth on protected write endpoints; see Known Limitations for owner and capability authorization gaps
- **Unauthorized writes** — HTTP Signature auth on all write endpoints
- **Tampered git objects** — CIDv1 content addressing detects modification
- **Identity spoofing** — DIDs derived from public keys, unforgeable without the private key
- **Centralized takedown** — no single point of control; data on IPFS + Arweave
-112
View File
@@ -1,112 +0,0 @@
# Dependency allowlist for the `gitlawb-core` crate.
#
# gitlawb-core is the shared-primitives crate every consumer embeds (the `gl`
# CLI, git-remote-gitlawb, the node daemon). It must stay lean so those
# consumers do not inherit daemon-weight dependencies. This file is the source
# of truth for the crates gitlawb-core's NORMAL (non-dev, non-build) transitive
# dependency closure is allowed to contain. CI (`scripts/check-gitlawb-core-deps.sh`,
# wired into the `core-deps-purity` job in .github/workflows/pr-checks.yml)
# fails if gitlawb-core's tree contains any crate not listed here.
#
# The allowlist is exhaustive-by-construction: a NEW dependency reds CI whether
# or not anyone thought to ban it, which a denylist cannot do.
#
# Measured in-workspace, so feature unification across the workspace can, in
# rare cases, surface a crate here that gitlawb-core would not pull standalone.
# An unexpected new entry is a prompt to check what pulled it in, not noise to
# rubber-stamp.
#
# Regenerate (from repo root, after an INTENTIONAL dependency change):
# cargo tree -p gitlawb-core --edges normal --prefix none \
# | sed -E 's/ v[0-9].*$//' \
# | grep -v '^gitlawb-core$' \
# | sort -u
# then paste the result below this header block. (The checker snapshots and
# restores Cargo.lock; a manual regen may refresh it — `git checkout Cargo.lock`
# afterward if you did not intend that.)
#
# Lines starting with `#` and blank lines are ignored by the checker.
aead
anyhow
base-x
base256emoji
base64
base64ct
block-buffer
cfg-if
chacha20
chacha20poly1305
chrono
cid
cipher
const-oid
const-str
core2
cpufeatures
crypto-common
crypto_box
crypto_secretbox
curve25519-dalek
curve25519-dalek-derive
data-encoding
data-encoding-macro
data-encoding-macro-internal
der
digest
ed25519
ed25519-dalek
equivalent
generic-array
getrandom
hashbrown
hex
iana-time-zone
indexmap
inout
itoa
libc
match-lookup
memchr
multibase
multihash
multihash-codetable
multihash-derive
multihash-derive-impl
num-traits
opaque-debug
pem-rfc7468
pkcs8
poly1305
ppv-lite86
proc-macro-crate
proc-macro2
quote
rand
rand_chacha
rand_core
salsa20
serde
serde_core
serde_derive
serde_json
sha2
signature
spki
subtle
syn
synstructure
thiserror
thiserror-impl
toml_datetime
toml_edit
toml_parser
typenum
unicode-ident
universal-hash
unsigned-varint
uuid
winnow
zerocopy
zeroize
zeroize_derive
zmij
+2 -19
View File
@@ -1,7 +1,7 @@
[package]
name = "git-remote-gitlawb"
description = "Git remote helper — enables 'git clone gitlawb://did:gitlawb:...'"
version = "0.7.1" # x-release-please-version
version = "0.5.1" # x-release-please-version
edition.workspace = true
rust-version.workspace = true
license.workspace = true
@@ -11,7 +11,7 @@ name = "git-remote-gitlawb"
path = "src/main.rs"
[dependencies]
gitlawb-core = { path = "../gitlawb-core", features = ["redirect"] }
gitlawb-core = { path = "../gitlawb-core" }
anyhow = { workspace = true }
reqwest = { workspace = true }
tracing = { workspace = true }
@@ -19,20 +19,3 @@ tracing-subscriber = { workspace = true }
[dev-dependencies]
mockito = "1"
tempfile = "3"
[target.'cfg(unix)'.dev-dependencies]
libc = "0.2"
[target.'cfg(windows)'.dev-dependencies]
# Job Object bindings for the real_git_fetch harness: the Windows analog of the
# unix process-group teardown, so the whole `git fetch` tree can be killed
# together on both the timeout and clean-exit paths (INV-22). Win32_Security is
# required by CreateJobObjectW, and Win32_System_Threading by
# JOBOBJECT_EXTENDED_LIMIT_INFORMATION (it embeds IO_COUNTERS).
windows-sys = { version = "0.61", features = [
"Win32_Foundation",
"Win32_Security",
"Win32_System_JobObjects",
"Win32_System_Threading",
] }
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
+1 -1
View File
@@ -1,7 +1,7 @@
[package]
name = "gitlawb-attest"
description = "External Attestation v1: pluggable provenance attachments for gitlawb ref-update certs"
version = "0.7.1" # x-release-please-version
version = "0.5.1" # x-release-please-version
edition.workspace = true
rust-version.workspace = true
license.workspace = true
+2 -37
View File
@@ -16,7 +16,7 @@
//! by exact match.
use base64::{engine::general_purpose::URL_SAFE_NO_PAD as B64U, Engine};
use ed25519_dalek::{Signature, Signer, SigningKey, VerifyingKey};
use ed25519_dalek::{Signature, Signer, SigningKey, Verifier, VerifyingKey};
use serde::{de::DeserializeOwned, Deserialize, Serialize};
use crate::error::{Error, Result};
@@ -111,7 +111,7 @@ impl Attestation {
.try_into()
.map_err(|_| Error::Signature("signature must be 64 bytes".to_string()))?;
let sig = Signature::from_bytes(&sig_bytes);
vk.verify_strict(&bytes, &sig)
vk.verify(&bytes, &sig)
.map_err(|e| Error::Signature(format!("ed25519: {e}")))?;
Ok(vk)
@@ -554,39 +554,4 @@ mod tests {
let err = tampered.verify_signature(cert_hash).unwrap_err();
assert!(matches!(err, Error::Signature(_)));
}
/// Regression guard for strict verification: a signature forged under a
/// weak (small-order) public key satisfies the verification equation
/// but must be rejected. The identity point is such a key: with R the
/// identity and S = 0, [S]B - [k]A is the identity for any message.
#[test]
fn verify_rejects_weak_key_signature() {
let mut weak_key_bytes = [0u8; 32];
weak_key_bytes[0] = 1; // compressed identity point (y = 1, x = 0)
let weak_vk = VerifyingKey::from_bytes(&weak_key_bytes).unwrap();
assert!(weak_vk.is_weak(), "identity point must be a weak key");
let cert_hash = sample_cert_hash();
let mut forged = [0u8; 64];
forged[0] = 1; // R = identity point, S = 0
let forged_sig = B64U.encode(forged);
// Build an attestation with the weak key as signer and the forged
// signature. The weak-key check happens at verify time.
let mut att = dummy_attestation(&SigningKey::generate(&mut OsRng), cert_hash);
let mut buf = Vec::with_capacity(ED25519_MULTICODEC.len() + 32);
buf.extend_from_slice(&ED25519_MULTICODEC);
buf.extend_from_slice(&weak_key_bytes);
att.signer = format!(
"did:key:{}",
multibase::encode(multibase::Base::Base58Btc, &buf)
);
att.sig = forged_sig;
let err = att.verify_signature(cert_hash).unwrap_err();
assert!(
matches!(err, Error::Signature(_)),
"signature under a weak (small-order) public key must be rejected"
);
}
}
+1 -8
View File
@@ -1,7 +1,7 @@
[package]
name = "gitlawb-core"
description = "Core cryptographic primitives for the gitlawb network: DIDs, CIDs, UCAN, HTTP Signatures, ref-update certificates"
version = "0.7.1" # x-release-please-version
version = "0.5.1" # x-release-please-version
edition.workspace = true
rust-version.workspace = true
license.workspace = true
@@ -22,7 +22,6 @@ multihash-codetable = { workspace = true }
cid = { workspace = true }
chrono = { workspace = true }
uuid = { workspace = true }
url = { workspace = true, optional = true }
zeroize = { version = "1", features = ["derive"] }
pkcs8 = { version = "0.10", features = ["pem", "std"] }
curve25519-dalek = "4"
@@ -31,9 +30,3 @@ chacha20poly1305 = "0.10"
[dev-dependencies]
tokio = { workspace = true }
# Not optional here: the redirect matrix must run under a bare
# `cargo test -p gitlawb-core`, with no feature selected.
url = { workspace = true }
[features]
redirect = ["dep:url"]
+2 -35
View File
@@ -6,8 +6,6 @@
//! The schema is frozen at v1. All fields are mandatory for forward compatibility.
//! Nodes that receive a certificate with an unknown version MUST reject it.
use std::collections::HashSet;
use chrono::{DateTime, Utc};
use serde::{Deserialize, Serialize};
use uuid::Uuid;
@@ -137,14 +135,10 @@ impl RefUpdateCert {
/// Check if this certificate satisfies a threshold of valid signatures
/// from the provided set of authorized maintainer DIDs.
///
/// Counts distinct signer DIDs, not signature entries: a repeated
/// signature from the same maintainer counts once.
pub fn satisfies_threshold(&self, maintainers: &[Did], threshold: usize) -> Result<bool> {
let valid = self.verify_all()?;
let distinct_signers: HashSet<&Did> =
valid.iter().filter(|d| maintainers.contains(d)).collect();
Ok(distinct_signers.len() >= threshold)
let count = valid.iter().filter(|d| maintainers.contains(d)).count();
Ok(count >= threshold)
}
/// Validate the certificate structure (not signatures).
@@ -350,33 +344,6 @@ mod tests {
assert!(!cert.satisfies_threshold(&maintainers, 1).unwrap());
}
#[test]
fn satisfies_threshold_rejects_duplicated_signature() {
let kp1 = Keypair::generate();
let kp2 = Keypair::generate();
let kp3 = Keypair::generate();
let repo_did = kp1.did();
let mut cert = RefUpdateCert::new(
repo_did,
"refs/heads/main".to_string(),
dummy_hash('0'),
dummy_hash('a'),
1,
&kp1,
)
.unwrap();
// Copy-paste the only real signature onto the certificate a second
// time. Same signer, same valid signature, still one real signer.
let dup = cert.signatures[0].clone();
cert.signatures.push(dup);
let maintainers = vec![kp1.did(), kp2.did(), kp3.did()];
// Two signature entries but a single distinct signer must not
// satisfy a 2-of-3 threshold.
assert!(!cert.satisfies_threshold(&maintainers, 2).unwrap());
}
#[test]
fn threshold_zero_is_always_satisfied() {
let kp = Keypair::generate();
-61
View File
@@ -80,19 +80,6 @@ impl Did {
)));
}
// Refuse an oversized id before decoding. base58 decoding is quadratic
// in its input, and both callers of this function take the string from
// an untrusted request: the unauthenticated peer-announce gate, and the
// HTTP-signature keyid. An ed25519 did:key method-id is a fixed 48
// characters, so this bound is slack rather than a behavior change, and
// it has to sit ahead of the decode to be worth anything.
const MAX_METHOD_ID_LEN: usize = 64;
if self.method_id().len() > MAX_METHOD_ID_LEN {
return Err(Error::InvalidDid(
"did:key method-specific id too long".to_string(),
));
}
let (_, bytes) =
multibase::decode(self.method_id()).map_err(|e| Error::InvalidDid(e.to_string()))?;
@@ -321,52 +308,4 @@ mod tests {
assert_eq!(Did::web("example.com").method_id(), "example.com");
assert_eq!(Did::gitlawb("z6MkSomeKey").method_id(), "z6MkSomeKey");
}
/// An oversized method-specific id is refused on its length, before the
/// base58 decode runs.
///
/// base58 decoding is quadratic in the input, and `to_verifying_key` is
/// reached from two places that take an attacker-supplied string: the
/// unauthenticated peer-announce gate, and the HTTP-signature keyid. A
/// 64k-character DID measured 8s through the router without this guard
/// against 4.6ms with it, on a tokio worker with no spawn_blocking, so a
/// single caller could buy CPU-seconds by the thousand.
///
/// Asserting the MESSAGE is what makes this load-bearing: an oversized id
/// errors either way, so a test that only checked `is_err()` would pass
/// with the guard removed, having merely paid the cost it exists to avoid.
#[test]
fn to_verifying_key_rejects_an_oversized_method_id_before_decoding() {
let oversized = format!("did:key:z{}", "z".repeat(64_000));
let did: Did = oversized.parse().expect("method is still `key`");
let start = std::time::Instant::now();
let err = did.to_verifying_key().expect_err("must be refused");
let elapsed = start.elapsed();
assert!(
err.to_string().contains("too long"),
"must be refused on length, not after the decode: {err}"
);
assert!(
elapsed < std::time::Duration::from_secs(1),
"refusal must not pay the quadratic decode: took {elapsed:?}"
);
}
/// The bound must not reject a real key. Every `did:key` this project
/// mints is a fixed 48 characters, so the limit is slack, not a behavior
/// change.
#[test]
fn to_verifying_key_still_accepts_a_real_did_key() {
use ed25519_dalek::SigningKey;
let signing = SigningKey::from_bytes(&[7u8; 32]);
let did = Did::from_verifying_key(&signing.verifying_key());
assert_eq!(
did.method_id().len(),
48,
"did:key method-id is fixed-width"
);
did.to_verifying_key().expect("a real did:key must resolve");
}
}
+2 -23
View File
@@ -78,9 +78,10 @@ impl Keypair {
/// Verify an Ed25519 signature.
pub fn verify(verifying_key: &VerifyingKey, msg: &[u8], sig_bytes: &[u8; 64]) -> Result<()> {
use ed25519_dalek::Verifier;
let sig = Signature::from_bytes(sig_bytes);
verifying_key
.verify_strict(msg, &sig)
.verify(msg, &sig)
.map_err(|_| Error::SignatureInvalid)
}
@@ -208,28 +209,6 @@ mod tests {
);
}
#[test]
fn verify_rejects_weak_key_signature() {
// Regression guard for strict verification: a signature forged under a
// weak (small-order) public key satisfies the verification equation
// but must be rejected. The identity point is such a key: with R the
// identity and S = 0, [S]B - [k]A is the identity for any message.
let mut weak_key_bytes = [0u8; 32];
weak_key_bytes[0] = 1; // compressed identity point (y = 1, x = 0)
let weak_vk = VerifyingKey::from_bytes(&weak_key_bytes).unwrap();
assert!(weak_vk.is_weak(), "identity point must be a weak key");
let msg = b"forged message";
let mut forged = [0u8; 64];
forged[0] = 1; // R = identity point, S = 0
let result = verify(&weak_vk, msg, &forged);
assert!(
result.is_err(),
"signature under a weak (small-order) public key must be rejected"
);
}
#[test]
fn signed_payload_round_trip() {
let kp = Keypair::generate();
-10
View File
@@ -5,16 +5,6 @@ pub mod encrypt;
pub mod error;
pub mod http_sig;
pub mod identity;
// `url` is the one dependency here that drags a tail (idna, then the icu
// crates), and gitlawb-core is allowlisted to stay embeddable. Every client that
// needs this predicate already parses URLs, so they opt in and nothing else
// pays. `test` is in the cfg so `cargo test -p gitlawb-core` still compiles and
// runs the matrix below with no feature selected; without it the tests would
// silently not run, which is the failure this module exists to prevent.
#[cfg(any(feature = "redirect", test))]
pub mod redirect;
pub mod sanitize;
pub mod scan_token;
pub mod ucan;
pub use error::Error;
-241
View File
@@ -1,241 +0,0 @@
//! Redirect policy shared by every gitlawb HTTP client that signs its requests.
//!
//! Both `gl` (async) and `git-remote-gitlawb` (blocking) attach RFC 9421
//! `Signature` and `Signature-Input` headers, and reqwest strips only
//! `Authorization`, `Cookie`, `Proxy-Authorization` and `WWW-Authenticate` when a
//! redirect crosses hosts. A signature would survive that hop, and it binds
//! `@method`, `@path` and `content-digest` with no authority component, so a node
//! answering 302 could hand a working credential to a host of its choosing and read
//! as the caller anywhere until the clock-skew window closes. On a 307/308 the
//! request body goes along with it, which for the remote helper is the pack.
//!
//! That same missing authority component is why the predicate also pins the
//! request-target: `@path` is signed as the client sent it and verified as the node
//! received it, so a same-origin hop that rewrites the path or the query (a
//! trailing-slash or query normalization) makes the signature cover a target the
//! node never saw and the read 401s. Only a hop that re-issues the identical target,
//! an http-to-https upgrade being the one that matters in practice, is followed.
//!
//! The decision lives here rather than in either client because the two used to
//! disagree: `gl` was scoped to the origin while the remote helper, the binary that
//! actually runs `git clone gitlawb://`, still ran reqwest's default and followed
//! anywhere. One predicate is what keeps a future third client from repeating that.
//!
//! The type is `url::Url`, which is what `reqwest::Url` re-exports, so both clients
//! pass their attempt URLs straight in.
/// Longest redirect chain followed. `reqwest::redirect::Policy::custom` replaces
/// reqwest's built-in limit, so the bound has to be restated by every client that
/// installs a custom policy; the value is reqwest's own default.
///
/// This counts FOLLOWS, matching `Policy::limited`: reqwest pushes the redirecting
/// URL onto `previous` before consulting the policy, and `Limit(max)` refuses once
/// `previous.len() > max`, so a caller comparing against this constant must use `>`
/// too or it permits one hop fewer than it says.
pub const MAX_REDIRECTS: usize = 10;
/// Follow a redirect only when it stays on the origin that issued it AND re-issues
/// the identical request-target.
///
/// `Policy::none()` would have been the simpler answer, but one same-origin redirect
/// shape is legitimate here: a node fronted by a proxy that upgrades http to https,
/// or otherwise re-issues the same path and query, so the policy is scoped rather
/// than switched off.
///
/// Path and query must match exactly, and that is the request-target clause rather
/// than an origin one. `@path` is signed as the client sent it and verified as the
/// node received it, so a hop that rewrites either half leaves a signature covering
/// a target nobody asked for and the node answers 401. Refusing the hop turns a
/// confusing 401 into the 3xx that names what actually happened. One policy covers
/// signed and unsigned callers alike, for the same reason the predicate is shared:
/// two rules would drift.
///
/// The clause pins `@path`, and only `@path`. A gitlawb signature also covers
/// `@method` and `content-digest`, and both of those are still open on a followed
/// hop: on a 301, 302 or 303, reqwest 0.12.28 delegates to tower-http's
/// `FollowRedirect`, which rewrites a POST to a GET and empties the body
/// (tower-http-0.6.8 `src/follow_redirect/mod.rs:273-285`), while its
/// `drop_payload_headers` removes only `Content-Type`, `Content-Length`,
/// `Content-Encoding` and `Transfer-Encoding`. So `Signature`, `Signature-Input` and
/// `Content-Digest` ride along on a request that no longer has the method or the body
/// they were computed over. Only 307 and 308 preserve both. This predicate returning
/// true therefore makes a GET-shaped hop safe to replay against the node's verifier
/// and says nothing about a bodied one: a signed write must not rely on it alone.
///
/// `Url::query` is `None` for `/a` and `Some("")` for `/a?`, and those are two
/// different request-targets on the node side too, so the comparison is strict and
/// needs no special case.
///
/// Host and port must match exactly. Port is compared as `Url::port`, which is
/// `None` for a scheme's default port, so http -> https on the same host compares
/// equal while http -> http on a different port does not. A downgrade from https to
/// http is refused as well: the target is the same host, but the signature would go
/// out in cleartext, which is the same credential leak by a slower route.
///
/// Host comparison rides on `url`'s parse-time normalization (lowercasing and IDN
/// -> punycode), so the spellings an attacker reaches for do not open a gap. That
/// is a property of the parsed `Url`, not of this function, which is why the test
/// matrix pins it: a move to raw string comparison would silently lose it.
pub fn may_follow(previous: &url::Url, next: &url::Url) -> bool {
let same_origin = next.host_str() == previous.host_str() && next.port() == previous.port();
let same_target = next.path() == previous.path() && next.query() == previous.query();
let downgraded = previous.scheme() == "https" && next.scheme() != "https";
same_origin && same_target && !downgraded
}
#[cfg(test)]
mod tests {
use super::*;
/// Every branch of the decision, both ways, plus the spellings that would slip
/// past a comparison less careful than `Url`'s own normalization.
#[test]
fn may_follow_covers_each_origin_branch() {
let url = |s: &str| url::Url::parse(s).unwrap();
let cases: &[(&str, &str, bool, &str)] = &[
(
"http://node.example/a",
"http://node.example/b",
false,
"same origin but the path changed: @path is signed as sent and verified as received",
),
(
"http://node.example/a",
"https://node.example/a",
true,
"http to https on one host with an identical request-target: both ports are \
the scheme default, so the proxy upgrade is still followed",
),
(
"https://node.example/a",
"https://node.example/a/",
false,
"trailing-slash normalization changes the request-target, so it is refused",
),
(
"https://node.example:8443/a",
"https://node.example:8443/a",
true,
"same explicit port",
),
(
"http://node.example/a",
"http://attacker.example/a",
false,
"different host",
),
(
"http://node.example/a",
"http://node.example:8080/a",
false,
"same host, different port",
),
(
"https://node.example/a",
"http://node.example/a",
false,
"https downgraded to cleartext on the same host",
),
(
"https://node.example/a",
"http://node.example:443/a",
false,
"a downgrade dressed up as the https port",
),
// The rows below pass today because `Url::parse` normalizes the host, not
// because anything here compares case-insensitively or decodes IDN. They
// are the variants an attacker reaches for, so they are pinned: swapping
// this predicate for a raw string comparison must break the suite.
// Each of these pairs an IDENTICAL path on both sides, deliberately. The
// request-target clause below would make every one of them false on the
// path alone, and a row that is false for two reasons has stopped pinning
// either. With the paths equal, the host or port comparison is the only
// thing left that can decide them.
(
"https://node.example/a",
"https://NODE.EXAMPLE/a",
true,
"same host in a different case: parse lowercases it",
),
(
"https://node.example/a",
"https://node.example./a",
false,
"a trailing dot is a different host to url, so the redirect is refused",
),
(
"https://exämple.test/a",
"https://xn--exmple-cua.test/a",
true,
"unicode host and its punycode spelling are one host after parse",
),
(
"https://node.example/a",
"https://user:[email protected]/a",
true,
"userinfo is not part of the origin: same host, still followed",
),
(
"https://node.example/a",
"https://[email protected]/a",
false,
"the node's name smuggled into userinfo: the host is the attacker's",
),
(
"https://node.example/a",
"https://attacker.example/a#node.example",
false,
"the node's name pushed into the fragment: the host is the attacker's",
),
// The request-target clause, both directions. `@path` is the only thing a
// gitlawb signature binds the request to, so a hop that rewrites it hands
// the node a signature over a target it never received.
(
"https://node.example/a?x=1",
"https://node.example/a?x=1",
true,
"identical request-target: the same-origin hop that is still followed",
),
(
"https://node.example/a?x=1",
"https://node.example/a?x=2",
false,
"same path but a different query: the request-target covers the query too",
),
(
"https://node.example/a",
"https://node.example/a?x=1",
false,
"a query added where there was none",
),
(
"http://node.example/a",
"http://node.example/a?",
false,
"an empty query added where there was none: a missing query and an empty \
one are different request-targets",
),
(
"https://node.example/a?x=1",
"https://node.example/a",
false,
"the query dropped where there was one: the comparison is symmetric, and \
nothing else in the matrix pins that direction",
),
(
"https://node.example/a#x",
"https://node.example/a#y",
true,
"a fragment-only difference is still followed: a fragment never reaches \
the wire, so it is no part of the request-target the node verifies",
),
];
for (previous, next, expected, why) in cases {
assert_eq!(
may_follow(&url(previous), &url(next)),
*expected,
"{previous} -> {next} ({why})"
);
}
}
}
-69
View File
@@ -1,69 +0,0 @@
//! Text sanitization predicates shared across the workspace.
//!
//! Untrusted peer/remote bytes that reach an operator's terminal (node error
//! bodies, node-advertised strings) must be stripped of BOTH `Cc` control bytes
//! (`char::is_control()`) AND `Cf` bidi/format controls before display: a
//! right-to-left override or directional isolate can visually reorder the line
//! and spoof the text (Trojan Source, CWE-451). This is INV-6.
//!
//! `is_bidi_format` is the single workspace definition of the second half of
//! that obligation. It was split out of `gl`'s local copy so the two terminal
//! sanitizers that shipped with only the `Cc` half (#183) — and any future
//! sanitizer — reuse one audited predicate instead of re-deriving the set.
/// True for the Unicode bidirectional/directional-isolate format characters
/// (the reordering subset of category `Cf`) that can visually reorder a terminal
/// line. These are NOT matched by [`char::is_control()`], which covers `Cc` only.
///
/// This is deliberately the reordering subset, not the whole `Cf` category:
/// legitimate `Cf` characters such as ZWJ (`U+200D`, emoji sequences) and the
/// soft hyphen (`U+00AD`) are NOT matched, so honest international text passes
/// through unchanged. Widening this set is a behavior change for every consumer.
///
/// Pair it with `!c.is_control()` in a terminal-bound sanitizer (INV-6); see the
/// wrappers in `gl`, `git-remote-gitlawb`, and `icaptcha-client`.
pub fn is_bidi_format(c: char) -> bool {
matches!(c,
'\u{200E}' | '\u{200F}' | '\u{061C}' // LRM, RLM, ALM
| '\u{202A}'..='\u{202E}' // LRE, RLE, PDF, LRO, RLO
| '\u{2066}'..='\u{2069}' // LRI, RLI, FSI, PDI
)
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn strips_every_reordering_code_point() {
// Asserted individually so a partial predicate (e.g. one that covers the
// marks but not the isolates) fails loudly rather than passing.
for c in [
'\u{200E}', '\u{200F}', '\u{061C}', // LRM, RLM, ALM
'\u{202A}', '\u{202B}', '\u{202C}', '\u{202D}',
'\u{202E}', // LRE, RLE, PDF, LRO, RLO
'\u{2066}', '\u{2067}', '\u{2068}', '\u{2069}', // LRI, RLI, FSI, PDI
] {
assert!(
is_bidi_format(c),
"U+{:04X} must be a bidi-format char",
c as u32
);
}
}
#[test]
fn passes_legitimate_and_control_chars() {
// Must NOT over-strip: plain text, a genuine RTL SCRIPT letter (Lo, not a
// format char), ZWJ (legitimate Cf), and soft hyphen (legitimate Cf) are
// not bidi-format. ESC is a Cc control char — owned by is_control(), not
// this predicate, which is bidi-only.
for c in ['a', '\u{0627}', '\u{200D}', '\u{00AD}', '\u{1b}'] {
assert!(
!is_bidi_format(c),
"U+{:04X} must NOT be a bidi-format char",
c as u32
);
}
}
}
-499
View File
@@ -1,499 +0,0 @@
//! Sealed continuation token for the node's bounded legacy CID scan (INV-13).
//!
//! The `/ipfs/{cid}` resolver's legacy scan stops at a row ceiling and sheds a
//! retryable 503. To let a holder buried past that ceiling still be reached, the
//! shed carries the scan position so the caller can echo it back and resume. The
//! whole point of the design is that the node keeps NO server-side scan state: the
//! position rides in the caller's token.
//!
//! That makes the token an EMITTED continuation derived from a FETCHED row, and on
//! a scan that served nothing every fetched row is by construction a private or
//! quarantined repo the caller may not read. The row's `created_at` leaks its
//! creation time and its `id` carries the owner's DID, so both halves are withheld
//! fields and the token must be CONFIDENTIAL, not merely tamper-evident:
//!
//! * AEAD-sealed (XChaCha20-Poly1305), never base64-of-plaintext and never
//! signed plaintext. Integrity is not confidentiality.
//! * A fresh `OsRng` nonce on EVERY seal. Under a stream cipher a repeated nonce
//! means repeated keystream, and an attacker who can force the node to seal a
//! position whose plaintext they know XORs two tokens and recovers a withheld
//! row's fields in full, strictly worse than emitting plaintext.
//! * FIXED-WIDTH plaintext. AEAD ciphertext is plaintext-length plus the tag, and
//! every field of a scan position varies in length, so a variable encoding would
//! make token LENGTH a side channel for the sealed row (a short name under a
//! short owner vs a long one) and for the candidate oid's width (40 hex on a
//! sha1 repo, 64 on a sha256 one). Every token this module mints is byte-identical
//! in length. Each field is padded to its own separate width, which is a per-field
//! constant and so still leaks nothing about a given row or candidate.
//! * The canonical CID as associated data, so a token minted while scanning for
//! one CID does not authenticate when replayed against another.
//!
//! Every failure to open (wrong key, tampered bytes, wrong CID, expired, malformed)
//! returns the same `None`. The caller treats that as "no token" and starts at the
//! front, so no failure class is distinguishable and the token is no oracle.
use base64::{engine::general_purpose::URL_SAFE_NO_PAD as B64URL, Engine};
use chacha20poly1305::{
aead::{Aead, KeyInit, OsRng, Payload},
XChaCha20Poly1305, XNonce,
};
use rand::RngCore;
/// Keyset position of the last row a truncated scan fetched.
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct ScanPosition {
/// The row's raw stored `created_at` text, the first half of the keyset cursor.
pub created_at_key: String,
/// The row's `id`, the tiebreaking half of the keyset cursor.
pub id: String,
/// The oid hex of the CANDIDATE this position resumes. One CID can map to several
/// git oids, so a bare row cursor names a row without naming whose walk it belongs
/// to. The candidate is named by IDENTITY rather than by position in the candidate
/// list: that list is ordered by hex and mutates between rungs, so an index would
/// name a different candidate the moment anything is pinned or unpinned.
pub sha256_hex: String,
}
/// Plaintext version byte, so a future layout change is a clean open-failure
/// (treated as absent) rather than a misparse.
///
/// Bumped to 2 when the two halves stopped sharing one width (see [`ID_WIDTH`]), and
/// to 3 when the position gained the candidate oid it resumes (see [`OID_WIDTH`]).
/// A token minted under an earlier layout is a different length and a different
/// framing, so it opens to `None` and the caller restarts at the front, which is the
/// safe direction: a misparse would resume at a fabricated row and skip coverage.
const VERSION: u8 = 3;
/// Byte width the `created_at` half is padded to. Every value stored here is a
/// serialized timestamp, about 30 bytes, so 64 is roomy for the field's whole domain.
/// It is deliberately NOT widened to match [`ID_WIDTH`]: padding both halves to the id
/// width would nearly double every token for a field that can never use the space.
const CREATED_WIDTH: usize = 64;
/// Byte width the `id` half is padded to.
///
/// The bound is set by the WRITERS, not by what a typical id happens to look like.
/// `upsert_mirror_repo` builds `repos.id` as `{owner}/{name}`, and the slug validators
/// in the node's `repo_store` admit an owner of up to 255 bytes and a name of up to
/// 100, so 356 bytes is reachable through the ordinary write path and repo names are
/// peer-controllable. 384 clears that with margin.
///
/// Under-sizing this is not a cosmetic bug. A row at a truncation boundary whose id
/// exceeds the width fails the seal, the handler sheds a 503 with no continuation, and
/// a tokenless shed is byte-identical to the wrapped-scan response whose contract is
/// "the absence of a token means the ladder is over". The boundary row is deterministic
/// for a stable inventory, so every retry reproduces it and every row past it becomes
/// permanently unreachable. Anything past the width still fails loudly rather than
/// silently truncating a cursor into one that resumes at the wrong row.
const ID_WIDTH: usize = 384;
/// Byte width the candidate oid half is padded to.
///
/// Git mints exactly two oid widths and this field carries BOTH. A production repo is
/// created by `store::init_bare` with `git init --bare --object-format=sha1`, so its
/// oids are 40 hex; only the sha256 test fixtures mint 64. The field is therefore
/// length-prefixed like the two row halves rather than a bare fixed 64: a 64-only
/// field would fail every seal on a real deployment, and a failed seal sheds a
/// tokenless 503 that is byte-identical to "your ladder is over".
///
/// The padding to 64 is what keeps the WIDTH off the wire. Without it a 40-hex token
/// is 24 bytes shorter than a 64-hex one, and token length would say which object
/// format the holder's repo uses.
const OID_WIDTH: usize = 64;
/// `version | created_len:u16 | created[CREATED_WIDTH] | id_len:u16 | id[ID_WIDTH] | oid_len:u16 | oid[OID_WIDTH] | expires:i64`
const PLAINTEXT_LEN: usize = 1 + 2 + CREATED_WIDTH + 2 + ID_WIDTH + 2 + OID_WIDTH + 8;
/// Nonce width for XChaCha20-Poly1305.
const NONCE_LEN: usize = 24;
/// A fresh random 32-byte sealing key from the OS CSPRNG.
pub fn new_key() -> [u8; 32] {
let mut key = [0u8; 32];
OsRng.fill_bytes(&mut key);
key
}
// The two halves below are deliberately separate and adjacent. The framing pair owns
// "every token is the same length"; the AEAD pair owns "the contents are confidential
// and CID-bound". Keeping them apart is what lets each property be exercised (and
// broken) without disturbing the other.
/// Encode a position into the FIXED-WIDTH plaintext:
/// `version | created_len:u16 | created[CREATED_WIDTH] | id_len:u16 | id[ID_WIDTH] | oid_len:u16 | oid[OID_WIDTH] | expires:i64`
///
/// The padding is the point. AEAD ciphertext is plaintext-length plus the tag, and every
/// field of a scan position varies in length, so a length-prefixed encoding with no
/// padding would make token LENGTH a side channel for the sealed row and for the
/// candidate's object format. Each field is padded to its OWN fixed width, which keeps
/// every minted token the same length while letting the id half carry the range the
/// write path actually admits.
fn encode_position(pos: &ScanPosition, expires_at_unix: i64) -> anyhow::Result<Vec<u8>> {
let mut out = vec![0u8; PLAINTEXT_LEN];
out[0] = VERSION;
let mut at = 1;
for (field, width) in [
(pos.created_at_key.as_bytes(), CREATED_WIDTH),
(pos.id.as_bytes(), ID_WIDTH),
(pos.sha256_hex.as_bytes(), OID_WIDTH),
] {
if field.len() > width {
// Loud rather than truncating: a clipped cursor resumes at the wrong row and
// silently skips coverage, which is the availability half of the bug this
// token exists to fix.
anyhow::bail!(
"scan token field is {} bytes, over the {width}-byte fixed width",
field.len()
);
}
out[at..at + 2].copy_from_slice(&(field.len() as u16).to_le_bytes());
at += 2;
out[at..at + field.len()].copy_from_slice(field);
at += width;
}
out[at..at + 8].copy_from_slice(&expires_at_unix.to_le_bytes());
Ok(out)
}
/// Decode what [`encode_position`] wrote. `None` on any structural mismatch.
fn decode_position(bytes: &[u8]) -> Option<(ScanPosition, i64)> {
if bytes.len() != PLAINTEXT_LEN || bytes[0] != VERSION {
return None;
}
let mut at = 1;
let mut fields = [const { String::new() }; 3];
for (slot, width) in fields.iter_mut().zip([CREATED_WIDTH, ID_WIDTH, OID_WIDTH]) {
let len = u16::from_le_bytes([bytes[at], bytes[at + 1]]) as usize;
at += 2;
if len > width {
return None;
}
*slot = String::from_utf8(bytes[at..at + len].to_vec()).ok()?;
at += width;
}
let expires_at = i64::from_le_bytes(bytes[at..at + 8].try_into().ok()?);
let [created_at_key, id, sha256_hex] = fields;
// A zero-length candidate is a third state the encoder never mints. The front-of-table
// sentinel is empty ROW halves with a real oid, so an empty oid would hand the resume
// path a candidate that names nothing; refuse it like every other malformed frame.
if sha256_hex.is_empty() {
return None;
}
Some((
ScanPosition {
created_at_key,
id,
sha256_hex,
},
expires_at,
))
}
/// AEAD-seal `plaintext` under `key`, bound to `cid`, framed as `nonce || ciphertext`.
fn seal_bytes(key: &[u8; 32], cid: &str, plaintext: &[u8]) -> anyhow::Result<Vec<u8>> {
let cipher = XChaCha20Poly1305::new_from_slice(key)
.map_err(|e| anyhow::anyhow!("scan token key: {e}"))?;
// A FRESH nonce per seal, from the OS CSPRNG. Under a stream cipher a repeated nonce
// repeats the keystream, and two tokens sealed under one nonce XOR to the difference
// of their plaintexts, which recovers a withheld row in full when the attacker can
// force one of the two positions. This draw is the property the whole confidentiality
// claim rests on.
let mut nonce = [0u8; NONCE_LEN];
OsRng.fill_bytes(&mut nonce);
let sealed = cipher
.encrypt(
XNonce::from_slice(&nonce),
Payload {
msg: plaintext,
// The canonical CID as associated data: a token minted while scanning for
// one CID does not authenticate against another, so it cannot be replayed
// to seed a different scan.
aad: cid.as_bytes(),
},
)
.map_err(|e| anyhow::anyhow!("scan token seal: {e}"))?;
let mut out = Vec::with_capacity(NONCE_LEN + sealed.len());
out.extend_from_slice(&nonce);
out.extend_from_slice(&sealed);
Ok(out)
}
/// Open what [`seal_bytes`] framed. `None` on any failure, including a wrong `cid`.
fn open_bytes(key: &[u8; 32], cid: &str, raw: &[u8]) -> Option<Vec<u8>> {
if raw.len() <= NONCE_LEN + 16 {
return None;
}
let (nonce, sealed) = raw.split_at(NONCE_LEN);
let cipher = XChaCha20Poly1305::new_from_slice(key).ok()?;
cipher
.decrypt(
XNonce::from_slice(nonce),
Payload {
msg: sealed,
aad: cid.as_bytes(),
},
)
.ok()
}
/// Seal `pos` under `key`, bound to `cid`, expiring at `expires_at_unix`.
///
/// Returns the base64url (no pad) token. Errors only when a field exceeds
/// its own fixed width ([`CREATED_WIDTH`], [`ID_WIDTH`], [`OID_WIDTH`]) or the AEAD
/// itself fails, never silently truncates.
pub fn seal_scan_token(
key: &[u8; 32],
cid: &str,
pos: &ScanPosition,
expires_at_unix: i64,
) -> anyhow::Result<String> {
let plaintext = encode_position(pos, expires_at_unix)?;
Ok(B64URL.encode(seal_bytes(key, cid, &plaintext)?))
}
/// Open a token minted by [`seal_scan_token`] under the same key and CID.
///
/// `None` for every failure class alike (wrong key, tampered, foreign CID, expired,
/// malformed, wrong version), so the caller can treat all of them as "absent" without
/// leaking which one occurred.
pub fn open_scan_token(
key: &[u8; 32],
cid: &str,
token: &str,
now_unix: i64,
) -> Option<ScanPosition> {
let raw = B64URL.decode(token).ok()?;
let plaintext = open_bytes(key, cid, &raw)?;
let (pos, expires_at) = decode_position(&plaintext)?;
if now_unix >= expires_at {
return None;
}
Some(pos)
}
#[cfg(test)]
mod tests {
use super::*;
/// A production-shaped candidate: `git init --bare --object-format=sha1`, so 40 hex.
const OID_40: &str = "a1b2c3d4e5f60718293a4b5c6d7e8f9012345678";
/// A test-fixture-shaped candidate: the sha256 repos the suite creates mint 64 hex.
const OID_64: &str = "0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef";
fn pos(created: &str, id: &str) -> ScanPosition {
pos_for(created, id, OID_40)
}
fn pos_for(created: &str, id: &str, sha256_hex: &str) -> ScanPosition {
ScanPosition {
created_at_key: created.to_string(),
id: id.to_string(),
sha256_hex: sha256_hex.to_string(),
}
}
/// Seal a hand-built plaintext through the AEAD half, so a test can frame bytes the
/// encoder would never mint (an old version byte, a zero-length oid) and still exercise
/// the real open path.
fn seal_raw(key: &[u8; 32], cid: &str, plaintext: &[u8]) -> String {
B64URL.encode(seal_bytes(key, cid, plaintext).unwrap())
}
/// Byte offset of the oid length prefix inside the plaintext, derived from the widths
/// rather than hardcoded so a width change moves it with the layout.
const OID_LEN_AT: usize = 1 + 2 + CREATED_WIDTH + 2 + ID_WIDTH;
/// Scenario 1: both of git's oid widths round trip. The 40-hex case is the PRODUCTION
/// shape (`git init --bare --object-format=sha1`), and an all-64 fixture suite would
/// never exercise it, which is exactly how a fixed-64 field would ship broken.
#[test]
fn round_trips_at_both_oid_widths() {
let key = new_key();
for hex in [OID_40, OID_64] {
let p = pos_for("2020-01-01T00:00:03+00:00", "z6MkOwner/private-repo", hex);
let t =
seal_scan_token(&key, "bafkcid", &p, 1 << 40).expect("both oid widths must seal");
assert_eq!(
open_scan_token(&key, "bafkcid", &t, 0),
Some(p),
"a {}-hex candidate must open to the identical position",
hex.len()
);
}
}
/// Scenario 2: a token framed under the OLD version opens to `None`, never a misparse.
/// Both legs matter: the version-2 plaintext was a different LENGTH, and a future
/// same-length layout would only be caught by the version byte itself.
#[test]
fn a_prior_layout_version_opens_to_none() {
let key = new_key();
// The version-2 layout verbatim: no oid field, so 461 bytes.
let mut old = vec![0u8; 1 + 2 + CREATED_WIDTH + 2 + ID_WIDTH + 8];
old[0] = 2;
assert_eq!(old.len(), 461, "the version-2 plaintext was 461 bytes");
assert_eq!(
open_scan_token(&key, "bafkcid", &seal_raw(&key, "bafkcid", &old), 0),
None,
"a version-2 token must open to None so the caller restarts at the front"
);
// Same length, stale version byte: only the version check can refuse this one.
let mut stamped = encode_position(
&pos("2020-01-01T00:00:03+00:00", "z6MkOwner/private-repo"),
1 << 40,
)
.unwrap();
stamped[0] = VERSION - 1;
assert_eq!(
open_scan_token(&key, "bafkcid", &seal_raw(&key, "bafkcid", &stamped), 0),
None,
"a stale version byte must open to None even at the current width"
);
}
/// Scenario 3: token length is invariant across candidate VALUE and candidate WIDTH.
/// The padding is what keeps the oid width off the wire: without it a 40-hex token is
/// 24 bytes shorter than a 64-hex one and the length says which repo format the
/// holder uses.
#[test]
fn token_length_is_invariant_across_oid_widths() {
let key = new_key();
let created = "2020-01-01T00:00:00+00:00";
let id = "z6MkOwner/private-repo";
let short =
seal_scan_token(&key, "bafkcid", &pos_for(created, id, OID_40), 1 << 40).unwrap();
let long =
seal_scan_token(&key, "bafkcid", &pos_for(created, id, OID_64), 1 << 40).unwrap();
assert_eq!(
short.len(),
long.len(),
"a 40-hex and a 64-hex candidate must mint tokens of identical length, or the \
oid width is a side channel"
);
// The absolute width, pinned by execution rather than by arithmetic on paper. The
// gl client's mock fixtures hardcode this number (`TOKEN_LEN` in
// crates/gl/src/ipfs_cmd.rs), and nothing in that crate seals a real token, so this
// assertion is the only executable check that the two agree.
assert_eq!(
short.len(),
756,
"24 nonce + {PLAINTEXT_LEN} plaintext + 16 tag, base64url no pad"
);
}
/// Scenario 4: the front-of-table sentinel. Empty row halves with a REAL candidate
/// round trip, which is what lets a seal say "this candidate, no row cursor yet".
#[test]
fn empty_row_fields_round_trip_with_a_real_candidate() {
let key = new_key();
let p = pos_for("", "", OID_40);
let t =
seal_scan_token(&key, "bafkcid", &p, 1 << 40).expect("the front sentinel must seal");
assert_eq!(open_scan_token(&key, "bafkcid", &t, 0), Some(p));
}
/// Scenario 6, third leg: a zero-length oid is a distinguishable third state that the
/// encoder never mints, and accepting it would hand the sentinel machinery a candidate
/// naming nothing. The decode path refuses it.
#[test]
fn a_zero_length_candidate_opens_to_none() {
let key = new_key();
let mut plaintext = encode_position(
&pos("2020-01-01T00:00:03+00:00", "z6MkOwner/private-repo"),
1 << 40,
)
.unwrap();
plaintext[OID_LEN_AT..OID_LEN_AT + 2].copy_from_slice(&0u16.to_le_bytes());
assert_eq!(
open_scan_token(&key, "bafkcid", &seal_raw(&key, "bafkcid", &plaintext), 0),
None,
"a zero-length candidate must open to None, not to a position naming nothing"
);
}
/// Scenario 6, second leg: an oid past the 64-byte width fails the seal loudly rather
/// than being clipped into a hex that names a different candidate.
#[test]
fn an_oid_over_the_fixed_width_fails_loudly() {
let key = new_key();
let p = pos_for(
"2020-01-01T00:00:03+00:00",
"z6MkOwner/private-repo",
&"a".repeat(OID_WIDTH + 1),
);
assert!(
seal_scan_token(&key, "bafkcid", &p, 1 << 40).is_err(),
"an over-wide candidate must fail the seal, never be truncated into a hex that \
resumes the wrong candidate"
);
}
#[test]
fn round_trips_under_the_same_key_and_cid() {
let key = new_key();
let p = pos("2020-01-01T00:00:03+00:00", "z6MkOwner/private-repo");
let t = seal_scan_token(&key, "bafkcid", &p, 1 << 40).unwrap();
assert_eq!(open_scan_token(&key, "bafkcid", &t, 0), Some(p));
}
#[test]
fn every_failure_class_opens_to_none() {
let key = new_key();
let other = new_key();
let p = pos("2020-01-01T00:00:03+00:00", "z6MkOwner/private-repo");
let t = seal_scan_token(&key, "bafkcid", &p, 1 << 40).unwrap();
assert_eq!(open_scan_token(&other, "bafkcid", &t, 0), None, "wrong key");
assert_eq!(
open_scan_token(&key, "bafkOTHER", &t, 0),
None,
"foreign CID"
);
assert_eq!(
open_scan_token(&key, "bafkcid", &t, 1 << 41),
None,
"expired"
);
assert_eq!(open_scan_token(&key, "bafkcid", "!!not b64", 0), None);
assert_eq!(open_scan_token(&key, "bafkcid", "", 0), None);
let mut flipped: Vec<u8> = t.bytes().collect();
let last = flipped.len() - 1;
flipped[last] = if flipped[last] == b'A' { b'B' } else { b'A' };
assert_eq!(
open_scan_token(&key, "bafkcid", &String::from_utf8(flipped).unwrap(), 0),
None,
"tampered"
);
}
/// The id half must clear the LARGEST repo id the node's own write path admits,
/// not merely a typical one. `upsert_mirror_repo` builds `repos.id` as
/// `{owner}/{name}`, and the slug validators in `repo_store` admit 255 bytes of
/// owner and 100 of name, so 356 is reachable. A width under that turns the
/// boundary row into a seal failure, which sheds a tokenless 503 that is
/// byte-identical to "your ladder is over" and strands every row past it forever.
#[test]
fn round_trips_a_repo_id_at_the_write_paths_maximum() {
let key = new_key();
let id = format!("{}/{}", "o".repeat(255), "n".repeat(100));
assert_eq!(id.len(), 356, "255 owner + '/' + 100 name");
let p = pos("2020-01-01T00:00:03+00:00", &id);
let t = seal_scan_token(&key, "bafkcid", &p, 1 << 40)
.expect("a repo id the write path admits must seal, never fail the width");
assert_eq!(open_scan_token(&key, "bafkcid", &t, 0), Some(p));
}
#[test]
fn a_field_over_the_fixed_width_fails_loudly() {
let key = new_key();
let p = pos("2020-01-01T00:00:03+00:00", &"x".repeat(ID_WIDTH + 1));
assert!(
seal_scan_token(&key, "bafkcid", &p, 1 << 40).is_err(),
"an over-wide field must fail the seal, never be truncated into a cursor \
that resumes at the wrong row"
);
}
}
+4 -67
View File
@@ -187,17 +187,11 @@ impl Ucan {
}
/// Check if this UCAN grants a specific capability on a resource.
///
/// Mirrors `Capability::is_attenuated_by`'s wildcard semantics: a stored
/// capability of `with: "*"` or `can: "*"` / `"repo/admin"` covers any
/// requested resource/action, since a valid delegation chain can produce
/// exactly that capability (see `is_attenuated_by`).
pub fn can(&self, resource: &str, action: &str) -> bool {
self.payload.att.iter().any(|cap| {
let resource_ok = cap.with == resource || cap.with == "*";
let action_ok = cap.can == action || cap.can == "*" || cap.can == caps::REPO_ADMIN;
resource_ok && action_ok
})
self.payload
.att
.iter()
.any(|cap| cap.with == resource && cap.can == action)
}
/// Encode to a compact JSON string (the wire format).
@@ -318,63 +312,6 @@ mod tests {
assert!(!ucan.can("gitlawb://repos/test/repo", caps::PR_MERGE));
}
#[test]
fn can_honors_resource_wildcard() {
let issuer = Keypair::generate();
let audience = Keypair::generate().did();
let ucan = Ucan::issue(
&issuer,
audience,
vec![Capability::new("*", caps::GIT_PUSH)],
None,
)
.unwrap();
assert!(ucan.can("gitlawb://repos/test/repo", caps::GIT_PUSH));
assert!(ucan.can("gitlawb://repos/other/repo", caps::GIT_PUSH));
assert!(!ucan.can("gitlawb://repos/test/repo", caps::PR_MERGE));
}
#[test]
fn can_honors_repo_admin_action_wildcard() {
let issuer = Keypair::generate();
let audience = Keypair::generate().did();
let ucan = Ucan::issue(
&issuer,
audience,
vec![Capability::new(
"gitlawb://repos/test/repo",
caps::REPO_ADMIN,
)],
None,
)
.unwrap();
assert!(ucan.can("gitlawb://repos/test/repo", caps::GIT_PUSH));
assert!(ucan.can("gitlawb://repos/test/repo", caps::PR_MERGE));
assert!(!ucan.can("gitlawb://repos/other/repo", caps::GIT_PUSH));
}
#[test]
fn can_honors_action_wildcard() {
let issuer = Keypair::generate();
let audience = Keypair::generate().did();
let ucan = Ucan::issue(
&issuer,
audience,
vec![Capability::new("gitlawb://repos/test/repo", "*")],
None,
)
.unwrap();
assert!(ucan.can("gitlawb://repos/test/repo", caps::GIT_PUSH));
assert!(ucan.can("gitlawb://repos/test/repo", caps::PR_MERGE));
assert!(!ucan.can("gitlawb://repos/other/repo", caps::GIT_PUSH));
}
#[test]
fn bootstrap_ucan() {
let issuer = Keypair::generate();
+1 -4
View File
@@ -1,7 +1,7 @@
[package]
name = "gitlawb-node"
description = "The gitlawb node daemon — git hosting over HTTP with DID auth"
version = "0.7.1" # x-release-please-version
version = "0.5.1" # x-release-please-version
edition.workspace = true
rust-version.workspace = true
license.workspace = true
@@ -77,6 +77,3 @@ libp2p-dns = { version = "0.44.0", features = ["tokio"] }
[dev-dependencies]
mockito = "1"
tempfile = "3"
# `test-util` for `#[tokio::test(start_paused = true)]`: lets the deadline tests assert
# on which timer fired instead of on wall clock.
tokio = { workspace = true, features = ["test-util"] }
+2 -49
View File
@@ -26,61 +26,14 @@ pub async fn list_anchors(
Query(q): Query<ListAnchorsQuery>,
) -> Result<Json<serde_json::Value>> {
let limit = q.limit.min(200);
// Bare `?` so connection-class sqlx failures downcast to `AppError::Db` and
// map to 503 `db_unavailable` (not 500 via `.map_err(AppError::Internal)`) (#251).
let anchors = state
.db
.list_arweave_anchors(q.repo.as_deref(), limit)
.await?;
.await
.map_err(crate::error::AppError::Internal)?;
Ok(Json(serde_json::json!({
"anchors": anchors,
"count": anchors.len(),
})))
}
#[cfg(test)]
mod closed_pool_tests {
use super::*;
use axum::http::{Request, StatusCode};
use axum::Router;
use serde_json::Value;
use sqlx::PgPool;
use tower::ServiceExt;
/// #251: a closed pool on /api/v1/arweave/anchors must be 503 db_unavailable.
#[sqlx::test]
async fn list_anchors_closed_pool_returns_503_db_unavailable(pool: PgPool) {
let state = crate::test_support::test_state(pool.clone()).await;
pool.close().await;
let resp = Router::new()
.route("/api/v1/arweave/anchors", axum::routing::get(list_anchors))
.with_state(state)
.oneshot(
Request::builder()
.uri("/api/v1/arweave/anchors")
.body(axum::body::Body::empty())
.unwrap(),
)
.await
.unwrap();
assert_eq!(
resp.status(),
StatusCode::SERVICE_UNAVAILABLE,
"closed-pool outage must be retryable 503, not 500"
);
let bytes = axum::body::to_bytes(resp.into_body(), usize::MAX)
.await
.expect("read body");
let v: Value = serde_json::from_slice(&bytes).expect("json body");
assert_eq!(
v,
serde_json::json!({
"error": crate::error::DB_UNAVAILABLE_CODE,
"message": crate::error::DB_UNAVAILABLE_MESSAGE,
})
);
}
}
+24 -89
View File
@@ -138,33 +138,9 @@ pub async fn list_ref_updates(
let caller = auth.as_ref().map(|e| e.0 .0.as_str());
let updates = collect_visible_ref_updates(&state.db, None, limit, caller).await?;
// Resolve the trusted display owner_did per row. The stored wire value is
// untrusted (arrives over gossip / unsigned peer notify), so it is echoed
// only when it matches the canonical owner of the local repo the slug
// names (#P1); legacy None rows are attributed via an exact unique local
// match (#P3). Both surfaces share this resolver so they cannot drift.
// The batch resolver issues at most one query per distinct local repo
// rather than one per event row (#P2).
let raw_dids: Vec<Option<String>> = state
.db
.resolve_ref_update_owner_dids(
&updates
.iter()
.map(|u| (u.repo.as_str(), u.owner_did.as_deref()))
.collect::<Vec<_>>(),
)
.await?;
let owner_dids: Vec<serde_json::Value> = raw_dids
.into_iter()
.map(|o| o.map_or(serde_json::Value::Null, |s| serde_json::json!(s)))
.collect();
let events: Vec<serde_json::Value> = updates
.iter()
.enumerate()
.map(|(i, u)| {
let owner_did = owner_dids[i].clone();
.map(|u| {
serde_json::json!({
"id": u.id,
"node_did": u.node_did,
@@ -177,7 +153,6 @@ pub async fn list_ref_updates(
"cert_id": u.cert_id,
"received_at": u.received_at,
"from_peer": u.from_peer,
"owner_did": owner_did,
})
})
.collect();
@@ -250,7 +225,6 @@ pub async fn list_repo_events(
"pusher_did": c.pusher_did,
"node_did": c.node_did,
"timestamp": c.issued_at,
"owner_did": record.owner_did,
"source": "local",
})
})
@@ -266,46 +240,28 @@ pub async fn list_repo_events(
// collect_visible_ref_updates drops any row whose slug matches a local repo the
// caller cannot read (fail-closed), and propagates DB errors instead of swallowing
// them.
let gossip_updates =
collect_visible_ref_updates(&state.db, Some(&repo_id_str), limit, caller).await?;
let gossip_raw: Vec<Option<String>> = state
.db
.resolve_ref_update_owner_dids(
&gossip_updates
.iter()
.map(|u| (u.repo.as_str(), u.owner_did.as_deref()))
.collect::<Vec<_>>(),
)
.await?;
let gossip_owner_dids: Vec<serde_json::Value> = gossip_raw
.into_iter()
.map(|o| o.map_or(serde_json::Value::Null, |s| serde_json::json!(s)))
.collect();
let gossip_events: Vec<serde_json::Value> = gossip_updates
.iter()
.enumerate()
.map(|(i, u)| {
let owner_did = gossip_owner_dids[i].clone();
serde_json::json!({
"type": "gossipsub",
"id": u.id,
"repo": u.repo,
"ref_name": u.ref_name,
"old_sha": u.old_sha,
"new_sha": u.new_sha,
"pusher_did": u.pusher_did,
"node_did": u.node_did,
"timestamp": u.timestamp,
"cert_id": u.cert_id,
"received_at": u.received_at,
"from_peer": u.from_peer,
"owner_did": owner_did,
"source": "gossipsub",
let gossip_events: Vec<serde_json::Value> =
collect_visible_ref_updates(&state.db, Some(&repo_id_str), limit, caller)
.await?
.iter()
.map(|u| {
serde_json::json!({
"type": "gossipsub",
"id": u.id,
"repo": u.repo,
"ref_name": u.ref_name,
"old_sha": u.old_sha,
"new_sha": u.new_sha,
"pusher_did": u.pusher_did,
"node_did": u.node_did,
"timestamp": u.timestamp,
"cert_id": u.cert_id,
"received_at": u.received_at,
"from_peer": u.from_peer,
"source": "gossipsub",
})
})
})
.collect();
.collect();
// Merge both lists
let mut all_events: Vec<serde_json::Value> = cert_events;
@@ -370,7 +326,6 @@ mod ref_updates_feed_tests {
cert_id: None,
received_at: Utc::now().to_rfc3339(),
from_peer: "peer1".into(),
owner_did: None,
}
}
@@ -1150,14 +1105,7 @@ mod ref_updates_feed_tests {
.await
.unwrap();
assert_eq!(resp.status(), StatusCode::OK);
let body = body_json(resp).await;
assert_eq!(count(&body), 2);
for event in body["events"].as_array().unwrap() {
assert_eq!(
event["owner_did"], OWNER,
"each event must carry the local owner_did"
);
}
assert_eq!(count(&body_json(resp).await), 2);
}
// Anon reads a quarantined mirror → 404 (withheld without disclosing existence
@@ -1440,8 +1388,7 @@ mod ref_updates_feed_tests {
// A DB error in the gate fails closed as 500, not swallowed into an empty 200 (the
// regression the old get_repo().ok().flatten() allowed). Inject by dropping a
// column get_repo selects so its query errors. Also pin the anonymous body to
// the exact opaque object (#226).
// column get_repo selects so its query errors.
#[sqlx::test]
async fn repo_events_db_error_fails_closed_500(pool: PgPool) {
let state = test_state(pool.clone()).await;
@@ -1464,18 +1411,6 @@ mod ref_updates_feed_tests {
StatusCode::INTERNAL_SERVER_ERROR,
"a DB error must fail closed (500), never serve an empty 200"
);
let body = axum::body::to_bytes(resp.into_body(), usize::MAX)
.await
.expect("read body");
let body = String::from_utf8_lossy(&body);
let v: serde_json::Value = serde_json::from_str(&body).expect("json body");
assert_eq!(
v,
serde_json::json!({
"error": "db_error",
"message": crate::error::DB_ERROR_MESSAGE,
})
);
}
// Symmetric to the gate DB-error test: a DB error in the CERT fetch (after the gate
File diff suppressed because it is too large. Load diff
-195
View File
@@ -449,198 +449,3 @@ mod lock_pool_shed_tests {
);
}
}
/// #426: end-to-end at the handler seam. A corrupt object store behind a
/// resolving issue ref used to look identical to an absent issue: get returned
/// a fake 404, close returned a fake 404 after releasing the write guard, and
/// list silently under-reported. All three must now surface a git error.
#[cfg(test)]
mod corrupt_store_tests {
use super::*;
use axum::response::IntoResponse;
use sqlx::PgPool;
use tempfile::TempDir;
fn seed_repo(owner_did: &str, name: &str) -> crate::db::RepoRecord {
let now = Utc::now();
crate::db::RepoRecord {
id: Uuid::new_v4().to_string(),
name: name.to_string(),
owner_did: owner_did.to_string(),
description: None,
is_public: true,
default_branch: "main".to_string(),
created_at: now,
updated_at: now,
disk_path: format!("/tmp/{name}"),
forked_from: None,
machine_id: None,
}
}
/// A state whose repo store roots in `repos_dir`, so the test can lay down
/// a real repo and then corrupt it without touching /tmp.
async fn state_with_repos_dir(pool: &PgPool, repos_dir: &std::path::Path) -> AppState {
let mut state = crate::test_support::test_state(pool.clone()).await;
state.repo_store =
crate::git::repo_store::RepoStore::for_testing(repos_dir.to_path_buf(), pool.clone());
state
}
/// Bare repo at the store's expected path, holding one issue ref.
fn bare_repo_with_issue(
repos_dir: &std::path::Path,
owner: &str,
name: &str,
) -> (std::path::PathBuf, String) {
let repo_path = repos_dir
.join(owner.replace([':', '/'], "_"))
.join(format!("{name}.git"));
std::fs::create_dir_all(&repo_path).unwrap();
let out = std::process::Command::new("git")
.args(["init", "--bare"])
.arg(&repo_path)
.output()
.unwrap();
assert!(out.status.success());
let issue_id = "aaaabbbb-0000-0000-0000-000000000000".to_string();
git_issues::create_issue(
&repo_path,
&issue_id,
&serde_json::json!({
"id": issue_id,
"title": "t",
"status": "open",
})
.to_string(),
)
.unwrap();
(repo_path, issue_id)
}
/// Delete the blob object behind the issue ref and put garbage in its
/// place: `cat-file` fails while `rev-parse` still resolves the ref.
fn corrupt_issue_object(repo_path: &std::path::Path, issue_id: &str) {
let out = std::process::Command::new("git")
.args(["rev-parse", &format!("refs/gitlawb/issues/{issue_id}")])
.current_dir(repo_path)
.output()
.unwrap();
let oid = String::from_utf8(out.stdout).unwrap().trim().to_string();
let obj = repo_path.join("objects").join(&oid[..2]).join(&oid[2..]);
std::fs::remove_file(&obj).unwrap();
std::fs::write(&obj, b"garbage").unwrap();
}
/// The regression: closing over a corrupt store used to return a fake 404
/// (or panic deeper in). It must reach the caller as a git 500, and the
/// handler must return rather than unwind.
#[sqlx::test]
async fn close_issue_on_corrupt_store_returns_git_error_not_panic(pool: PgPool) {
let owner = "did:key:zISSUECORRUPTCCCCCCCCCCCCCCCCCCCCCCCCC";
let dir = TempDir::new().unwrap();
let state = state_with_repos_dir(&pool, dir.path()).await;
state
.db
.create_repo(&seed_repo(owner, "corrupt-close"))
.await
.expect("seed repo");
let (repo_path, issue_id) = bare_repo_with_issue(dir.path(), owner, "corrupt-close");
corrupt_issue_object(&repo_path, &issue_id);
let err = close_issue(
State(state),
Extension(AuthenticatedDid(owner.to_string())),
Path((owner.to_string(), "corrupt-close".to_string(), issue_id)),
)
.await
.expect_err("a corrupt store must error, not panic or 404");
assert_eq!(
err.into_response().status(),
StatusCode::INTERNAL_SERVER_ERROR,
"a read failure is a git 500, not a fake not-found"
);
}
/// get on a corrupt store: 500, not the 404 the fold used to produce.
#[sqlx::test]
async fn get_issue_on_corrupt_store_returns_500_not_404(pool: PgPool) {
let owner = "did:key:zISSUECORRUPTGETDDDDDDDDDDDDDDDDDDDDDDD";
let dir = TempDir::new().unwrap();
let state = state_with_repos_dir(&pool, dir.path()).await;
state
.db
.create_repo(&seed_repo(owner, "corrupt-get"))
.await
.expect("seed repo");
let (repo_path, issue_id) = bare_repo_with_issue(dir.path(), owner, "corrupt-get");
corrupt_issue_object(&repo_path, &issue_id);
let err = get_issue(
State(state),
Path((owner.to_string(), "corrupt-get".to_string(), issue_id)),
None,
)
.await
.expect_err("a corrupt store must error, not 404");
assert_eq!(
err.into_response().status(),
StatusCode::INTERNAL_SERVER_ERROR,
);
}
/// list on a corrupt store must fail outright; a 200 that silently drops
/// the unreadable issue is the other half of the fold.
#[sqlx::test]
async fn list_issues_on_corrupt_store_returns_git_error(pool: PgPool) {
let owner = "did:key:zISSUECORRUPTLISTEEEEEEEEEEEEEEEEEEEEEE";
let dir = TempDir::new().unwrap();
let state = state_with_repos_dir(&pool, dir.path()).await;
state
.db
.create_repo(&seed_repo(owner, "corrupt-list"))
.await
.expect("seed repo");
let (repo_path, issue_id) = bare_repo_with_issue(dir.path(), owner, "corrupt-list");
corrupt_issue_object(&repo_path, &issue_id);
let err = list_issues(
State(state),
Path((owner.to_string(), "corrupt-list".to_string())),
None,
)
.await
.expect_err("a corrupt store must error, not under-report");
assert_eq!(
err.into_response().status(),
StatusCode::INTERNAL_SERVER_ERROR,
);
}
/// Must-not direction: a genuinely absent issue still gets the 404.
#[sqlx::test]
async fn get_issue_on_absent_ref_still_returns_404(pool: PgPool) {
let owner = "did:key:zISSUEABSENTFFFFFFFFFFFFFFFFFFFFFFFFFFF";
let dir = TempDir::new().unwrap();
let state = state_with_repos_dir(&pool, dir.path()).await;
state
.db
.create_repo(&seed_repo(owner, "absent-get"))
.await
.expect("seed repo");
bare_repo_with_issue(dir.path(), owner, "absent-get");
let err = get_issue(
State(state),
Path((
owner.to_string(),
"absent-get".to_string(),
"ffffffff-0000-0000-0000-000000000000".to_string(),
)),
None,
)
.await
.expect_err("a missing issue is still not-found");
assert_eq!(err.into_response().status(), StatusCode::NOT_FOUND);
}
}
-4
View File
@@ -71,10 +71,6 @@ pub async fn remove_label(
}
/// GET /api/v1/repos/:owner/:repo/labels
///
/// Read-visibility-gated (INV-2 root listing): a public repo's labels stay
/// anonymously listable; a private repo's label names are hidden (404) from
/// anyone who cannot read it at the root.
pub async fn list_labels(
State(state): State<AppState>,
Path((owner, name)): Path<(String, String)>,
+8 -20
View File
@@ -172,7 +172,6 @@ mod authz_guard {
let issues = include_str!("issues.rs");
let bounties = include_str!("bounties.rs");
let replicas = include_str!("replicas.rs");
let events = include_str!("events.rs");
let tasks = include_str!("tasks.rs");
let stars = include_str!("stars.rs");
let protect = include_str!("protect.rs");
@@ -188,13 +187,6 @@ mod authz_guard {
(pulls, "merge_pr", "require_repo_owner("),
(webhooks, "create_webhook", "require_repo_owner("),
(webhooks, "delete_webhook", "require_repo_owner("),
// Bucket A/B hybrid — list_webhooks is read-visibility THEN owner:
// authorize_repo_read 404s a non-reader of a private repo, then
// require_repo_owner 403s a non-owner of a public one. Both halves
// are pinned: the authorize_repo_read marker guards the read half
// (existence hiding) and require_repo_owner guards the owner half.
(webhooks, "list_webhooks", "authorize_repo_read("),
(webhooks, "list_webhooks", "require_repo_owner("),
(labels, "add_label", "require_repo_owner("),
(labels, "remove_label", "require_repo_owner("),
// Bucket A' — owner OR author (did_matches against the author)
@@ -209,18 +201,10 @@ mod authz_guard {
(bounties, "create_bounty", "authorize_repo_read("),
(repos, "fork_repo", "authorize_repo_read("),
// get_by_cid resolves each candidate (provenance path + legacy scan) through
// the shared `gate_and_serve`; the gate markers themselves are asserted
// below. This row proves the delegation is real, so the gate is actually
// reached rather than dead code. The delegated gate still calls
// `visibility_check` directly and never `authorize_repo_read`, so it keeps
// the property the pre-merge marker enforced: no fuzzy re-resolve.
// the shared `gate_and_serve` (#173 round 2); the gate markers themselves are
// asserted below. This row proves the delegation is real — the gate is
// actually reached, not dead code.
(ipfs, "get_by_cid", "gate_and_serve("),
// Sibling read surfaces: gate private-repo metadata on read visibility
// (public repos stay anonymous; private repos 404).
(replicas, "list_replicas", "authorize_repo_read("),
(protect, "list_protected_branches", "authorize_repo_read("),
(labels, "list_labels", "authorize_repo_read("),
(events, "list_repo_events", "authorize_repo_read("),
// Bucket C — signer-self: the acting DID is matched/bound to auth.0
(tasks, "create_task", "did_matches("),
(tasks, "claim_task", "did_matches("),
@@ -543,7 +527,11 @@ mod authz_guard {
// Repo-scoped reads known to be ungated today, each tracked by an issue.
// Remove an entry the moment its gate lands (the staleness assert enforces it).
let known_ungated: &[(&str, &str)] = &[];
let known_ungated: &[(&str, &str)] = &[
("list_webhooks", "#94 (PR #113)"),
("list_replicas", "PR #113"),
("list_protected_branches", "PR #113"),
];
let is_known = |n: &str| known_ungated.iter().any(|(k, _)| *k == n);
// Any one of these = the handler binds the caller to an authz decision: the
// first two are read gates, the rest are the write/owner/self-binding forms.
File diff suppressed because it is too large. Load diff
+5 -7
View File
@@ -79,14 +79,12 @@ pub async fn unprotect_branch(
pub async fn list_protected_branches(
State(state): State<AppState>,
Path((owner, repo)): Path<(String, String)>,
auth: Option<Extension<AuthenticatedDid>>,
) -> Result<Json<serde_json::Value>> {
// Read-visibility-gated (INV-2 root listing): a public repo's protected
// branches stay anonymously listable; a private repo's branch names are
// hidden (404) from anyone who cannot read it at the root.
let caller = auth.as_ref().map(|e| e.0 .0.as_str());
let (record, _rules) =
crate::api::authorize_repo_read(&state, &owner, &repo, caller, "/").await?;
let record = state
.db
.get_repo(&owner, &repo)
.await?
.ok_or_else(|| AppError::RepoNotFound(format!("{owner}/{repo}")))?;
let branches = state.db.list_protected_branches(&record.id).await?;
+6 -7
View File
@@ -113,17 +113,16 @@ pub async fn unregister_replica(
}
/// GET /api/v1/repos/:owner/:repo/replicas
/// Read-visibility-gated: a PUBLIC repo's replica list stays anonymously
/// listable (mirror-discovery), but a PRIVATE repo's replica URLs are hidden
/// (404) from anyone who cannot read the repo at its root.
/// Public — returns the list of replicas (DID + URL + registration timestamp).
pub async fn list_replicas(
State(state): State<AppState>,
Path((owner, repo)): Path<(String, String)>,
auth: Option<Extension<AuthenticatedDid>>,
) -> Result<Json<serde_json::Value>> {
let caller = auth.as_ref().map(|e| e.0 .0.as_str());
let (record, _rules) =
crate::api::authorize_repo_read(&state, &owner, &repo, caller, "/").await?;
let record = state
.db
.get_repo(&owner, &repo)
.await?
.ok_or_else(|| AppError::RepoNotFound(format!("{owner}/{repo}")))?;
let replicas = state.db.list_replicas(&record.id).await?;
File diff suppressed because it is too large. Load diff
+8 -22
View File
@@ -1,8 +1,8 @@
//! Webhook CRUD API.
//!
//! POST /api/v1/repos/:owner/:repo/hooks — create (owner only)
//! GET /api/v1/repos/:owner/:repo/hooks — list (owner only; auth required)
//! DELETE /api/v1/repos/:owner/:repo/hooks/:id — delete (owner only)
//! POST /api/v1/repos/:owner/:repo/hooks — create (auth required)
//! GET /api/v1/repos/:owner/:repo/hooks — list
//! DELETE /api/v1/repos/:owner/:repo/hooks/:id — delete (auth required)
use axum::extract::{Extension, Path, State};
use axum::http::StatusCode;
@@ -73,26 +73,12 @@ pub async fn create_webhook(
pub async fn list_webhooks(
State(state): State<AppState>,
Path((owner, name)): Path<(String, String)>,
auth: Option<Extension<AuthenticatedDid>>,
) -> Result<Json<serde_json::Value>> {
// This route sits on `optional_signature`, so the DID is optional. Webhook
// callback URLs are owner-secret config and there is no anonymous form, so a
// headerless caller is rejected before any lookup (401, which fires for an
// existing-private and an absent repo alike, so it leaks no existence).
let Some(Extension(AuthenticatedDid(caller))) = auth else {
return Err(AppError::Unauthorized(
"listing webhooks requires authentication".into(),
));
};
// Read-visibility first, then owner. authorize_repo_read returns 404 on a
// visibility deny, so a non-reader of a private repo cannot tell it exists
// (uniform with the sibling read surfaces); require_repo_owner then yields
// 403 only for a non-owner of a public/readable repo, where existence is not
// secret.
let (record, _rules) =
crate::api::authorize_repo_read(&state, &owner, &name, Some(&caller), "/").await?;
crate::api::require_repo_owner(&record, &caller)?;
let record = state
.db
.get_repo(&owner, &name)
.await?
.ok_or_else(|| AppError::RepoNotFound(format!("{owner}/{name}")))?;
let hooks = state.db.list_webhooks(&record.id).await?;
// Redact secrets in list response
-8
View File
@@ -490,7 +490,6 @@ mod tests {
use clap::Parser;
let keypair = Keypair::generate();
let scan_token_key = crate::state::AppState::derive_scan_token_key(&keypair);
let (ref_tx, _) = tokio::sync::broadcast::channel(1);
let (task_tx, _) = tokio::sync::broadcast::channel(1);
let pool = sqlx::postgres::PgPoolOptions::new()
@@ -521,11 +520,6 @@ mod tests {
ipfs_work_rate_limiter: RateLimiter::new(600, Duration::from_secs(3600)),
ipfs_max_history_walks: crate::api::ipfs::MAX_HISTORY_WALKS_PER_REQUEST,
ipfs_max_legacy_probes: crate::api::ipfs::MAX_LEGACY_PROBES_PER_REQUEST,
ipfs_legacy_scan_page_rows: crate::api::ipfs::LEGACY_SCAN_PAGE_ROWS,
ipfs_max_legacy_scan_rows: crate::api::ipfs::MAX_LEGACY_SCAN_ROWS_PER_REQUEST,
ipfs_max_legacy_scan_rule_bytes:
crate::api::ipfs::MAX_LEGACY_SCAN_RULE_BYTES_PER_REQUEST,
ipfs_scan_token_key: Arc::new(scan_token_key),
ipfs_max_served_object_bytes: crate::api::ipfs::MAX_SERVED_OBJECT_BYTES,
push_limiter_trust: crate::rate_limit::TrustedProxy::None,
sync_trigger_rate_limiter: RateLimiter::new(60, Duration::from_secs(3600)),
@@ -535,9 +529,7 @@ mod tests {
git_write_semaphore: Arc::new(tokio::sync::Semaphore::new(64)),
git_push_advert_semaphore: Arc::new(tokio::sync::Semaphore::new(64)),
git_encrypt_semaphore: Arc::new(tokio::sync::Semaphore::new(64)),
pin_semaphore: Arc::new(tokio::sync::Semaphore::new(64)),
encrypt_inflight: crate::state::EncryptInflight::new(),
repo_write_leases: crate::state::RepoWriteLeases::new(8),
git_read_per_caller: crate::rate_limit::PerCallerConcurrency::with_default_max_keys(16),
git_push_advert_per_caller:
crate::rate_limit::PerCallerConcurrency::with_default_max_keys(8),
+56 -732
View File
@@ -1,33 +1,6 @@
use clap::Parser;
use std::path::PathBuf;
/// Upper bound on `git_service_timeout_secs`, `ipfs_request_budget_secs`, and
/// `ipfs_resolve_budget_secs`, in seconds (100 years).
///
/// Three consumers now, so a future tightening moves all of them. `ipfs_request_budget_secs`
/// and `ipfs_resolve_budget_secs` derive only the `Instant` addition in `get_by_cid`, not the
/// lease-steal multiply below, but they share this ceiling because the defect class and the
/// "set it very large to disable" contract are the same.
///
/// The knob is not just stored, it is arithmetic input: the write path derives the
/// per-repo lease steal bound from it (`* 2 + 60`), and #174 routed it into
/// `build_filtered_pack` and `blob_paths`, which each build a deadline as
/// `Instant::now() + Duration::from_secs(this)`. That addition panics on overflow in
/// RELEASE as well as debug, so an unbounded `u64` here turns an operator typo into a
/// serve-path crash rather than a very long timeout. Bounding at parse time keeps every
/// derived duration in range at once, instead of hardening each site as it is found.
///
/// The ceiling is representability, NOT a policy view of a sane timeout, and that
/// distinction is what sets the number. The help text has always told operators to set
/// this very large to disable the bound, so values like `999999999` (~31 years) are
/// working production "off" settings; a tighter, tidier cap would fail those nodes at
/// boot on upgrade over a value that was never the defect. 100 years clears every such
/// setting while staying a factor of about 5.85 under the ~584-year ceiling of a
/// `u64`-nanosecond `Instant`, which is the tightest representation on any platform we
/// build for. Every value that worked before still parses; only the ones that would have
/// panicked are rejected.
pub const GIT_SERVICE_TIMEOUT_SECS_MAX: u64 = 100 * 365 * 24 * 60 * 60;
#[derive(Parser, Debug, Clone)]
#[command(name = "gitlawb-node", about = "gitlawb node daemon", version)]
pub struct Config {
@@ -78,26 +51,10 @@ pub struct Config {
/// Require the authenticated pusher to be the repo owner on `git-receive-pack`.
/// Authentication (a valid did:key signature) is not authorization on its own:
/// any party can mint a did:key and sign as it, so with this off every signed
/// caller may push to every repository, private ones included. On by default.
///
/// Turn it off only for a rolling upgrade whose pushers are not yet the repo
/// owner. Both `GITLAWB_ENFORCE_OWNER_PUSH=false` and `--enforce-owner-push
/// false` disable it, and the bare `--enforce-owner-push` form still means
/// `true`.
///
/// The value-taking action is what makes the CLI form parse at all: as a
/// presence-only flag, `--enforce-owner-push false` is an "unexpected argument"
/// error. The env form resolved correctly either way, so it is the CLI escape
/// hatch this buys, not the environment one.
#[arg(
long,
env = "GITLAWB_ENFORCE_OWNER_PUSH",
action = clap::ArgAction::Set,
num_args = 0..=1,
default_value_t = true,
default_missing_value = "true"
)]
/// any party can sign as their own DID. When true, pushes whose authenticated
/// DID is not the repo owner are rejected. Keep false during rolling upgrades;
/// flip it on once owners are ready for owner-only writes.
#[arg(long, env = "GITLAWB_ENFORCE_OWNER_PUSH", default_value_t = false)]
pub enforce_owner_push: bool,
/// URL of local IPFS/Kubo node HTTP API (e.g. http://127.0.0.1:5001)
@@ -213,19 +170,16 @@ pub struct Config {
/// Maximum wall-clock time a single served git operation (upload-pack /
/// receive-pack through `run_git_service`) may run before it is aborted and
/// its process group torn down, in seconds. Bounds a git that neither
/// finishes nor disconnects. Must be positive and at most
/// [`GIT_SERVICE_TIMEOUT_SECS_MAX`] (100 years, the largest value every derived
/// deadline can represent); setting it very large is still the way to disable the
/// bound. Default: 600s (10 min), generous for large clones. Also bounds the ref
/// advertisement
/// (`info/refs`) and the withheld-blob pack build (`upload_pack_excluding`'s
/// pack-objects stage), which now share the same timeout + process-group
/// teardown (#174).
/// finishes nor disconnects. Must be positive; set it very large to
/// effectively disable the bound. Default: 600s (10 min), generous for large
/// clones. Also bounds the ref advertisement (`info/refs`) and the withheld-blob
/// pack build (`upload_pack_excluding`'s pack-objects stage), which now share the
/// same timeout + process-group teardown (#174).
#[arg(
long,
env = "GITLAWB_GIT_SERVICE_TIMEOUT_SECS",
default_value_t = 600,
value_parser = clap::value_parser!(u64).range(1..=GIT_SERVICE_TIMEOUT_SECS_MAX)
value_parser = clap::value_parser!(u64).range(1..)
)]
pub git_service_timeout_secs: u64,
@@ -252,15 +206,11 @@ pub struct Config {
/// Maximum connections in the PostgreSQL pool. This is a cap, not a floor
/// (connections open lazily). Size against the database server's
/// max_connections, remembering admin tooling opens its own pool. Each
/// concurrent write pins one pooled connection for its whole duration (the
/// advisory lock in `repo_store::acquire_write` is connection-affine), so this
/// must exceed `max_concurrent_git_pushes` by `DB_POOL_APP_HEADROOM` or slow
/// pushes starve every other DB path — enforced by `Config::validate`.
/// max_connections, remembering admin tooling opens its own pool.
#[arg(
long,
env = "GITLAWB_DB_MAX_CONNECTIONS",
default_value_t = 48,
default_value_t = 20,
value_parser = clap::value_parser!(u32).range(1..)
)]
pub db_max_connections: u32,
@@ -361,8 +311,8 @@ pub struct Config {
/// CONNECTION BUDGET. A push holds a Postgres connection from the node's separate
/// advisory-lock pool for the whole receive-pack, and that pool is sized from this
/// knob (this value + 8, clamped to 64 in `main.rs`). The node's total ceiling is
/// therefore `db_max_connections` (default 48) + the lock pool (default 40), i.e.
/// 88 by default, and at most `db_max_connections` + 64. Size BOTH against the
/// therefore `db_max_connections` (default 20) + the lock pool (default 40), i.e.
/// 60 by default, and at most `db_max_connections` + 64. Size BOTH against the
/// database server's `max_connections`: `db_max_connections`' own doc predates the
/// lock pool and no longer covers most of the node's connections. The +8 headroom
/// is shared with the three non-push `acquire_write` callers (`api/issues.rs` x2,
@@ -376,52 +326,6 @@ pub struct Config {
)]
pub max_concurrent_git_pushes: usize,
/// Maximum number of pushes that may be PARKED on one repo's in-process write lease
/// at once. Same-repo pushes are serialized (block-and-wait), and a parked push has
/// already had its entire pack buffered by axum, so an unbounded queue on a contended
/// repo is unbounded buffered memory held for up to `git_service_timeout_secs * 2 +
/// 60` (1260s at defaults). Past this cap the newest push sheds a clean 503 +
/// Retry-After instead of joining the queue.
///
/// The trade: raising it lets more same-repo pushes wait their turn (fewer 503s for a
/// hot repo, more memory pinned by waiters); lowering it sheds sooner. Only pushes to
/// the SAME repo count, and only ones parked right now, so the cap can never deny a
/// push to a different repo. The holder is deliberately not counted: a holder whose
/// cleanup never ran would otherwise pin a slot forever and wedge the repo, which is
/// the failure the `steal_after` reclaim exists to survive.
///
/// Default: 8, a quarter of the default `max_concurrent_git_pushes` (32). Raising the
/// push pool does not raise this; set it explicitly. Must be between 1 and 1_048_576.
#[arg(
long,
env = "GITLAWB_REPO_LEASE_MAX_WAITERS",
default_value_t = 8,
value_parser = clap::builder::RangedU64ValueParser::<usize>::new().range(1..=1_048_576)
)]
pub repo_lease_max_waiters: usize,
/// Max concurrent post-push pin loops (`ipfs_pin` and `pinata`
/// `pin_new_objects`) across all repos. `EncryptInflight` bounds the outstanding
/// pin-task COUNT to one per repo, but each pin loop holds a full per-push
/// object-id list (up to `git_max_pack_bytes` worth of OIDs) while it walks it,
/// so N distinct repos could hold N such lists at once. This caps how many run
/// concurrently (#174 F6). Beyond it a pin loop DEFERS (waits) and never drops,
/// since a dropped pin would lose the object's replication copy.
///
/// It does not cap the memory itself: the local IPFS path builds its list before
/// taking a permit, so tasks parked on this pool still hold theirs, and how many
/// park is capped only per repo. Lowering this knob bounds concurrent pinning, not
/// how much an actor pushing to many repos can retain.
///
/// Default: 8. Must be between 1 and 1_048_576.
#[arg(
long,
env = "GITLAWB_MAX_CONCURRENT_PIN_TASKS",
default_value_t = 8,
value_parser = clap::builder::RangedU64ValueParser::<usize>::new().range(1..=1_048_576)
)]
pub max_concurrent_pin_tasks: usize,
/// Maximum concurrent read operations (`upload-pack` and the upload-pack
/// `info/refs` advertisement) a single caller may hold at once, so one caller
/// cannot monopolize the `max_concurrent_git_ops` read pool (#174). Callers are
@@ -481,181 +385,22 @@ pub struct Config {
/// Per-request ceiling on the number of legacy (NULL-provenance) repos the
/// `/ipfs/{cid}` resolver's scan fallback will PROBE (`acquire` + `git cat-file
/// -t`) before giving up. The provenance path targets its recorded sources; the
/// legacy scan, absent this bound, fans one anonymous request out to O(repos)
/// subprocess spawns and cold-cache fetches for a CID enumerable from the public
/// pins index. A truncated scan surfaces as a retryable 503, never a false 404.
/// Wired into `AppState::ipfs_max_legacy_probes` at construction. This knob does
/// not govern the history-walk ceiling; see `ipfs_max_repos_walked` for that.
/// Must be between 1 and 1_048_576. Default: 256.
#[arg(
long,
env = "GITLAWB_IPFS_MAX_LEGACY_PROBES",
default_value_t = crate::api::ipfs::MAX_LEGACY_PROBES_PER_REQUEST as usize,
value_parser = clap::builder::RangedU64ValueParser::<usize>::new().range(1..=1_048_576)
)]
pub ipfs_max_legacy_probes: usize,
/// Per-request ceiling on how many repo ROWS the `/ipfs/{cid}` resolver's legacy
/// scan may fetch from the database. The probe ceiling above only starts counting
/// once a probe runs, and the two denial classes that dominate a hostile inventory
/// (quarantine, and a root-scope visibility deny) return before a probe or a visit
/// is spent, so without this an all-denying node paged its ENTIRE repo table for one
/// anonymous request while holding a scarce walk permit.
///
/// Reach bound: a holder buried past the ceiling is servable in
/// `ceil(repos / ceiling) + 1` token-echoing retries. A truncated scan sheds a
/// retryable 503 carrying a sealed continuation token; the caller echoes it as
/// `?scan=` and the scan resumes where it stopped. No server-side scan state.
///
/// Floor coupling: raising this knob raises every caller's per-window `/ipfs` work
/// allowance whenever the route limit sits below the derived floor, because the
/// floor must fit one full deep scan's page toll (see `AppState::ipfs_work_budget`).
///
/// Tuning DOWN trade: token presence is a coarse inventory-size oracle. A ceiling
/// truncation emits a token and a wrapped scan does not, so laddering to the
/// `scan-wrapped` taint tells an anonymous caller the node's total repo count,
/// private and quarantined included, to within one ceiling. Tolled and coarse at
/// the 2048 default; it sharpens as the ceiling is lowered.
///
/// Must be between 1 and 1_048_576. Default: 2048.
#[arg(
long,
env = "GITLAWB_IPFS_MAX_LEGACY_SCAN_ROWS",
default_value_t = crate::api::ipfs::MAX_LEGACY_SCAN_ROWS_PER_REQUEST,
value_parser = clap::builder::RangedU64ValueParser::<usize>::new().range(1..=1_048_576)
)]
pub ipfs_max_legacy_scan_rows: usize,
/// Upper bound on the number of EXPENSIVE visibility walks
/// (`allowed_blob_set_for_caller_bounded`, a full-history git walk in a
/// blocking thread) a single `/ipfs/{cid}` request may run. Only a blob in a
/// path-scoped repo costs a walk, so the cap counts exactly those candidates
/// — cheap probe-only visits are bounded by `ipfs_max_repo_visits` instead
/// (counting them here would starve a plain public copy past the cap out of
/// its 200). On exhaustion the walk-needing repo is skipped WITHOUT a verdict
/// and the scan continues; if the request then finds the object nowhere it
/// sheds a retryable 503 + Retry-After rather than misreport existing content
/// absent with a 404. The handler still short-circuits the moment it serves.
/// Must be between 1 and 1_048_576. Default: 64.
///
/// The effective ceiling is the TIGHTER of this knob and the node's internal
/// history-walk ceiling, `MAX_PIN_SOURCES + 1` = 17 (see
/// `api::ipfs::MAX_HISTORY_WALKS_PER_REQUEST` and the `min()` that combines the
/// two in the resolver). Setting this above 17 changes nothing, because the
/// internal ceiling already binds. Setting it below 17 does lower the cap: the
/// constant side of the `min()` is what keeps a request from being truncated
/// before its whole bounded provenance source set has been tried, so an operator
/// who goes under it is choosing a tighter cap that can 503 a provenanced
/// request, which is allowed.
///
/// That combined cap is charged PER PHASE, not per request: the provenance phase and
/// the legacy-scan fallback each get their own equal budget, so one request can run
/// up to twice it in total (see `MAX_HISTORY_WALKS_PER_REQUEST`, which explains why
/// the split is what keeps the fallback from inheriting a provenance phase's spent
/// remainder).
/// -t`) before giving up. The provenance path targets one repo; the legacy scan,
/// absent this bound, fans one anonymous request out to O(repos) subprocess spawns
/// and cold-cache fetches for a CID enumerable from the public pins index (#173,
/// INV-10). A truncated scan surfaces as a retryable 503, never a false 404. Wired
/// into `AppState::ipfs_max_legacy_probes` at construction; the history-walk ceiling
/// stays constant (`MAX_HISTORY_WALKS_PER_REQUEST`) and is NOT governed by this knob
/// (a smaller value would falsely 503 a provenanced request). Must be between 1 and
/// 1_048_576. Default: 256.
#[arg(
long,
env = "GITLAWB_IPFS_MAX_REPOS_WALKED",
default_value_t = 64,
default_value_t = crate::api::ipfs::MAX_LEGACY_PROBES_PER_REQUEST as usize,
value_parser = clap::builder::RangedU64ValueParser::<usize>::new().range(1..=1_048_576)
)]
pub ipfs_max_repos_walked: usize,
/// Ceiling on the number of repos a single `/ipfs/{cid}` request may VISIT —
/// pass the repo-level visibility gate into the acquire + `cat-file` probe.
/// Each visit costs a `RepoStore::acquire` (on a Tigris cache miss that is a
/// full repo-archive download from object storage, so the worst-case
/// object-store fetch count for one request equals this ceiling) plus a git
/// probe subprocess. On exhaustion the scan STOPS — unlike
/// `ipfs_max_repos_walked`, which skips just the walk-needing repo, there is
/// no cheaper way to keep scanning — and the request sheds a retryable 503 +
/// Retry-After rather than a false 404. Must be between 1 and 1_048_576.
/// Default: 1024.
#[arg(
long,
env = "GITLAWB_IPFS_MAX_REPO_VISITS",
default_value_t = 1024,
value_parser = clap::builder::RangedU64ValueParser::<usize>::new().range(1..=1_048_576)
)]
pub ipfs_max_repo_visits: usize,
/// Absolute wall-clock budget for one admitted `GET /ipfs/{cid}` request's
/// acquire+walk lifetime, in seconds. `max_concurrent_ipfs_walks` bounds how
/// MANY requests hold walk slots; this bounds how LONG one admitted request
/// may keep its slot. Without it, each repo iteration draws a fresh
/// `git_acquire_timeout_secs` and each expensive walk a fresh
/// `git_service_timeout_secs`, so one request scanning many repos could hold
/// a scarce walk slot for hours. Every stage (acquire, `cat-file` probe,
/// visibility walk, content read) starts only while budget remains, and the
/// acquire wait and walk deadline are clamped to `min(their own timeout,
/// remaining budget)`; a stage is never started with zero remaining. On
/// exhaustion the scan stops without a verdict and the request sheds a
/// retryable 503 + Retry-After rather than a false 404. The clamps bound
/// only the acquire and walk stages (overshoot there is the walk watchdog's
/// SIGTERM grace + SIGKILL settle); the `object_type` /
/// `read_object_content` probe subprocesses are budget-checked before they
/// start AND each run under their own deadline (the lesser of
/// `git_service_timeout_secs` and the remaining budget), reaped by
/// process-group teardown, so a hung `cat-file` cannot hold the request's walk
/// slot past it. Still unbounded: the probe's `object_store_readable` check is a
/// synchronous filesystem sweep with nothing to reap, so a wedged filesystem can
/// hold the slot past the deadline.
/// Must be positive, and no larger than `GIT_SERVICE_TIMEOUT_SECS_MAX`. The ceiling is
/// representability, NOT a policy view of a sane budget: `get_by_cid` derives the
/// request deadline as `Instant::now() + Duration::from_secs(this)`, and that addition
/// is an explicit overflow check rather than a debug-only one, so a value near the top
/// of the `u64` range aborts every `/ipfs/{cid}` request in a release build instead of
/// setting a very long budget. The ceiling sits well below where that starts, about a
/// factor of 5.85 (see the constant's own note), so it is a conservative margin rather than the
/// exact overflow point; rejecting at parse time keeps the unrepresentable values out
/// of every reachable configuration. Setting it very large is still the way to
/// effectively disable the budget, and the documented sentinels (`999999999`,
/// `1000000000`) are well inside the range.
/// Default: 600s (10 min), matching `git_service_timeout_secs` so a single full-length
/// walk still fits.
#[arg(
long,
env = "GITLAWB_IPFS_REQUEST_BUDGET_SECS",
default_value_t = 600,
value_parser = clap::value_parser!(u64).range(1..=GIT_SERVICE_TIMEOUT_SECS_MAX)
)]
pub ipfs_request_budget_secs: u64,
/// Budget for the PRE-WALK CID resolve inside `get_by_cid`, in seconds: the
/// `oids_for_cid` lookup that maps the requested CID to its git oid(s), which runs
/// while the scarce walk admission (the global pool permit plus the per-source
/// sub-permit) is already held.
///
/// It exists because that one await decides whether the request does any admitted
/// work at all. A syntactically valid CID with no `pinned_cids` row runs zero probes
/// and zero walks, so under a stalled or saturated pool it would otherwise occupy a
/// walk slot for the whole `ipfs_request_budget_secs` window (600s by default) while
/// nothing is walking, and enough distinct source keys doing that reject every real
/// `/ipfs` retrieval at admission. The other repair, resolving the CID before taking
/// admission, was rejected: admission stays FIRST so an anonymous flood sheds before
/// touching the database at all, and moving the read ahead of it would let arbitrarily
/// many unadmitted permissionless callers stack concurrent DB queries.
///
/// The effective deadline is the lesser of this and the remaining request budget, so a
/// value larger than `ipfs_request_budget_secs` degrades to the request budget rather
/// than extending it. Only the resolve is on this clock; every later stage stays on the
/// full request budget, because from the second oid candidate on those run after real
/// probe and walk work and a short deadline anchored at admission would shed a
/// legitimately slow but progressing scan.
///
/// Must be positive, and no larger than `GIT_SERVICE_TIMEOUT_SECS_MAX`, for the same
/// representability reason as the request budget above: `get_by_cid` derives the
/// resolve deadline as `Instant::now() + Duration::from_secs(this)`, and that addition
/// panics on overflow in release builds too. Default: 10s.
#[arg(
long,
env = "GITLAWB_IPFS_RESOLVE_BUDGET_SECS",
default_value_t = 10,
value_parser = clap::value_parser!(u64).range(1..=GIT_SERVICE_TIMEOUT_SECS_MAX)
)]
pub ipfs_resolve_budget_secs: u64,
/// Per-client-IP rate limit for `GET /ipfs/{cid}`, in requests per hour. The
/// route is publicly reachable (`optional_signature`) and each request can drive
/// a full-history git walk, so it carries a per-IP flood brake in addition to the
@@ -718,36 +463,6 @@ impl Config {
}
PathBuf::from(&self.key_path)
}
/// DB connections reserved for everything other than held write-locks: auth
/// lookups, visibility-rule reads, the post-receive tail's own DB writes, and
/// admin tooling. A write pins one pooled connection for its whole duration, so
/// the pool must clear the concurrent-write cap by at least this margin.
pub const DB_POOL_APP_HEADROOM: u32 = 8;
/// Cross-field boot validation. Single-field ranges are enforced by clap; this
/// catches combinations that ship a denial-of-service under otherwise-valid
/// values. Call once at startup and fail fast on `Err`.
pub fn validate(&self) -> Result<(), String> {
// A write pins one pooled connection for its whole duration (the
// connection-affine advisory lock in repo_store::acquire_write), and
// concurrent writes are capped at max_concurrent_git_pushes. If the pool
// does not exceed that cap by DB_POOL_APP_HEADROOM, a burst of slow pushes
// drains every connection and every other DB path 503s. (#174 F1)
let floor = (self.max_concurrent_git_pushes as u64) + (Self::DB_POOL_APP_HEADROOM as u64);
if (self.db_max_connections as u64) < floor {
return Err(format!(
"GITLAWB_DB_MAX_CONNECTIONS ({}) must be at least max_concurrent_git_pushes ({}) \
+ {} headroom = {}: each concurrent write pins one pooled connection for its whole \
duration, so a smaller pool lets a burst of slow pushes starve every other DB path.",
self.db_max_connections,
self.max_concurrent_git_pushes,
Self::DB_POOL_APP_HEADROOM,
floor
));
}
Ok(())
}
}
#[cfg(test)]
@@ -771,88 +486,6 @@ mod tests {
);
}
/// #174 (RED-before/GREEN-after): the upper bound is what keeps every duration
/// derived from this knob in range — the lease steal bound's `* 2 + 60` on the write
/// path, and the `Instant::now() + Duration::from_secs(..)` deadlines in
/// `build_filtered_pack` / `blob_paths` on the serve path, which panic on overflow in
/// release builds too. Checked at parse time so no reachable configuration can carry a
/// value those sites cannot represent.
///
/// The pre-existing "set it very large to disable the bound" settings are asserted
/// alongside the rejections on purpose. The bound exists to exclude unrepresentable
/// values, not to impose a view of a reasonable timeout, so a node that has been
/// running on ~31 years must not start failing at boot on upgrade.
#[test]
fn git_service_timeout_rejects_values_no_derived_duration_can_represent() {
let parse = |secs: u64| {
Config::try_parse_from([
"gitlawb-node",
"--git-service-timeout-secs",
&secs.to_string(),
])
};
// Large "disable the bound" values that predate the ceiling still parse.
for disable in [1_000_000_000, 999_999_999] {
assert_eq!(
parse(disable)
.unwrap_or_else(|e| panic!("{disable} was a working setting: {e}"))
.git_service_timeout_secs,
disable
);
}
// At the ceiling: accepted, and every derived duration still fits.
let at_max =
parse(GIT_SERVICE_TIMEOUT_SECS_MAX).expect("the documented maximum must parse");
assert_eq!(
at_max.git_service_timeout_secs,
GIT_SERVICE_TIMEOUT_SECS_MAX
);
assert!(at_max
.git_service_timeout_secs
.checked_mul(2)
.and_then(|v| v.checked_add(60))
.is_some());
assert!(std::time::Instant::now()
.checked_add(std::time::Duration::from_secs(
at_max.git_service_timeout_secs
))
.is_some());
// Past the ceiling, and the top of the u64 range clap used to accept — the value
// that panics `Instant::now() + Duration::from_secs(..)` outright.
for over in [GIT_SERVICE_TIMEOUT_SECS_MAX + 1, u64::MAX] {
assert!(
parse(over).is_err(),
"{over} is past the representable ceiling and must be rejected at parse time"
);
}
assert!(std::time::Instant::now()
.checked_add(std::time::Duration::from_secs(u64::MAX))
.is_none());
}
#[test]
fn max_concurrent_pin_tasks_defaults_and_rejects_out_of_range() {
assert_eq!(
Config::parse_from(["gitlawb-node"]).max_concurrent_pin_tasks,
8
);
assert_eq!(
Config::parse_from(["gitlawb-node", "--max-concurrent-pin-tasks", "2"])
.max_concurrent_pin_tasks,
2
);
assert!(
Config::try_parse_from(["gitlawb-node", "--max-concurrent-pin-tasks", "0"]).is_err()
);
assert!(
Config::try_parse_from(["gitlawb-node", "--max-concurrent-pin-tasks", "1048577"])
.is_err()
);
}
#[test]
fn max_concurrent_git_ops_defaults_and_rejects_out_of_range() {
assert_eq!(
@@ -978,40 +611,34 @@ mod tests {
);
}
/// The legacy-probe budget and the expensive-walk cap are SEPARATE knobs with
/// different defaults. They were one field until the probe budget and the walk cap
/// were split apart, so assert both defaults here: a future collapse back into one
/// field silently gives one of the two the other's default.
#[test]
fn ipfs_probe_and_walk_knobs_default_apart_and_reject_out_of_range() {
let default = Config::parse_from(["gitlawb-node"]);
assert_eq!(default.ipfs_max_legacy_probes, 256, "legacy-probe budget");
assert_eq!(default.ipfs_max_repos_walked, 64, "expensive-walk cap");
fn ipfs_max_repos_walked_defaults_and_rejects_out_of_range() {
assert_eq!(
Config::parse_from(["gitlawb-node", "--ipfs-max-legacy-probes", "8"])
.ipfs_max_legacy_probes,
Config::parse_from(["gitlawb-node"]).ipfs_max_repos_walked,
256
);
assert_eq!(
Config::parse_from(["gitlawb-node", "--ipfs-max-repos-walked", "8"])
.ipfs_max_repos_walked,
8
);
// 0 would probe no repos (serve nothing); clap must reject it.
assert!(Config::try_parse_from(["gitlawb-node", "--ipfs-max-legacy-probes", "0"]).is_err());
assert!(
Config::try_parse_from(["gitlawb-node", "--ipfs-max-legacy-probes", "1048577"])
.is_err()
);
// 0 would walk no repos (serve nothing); clap must reject it.
assert!(Config::try_parse_from(["gitlawb-node", "--ipfs-max-repos-walked", "0"]).is_err());
assert!(
Config::try_parse_from(["gitlawb-node", "--ipfs-max-repos-walked", "1048577"]).is_err()
);
}
/// The `GITLAWB_IPFS_MAX_LEGACY_PROBES` knob must actually reach the legacy-probe
/// budget it advertises: production seeds `ipfs_max_legacy_probes` from this helper,
/// so the knob is a no-op unless the helper reflects it. RED while the helper returns
/// the hardcoded `MAX_LEGACY_PROBES_PER_REQUEST` (256 regardless of the knob), GREEN
/// once it reads the knob.
/// The `GITLAWB_IPFS_MAX_REPOS_WALKED` knob must actually reach the legacy-probe
/// budget it advertises (R5, KTD5): production seeds `ipfs_max_legacy_probes` from
/// this helper, so the knob is a no-op unless the helper reflects it. RED while the
/// helper returns the hardcoded `MAX_LEGACY_PROBES_PER_REQUEST` (256 regardless of
/// the knob), GREEN once it reads the knob.
#[test]
fn ipfs_max_legacy_probes_wires_the_legacy_probe_budget() {
fn ipfs_max_repos_walked_wires_the_legacy_probe_budget() {
use crate::state::AppState;
// Knob set to 1 → a one-probe legacy budget.
let one = Config::parse_from(["gitlawb-node", "--ipfs-max-legacy-probes", "1"]);
let one = Config::parse_from(["gitlawb-node", "--ipfs-max-repos-walked", "1"]);
assert_eq!(
AppState::ipfs_legacy_probe_budget(&one),
1,
@@ -1038,13 +665,11 @@ mod tests {
}
/// The `/ipfs` work-budget capacity is DERIVED from the route limit (R6, KTD6), with
/// a hard floor of one complete COMBINED resolution per window: the provenance
/// phase's walk term plus a full legacy search (the effective
/// `ipfs_max_legacy_probes` plus the row ceiling's page toll). This guards the
/// derived default so a single default-config deep search never self-throttles
/// mid-scan and recreates the F6 admit-then-429 for a legitimate caller. A
/// `RateLimiter` sized to the derived budget must admit the whole budget back to
/// back.
/// a hard floor of one full legacy search per window (the effective
/// `ipfs_max_legacy_probes`). This guards the derived default so a single
/// default-config deep search never self-throttles mid-scan and recreates the F6
/// admit-then-429 for a legitimate caller. A `RateLimiter` sized to the derived
/// budget must admit the whole probe budget back to back.
#[test]
fn ipfs_work_budget_derives_from_route_limit_and_clears_the_probe_floor() {
use crate::state::AppState;
@@ -1059,80 +684,28 @@ mod tests {
"the work budget must clear one full legacy search per window"
);
// Tight route limit (1): the floor lifts the work budget to one complete
// COMBINED resolution, the 256-probe budget PLUS the page toll a 2048-row
// ceiling costs at 128 rows per page (16) PLUS the provenance phase's walk term
// min(17, 64) = 17, so 289, NOT down to 1. The provenance walks come off the
// same bucket before the fallback runs, so a floor without that term hands the
// legacy search a bucket the provenance phase already spent from. This case
// also carries the walk term's ABOVE-constant direction: the repos-walked knob
// is at its default 64, so `MAX_HISTORY_WALKS_PER_REQUEST` (17) is what binds.
// Tight route limit (1): the floor lifts the work budget to the probe budget
// (256), NOT down to 1 — a single deep search still completes its full scan.
let tight = Config::parse_from(["gitlawb-node", "--ipfs-rate-limit", "1"]);
assert_eq!(
AppState::ipfs_work_budget(&tight),
289,
"a tight route limit is floored at probes + pages + walks \
(256 + 16 + min(17, 64) = 17), not clamped to 1"
);
// The walk term's BELOW-constant direction: a repos-walked knob under the
// history-walk constant is what the resolver's own `walk_cap` min() selects, so
// it is what the floor must carry too. 256 + 16 + min(17, 3) = 275.
let narrow_walk = Config::parse_from([
"gitlawb-node",
"--ipfs-rate-limit",
"1",
"--ipfs-max-repos-walked",
"3",
]);
assert_eq!(
AppState::ipfs_work_budget(&narrow_walk),
275,
"the walk term takes min(17, repos-walked 3) = 3, the resolver's own \
walk_cap, so the floor is 256 + 16 + 3"
256,
"a tight route limit is floored at the 256-probe budget, not clamped to 1"
);
// Raised probe budget lifts the floor with it (the work budget tracks the
// effective probe budget, not the constant). The walk cap here is a SECOND
// below-constant proof at a different pair of values: min(17, 7) = 7, and the
// probe knob is raised at the same time so a floor that folded the two terms
// together (they were one field before the split) reads visibly wrong rather
// than plausibly right.
// effective probe budget, not the constant).
let raised = Config::parse_from([
"gitlawb-node",
"--ipfs-rate-limit",
"10",
"--ipfs-max-legacy-probes",
"1000",
"--ipfs-max-repos-walked",
"7",
"1000",
]);
assert_eq!(
AppState::ipfs_work_budget(&raised),
1023,
"the floor tracks the operator-raised legacy-probe budget (1000) plus the \
default row ceiling's page toll (16) plus the walk term min(17, 7) = 7"
);
// The scan-rows knob is coupled to the floor too, and this EXECUTES the coupling
// rather than describing it: every page the ceiling permits is charged to the
// caller's work bucket, so a raised ceiling that did not lift the floor would
// 429 an honest caller part-way down their own token ladder. 4096 rows at 128
// rows per page is 32 pages, so the floor is 256 + 32 + the default walk term
// of 17.
let wide_scan = Config::parse_from([
"gitlawb-node",
"--ipfs-rate-limit",
"10",
"--ipfs-max-legacy-scan-rows",
"4096",
]);
assert_eq!(
AppState::ipfs_work_budget(&wide_scan),
305,
"raising the row ceiling must raise the work floor by the pages it buys \
(256 probes + 4096/128 = 32 pages + min(17, 64) = 17 walks), or a full \
deep scan self-throttles"
1000,
"the floor tracks the operator-raised legacy-probe budget"
);
// 0 route limit disables the derived bucket too (a 0-capacity limiter admits all).
@@ -1144,8 +717,7 @@ mod tests {
);
// Behavioral floor: a limiter sized to the derived (tight-route) budget admits
// a whole combined resolution's worth of charges back to back for one source,
// then sheds the next.
// the whole probe budget back to back for one source, then sheds the next.
let budget = AppState::ipfs_work_budget(&tight);
let limiter =
crate::rate_limit::RateLimiter::new(budget, std::time::Duration::from_secs(3600));
@@ -1157,7 +729,7 @@ mod tests {
for i in 0..budget {
assert!(
limiter.check("1.2.3.4").await,
"charge {i} of one full combined resolution must be admitted (no mid-scan throttle)"
"probe {i} of one full default-config scan must be admitted (no mid-scan throttle)"
);
}
assert!(
@@ -1167,169 +739,6 @@ mod tests {
});
}
#[test]
fn ipfs_max_repos_walked_defaults_and_rejects_out_of_range() {
assert_eq!(
Config::parse_from(["gitlawb-node"]).ipfs_max_repos_walked,
64
);
assert_eq!(
Config::parse_from(["gitlawb-node", "--ipfs-max-repos-walked", "8"])
.ipfs_max_repos_walked,
8
);
// 0 would walk no repos (serve nothing); clap must reject it.
assert!(Config::try_parse_from(["gitlawb-node", "--ipfs-max-repos-walked", "0"]).is_err());
assert!(
Config::try_parse_from(["gitlawb-node", "--ipfs-max-repos-walked", "1048577"]).is_err()
);
}
#[test]
fn ipfs_max_repo_visits_defaults_and_rejects_out_of_range() {
assert_eq!(
Config::parse_from(["gitlawb-node"]).ipfs_max_repo_visits,
1024
);
assert_eq!(
Config::parse_from(["gitlawb-node", "--ipfs-max-repo-visits", "8"])
.ipfs_max_repo_visits,
8
);
// 0 would visit no repos (serve nothing); clap must reject it.
assert!(Config::try_parse_from(["gitlawb-node", "--ipfs-max-repo-visits", "0"]).is_err());
assert!(
Config::try_parse_from(["gitlawb-node", "--ipfs-max-repo-visits", "1048577"]).is_err()
);
}
#[test]
fn ipfs_request_budget_secs_defaults_to_600_and_rejects_zero() {
assert_eq!(
Config::parse_from(["gitlawb-node"]).ipfs_request_budget_secs,
600
);
assert_eq!(
Config::parse_from(["gitlawb-node", "--ipfs-request-budget-secs", "30"])
.ipfs_request_budget_secs,
30
);
// 0 would expire every /ipfs request at its first stage (unconditional
// 503); clap must reject it.
assert!(
Config::try_parse_from(["gitlawb-node", "--ipfs-request-budget-secs", "0"]).is_err()
);
}
#[test]
fn ipfs_resolve_budget_secs_defaults_to_10_and_rejects_zero() {
assert_eq!(
Config::parse_from(["gitlawb-node"]).ipfs_resolve_budget_secs,
10
);
assert_eq!(
Config::parse_from(["gitlawb-node", "--ipfs-resolve-budget-secs", "3"])
.ipfs_resolve_budget_secs,
3
);
// 0 would shed every /ipfs request at the pre-walk resolve (unconditional
// 503); clap must reject it.
assert!(
Config::try_parse_from(["gitlawb-node", "--ipfs-resolve-budget-secs", "0"]).is_err()
);
// The ceiling is shared with the request budget: at the max it parses and the
// derived deadline is still representable, past it clap rejects.
let at_max = Config::try_parse_from([
"gitlawb-node",
"--ipfs-resolve-budget-secs",
&GIT_SERVICE_TIMEOUT_SECS_MAX.to_string(),
])
.expect("the documented maximum must parse");
assert_eq!(
at_max.ipfs_resolve_budget_secs,
GIT_SERVICE_TIMEOUT_SECS_MAX
);
assert!(std::time::Instant::now()
.checked_add(std::time::Duration::from_secs(
at_max.ipfs_resolve_budget_secs
))
.is_some());
for over in [GIT_SERVICE_TIMEOUT_SECS_MAX + 1, u64::MAX] {
assert!(
Config::try_parse_from([
"gitlawb-node",
"--ipfs-resolve-budget-secs",
&over.to_string(),
])
.is_err(),
"{over} is past the representable ceiling and must be rejected at parse time"
);
}
}
/// #174 (RED-before/GREEN-after): the upper bound is what keeps the deadline derived
/// from this knob in range. `get_by_cid` builds the request budget as
/// `Instant::now() + Duration::from_secs(this)` (api/ipfs.rs), and that addition is an
/// explicit overflow check rather than a debug-only one, so an oversized value aborts
/// every `/ipfs/{cid}` request in a release build instead of setting a very long budget.
/// The route is anon-reachable, so the failure is operator-triggered but publicly felt.
/// Checked at parse time so no reachable configuration can carry a value the deadline
/// cannot represent.
///
/// The large "disable the bound" settings are asserted alongside the rejections on
/// purpose, the same way the `git_service_timeout_secs` sibling does it. The ceiling
/// exists to exclude unrepresentable values, not to impose a view of a reasonable
/// budget, so a node already running on such a value must not start failing at boot on
/// upgrade. A test that only checked the boundary would pass with a far tighter cap.
#[test]
fn ipfs_request_budget_rejects_values_no_derived_duration_can_represent() {
let parse = |secs: u64| {
Config::try_parse_from([
"gitlawb-node",
"--ipfs-request-budget-secs",
&secs.to_string(),
])
};
// Large "disable the bound" values that predate the ceiling still parse.
for disable in [1_000_000_000, 999_999_999] {
assert_eq!(
parse(disable)
.unwrap_or_else(|e| panic!("{disable} was a working setting: {e}"))
.ipfs_request_budget_secs,
disable
);
}
// At the ceiling: accepted, and the derived deadline still fits. This knob feeds
// only the `Instant` addition (no multiply derivation like the lease steal bound),
// so there is no `checked_mul` clause to carry over from the sibling test.
let at_max =
parse(GIT_SERVICE_TIMEOUT_SECS_MAX).expect("the documented maximum must parse");
assert_eq!(
at_max.ipfs_request_budget_secs,
GIT_SERVICE_TIMEOUT_SECS_MAX
);
assert!(std::time::Instant::now()
.checked_add(std::time::Duration::from_secs(
at_max.ipfs_request_budget_secs
))
.is_some());
// Past the ceiling, and the top of the u64 range clap used to accept. Only the
// latter actually panics `Instant::now() + Duration::from_secs(..)`; the ceiling
// sits well below that, which is the conservative margin the constant documents.
for over in [GIT_SERVICE_TIMEOUT_SECS_MAX + 1, u64::MAX] {
assert!(
parse(over).is_err(),
"{over} is past the representable ceiling and must be rejected at parse time"
);
}
assert!(std::time::Instant::now()
.checked_add(std::time::Duration::from_secs(u64::MAX))
.is_none());
}
#[test]
fn max_concurrent_reads_per_caller_defaults_and_rejects_out_of_range() {
assert_eq!(
@@ -1362,89 +771,4 @@ mod tests {
1_048_576
);
}
/// #174 F1: a connection-affine write lock pins a pooled connection per
/// concurrent write, so the pool must clear `max_concurrent_git_pushes` by
/// `DB_POOL_APP_HEADROOM` or a push burst starves every other DB path.
/// `validate()` must reject an under-sized pool at boot.
#[test]
fn db_pool_must_clear_the_git_push_cap() {
// Shipped defaults validate (48 >= 32 + 8).
Config::parse_from(["gitlawb-node"])
.validate()
.expect("default config must validate");
// An under-sized pool relative to the push cap is rejected (20 < 32 + 8).
let under = Config::parse_from([
"gitlawb-node",
"--db-max-connections",
"20",
"--max-concurrent-git-pushes",
"32",
]);
assert!(
under.validate().is_err(),
"db_max_connections 20 below max_concurrent_git_pushes 32 + headroom must be rejected"
);
// Exactly at the floor validates (40 == 32 + 8).
let at_floor = Config::parse_from([
"gitlawb-node",
"--db-max-connections",
"40",
"--max-concurrent-git-pushes",
"32",
]);
assert!(
at_floor.validate().is_ok(),
"db_max_connections at the floor (pushes + headroom) must validate"
);
}
/// The DECLARED default, read off the parser rather than out of a parse.
///
/// `Config::parse_from` consults the process environment, so on a host that
/// exports `GITLAWB_ENFORCE_OWNER_PUSH=false` a parse-based assertion says
/// nothing about what this crate declares — it reports the operator's setting.
/// Asserting the declaration is the env-independent form, and it is the one that
/// actually fails if someone flips `default_value_t` back.
#[test]
fn enforce_owner_push_is_declared_true_independent_of_the_environment() {
use clap::CommandFactory;
let cmd = Config::command();
let arg = cmd
.get_arguments()
.find(|a| a.get_id() == "enforce_owner_push")
.expect("the argument must exist");
assert_eq!(
arg.get_default_values(),
["true"],
"owner-only push must be the declared default; a node started with no \
configuration cannot accept a push from a self-minted key"
);
}
/// The flip must not strand an operator mid-upgrade.
///
/// Turning the gate on is a breaking change for any deployment whose pushers are
/// not yet the repo owner, so the escape hatch has to keep working. As a
/// presence-only flag `--enforce-owner-push false` is not "false", it is an
/// "unexpected argument" error; the value-taking action is what makes that form
/// parse.
///
/// This pins the CLI form only. The environment form resolved to `false` under
/// the presence-only declaration too, so it is not what this change fixed, and it
/// is not exercised here because the process environment is global and these
/// tests run in parallel.
#[test]
fn enforce_owner_push_stays_disableable_for_rolling_upgrades() {
assert!(
!Config::parse_from(["gitlawb-node", "--enforce-owner-push", "false"])
.enforce_owner_push,
"operators must still be able to opt out during a rolling upgrade"
);
assert!(
Config::parse_from(["gitlawb-node", "--enforce-owner-push", "true"]).enforce_owner_push
);
}
}
File diff suppressed because it is too large. Load diff
+1 -1
View File
@@ -170,7 +170,7 @@ pub async fn encrypt_and_pin(
continue;
}
};
let cid = match crate::ipfs_pin::pin_git_object(ipfs_api, oid, &envelope, None).await {
let cid = match crate::ipfs_pin::pin_git_object(ipfs_api, oid, &envelope).await {
Ok(c) if !c.is_empty() => c,
Ok(_) => {
tracing::warn!(oid = %oid, "pin_git_object returned no cid; skipping");
+14 -149
View File
@@ -35,30 +35,14 @@ pub enum AppError {
#[error("invalid request: {0}")]
BadRequest(String),
/// A DID was well-formed enough to carry to a resolver but no verifying key
/// could be derived from it. Its own code rather than plain `bad_request`
/// because the auth middleware already answers `unresolvable_did` for the
/// same failure on a request's keyid, and a client should not have to
/// substring-match a message to tell this apart from the other validation
/// failures on the same route.
#[error("unresolvable did: {0}")]
UnresolvableDid(String),
#[error("too many requests: {0}")]
TooManyRequests(String),
#[error("incomplete: {0}")]
Incomplete(String),
/// A bounded search that could not complete. `continuation`, when present, is the
/// sealed scan position the caller echoes as `?scan=` to resume where the search
/// stopped (#173 round 13, F2). It is AEAD-sealed at the mint site, never plaintext:
/// the row it names is by construction one the caller was denied (INV-13).
#[error("search incomplete: {message}")]
SearchIncomplete {
message: String,
continuation: Option<String>,
},
#[error("search incomplete: {0}")]
SearchIncomplete(String),
#[error("git error: {0}")]
Git(String),
@@ -82,14 +66,6 @@ pub enum AppError {
pub const DB_UNAVAILABLE_CODE: &str = "db_unavailable";
pub const DB_UNAVAILABLE_MESSAGE: &str = "database is temporarily unavailable";
/// Generic client-facing message for `AppError::Internal`. The real error is
/// logged server-side; never put sqlx/anyhow detail in the HTTP body (#226).
pub const INTERNAL_ERROR_MESSAGE: &str = "an internal error occurred";
/// Generic client-facing message for non-unavailable `AppError::Db`. Query /
/// schema errors stay in logs; the HTTP body must not leak them (#226).
pub const DB_ERROR_MESSAGE: &str = "a database error occurred";
/// Connection-level sqlx failures that mean the database is unreachable right
/// now (retryable, 503), as opposed to server-reported query errors.
fn db_unavailable(e: &sqlx::Error) -> bool {
@@ -162,9 +138,6 @@ impl IntoResponse for AppError {
// IcaptchaProofRequired is handled above (it carries extra headers/fields).
AppError::IcaptchaProofRequired { .. } => unreachable!("handled before this match"),
AppError::BadRequest(msg) => (StatusCode::BAD_REQUEST, "bad_request", msg.clone()),
AppError::UnresolvableDid(msg) => {
(StatusCode::BAD_REQUEST, "unresolvable_did", msg.clone())
}
AppError::TooManyRequests(msg) => {
(StatusCode::TOO_MANY_REQUESTS, "rate_limited", msg.clone())
}
@@ -175,10 +148,10 @@ impl IntoResponse for AppError {
// legacy-probe or walk ceiling), distinct from the 404 that asserts a
// definitive not-found: absence was NOT proven, so the caller should
// retry rather than treat it as gone (#173, F2). 503, retryable.
AppError::SearchIncomplete { message, .. } => (
AppError::SearchIncomplete(msg) => (
StatusCode::SERVICE_UNAVAILABLE,
"search_incomplete",
message.clone(),
msg.clone(),
),
AppError::Git(msg) => (StatusCode::INTERNAL_SERVER_ERROR, "git_error", msg.clone()),
// 504, distinct from the 500 git_error and from the read-gate's 404 /
@@ -195,56 +168,27 @@ impl IntoResponse for AppError {
AppError::Overloaded(msg) => {
(StatusCode::SERVICE_UNAVAILABLE, "overloaded", msg.clone())
}
// Opaque body + server log: bare `?` on sqlx paths becomes `AppError::Db`
// via `From`, so this arm (not `Internal`) is the common leak for open
// routes like GET /api/v1/repos and GET /api/v1/peers (#226).
AppError::Db(e) => {
tracing::error!(error = %e, "database error");
(
StatusCode::INTERNAL_SERVER_ERROR,
"db_error",
DB_ERROR_MESSAGE.into(),
)
}
// Opaque body: handlers that map with `.map_err(AppError::Internal)`
// (e.g. GET /ipfs/{cid}) land here; other DB failures usually hit `Db`.
// Log `{e:#}` so context-wrapped anyhow chains keep the leaf cause
// (Display alone is only the outermost layer; see api/repos.rs).
AppError::Internal(e) => {
tracing::error!(error = %format!("{e:#}"), "internal error");
(
StatusCode::INTERNAL_SERVER_ERROR,
"internal_error",
INTERNAL_ERROR_MESSAGE.into(),
)
}
AppError::Db(e) => (StatusCode::INTERNAL_SERVER_ERROR, "db_error", e.to_string()),
AppError::Internal(e) => (
StatusCode::INTERNAL_SERVER_ERROR,
"internal_error",
e.to_string(),
),
};
let mut body = json!({
let body = Json(json!({
"error": code,
"message": message,
});
// A truncated CID search may carry the sealed position the caller echoes as
// `?scan=` to resume. Rendered as a third body field, present only when the
// shed actually left something to resume: a wrapped scan and a throttled
// request both omit it, and its ABSENCE is what tells a caller the ladder is
// over. It is opaque ciphertext; see `gitlawb_core::scan_token`.
if let AppError::SearchIncomplete {
continuation: Some(token),
..
} = &self
{
body["continuation"] = json!(token);
}
}));
let mut resp = (status, Json(body)).into_response();
let mut resp = (status, body).into_response();
// Both retryable 503s advertise when to retry: Overloaded (capacity shed) and
// SearchIncomplete (a bounded CID search cut short by a cap — retry may complete
// it). They ride the shared tail above for body/status, so the header is attached
// here rather than in bespoke early returns, keeping each variant handled once.
if matches!(
self,
AppError::Overloaded(_) | AppError::SearchIncomplete { .. }
AppError::Overloaded(_) | AppError::SearchIncomplete(_)
) {
resp.headers_mut().insert(
axum::http::header::RETRY_AFTER,
@@ -283,83 +227,4 @@ mod tests {
"1"
);
}
/// #226: raw sqlx/DB detail must never appear in the Internal 500 body.
#[tokio::test]
async fn internal_error_body_is_opaque() {
use serde_json::{json, Value};
let leak = "error returned from database: relation \"repos\" does not exist";
let resp = AppError::Internal(anyhow::anyhow!("{leak}")).into_response();
assert_eq!(resp.status(), StatusCode::INTERNAL_SERVER_ERROR);
let bytes = axum::body::to_bytes(resp.into_body(), usize::MAX)
.await
.expect("read body");
let v: Value = serde_json::from_slice(&bytes).expect("json body");
// Exact object: a new `detail` field with different sensitive text must
// also fail, not only a repeat of the original error string.
assert_eq!(
v,
json!({
"error": "internal_error",
"message": INTERNAL_ERROR_MESSAGE,
})
);
}
/// #226: `AppError::Db` query errors (the common `?` path) must also be opaque.
#[tokio::test]
async fn db_error_body_is_opaque() {
use serde_json::{json, Value};
let resp = AppError::Db(sqlx::Error::Protocol(
"error returned from database: column \"is_public\" does not exist".into(),
))
.into_response();
assert_eq!(resp.status(), StatusCode::INTERNAL_SERVER_ERROR);
let bytes = axum::body::to_bytes(resp.into_body(), usize::MAX)
.await
.expect("read body");
let v: Value = serde_json::from_slice(&bytes).expect("json body");
assert_eq!(
v,
json!({
"error": "db_error",
"message": DB_ERROR_MESSAGE,
})
);
}
/// Connection-level failures must stay 503 `db_unavailable`, not collapse
/// into the opaque 500 `db_error` arm if `db_unavailable` loses a variant.
#[tokio::test]
async fn db_pool_timeout_stays_503_unavailable() {
use serde_json::{json, Value};
let resp = AppError::Db(sqlx::Error::PoolTimedOut).into_response();
assert_eq!(resp.status(), StatusCode::SERVICE_UNAVAILABLE);
let bytes = axum::body::to_bytes(resp.into_body(), usize::MAX)
.await
.expect("read body");
let v: Value = serde_json::from_slice(&bytes).expect("json body");
assert_eq!(
v,
json!({
"error": DB_UNAVAILABLE_CODE,
"message": DB_UNAVAILABLE_MESSAGE,
})
);
}
/// #251: bare `?` on anyhow-wrapped sqlx relies on this downcast so a
/// closed pool becomes 503 `db_unavailable`, not 500 `internal_error`.
#[test]
fn pool_closed_via_anyhow_from_is_503_db_unavailable() {
let err: AppError = anyhow::Error::from(sqlx::Error::PoolClosed).into();
let resp = err.into_response();
assert_eq!(resp.status(), StatusCode::SERVICE_UNAVAILABLE);
}
}
+27 -240
View File
@@ -61,119 +61,6 @@ pub fn create_issue(repo_path: &Path, issue_id: &str, json: &str) -> Result<()>
Ok(())
}
/// Issue ids are UUIDs; refs under `refs/gitlawb/issues/` only ever carry
/// `[0-9a-zA-Z_-]` names. Anything else is not a value we pass to git.
fn issue_ref_name_is_safe(ref_name: &str) -> bool {
let Some(id) = ref_name.strip_prefix("refs/gitlawb/issues/") else {
return false;
};
!id.is_empty()
&& id
.chars()
.all(|c| c.is_ascii_alphanumeric() || c == '-' || c == '_')
}
/// Resolve a ref name without putting it on a command line: enumerate refs
/// (constant argv) and match in memory. A nonzero exit propagates, so "cannot
/// determine" is an error rather than a quiet `false`.
fn ref_name_resolves(repo_path: &Path, ref_name: &str) -> Result<bool> {
// allow-unbounded-git: failure-path existence recheck, module convention.
let out = Command::new("git")
.args(["for-each-ref", "--format=%(refname)"])
.current_dir(repo_path)
.output()
.context("failed to run git for-each-ref")?;
if !out.status.success() {
let stderr = String::from_utf8_lossy(&out.stderr);
anyhow::bail!("git for-each-ref failed: {}", stderr.trim());
}
let stdout = String::from_utf8_lossy(&out.stdout);
Ok(stdout.lines().any(|line| line.trim() == ref_name))
}
/// Read the blob behind an issue ref.
///
/// `Ok(None)` means the ref itself is absent. A ref that resolves but whose
/// object cannot be read (corrupt object, unreadable store, a mid-read gc) is
/// an error, not an absence (#426).
///
/// The ref name reaches git on stdin, never argv: `cat-file --batch` answers
/// `<oid> <type> <size>\n<body>\n` for a readable object and
/// `<spec> missing\n` for an absent ref or an unreadable object alike, so the
/// `missing` case is re-resolved by name enumeration to tell them apart.
fn read_issue_blob(repo_path: &Path, ref_name: &str) -> Result<Option<String>> {
if !issue_ref_name_is_safe(ref_name) {
anyhow::bail!("refusing issue ref outside the issues namespace: {ref_name}");
}
// allow-unbounded-git: this module's spawns predate the bounded runner and
// its callers run inside issue handlers that already hold the repo guard.
let mut child = Command::new("git")
.args(["cat-file", "--batch"])
.current_dir(repo_path)
.stdin(std::process::Stdio::piped())
.stdout(std::process::Stdio::piped())
.stderr(std::process::Stdio::piped())
.spawn()
.context("failed to spawn git cat-file --batch")?;
use std::io::Write;
// Reap the child even when the write fails so a stdin error cannot drop an
// unwaited Child and leak a zombie (#53).
let write_result = match child.stdin.take() {
Some(mut stdin) => stdin
.write_all(ref_name.as_bytes())
.and_then(|()| stdin.write_all(b"\n")),
None => Err(std::io::Error::new(
std::io::ErrorKind::BrokenPipe,
"git cat-file --batch stdin unavailable",
)),
};
let output = child
.wait_with_output()
.context("git cat-file --batch failed")?;
write_result.context("failed to write ref name to git cat-file stdin")?;
if !output.status.success() {
let stderr = String::from_utf8_lossy(&output.stderr);
anyhow::bail!("git cat-file --batch failed: {}", stderr.trim());
}
// "<spec> missing\n" covers an absent ref and an unreadable object; only a
// ref that still resolves is the error case.
let missing = format!("{ref_name} missing\n");
if output.stdout.as_slice() == missing.as_bytes() {
return match ref_name_resolves(repo_path, ref_name)? {
true => anyhow::bail!("issue ref resolves but its object is unreadable: {ref_name}"),
false => Ok(None),
};
}
// Header: "<oid> <type> <size>\n" then exactly <size> bytes plus a newline.
let header_end = output
.stdout
.iter()
.position(|b| *b == b'\n')
.context("unexpected git cat-file --batch output")?;
let header = String::from_utf8_lossy(&output.stdout[..header_end]);
let mut parts = header.split(' ');
parts.next();
let obj_type = parts.next().unwrap_or("");
if obj_type != "blob" {
anyhow::bail!("issue ref {ref_name} resolves to a non-blob object");
}
let size: usize = parts
.next()
.and_then(|s| s.parse().ok())
.context("unexpected git cat-file --batch header")?;
let body = &output.stdout[header_end + 1..];
anyhow::ensure!(
body.len() >= size,
"truncated git cat-file --batch body for {ref_name}"
);
Ok(Some(String::from_utf8_lossy(&body[..size]).to_string()))
}
/// List all issue refs and return their JSON content.
pub fn list_issues(repo_path: &Path) -> Result<Vec<String>> {
// List all refs under refs/gitlawb/issues/
@@ -187,11 +74,9 @@ pub fn list_issues(repo_path: &Path) -> Result<Vec<String>> {
.output()
.context("failed to run git for-each-ref")?;
// for-each-ref exits 0 with empty output when no refs match, so a nonzero
// exit is a real enumeration failure, not "no issues yet".
if !list_output.status.success() {
let stderr = String::from_utf8_lossy(&list_output.stderr);
anyhow::bail!("git for-each-ref failed: {}", stderr.trim());
// No issues yet
return Ok(vec![]);
}
let refs_str = String::from_utf8_lossy(&list_output.stdout);
@@ -203,12 +88,16 @@ pub fn list_issues(repo_path: &Path) -> Result<Vec<String>> {
continue;
}
// Read the blob content. A ref that vanished between for-each-ref and
// here is skipped; a ref that resolves but fails to read propagates so
// a degraded object store errors the listing instead of under-reporting.
match read_issue_blob(repo_path, ref_name)? {
Some(content) => issues.push(content),
None => continue,
// Read the blob content
let cat_output = Command::new("git")
.args(["cat-file", "blob", ref_name])
.current_dir(repo_path)
.output()
.context("failed to run git cat-file")?;
if cat_output.status.success() {
let content = String::from_utf8_lossy(&cat_output.stdout).to_string();
issues.push(content);
}
}
@@ -238,12 +127,8 @@ pub fn resolve_issue_id(repo_path: &Path, id_or_prefix: &str) -> Result<Option<S
.output()
.context("failed to run git for-each-ref")?;
// for-each-ref exits 0 with empty output when nothing matches, so a
// nonzero exit is a real enumeration failure (e.g. not a repo), not
// "issue not found".
if !list.status.success() {
let stderr = String::from_utf8_lossy(&list.stderr);
anyhow::bail!("git for-each-ref failed: {}", stderr.trim());
return Ok(None);
}
let output = String::from_utf8_lossy(&list.stdout);
@@ -276,12 +161,8 @@ pub fn close_issue(repo_path: &Path, issue_id: &str) -> Result<Option<String>> {
None => return Ok(None),
};
// A ref that resolved and is now absent was deleted between the two calls;
// any other read failure propagates as an error rather than panic (#426).
let raw = match get_issue(repo_path, &full_id)? {
Some(raw) => raw,
None => return Ok(None),
};
let raw = get_issue(repo_path, &full_id)?
.expect("ref existed in resolve but not in get — should be impossible");
let mut issue: serde_json::Value =
serde_json::from_str(&raw).context("invalid issue JSON in git ref")?;
@@ -300,7 +181,18 @@ pub fn get_issue(repo_path: &Path, issue_id: &str) -> Result<Option<String>> {
};
let ref_name = format!("refs/gitlawb/issues/{full_id}");
read_issue_blob(repo_path, &ref_name)
let cat_output = Command::new("git")
.args(["cat-file", "blob", &ref_name])
.current_dir(repo_path)
.output()
.context("failed to run git cat-file")?;
if !cat_output.status.success() {
return Ok(None);
}
let content = String::from_utf8_lossy(&cat_output.stdout).to_string();
Ok(Some(content))
}
#[cfg(test)]
@@ -402,109 +294,4 @@ mod tests {
let v: serde_json::Value = serde_json::from_str(&updated).unwrap();
assert_eq!(v["status"], "closed");
}
/// Overwrite the loose object behind an issue ref with garbage, so
/// `cat-file` fails while `rev-parse` still resolves the ref (#426).
fn corrupt_issue_object(dir: &TempDir, full_id: &str) {
let oid_out = Command::new("git")
.args(["rev-parse", &format!("refs/gitlawb/issues/{full_id}")])
.current_dir(dir.path())
.output()
.unwrap();
let oid = String::from_utf8(oid_out.stdout)
.unwrap()
.trim()
.to_string();
let obj = dir
.path()
.join(".git/objects")
.join(&oid[..2])
.join(&oid[2..]);
std::fs::remove_file(&obj).unwrap();
std::fs::write(&obj, b"garbage").unwrap();
}
#[test]
fn test_get_and_close_missing_issue_return_none() {
let dir = TempDir::new().unwrap();
init_repo(&dir);
assert_eq!(get_issue(dir.path(), "nosuch00").unwrap(), None);
assert_eq!(close_issue(dir.path(), "nosuch00").unwrap(), None);
}
// #426: a directory that is not a repo makes `cat-file --batch` exit
// nonzero; that failure is an error, not a quiet None.
#[test]
fn test_issue_read_on_non_repo_errors_instead_of_none() {
let dir = TempDir::new().unwrap();
let not_a_repo = dir.path().join("not-a-repo");
std::fs::create_dir_all(&not_a_repo).unwrap();
assert!(read_issue_blob(&not_a_repo, "refs/gitlawb/issues/deadbeef").is_err());
}
// #426: the ref name is constrained to the issues namespace and a safe
// charset before it is fed to git, even on stdin (a newline would smuggle
// a second `--batch` spec).
#[test]
fn test_issue_read_rejects_ref_outside_issues_namespace() {
let dir = TempDir::new().unwrap();
init_repo(&dir);
assert!(read_issue_blob(dir.path(), "refs/heads/main").is_err());
assert!(read_issue_blob(dir.path(), "refs/gitlawb/issues/../x").is_err());
}
// #426: a failed for-each-ref enumeration is an error, not an empty list.
#[test]
fn test_list_issues_on_non_repo_errors_instead_of_empty() {
let dir = TempDir::new().unwrap();
let not_a_repo = dir.path().join("not-a-repo");
std::fs::create_dir_all(&not_a_repo).unwrap();
assert!(list_issues(&not_a_repo).is_err());
}
// #426: the same enumeration failure inside resolve_issue_id must reach
// get_issue/close_issue as an error, not a "not found" None.
#[test]
fn test_get_and_close_on_non_repo_error_instead_of_none() {
let dir = TempDir::new().unwrap();
let not_a_repo = dir.path().join("not-a-repo");
std::fs::create_dir_all(&not_a_repo).unwrap();
assert!(get_issue(&not_a_repo, "nosuch00").is_err());
assert!(close_issue(&not_a_repo, "nosuch00").is_err());
}
// #426: a ref that resolves but whose object is corrupt is a read error,
// not an absent issue. `close_issue` used to `expect()` on that None and
// panic inside a request handler holding the write guard.
#[test]
fn test_corrupt_issue_object_errors_instead_of_panicking() {
let dir = TempDir::new().unwrap();
init_repo(&dir);
let full_id = "bad00000-0000-0000-0000-000000000000";
create_issue(dir.path(), full_id, r#"{"status":"open"}"#).unwrap();
corrupt_issue_object(&dir, full_id);
assert!(get_issue(dir.path(), full_id).is_err());
assert!(close_issue(dir.path(), full_id).is_err());
assert!(list_issues(dir.path()).is_err());
}
#[test]
fn test_list_issues_returns_all() {
let dir = TempDir::new().unwrap();
init_repo(&dir);
create_issue(
dir.path(),
"aaa00000-0000-0000-0000-000000000000",
r#"{"status":"open"}"#,
)
.unwrap();
create_issue(
dir.path(),
"bbb00000-0000-0000-0000-000000000000",
r#"{"status":"open"}"#,
)
.unwrap();
assert_eq!(list_issues(dir.path()).unwrap().len(), 2);
}
}
+1 -221
View File
@@ -282,61 +282,19 @@ pub struct PinCandidateSet {
/// it can never leak because the withheld/fail-closed filter still runs on
/// whatever set is returned. `full_scan` rides on the returned set so the caller
/// knows when the dangling-inclusive filter is required.
///
/// `scan_sem` is the post-receive scan admission pool (`git_encrypt_semaphore`,
/// #174 F4): both git-spawning stages — the per-tip `cat-file` probe + delta
/// rev-list, and the full-scan fallback — run under ONE permit held for the
/// whole blocking scan. The deletion-only fast path (no new tips) spawns no git
/// and never parks.
///
/// `force_full_scan` forces the full-scan fallback regardless of the tips — the
/// coalesced-drain `PendingWork::FullScan` marker (#174 F5). It composes with
/// the env kill-switch (either forces), and it disqualifies the empty-tips fast
/// path: a forced-scan call with no tips must still enumerate the repo, never
/// silently resolve to an empty delta (which would pin nothing and lose the
/// drained pushes' work — the F5 bug shape).
pub async fn resolve_candidates_for_push(
scan_sem: std::sync::Arc<tokio::sync::Semaphore>,
repo_path: PathBuf,
new_tips: Vec<String>,
old_tips: Vec<String>,
git_bin: String,
timeout: Duration,
force_full_scan: bool,
) -> PinCandidateSet {
let force = force_full_scan || self::force_full_scan();
// No-git fast path: a deletion-only push resolves to an empty delta without
// spawning any git child, so it must not park on the scan pool. A forced
// full scan (caller flag or kill-switch) disqualifies it — the scan spawns git.
if new_tips.is_empty() && !force {
tracing::info!(delta = 0usize, repo = %repo_path.display(), "pin candidate set from push delta");
return PinCandidateSet {
candidates: Vec::new(),
full_scan: false,
};
}
// Scan admission (#174 F4): DEFER, never shed — a dropped scan would
// silently under-pin this push. The permit moves into the blocking closure
// so a started scan always completes holding it. Residuals at
// `acquire_scan_permit`.
let permit =
crate::state::acquire_scan_permit(scan_sem, &repo_path, "pin-candidate scan").await;
tokio::task::spawn_blocking(move || {
let _permit = permit;
// ONE shared deadline for the whole scan, per jatmn ("the same deadline").
let deadline = Instant::now() + timeout;
let new_refs: Vec<&str> = new_tips.iter().map(String::as_str).collect();
let old_refs: Vec<&str> = old_tips.iter().map(String::as_str).collect();
// `force` already folds in the env kill-switch, so the forced arm skips the
// delta machinery outright; the unforced arm goes through the normal
// resolver (whose own env read is false here by construction).
let resolved = if force {
tracing::debug!("full scan forced (coalesced-drain marker or kill-switch)");
PinCandidates::FullScanRequired
} else {
resolve_push_delta(&repo_path, &new_refs, &old_refs, &git_bin, deadline)
};
match resolved {
match resolve_push_delta(&repo_path, &new_refs, &old_refs, &git_bin, deadline) {
PinCandidates::Delta(objs) => {
tracing::info!(delta = objs.len(), repo = %repo_path.display(), "pin candidate set from push delta");
PinCandidateSet { candidates: objs, full_scan: false }
@@ -589,13 +547,11 @@ mod tests {
let c1 = repo.commit_file("a.txt", "one\n");
let c2 = repo.commit_file("b.txt", "two\n");
let set = resolve_candidates_for_push(
std::sync::Arc::new(tokio::sync::Semaphore::new(64)),
repo.path.clone(),
vec![c2.clone()],
vec![c1.clone()],
"git".to_string(),
std::time::Duration::from_secs(600),
false,
)
.await;
assert!(!set.full_scan, "happy-path delta is not a full scan");
@@ -620,13 +576,11 @@ mod tests {
.into_iter()
.collect();
let set = resolve_candidates_for_push(
std::sync::Arc::new(tokio::sync::Semaphore::new(64)),
repo.path.clone(),
vec![blob],
vec![],
"git".to_string(),
std::time::Duration::from_secs(600),
false,
)
.await;
assert!(set.full_scan, "non-commit tip is signalled as a full scan");
@@ -634,180 +588,6 @@ mod tests {
assert_eq!(got, all, "non-commit tip falls back to full repo scan");
}
/// F4 defer proof 2: `resolve_candidates_for_push`'s git stages (the per-tip
/// cat-file type probe + delta rev-list, and the full-scan fallback) run under a
/// scan-admission permit: with a zero-permit pool the call parks and spawns no
/// git; once a permit is available the SAME call runs (defer, not shed). On
/// ungated code the git runs regardless of the pool (RED).
#[cfg(unix)]
#[tokio::test]
async fn resolve_candidates_defers_when_scan_pool_exhausted() {
use std::os::unix::fs::PermissionsExt;
use std::sync::Arc;
use std::time::Duration;
use tokio::sync::Semaphore;
let dir = tempfile::TempDir::new().unwrap();
let marker = dir.path().join("git.ran");
// Fake git records ANY invocation; cat-file reports a commit tip so the
// delta stage proceeds, rev-list yields an empty delta.
let fake = dir.path().join("fakegit");
std::fs::write(
&fake,
format!(
"#!/bin/sh\necho ran >> \"{}\"\ncase \"$1\" in\n cat-file) echo commit ;;\n *) : ;;\nesac\nexit 0\n",
marker.display()
),
)
.unwrap();
let mut perm = std::fs::metadata(&fake).unwrap().permissions();
perm.set_mode(0o755);
std::fs::set_permissions(&fake, perm).unwrap();
let git_bin = fake.to_str().unwrap().to_string();
let tip = "deadbeefdeadbeefdeadbeefdeadbeefdeadbeef".to_string();
let sem: Arc<Semaphore> = Arc::new(Semaphore::new(0));
let blocked = tokio::time::timeout(
Duration::from_millis(500),
resolve_candidates_for_push(
sem.clone(),
dir.path().to_path_buf(),
vec![tip.clone()],
vec![],
git_bin.clone(),
Duration::from_secs(5),
false,
),
)
.await;
assert!(
blocked.is_err(),
"the pin-candidate scan must defer (park on admission) when the pool is exhausted"
);
assert!(
!marker.exists(),
"the scan's git must not spawn while its admission permit is unavailable (F4)"
);
// Release admission: the SAME scan now runs (defer, not shed).
sem.add_permits(1);
let set = resolve_candidates_for_push(
sem,
dir.path().to_path_buf(),
vec![tip],
vec![],
git_bin,
Duration::from_secs(5),
false,
)
.await;
assert!(
marker.exists(),
"once admission is available the deferred scan runs its git"
);
assert!(!set.full_scan, "commit tip + empty rev-list is a delta");
assert!(set.candidates.is_empty());
}
/// F4 fast-path negative arm: a deletion-only push (no new tips, kill-switch
/// off) computes its empty delta without spawning ANY git child, so it must
/// complete without acquiring from the scan pool — even at zero permits. The
/// per-tip cat-file probe means every push with a non-empty new tip DOES spawn
/// git, so this is the only genuinely git-free stage.
#[cfg(unix)]
#[tokio::test]
async fn resolve_candidates_no_git_fast_path_skips_admission() {
use std::os::unix::fs::PermissionsExt;
use std::sync::Arc;
use std::time::Duration;
use tokio::sync::Semaphore;
let dir = tempfile::TempDir::new().unwrap();
let marker = dir.path().join("git.ran");
let fake = dir.path().join("fakegit");
std::fs::write(
&fake,
format!("#!/bin/sh\necho ran >> \"{}\"\nexit 0\n", marker.display()),
)
.unwrap();
let mut perm = std::fs::metadata(&fake).unwrap().permissions();
perm.set_mode(0o755);
std::fs::set_permissions(&fake, perm).unwrap();
let set = tokio::time::timeout(
Duration::from_millis(500),
resolve_candidates_for_push(
Arc::new(Semaphore::new(0)),
dir.path().to_path_buf(),
vec![],
vec!["deadbeefdeadbeefdeadbeefdeadbeefdeadbeef".to_string()],
fake.to_str().unwrap().to_string(),
Duration::from_secs(5),
false,
),
)
.await
.expect("the no-new-tips fast path must not park on the scan pool");
assert_eq!(
set,
PinCandidateSet {
candidates: Vec::new(),
full_scan: false
}
);
assert!(!marker.exists(), "the fast path must spawn no git at all");
}
/// #174 F5: the coalesced-drain FullScan marker is signalled via the explicit
/// `force_full_scan` flag, and a flagged call with NO tips must still enumerate
/// the whole repo (full_scan=true, non-empty candidates). The RED arm of the
/// encoding question: if the marker were encoded as a plain empty-tips call
/// (flag off — the fast path), the drain would resolve to an empty delta and
/// pin nothing, silently losing the coalesced pushes' work.
#[tokio::test]
async fn forced_full_scan_with_no_tips_enumerates_the_repo() {
let repo = Repo::new();
repo.commit_file("a.txt", "one\n");
let all: HashSet<String> = list_all_objects(&repo.path, "git", td())
.unwrap()
.into_iter()
.collect();
assert!(!all.is_empty(), "fixture repo has objects");
let set = resolve_candidates_for_push(
std::sync::Arc::new(tokio::sync::Semaphore::new(64)),
repo.path.clone(),
vec![],
vec![],
"git".to_string(),
std::time::Duration::from_secs(600),
true,
)
.await;
assert!(set.full_scan, "the forced call is signalled as a full scan");
let got: HashSet<String> = set.candidates.into_iter().collect();
assert_eq!(
got, all,
"a forced full scan with no tips enumerates the repo — never an empty delta"
);
// The discriminator: the SAME empty-tips call without the flag is the
// deletion-only fast path (empty delta, pin nothing). The two must differ,
// or the marker encoding has collapsed into the silent-loss shape.
let unforced = resolve_candidates_for_push(
std::sync::Arc::new(tokio::sync::Semaphore::new(64)),
repo.path.clone(),
vec![],
vec![],
"git".to_string(),
std::time::Duration::from_secs(600),
false,
)
.await;
assert!(!unforced.full_scan);
assert!(unforced.candidates.is_empty());
}
#[test]
fn missing_oid_tip_forces_full_scan() {
let repo = Repo::new();
File diff suppressed because it is too large. Load diff
+82 -618
View File
@@ -24,9 +24,7 @@ use tokio::process::Command;
///
/// The `be0cdd6` path-scoped upload-pack walk already applies this discipline by
/// moving its permits into the `spawn_blocking`; this generalizes it to the plain
/// (non-path-scoped) `info_refs` / `run_git_service` spawn paths and, via the
/// hand-back contract on `drive_git_child`, across both stages of the filtered
/// (`upload_pack_excluding`) pack build (F1).
/// (non-path-scoped) `info_refs` / `run_git_service` spawn paths.
pub struct AdmissionGuard {
// Boxed so the concrete permit types stay private to the caller (repos.rs) — the
// guard only needs to hold them and drop them, never inspect them. `Send +
@@ -35,29 +33,16 @@ pub struct AdmissionGuard {
_caller: Option<Box<dyn Send + 'static>>,
// Any further work-scoped hold that must outlive the process group; see `with_hold`.
_hold: Option<Box<dyn Send + 'static>>,
// Per-repo write lease (#174 U2/F3), `Some` ONLY on the receive-pack write path
// (via [`with_lease`](Self::with_lease)); `None` on every read path and every
// non-receive-pack write path. It rides this guard into `KillGroupOnDrop`'s detached
// reaper, so on a client disconnect the lease frees only after the disconnected
// push's git group is reaped — serializing a second same-node push against the
// still-writing group. A `RepoWriteLease` clone is `Send + 'static` and holds no pg
// connection, so it travels with the reaper cleanly. A stray `Some` on a READ path
// would wrongly serialize upload-pack against pushes, hence the None-everywhere-else
// discipline.
_lease: Option<crate::state::RepoWriteLease>,
}
impl AdmissionGuard {
/// Take ownership of the global permit and an optional per-caller permit. Both are
/// erased to `Box<dyn Send>` — the guard's only job is to hold them until it drops.
/// No lease is attached here; the receive-pack write path adds one via
/// [`with_lease`](Self::with_lease), so every other call site is lease-free.
pub fn new(global: impl Send + 'static, caller: Option<impl Send + 'static>) -> Self {
Self {
_global: Some(Box::new(global)),
_caller: caller.map(|c| Box::new(c) as Box<dyn Send + 'static>),
_hold: None,
_lease: None,
}
}
@@ -75,13 +60,6 @@ impl AdmissionGuard {
self._hold = Some(Box::new(hold));
self
}
/// Attach the per-repo write lease (#174 U2/F3). Called ONLY on the receive-pack
/// write path, so the lease rides the disconnect reaper; read paths never call this.
pub fn with_lease(mut self, lease: crate::state::RepoWriteLease) -> Self {
self._lease = Some(lease);
self
}
}
/// Handle `GET /:owner/:repo/info/refs?service=git-upload-pack`
@@ -110,12 +88,10 @@ pub async fn info_refs(
.arg("--stateless-rpc")
.arg("--advertise-refs")
.arg(repo_path);
// No request body: advertise-refs does not read stdin.
let (stdout, admission) =
// No request body — advertise-refs does not read stdin. Single stage: the returned
// guard is dropped here (advertise-refs holds admission for its own child only).
let (stdout, _admission) =
drive_git_child(command, Bytes::new(), timeout, "advertise-refs", admission).await?;
// Single-stage op: the advertisement's group is reaped by now; release admission
// rather than holding it across the pure response framing below.
drop(admission);
let content_type = format!("application/x-{service}-advertisement");
@@ -139,18 +115,13 @@ pub async fn info_refs(
///
/// Serves pack data for a clone or fetch. This is stateless — the entire
/// negotiation happens in a single request/response.
///
/// Returns `(response, served_pack)` where `served_pack` is whether the
/// git-upload-pack-result actually delivered a packfile (vs a negotiation-only
/// ACK/NAK round). The handler counts a completed fetch from this outcome rather
/// than from the request's `done` flag (#192 F1).
pub async fn upload_pack(
git_bin: &str,
repo_path: &Path,
request_body: Bytes,
timeout: Duration,
admission: Option<AdmissionGuard>,
) -> Result<(Response, bool)> {
) -> Result<Response> {
let output = run_git_service(
git_bin,
"git-upload-pack",
@@ -161,59 +132,11 @@ pub async fn upload_pack(
)
.await?;
let served_pack = response_served_pack(&output);
let resp = Response::builder()
Ok(Response::builder()
.status(StatusCode::OK)
.header("Content-Type", "application/x-git-upload-pack-result")
.header("Cache-Control", "no-cache")
.body(Body::from(output))?;
Ok((resp, served_pack))
}
/// True when a `git-upload-pack-result` stream actually delivered a packfile,
/// as opposed to a negotiation-only response (ACK/NAK with no pack).
///
/// Lets the fetch-completion metric key off the response outcome instead of the
/// request's `done` flag (#192 F1): a `no-done` fetch that the server answers
/// with `ACK <oid> ready` + pack is a completion; a flush-terminated negotiation
/// round (the server replies only ACK/NAK, or nothing) is not.
///
/// Handles both framings the serve paths emit:
/// - side-band-64k: the pack rides in band-1 (`0x01`) pkt-lines; the first such
/// chunk begins with the `PACK` magic (band-2 progress lines can precede it).
/// - non-sideband: the raw packfile follows the ACK/NAK pkt-lines and begins with
/// the `PACK` magic (which is not valid pkt-line hex, so parsing falls through
/// to the raw-stream check).
///
/// Fails closed (returns false) on a truncated or malformed stream.
pub fn response_served_pack(output: &[u8]) -> bool {
let mut i = 0;
while i + 4 <= output.len() {
let Ok(hdr) = std::str::from_utf8(&output[i..i + 4]) else {
// Not a pkt-line header: the raw (non-sideband) pack stream has begun.
return output[i..].starts_with(b"PACK");
};
let Ok(len) = usize::from_str_radix(hdr, 16) else {
// `PACK` (and any other non-hex 4 bytes) lands here: the raw pack
// follows the NAK pkt-line in the non-sideband framing.
return output[i..].starts_with(b"PACK");
};
// 0000/0001/0002 are flush/delim/response-end markers, no payload.
if len < 4 {
i += 4;
continue;
}
if i + len > output.len() {
return false; // truncated
}
let payload = &output[i + 4..i + len];
// side-band-64k: band 1 carries pack data; its first chunk is the PACK magic.
if payload.first() == Some(&0x01) && payload[1..].starts_with(b"PACK") {
return true;
}
i += len;
}
false
.body(Body::from(output))?)
}
/// Handle `POST /:owner/:repo/git-receive-pack`
@@ -270,12 +193,11 @@ struct KillGroupOnDrop {
// own SIGCHLD-driven orphan reaper.
child: Option<tokio::process::Child>,
pgid: Option<i32>,
// The global + per-source admission permits for this op, if any. repos.rs moves
// its permits in here (directly on the plain spawn paths, via the stage threading
// on the filtered rev-list/pack-objects ones) so admission is released only when
// the group is confirmed reaped, on every exit: complete, timeout, or
// client-disconnect (#174 P1-a, F1). `None` when the op carries no admission
// (e.g. the smart_http test harness).
// The global + per-source admission permits for this op, if any. On the plain
// (non-path-scoped) spawn paths repos.rs moves its permits in here so admission is
// released only when the group is confirmed reaped, on every exit — complete,
// timeout, or client-disconnect (#174 P1-a). The rev-list/pack-objects
// visibility-walk callers hold no admission and pass `None`.
admission: Option<AdmissionGuard>,
}
@@ -288,10 +210,9 @@ impl KillGroupOnDrop {
/// Disarm on the success/timeout path: the body has already reaped the child (its
/// `wait()` returned, or `reap_group_on_timeout` ran), so drop the handle and clear
/// the pgid, leaving the guard's Drop a no-op. Returns the admission guard: the
/// group is already reaped on both callers of this, so the success path hands it
/// back to drive_git_child's caller (a multi-stage build carries it to the next
/// stage) and the timeout path drops it there and then.
/// the pgid, leaving the guard's Drop a no-op. Returns the admission guard so the
/// caller drops it at the earliest provably-free point (the group is already reaped
/// on both callers of this) rather than at end-of-bookkeeping.
fn disarm(&mut self) -> Option<AdmissionGuard> {
self.pgid = None;
self.child = None;
@@ -429,29 +350,27 @@ async fn run_git_service(
.arg(service_to_command(service))
.arg("--stateless-rpc")
.arg(repo_path);
let (out, admission) = drive_git_child(command, input, timeout, service, admission).await?;
// Single-stage op: the child group is already reaped when drive_git_child
// returns, so releasing admission here keeps the permits held exactly for the
// process lifetime, no longer.
drop(admission);
// Single stage: drop the returned guard when this function returns (its permits are
// released once the child's group is reaped, per drive_git_child).
let (out, _admission) = drive_git_child(command, input, timeout, service, admission).await?;
Ok(out)
}
/// Drive a spawned git child under `timeout` with process-group teardown, returning
/// its stdout together with the admission guard. Shared core for
/// [`run_git_service`], [`info_refs`], and the filtered-pack stages: the caller
/// passes a `Command` with its args set; this adds piped stdio and `process_group(0)`.
/// On the deadline the whole group is torn down and reaped before returning
/// [`GitServiceTimeout`]; on a dropped future (client disconnect) the
/// [`KillGroupOnDrop`] guard fires. `input` is written to the child's stdin (empty
/// for the advertise-refs path, which has no request body); `what` labels errors.
/// its stdout AND the disarmed admission guard on success. Shared core for
/// [`run_git_service`] and [`info_refs`]: the caller passes a `Command` with its args
/// set; this adds piped stdio and `process_group(0)`. On the deadline the whole group
/// is torn down and reaped before returning [`GitServiceTimeout`]; on a dropped future
/// (client disconnect) the [`KillGroupOnDrop`] guard fires. `input` is written to the
/// child's stdin (empty for the advertise-refs path, which has no request body); `what`
/// labels errors.
///
/// Admission contract: on success the guard is handed BACK so a multi-stage caller
/// (the filtered-pack build) can carry one admission across consecutive children
/// instead of releasing it between stages. On every Err return (timeout, spawn
/// failure, interaction error, non-zero exit) the guard is dropped internally, and
/// only after the child is reaped: an Err cannot carry the guard, and by then the
/// group is confirmed gone, so releasing admission there is correct.
/// On success the guard is RETURNED rather than dropped internally (#174 KTD3), so a
/// caller running two sequential git stages under one admission — `build_filtered_pack`'s
/// rev-list then pack-objects — can hand the same guard from the first stage to the
/// second and keep the permits held across both, releasing them only when the second
/// stage's process group is reaped. Callers that run a single stage just let the
/// returned guard drop.
async fn drive_git_child(
mut command: Command,
input: Bytes,
@@ -493,10 +412,10 @@ async fn drive_git_child(
pgid,
admission,
};
// On non-unix there is no process-group teardown; the `admission` parameter
// itself stays live for the child's whole interaction (nothing moves it until
// the match below), so the success path hands it back like the unix path and
// the timeout path drops it after the child is waited on.
// On non-unix there is no process-group teardown, so hold the admission guard here
// for the child's whole interaction; it is returned to the caller below.
#[cfg(not(unix))]
let admission_holder = admission;
let mut out = Vec::new();
let mut err = Vec::new();
@@ -530,18 +449,25 @@ async fn drive_git_child(
};
let timed = tokio::time::timeout(timeout, interact).await;
let (write_result, status, admission) = match timed {
// The disarmed admission guard, handed back to the caller on success so it can ride
// a following sequential git stage (#174 KTD3) instead of releasing between stages.
let admission_out: Option<AdmissionGuard>;
let (write_result, status) = match timed {
Ok(result) => {
// The join runs all arms to completion, so the child is reaped: disarm
// before surfacing any interaction error (a read/wait error), else the
// guard's drop would reap an already-reaped child / signal a reused pgid.
// Disarming hands the admission guard back; it rides to the Ok return
// below, and every error return between here and there drops it, always
// AFTER the reap, the earliest provably-free point.
// The guard is returned (not dropped) so a two-stage caller keeps the
// permits held across both stages; a single-stage caller drops it on return.
#[cfg(unix)]
let admission = group_guard.disarm();
let (write_result, status) = result?;
(write_result, status, admission)
{
admission_out = group_guard.disarm();
}
#[cfg(not(unix))]
{
admission_out = admission_holder;
}
result?
}
Err(_elapsed) => {
// Timeout: tear the whole group down and reap it before returning so a
@@ -557,7 +483,6 @@ async fn drive_git_child(
{
let _ = child.start_kill();
let _ = tokio::time::timeout(Duration::from_secs(2), child.wait()).await;
drop(admission);
}
return Err(GitServiceTimeout.into());
}
@@ -566,8 +491,7 @@ async fn drive_git_child(
// Surface git's own failure (its stderr, which the handler may classify as a
// 400) before any stdin-write error: when git rejects a malformed body it
// exits non-zero and closes stdin, so the write's EPIPE would otherwise mask
// the real cause. `admission` drops on these error returns; the child is
// already reaped by the completed join above.
// the real cause.
if !status.success() {
let stderr = String::from_utf8_lossy(&err);
bail!("{what} failed: {stderr}");
@@ -575,7 +499,7 @@ async fn drive_git_child(
write_result.context("failed to write to git stdin")?;
Ok((out, admission))
Ok((out, admission_out))
}
fn service_to_command(service: &str) -> &str {
@@ -619,9 +543,10 @@ async fn rev_list_keep(
command
.args(["rev-list", "--objects", "--all"])
.current_dir(repo_path);
// The filtered-serve caller's admission rides through drive_git_child so a
// disconnect mid-enumeration keeps the permits held until the rev-list group is
// reaped; on success the guard comes back for the pack-objects stage (F1).
// First of `build_filtered_pack`'s two sequential stages: it holds the caller's
// admission for the rev-list child and hands the disarmed guard back so the same
// permits ride the pack-objects stage (#174 KTD3, R2). The inter-stage window holds
// no live git group, so nothing is reaped-late by carrying the guard between them.
let (stdout, admission) =
drive_git_child(command, Bytes::new(), timeout, "rev-list", admission).await?;
let mut keep = Vec::new();
@@ -645,24 +570,20 @@ async fn rev_list_keep(
/// disconnect. An outer `tokio::time::timeout` around a `spawn_blocking` cannot
/// cancel the blocking thread, so neither stage may live off the async side
/// (#174, KTD5).
///
/// `admission` carries the filtered-serve permits across BOTH stages: stage 1 hands
/// it back on success, stage 2 takes it, and the caller receives it after stage 2 so
/// release happens only once the last child group is reaped. On a disconnect the
/// guard rides the active stage's detached reaper instead of dropping with the
/// future (F1). A stage that fails or times out (including a stage 2 whose
/// remaining budget has saturated to zero) drops the guard inside drive_git_child,
/// after that stage's reap.
pub async fn build_filtered_pack(
git_bin: &str,
repo_path: &Path,
withheld: &HashSet<String>,
timeout: Duration,
admission: Option<AdmissionGuard>,
) -> Result<(Vec<u8>, Option<AdmissionGuard>)> {
) -> Result<Vec<u8>> {
// One deadline spans both git stages so a slow rev-list eats into the pack
// budget rather than granting each stage a fresh `timeout` (2x the permit hold).
let deadline = Instant::now() + timeout;
// The caller's admission rides both stages: rev-list holds it, hands it back, then
// pack-objects holds it until ITS process group is reaped on disconnect. That closes
// the path-scoped cap bypass — permits release on group reap, not the instant the
// request future drops (#174 R2, KTD3). The plain upload-pack path already does this.
let (keep, admission) = rev_list_keep(
git_bin,
repo_path,
@@ -677,7 +598,7 @@ pub async fn build_filtered_pack(
command
.args(["pack-objects", "--stdout"])
.current_dir(repo_path);
let (out, admission) = drive_git_child(
let (out, _admission) = drive_git_child(
command,
Bytes::from(data),
deadline.saturating_duration_since(Instant::now()),
@@ -685,7 +606,7 @@ pub async fn build_filtered_pack(
admission,
)
.await?;
Ok((out, admission))
Ok(out)
}
/// Serve a clone/fetch with the withheld blobs removed from the response pack.
@@ -712,24 +633,19 @@ pub async fn build_filtered_pack(
/// instead of a thin delta. Honoring negotiation for smaller fetch packs is an
/// optimization follow-up, not a correctness requirement.
pub async fn upload_pack_excluding(
git_bin: &str,
repo_path: &Path,
request_body: Bytes,
withheld: &HashSet<String>,
timeout: Duration,
admission: Option<AdmissionGuard>,
) -> Result<(Response, bool)> {
// Both filtered-pack stages run async under drive_git_child (duration-bounded,
// process group reaped on disconnect, #174), and `admission` threads through
// them so the caller's permits release only after the active stage's group is
// reaped, never the instant a disconnect drops this future (F1). `git_bin` is
// injectable for the same fake-git testing reason as `run_git_service`
// (production passes the configured git binary).
let (pack, admission) =
build_filtered_pack(git_bin, repo_path, withheld, timeout, admission).await?;
// Both git stages are done and their groups reaped; release admission before the
// pure in-memory response framing below.
drop(admission);
) -> Result<Response> {
// Both git stages are duration-bounded and their process groups are reaped on
// disconnect via drive_git_child (#174), so a hung build no longer pins its
// concurrency slot and a client disconnect no longer orphans the git child. The
// caller's read + per-caller admission is threaded through both stages so the
// permits are held until the pack-objects group is reaped, matching the plain path
// (#174 R2).
let pack = build_filtered_pack("git", repo_path, withheld, timeout, admission).await?;
// The client lists its capabilities on the first `want` line. Honor
// side-band-64k when offered (every modern smart-HTTP client offers it);
@@ -753,19 +669,11 @@ pub async fn upload_pack_excluding(
body.extend_from_slice(&pack);
}
// The filtered path always builds and serves a self-contained pack, so this
// is always true; computing it (rather than hardcoding) keeps the signal
// honest if the framing ever changes. The handler does NOT count off this
// flag alone — a filtered response can't distinguish an accepted fresh clone
// from a rejected mid-negotiation one, so the count is gated on `done` too
// (#192 F2, see `should_count_fetch`).
let served_pack = response_served_pack(&body);
let resp = Response::builder()
Ok(Response::builder()
.status(StatusCode::OK)
.header("Content-Type", "application/x-git-upload-pack-result")
.header("Cache-Control", "no-cache")
.body(Body::from(body))?;
Ok((resp, served_pack))
.body(Body::from(body))?)
}
/// True if `needle` occurs anywhere in `haystack`. Small substring scan used to
@@ -779,87 +687,6 @@ fn memmem(haystack: &[u8], needle: &[u8]) -> bool {
.any(|window| window == needle)
}
/// Build real `git upload-pack --stateless-rpc` result fixtures for tests:
/// `(pack_bearing, negotiation_only)`. The first is a completing clone request
/// (`want` + flush + `done`) which git answers with `NAK` + a packfile; the
/// second is a negotiation round (`want` + flush + an unknown `have` + flush, no
/// `done`) which git answers with `NAK` and no pack. Used to exercise
/// [`response_served_pack`] and the completion-count rule against real protocol
/// output rather than a hand-rolled approximation.
#[cfg(test)]
pub(crate) fn upload_pack_result_fixtures() -> (Vec<u8>, Vec<u8>) {
use std::io::Write as _;
let dir = tempfile::TempDir::new().unwrap();
let work = dir.path().join("work");
let bare = dir.path().join("bare.git");
std::fs::create_dir_all(work.join("sub")).unwrap();
std::fs::write(work.join("a.txt"), b"hello\n").unwrap();
std::fs::write(work.join("sub/b.txt"), b"world\n").unwrap();
let g = |args: &[&str], d: &Path| {
assert!(std::process::Command::new("git")
.args(args)
.current_dir(d)
.status()
.unwrap()
.success());
};
g(&["init", "-q"], &work);
g(&["config", "user.email", "t@t"], &work);
g(&["config", "user.name", "t"], &work);
g(&["add", "."], &work);
g(&["commit", "-qm", "init"], &work);
g(
&[
"clone",
"-q",
"--bare",
work.to_str().unwrap(),
bare.to_str().unwrap(),
],
dir.path(),
);
let sha = {
let o = std::process::Command::new("git")
.args(["rev-parse", "HEAD"])
.current_dir(&work)
.output()
.unwrap();
String::from_utf8_lossy(&o.stdout).trim().to_string()
};
let caps = "multi_ack_detailed side-band-64k thin-pack ofs-delta agent=git/test";
let run = |req: &[u8]| -> Vec<u8> {
let mut child = std::process::Command::new("git")
.args(["upload-pack", "--stateless-rpc"])
.arg(&bare)
.stdin(Stdio::piped())
.stdout(Stdio::piped())
.stderr(Stdio::piped())
.spawn()
.unwrap();
child.stdin.take().unwrap().write_all(req).unwrap();
let out = child.wait_with_output().unwrap();
assert!(
out.status.success(),
"upload-pack failed: {}",
String::from_utf8_lossy(&out.stderr)
);
out.stdout
};
// Completing: want + flush + done -> NAK + pack.
let mut completing = pkt_line(&format!("want {sha} {caps}\n"));
completing.extend_from_slice(b"0000");
completing.extend_from_slice(&pkt_line("done\n"));
let pack_bearing = run(&completing);
// Negotiation-only: want + flush + an unknown have + flush, no done -> NAK,
// no pack.
let mut nego = pkt_line(&format!("want {sha} {caps}\n"));
nego.extend_from_slice(b"0000");
nego.extend_from_slice(&pkt_line("have 0000000000000000000000000000000000000000\n"));
nego.extend_from_slice(b"0000");
let negotiation_only = run(&nego);
(pack_bearing, negotiation_only)
}
#[cfg(test)]
mod tests {
use super::*;
@@ -937,10 +764,9 @@ mod tests {
let mut withheld = std::collections::HashSet::new();
withheld.insert(secret.clone());
let (pack, _admission) =
build_filtered_pack("git", &bare, &withheld, Duration::from_secs(30), None)
.await
.unwrap();
let pack = build_filtered_pack("git", &bare, &withheld, Duration::from_secs(30), None)
.await
.unwrap();
let ids = pack_object_ids(&pack);
assert!(ids.contains(&public), "public blob must be in the pack");
assert!(
@@ -1002,14 +828,9 @@ mod tests {
b"0098want 0000000000000000000000000000000000000000 \
side-band-64k ofs-delta agent=git/2\n00000009done\n",
);
let (resp, served_pack) =
upload_pack_excluding("git", &bare, req, &withheld, Duration::from_secs(30), None)
.await
.unwrap();
assert!(
served_pack,
"the filtered serve path always delivers a pack"
);
let resp = upload_pack_excluding(&bare, req, &withheld, Duration::from_secs(30), None)
.await
.unwrap();
let body = to_bytes(resp.into_body(), usize::MAX).await.unwrap();
let ids = pack_object_ids(&extract_pack(&body));
assert!(
@@ -1068,17 +889,9 @@ mod tests {
axum::extract::State(st): axum::extract::State<std::sync::Arc<FilterState>>,
body: Bytes,
) -> Response {
upload_pack_excluding(
"git",
&st.repo,
body,
&st.withheld,
Duration::from_secs(30),
None,
)
.await
.unwrap()
.0
upload_pack_excluding(&st.repo, body, &st.withheld, Duration::from_secs(30), None)
.await
.unwrap()
}
/// Spawn the server for `bare`, withholding `withheld`. Returns the clone URL
@@ -1605,12 +1418,7 @@ mod tests {
"build_filtered_pack must return within the watchdog — the git stage \
must be timeout-bounded, not an uncancellable spawn_blocking",
);
// `match` rather than `expect_err`: the Ok arm carries an
// AdmissionGuard, which has no Debug impl for expect_err to print.
let err = match result {
Ok(_) => panic!("a hung git stage must return an error, not hang"),
Err(e) => e,
};
let err = result.expect_err("a hung git stage must return an error, not hang");
if is_transient_exec_race(&err) {
// Fresh-fake-git ETXTBSY: back off (growing) so a bursty fork-pressure
// spike subsides before retrying, per fake_git_run_with_pids.
@@ -2036,296 +1844,6 @@ mod tests {
);
}
// ── F1: filtered-serve admission threaded through both pack stages ──────
//
// The handler-level disconnect regression lives in api/repos.rs
// (upload_pack_filtered_permit_held_through_group_reap_after_disconnect); these
// exercise the guard contract at the smart_http seam: held through a
// mid-rev-list disconnect, handed back on success, and recovered (post-reap)
// on every error return.
/// F1: a client disconnect mid-REV-LIST (stage 1 of the filtered build) must
/// keep the threaded admission held until the rev-list group is reaped, and
/// stage 2 (pack-objects) must never spawn, since the dropped future cannot
/// advance to it.
#[cfg(unix)]
#[tokio::test]
async fn build_filtered_pack_holds_admission_through_rev_list_reap_on_disconnect() {
let tmp = tempfile::TempDir::new().unwrap();
let descfile = tmp.path().join("desc.pid");
let packfile = tmp.path().join("pack.ran");
// rev-list: SIGTERM-trapping descendant records its pid and loops (bounded
// so a broken fix leaks no permanent orphan); the leader waits, keeping the
// interaction pending. pack-objects: records that it ran (it must never run).
let body = format!(
"#!/bin/sh\n\
case \"$1\" in\n\
rev-list)\n\
sh -c 'trap \"\" TERM; echo $$ > \"{desc}\"; i=0; while [ $i -lt 20 ]; do sleep 1; i=$((i+1)); done' &\n\
wait ;;\n\
pack-objects) : > \"{pack}\" ;;\n\
*) : ;;\n\
esac\n\
exit 0\n",
desc = descfile.display(),
pack = packfile.display()
);
let git_bin = write_fake_git(tmp.path(), &body);
let sem = std::sync::Arc::new(tokio::sync::Semaphore::new(1));
let guard = AdmissionGuard::new(sem.clone().try_acquire_owned().unwrap(), None::<()>);
let withheld = HashSet::new();
let mut fut = Box::pin(build_filtered_pack(
git_bin.to_str().unwrap(),
tmp.path(),
&withheld,
Duration::from_secs(60),
Some(guard),
));
// Drive until the rev-list descendant records its pid.
let mut desc: Option<i32> = None;
for _ in 0..500 {
let _ = tokio::time::timeout(Duration::from_millis(10), &mut fut).await;
if let Some(p) = std::fs::read_to_string(&descfile)
.ok()
.and_then(|s| s.trim().parse::<i32>().ok())
{
desc = Some(p);
break;
}
}
let desc = desc.expect("the fake rev-list must have spawned its descendant");
let _cleanup = ReapOnPanic(vec![desc]);
assert_eq!(
sem.available_permits(),
0,
"admission held while rev-list runs"
);
// Client disconnect mid-enumeration.
drop(fut);
// Load-bearing: the permit must stay held while the SIGTERM-ignoring group
// member is still alive (check before the reaper's ~2s SIGKILL escalation).
tokio::time::sleep(Duration::from_millis(100)).await;
assert!(
alive(desc),
"the SIGTERM-ignoring descendant must still be alive during the hold window"
);
assert_eq!(
sem.available_permits(),
0,
"on disconnect mid-rev-list the admission must be held until the group is \
reaped, not released the instant the future drops (F1)"
);
let mut freed = false;
for _ in 0..400 {
if sem.available_permits() == 1 {
freed = true;
break;
}
tokio::time::sleep(Duration::from_millis(50)).await;
}
assert!(
freed,
"the reaper must release admission once the rev-list group is gone"
);
assert!(
!packfile.exists(),
"pack-objects must never spawn after a disconnect mid-rev-list"
);
}
/// F1 success path: a filtered serve that completes hands the guard back
/// through both stages and releases admission when the serve returns. No leak,
/// no double-release.
#[tokio::test]
async fn upload_pack_excluding_releases_admission_after_success() {
let td = TempDir::new().unwrap();
let (_work, bare, secret_oid, _public_oid) = fixture_with_secret(&td);
let sem = std::sync::Arc::new(tokio::sync::Semaphore::new(1));
let guard = AdmissionGuard::new(sem.clone().try_acquire_owned().unwrap(), None::<()>);
let withheld = HashSet::from([secret_oid]);
let (resp, _served_pack) = upload_pack_excluding(
"git",
&bare,
Bytes::from_static(b"0000"),
&withheld,
Duration::from_secs(30),
Some(guard),
)
.await
.unwrap();
assert_eq!(resp.status(), StatusCode::OK);
assert_eq!(
sem.available_permits(),
1,
"admission must be released once the filtered serve returns"
);
}
/// F1 error path: a rev-list that exits non-zero fails the build AND recovers
/// the threaded admission: drive_git_child drops the guard only after the
/// completed join has reaped the child, and the error return must not leak it.
#[cfg(unix)]
#[tokio::test]
async fn build_filtered_pack_releases_admission_after_rev_list_failure() {
let tmp = tempfile::TempDir::new().unwrap();
let body =
"#!/bin/sh\ncase \"$1\" in\n rev-list) echo boom >&2; exit 2 ;;\n *) : ;;\nesac\nexit 0\n";
let git_bin = write_fake_git(tmp.path(), body);
let withheld = HashSet::new();
let sem = std::sync::Arc::new(tokio::sync::Semaphore::new(1));
for i in 0..FAKE_GIT_RETRY_ATTEMPTS {
let guard = AdmissionGuard::new(sem.clone().try_acquire_owned().unwrap(), None::<()>);
let result = build_filtered_pack(
git_bin.to_str().unwrap(),
tmp.path(),
&withheld,
Duration::from_secs(30),
Some(guard),
)
.await;
let err = match result {
Ok(_) => panic!("a non-zero rev-list exit must fail the build"),
Err(e) => e,
};
// Load-bearing on EVERY attempt: even a transient spawn-failure return
// must recover the permit (the guard drops before the Err surfaces).
assert_eq!(
sem.available_permits(),
1,
"the admission permit must be recovered on the error return, not leaked"
);
if is_transient_exec_race(&err) {
tokio::time::sleep(Duration::from_millis(FAKE_GIT_BACKOFF_STEP_MS * (i + 1))).await;
continue;
}
assert!(
err.to_string().contains("rev-list failed"),
"the failure must surface rev-list's own error, got: {err}"
);
return;
}
panic!(
"fake git kept hitting ETXTBSY after {FAKE_GIT_RETRY_ATTEMPTS} attempts \
(persistent exec failure, not a transient parallel-runner miss)"
);
}
/// F1: a pack-objects stage that times out (stage 2 of the filtered build,
/// guard threaded through) recovers the admission after the reap: the timeout
/// Err cannot carry the guard, so drive_git_child must drop it post-reap.
#[cfg(unix)]
#[tokio::test]
async fn build_filtered_pack_recovers_admission_after_pack_objects_timeout() {
let tmp = tempfile::TempDir::new().unwrap();
// rev-list returns one oid fast (guard handed back); pack-objects hangs.
let body = "#!/bin/sh\ncase \"$1\" in\n rev-list) echo deadbeefdeadbeefdeadbeefdeadbeefdeadbeef ;;\n pack-objects) sleep 300 ;;\n *) exit 1 ;;\nesac\n";
let git_bin = write_fake_git(tmp.path(), body);
let withheld = HashSet::new();
let sem = std::sync::Arc::new(tokio::sync::Semaphore::new(1));
for i in 0..FAKE_GIT_RETRY_ATTEMPTS {
let guard = AdmissionGuard::new(sem.clone().try_acquire_owned().unwrap(), None::<()>);
let result = tokio::time::timeout(
Duration::from_secs(10),
build_filtered_pack(
git_bin.to_str().unwrap(),
tmp.path(),
&withheld,
Duration::from_millis(200),
Some(guard),
),
)
.await
.expect(
"build_filtered_pack must return within the watchdog; the git stage \
must be timeout-bounded",
);
let err = match result {
Ok(_) => panic!("a hung pack-objects must fail the build"),
Err(e) => e,
};
assert_eq!(
sem.available_permits(),
1,
"admission must be recovered after the stage-2 timeout reap, not leaked"
);
if is_transient_exec_race(&err) {
tokio::time::sleep(Duration::from_millis(FAKE_GIT_BACKOFF_STEP_MS * (i + 1))).await;
continue;
}
assert!(
err.downcast_ref::<GitServiceTimeout>().is_some(),
"a hung pack-objects must abort with GitServiceTimeout, got: {err}"
);
return;
}
panic!(
"fake git kept hitting ETXTBSY after {FAKE_GIT_RETRY_ATTEMPTS} attempts \
(persistent exec failure, not a transient parallel-runner miss)"
);
}
/// F1 zero-budget contract: build_filtered_pack hands pack-objects
/// `deadline.saturating_duration_since(now)`, which saturates to ZERO once
/// rev-list has consumed the whole budget. A zero-budget stage must still route
/// the guard through the timeout's reap-then-drop rather than leak it: the
/// child spawns, the deadline fires immediately, the group is reaped, and only
/// then does the guard drop.
#[cfg(unix)]
#[tokio::test]
async fn drive_git_child_zero_budget_drops_admission_after_reap() {
let tmp = tempfile::TempDir::new().unwrap();
let git_bin = write_fake_git(tmp.path(), "#!/bin/sh\nsleep 300\n");
let sem = std::sync::Arc::new(tokio::sync::Semaphore::new(1));
for i in 0..FAKE_GIT_RETRY_ATTEMPTS {
let guard = AdmissionGuard::new(sem.clone().try_acquire_owned().unwrap(), None::<()>);
let mut command = tokio::process::Command::new(git_bin.to_str().unwrap());
command
.args(["pack-objects", "--stdout"])
.current_dir(tmp.path());
let result = tokio::time::timeout(
Duration::from_secs(10),
drive_git_child(
command,
Bytes::new(),
Duration::ZERO,
"pack-objects",
Some(guard),
),
)
.await
.expect("a zero-budget stage must abort via its own deadline, not hang");
let err = match result {
Ok(_) => panic!("a zero-budget stage must return an error"),
Err(e) => e,
};
assert_eq!(
sem.available_permits(),
1,
"the guard must drop after the zero-budget timeout reap, not leak"
);
if is_transient_exec_race(&err) {
tokio::time::sleep(Duration::from_millis(FAKE_GIT_BACKOFF_STEP_MS * (i + 1))).await;
continue;
}
assert!(
err.downcast_ref::<GitServiceTimeout>().is_some(),
"a zero-budget stage must abort with GitServiceTimeout, got: {err}"
);
return;
}
panic!(
"fake git kept hitting ETXTBSY after {FAKE_GIT_RETRY_ATTEMPTS} attempts \
(persistent exec failure, not a transient parallel-runner miss)"
);
}
// A request that runs to completion must DISARM the guard after reaping, so
// no stray group SIGTERM fires. The fake exits non-zero (surfacing as Err)
// but leaves a grandchild alive; the grandchild must survive. Goes RED if the
@@ -2620,58 +2138,4 @@ mod tests {
stdin-write EPIPE (a generic 500); got: {msg}"
);
}
// ── #192 F1: response_served_pack detector (real git output) ────────────
// A completing request (want+flush+done) yields NAK + a real packfile;
// response_served_pack must detect the delivered pack. A negotiation-only
// round (want+flush+have+flush, no done) yields NAK with no pack and must
// detect false. Fixtures are captured from real `git upload-pack
// --stateless-rpc`, so the detector is validated against protocol output, not
// a hand-rolled approximation.
#[test]
fn response_served_pack_detects_real_pack_vs_negotiation_only() {
let (pack_bearing, negotiation_only) = upload_pack_result_fixtures();
// Sanity-check the fixtures are the shapes we think (a served pack carries
// the PACK magic; a NAK-only round does not).
assert!(
pack_bearing.windows(4).any(|w| w == b"PACK"),
"the completing fixture must carry a packfile"
);
assert!(
!negotiation_only.windows(4).any(|w| w == b"PACK"),
"the negotiation-only fixture must carry no packfile"
);
assert!(
response_served_pack(&pack_bearing),
"a real NAK + pack response must be detected as a served pack"
);
assert!(
!response_served_pack(&negotiation_only),
"a real ACK/NAK-only negotiation round must be detected as no pack"
);
// An empty response (git produced nothing for a want+flush with no done)
// is a negotiation round, not a completion.
assert!(!response_served_pack(b""));
}
// The non-sideband framing (raw pack after the NAK pkt-line) must also detect
// true: the raw PACK magic is not valid pkt-line hex, so parsing falls through
// to the raw-stream check. Built by hand to pin that branch independent of the
// client's advertised capabilities.
#[test]
fn response_served_pack_detects_non_sideband_raw_pack() {
let mut raw = pkt_line("NAK\n");
raw.extend_from_slice(b"PACK\x00\x00\x00\x02\x00\x00\x00\x00");
assert!(
response_served_pack(&raw),
"a raw (non-sideband) pack after NAK must be detected"
);
// NAK with no following pack is a negotiation round.
let nak_only = pkt_line("NAK\n");
assert!(!response_served_pack(&nak_only));
}
}
File diff suppressed because it is too large. Load diff
-19
View File
@@ -31,25 +31,6 @@ impl TigrisClient {
})
}
/// Test-only constructor with an explicit S3 endpoint, region, and static
/// credentials — no env-var reads, so parallel tests cannot race each other's
/// `AWS_*` environment the way the env-based `new` would. Lets a test point
/// the client at a non-routable endpoint to exercise acquire-stall paths.
#[cfg(test)]
pub(crate) async fn for_testing_with_endpoint(bucket: &str, endpoint_url: &str) -> Self {
let creds = aws_sdk_s3::config::Credentials::new("test", "test", None, None, "test");
let config = aws_config::defaults(aws_config::BehaviorVersion::latest())
.endpoint_url(endpoint_url)
.region(aws_config::Region::new("auto"))
.credentials_provider(creds)
.load()
.await;
Self {
s3: S3Client::new(&config),
bucket: bucket.to_string(),
}
}
/// S3 key for a given repo: `repos/v1/{owner_slug}/{repo_name}.tar.zst`
fn repo_key(owner_slug: &str, repo_name: &str) -> String {
format!("repos/v1/{owner_slug}/{repo_name}.tar.zst")
+6 -51
View File
@@ -67,13 +67,13 @@ fn child_terminated_without_reaping(pid: i32) -> bool {
}
#[cfg(unix)]
pub(crate) fn run_bounded_git_raw(
pub(crate) fn run_bounded_git(
git_bin: &str,
args: &[&str],
repo_path: &Path,
stdin_bytes: &[u8],
deadline: Instant,
) -> Result<(std::process::ExitStatus, Vec<u8>, Vec<u8>)> {
) -> Result<Vec<u8>> {
use std::io::{Read, Write};
use std::os::unix::process::CommandExt;
use std::sync::mpsc::RecvTimeoutError;
@@ -191,23 +191,6 @@ pub(crate) fn run_bounded_git_raw(
if killed && !status.success() {
return Err(crate::git::smart_http::GitServiceTimeout.into());
}
Ok((status, out, err))
}
/// Bounded git returning only stdout, `bail!`ing on any nonzero exit. The thin
/// wrapper the walk callers use. Probes that must distinguish exit classes —
/// `git cat-file` absence vs an object-store access failure — call
/// [`run_bounded_git_raw`] and classify the status/stderr themselves.
#[cfg(unix)]
pub(crate) fn run_bounded_git(
git_bin: &str,
args: &[&str],
repo_path: &Path,
stdin_bytes: &[u8],
deadline: Instant,
) -> Result<Vec<u8>> {
let label = args.first().copied().unwrap_or("git");
let (status, out, err) = run_bounded_git_raw(git_bin, args, repo_path, stdin_bytes, deadline)?;
if !status.success() {
anyhow::bail!("git {label} failed: {}", String::from_utf8_lossy(&err));
}
@@ -227,13 +210,13 @@ pub(crate) fn run_bounded_git(
/// the Unix version's signature and result semantics so every caller compiles on all
/// targets (#174).
#[cfg(not(unix))]
pub(crate) fn run_bounded_git_raw(
pub(crate) fn run_bounded_git(
git_bin: &str,
args: &[&str],
repo_path: &Path,
stdin_bytes: &[u8],
deadline: Instant,
) -> Result<(std::process::ExitStatus, Vec<u8>, Vec<u8>)> {
) -> Result<Vec<u8>> {
use std::io::{Read, Write};
use std::sync::mpsc::RecvTimeoutError;
@@ -300,20 +283,6 @@ pub(crate) fn run_bounded_git_raw(
if killed && !status.success() {
return Err(crate::git::smart_http::GitServiceTimeout.into());
}
Ok((status, out, err))
}
/// Non-Unix thin wrapper matching the Unix [`run_bounded_git`] semantics.
#[cfg(not(unix))]
pub(crate) fn run_bounded_git(
git_bin: &str,
args: &[&str],
repo_path: &Path,
stdin_bytes: &[u8],
deadline: Instant,
) -> Result<Vec<u8>> {
let label = args.first().copied().unwrap_or("git");
let (status, out, err) = run_bounded_git_raw(git_bin, args, repo_path, stdin_bytes, deadline)?;
if !status.success() {
anyhow::bail!("git {label} failed: {}", String::from_utf8_lossy(&err));
}
@@ -1272,10 +1241,7 @@ mod tests {
assert!(
err.downcast_ref::<crate::git::smart_http::GitServiceTimeout>()
.is_some(),
// `{err:#}` prints the whole anyhow chain. Plain `{err}` shows only the top
// context ("failed to spawn git for-each-ref") and drops the underlying io
// error, which left a real beta-lane CI failure undiagnosable.
"a hung walk must abort with GitServiceTimeout (mapped to 504), got: {err:#}"
"a hung walk must abort with GitServiceTimeout (mapped to 504), got: {err}"
);
// The recorded process-group leader must be gone: the watchdog reaps the
@@ -1466,17 +1432,6 @@ esac\n";
// rev-list records its (leader) pid, closes stdout+stderr so the drain EOFs
// immediately, then sleeps without trapping TERM. `sleep 30` bounds the worst
// case so a RED run cannot wedge the suite; the recv budget fires first.
//
// The deadline below is deliberately far larger than the work it bounds: the
// watchdog tears the whole group down at the deadline, so a deadline tight
// enough to race `/bin/sh` reaching its first statement kills the leader
// BEFORE it records its pid, and the `leader.pid` read below then fails with
// NotFound. That is a harness race, not a product failure, and it is exactly
// what a loaded CI runner reproduced (test (beta), 2026-08-04). Sizing it at
// 2s keeps the property under test intact, since the pre-fix build blocks on
// `child.wait()` for the child's full `sleep 30` and still trips the 10s recv
// budget below (RED), while leaving a 20x margin over the ~100ms that a
// healthy start actually needs.
let body = "#!/bin/sh\ncase \"$1\" in\n rev-list) echo $$ > leader.pid; exec 1>&- 2>&-; sleep 30 ;;\n *) : ;;\nesac\nexit 0\n";
let git_bin = write_fake_git(tmp.path(), body);
let (tx, rx) = std::sync::mpsc::channel();
@@ -1487,7 +1442,7 @@ esac\n";
&["rev-list"],
&path,
b"",
Instant::now() + Duration::from_secs(2),
Instant::now() + Duration::from_millis(100),
));
});
let out = rx.recv_timeout(Duration::from_secs(10)).expect(
-164
View File
@@ -14,68 +14,6 @@ use subscription::SubscriptionRoot;
pub type GitlawbSchema = Schema<QueryRoot, MutationRoot, SubscriptionRoot>;
/// Client-facing message for GraphQL resolver failures that wrap a real
/// `sqlx::Error`. The real error is logged server-side; never put sqlx/Postgres
/// detail in the GraphQL `errors` array (#250).
///
/// Kept as its own constant on this PR's base (main still renders
/// `AppError::Db` with `e.to_string()`). If/when #247's `DB_ERROR_MESSAGE`
/// lands, fold this into that shared constant.
pub const GRAPHQL_DB_ERROR_MESSAGE: &str = "a database error occurred";
fn anyhow_has_sqlx(e: &anyhow::Error) -> bool {
e.chain().any(|c| c.downcast_ref::<sqlx::Error>().is_some())
}
/// Map an `anyhow` failure from the db layer to a GraphQL error.
///
/// - Real DB faults (`sqlx::Error` anywhere in the chain) → opaque client
/// message + `error!` log with the full `{e:#}` cause chain.
/// - Application/business errors (e.g. claim race, not-in-claimed-state) →
/// keep the actionable message; log at `warn!` so they are not mistaken for
/// infrastructure failures (#250 review).
pub(crate) fn graphql_db_err(e: anyhow::Error) -> async_graphql::Error {
if anyhow_has_sqlx(&e) {
tracing::error!(error = %format!("{e:#}"), "graphql database error");
async_graphql::Error::new(GRAPHQL_DB_ERROR_MESSAGE)
} else {
tracing::warn!(error = %format!("{e:#}"), "graphql application error");
async_graphql::Error::new(e.to_string())
}
}
/// Map an `AppError` from a shared collector (e.g. ref-update feed) to a
/// GraphQL error.
///
/// Fail closed: only explicitly curated variants surface their `Display`
/// text. Unnamed variants (including `Git`, which may embed on-disk paths)
/// render opaque so a future addition cannot leak by default (#255 review).
pub(crate) fn graphql_app_err(e: crate::error::AppError) -> async_graphql::Error {
match e {
crate::error::AppError::Db(sql) => graphql_db_err(sql.into()),
crate::error::AppError::Internal(err) => {
tracing::error!(error = %format!("{err:#}"), "graphql internal error");
async_graphql::Error::new(GRAPHQL_DB_ERROR_MESSAGE)
}
// Curated client-safe variants — `Display` is intentional API text.
safe @ (crate::error::AppError::RepoNotFound(_)
| crate::error::AppError::RepoExists(_)
| crate::error::AppError::NotFound(_)
| crate::error::AppError::Unauthorized(_)
| crate::error::AppError::Forbidden(_)
| crate::error::AppError::BadRequest(_)
| crate::error::AppError::TooManyRequests(_)
| crate::error::AppError::Incomplete(_)) => {
tracing::warn!(error = %safe, "graphql application error");
async_graphql::Error::new(safe.to_string())
}
other => {
tracing::error!(error = %other, "graphql unclassified AppError (opaque)");
async_graphql::Error::new(GRAPHQL_DB_ERROR_MESSAGE)
}
}
}
pub fn build_schema(
db: Arc<Db>,
ref_update_tx: tokio::sync::broadcast::Sender<RefUpdateBroadcast>,
@@ -87,105 +25,3 @@ pub fn build_schema(
.data(task_event_tx)
.finish()
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn graphql_db_err_opaques_sqlx_chain() {
let leak = "error returned from database: column \"is_public\" does not exist";
// Context layer must not hide sqlx from the chain walk (db helpers
// wrap with `.context(...)` in several places).
let err = graphql_db_err(
anyhow::Error::from(sqlx::Error::Protocol(leak.into())).context("loading repos"),
);
assert_eq!(err.message, GRAPHQL_DB_ERROR_MESSAGE);
assert!(!err.message.contains("is_public"));
assert!(!err.message.contains(leak));
assert!(!err.message.contains("loading repos"));
}
#[test]
fn graphql_db_err_keeps_business_message() {
let msg = "task not claimable: not found or already claimed";
let err = graphql_db_err(anyhow::anyhow!("{msg}"));
assert_eq!(err.message, msg);
}
#[test]
fn graphql_app_err_opaques_db_and_internal() {
let leak = "column \"is_public\" does not exist";
let db_err = graphql_app_err(crate::error::AppError::Db(sqlx::Error::Protocol(
leak.into(),
)));
assert_eq!(db_err.message, GRAPHQL_DB_ERROR_MESSAGE);
assert!(!db_err.message.contains("is_public"));
let internal = graphql_app_err(crate::error::AppError::Internal(anyhow::anyhow!(
"loading repo: {leak}"
)));
assert_eq!(internal.message, GRAPHQL_DB_ERROR_MESSAGE);
assert!(!internal.message.contains("is_public"));
}
#[test]
fn graphql_app_err_keeps_safe_variant_messages() {
let err = graphql_app_err(crate::error::AppError::NotFound("widget".into()));
assert!(
err.message.contains("widget"),
"safe NotFound message must reach the client: {}",
err.message
);
assert_ne!(err.message, GRAPHQL_DB_ERROR_MESSAGE);
let err = graphql_app_err(crate::error::AppError::BadRequest("bad cid".into()));
assert!(
err.message.contains("bad cid"),
"safe BadRequest message must reach the client: {}",
err.message
);
assert_ne!(err.message, GRAPHQL_DB_ERROR_MESSAGE);
}
#[test]
fn graphql_app_err_opaques_unclassified_variants() {
// `Git` may embed on-disk paths from libgit2; fail closed.
let path = "/var/lib/gitlawb/repos/owner/secret.git";
let err = graphql_app_err(crate::error::AppError::Git(format!(
"failed to open '{path}'"
)));
assert_eq!(err.message, GRAPHQL_DB_ERROR_MESSAGE);
assert!(!err.message.contains(path));
assert!(!err.message.contains("failed to open"));
}
/// Every `.map_err(` in the GraphQL query/mutation resolvers must route
/// through the opaque helpers, or discard the error (`|_|`). Same source-
/// scrape pattern as `api::authz_guard` (#255 review).
#[test]
fn every_graphql_map_err_uses_opaque_helpers() {
for (file, src) in [
("query.rs", include_str!("query.rs")),
("mutation.rs", include_str!("mutation.rs")),
] {
for (lineno, line) in src.lines().enumerate() {
let code = line.split("//").next().unwrap_or(line);
let Some(idx) = code.find(".map_err(") else {
continue;
};
let after = code[idx + ".map_err(".len()..].trim_start();
let ok = after.starts_with("crate::graphql::graphql_db_err")
|| after.starts_with("crate::graphql::graphql_app_err")
|| after.starts_with("|_|")
|| after.starts_with("|_ ");
assert!(
ok,
"{file}:{}: `.map_err(` must use graphql_db_err / graphql_app_err \
or discard (`|_|`): {line}",
lineno + 1
);
}
}
}
}
+8 -50
View File
@@ -54,7 +54,7 @@ impl MutationRoot {
};
db.create_task(&task)
.await
.map_err(crate::graphql::graphql_db_err)?;
.map_err(|e| async_graphql::Error::new(e.to_string()))?;
Ok(AgentTaskType::from(task))
}
@@ -76,7 +76,7 @@ impl MutationRoot {
let task = db
.claim_task(&id, &assignee_did)
.await
.map_err(crate::graphql::graphql_db_err)?;
.map_err(|e| async_graphql::Error::new(e.to_string()))?;
let _ = tx.send(TaskEventBroadcast {
task_id: id,
old_status: "pending".to_string(),
@@ -108,7 +108,7 @@ impl MutationRoot {
let existing = db
.get_task(&id)
.await
.map_err(crate::graphql::graphql_db_err)?
.map_err(|e| async_graphql::Error::new(e.to_string()))?
.ok_or_else(|| async_graphql::Error::new("task not found"))?;
if !crate::api::did_matches(caller, existing.assignee_did.as_deref().unwrap_or_default()) {
return Err(async_graphql::Error::new(
@@ -118,7 +118,7 @@ impl MutationRoot {
let task = db
.finish_task(&id, "completed", input.result.as_deref())
.await
.map_err(crate::graphql::graphql_db_err)?;
.map_err(|e| async_graphql::Error::new(e.to_string()))?;
let _ = tx.send(TaskEventBroadcast {
task_id: id,
old_status: "claimed".to_string(),
@@ -149,7 +149,7 @@ impl MutationRoot {
let existing = db
.get_task(&id)
.await
.map_err(crate::graphql::graphql_db_err)?
.map_err(|e| async_graphql::Error::new(e.to_string()))?
.ok_or_else(|| async_graphql::Error::new("task not found"))?;
if !crate::api::did_matches(caller, existing.assignee_did.as_deref().unwrap_or_default()) {
return Err(async_graphql::Error::new(
@@ -160,7 +160,7 @@ impl MutationRoot {
let task = db
.finish_task(&id, "failed", Some(&reason))
.await
.map_err(crate::graphql::graphql_db_err)?;
.map_err(|e| async_graphql::Error::new(e.to_string()))?;
let _ = tx.send(TaskEventBroadcast {
task_id: id,
old_status: "claimed".to_string(),
@@ -218,9 +218,8 @@ mod tests {
errors(&resp)
);
// 3. Signed as the claimed assignee → passes the auth gate. The missing
// task is a business error from claim_task, not a sqlx fault, so the
// actionable message must survive (not the opaque DB string) (#250).
// 3. Signed as the claimed assignee → passes the auth gate (any remaining
// error is the missing task, not an auth error).
let resp = schema
.execute(Request::new(&q).data(AuthenticatedDid(assignee.into())))
.await;
@@ -229,47 +228,6 @@ mod tests {
!errs.contains("authentication required") && !errs.contains("authenticated signer"),
"matching signer must pass the auth gate: {errs}"
);
assert!(
errs.contains("task not claimable"),
"claim race / missing task must keep its business message: {errs}"
);
assert!(
!errs.contains(crate::graphql::GRAPHQL_DB_ERROR_MESSAGE),
"business error must not be rewritten as opaque DB error: {errs}"
);
}
/// #250: mutation DB faults must be opaque; create_task hits agent_tasks.
#[sqlx::test]
async fn create_task_db_error_message_is_opaque(pool: PgPool) {
let state = crate::test_support::test_state(pool.clone()).await;
sqlx::query("ALTER TABLE agent_tasks DROP COLUMN status")
.execute(&pool)
.await
.unwrap();
let delegator = "did:key:zGQLDELEGATORAAAAAAAAAAAAAAAAAAAAAAAAAA";
let q = format!(
r#"mutation {{
createTask(
delegatorDid: "{delegator}",
input: {{ kind: "build", capability: "repo:write" }}
) {{ id }}
}}"#
);
let resp = state
.graphql_schema
.execute(Request::new(&q).data(AuthenticatedDid(delegator.into())))
.await;
let errs = errors(&resp);
assert!(
errs.contains(crate::graphql::GRAPHQL_DB_ERROR_MESSAGE),
"sqlx fault must be opaque: {errs}"
);
assert!(
!errs.contains("column") && !errs.contains("status"),
"schema text leaked: {errs}"
);
}
/// Adversarial-review GATE-1 (GraphQL): completing a task requires being its
+11 -130
View File
@@ -14,7 +14,7 @@ impl QueryRoot {
let repos = db
.list_all_repos_deduped()
.await
.map_err(crate::graphql::graphql_db_err)?;
.map_err(|e| async_graphql::Error::new(e.to_string()))?;
// Apply the same "/" visibility gate the REST/per-repo endpoints use so
// this surface does not enumerate private repos (#97). The caller DID is
@@ -27,7 +27,7 @@ impl QueryRoot {
let rules_by_repo = db
.list_visibility_rules_for_repos(&ids)
.await
.map_err(crate::graphql::graphql_db_err)?;
.map_err(|e| async_graphql::Error::new(e.to_string()))?;
Ok(repos
.into_iter()
@@ -71,27 +71,11 @@ impl QueryRoot {
let updates =
crate::api::events::collect_visible_ref_updates(db, repo.as_deref(), limit, caller)
.await
.map_err(crate::graphql::graphql_app_err)?;
.map_err(|e| async_graphql::Error::new(e.to_string()))?;
// Resolve the trusted display owner_did per row, identical to the REST
// feed: the stored wire value is untrusted, so it is echoed only when it
// matches the canonical owner of the local repo the slug names (#P1);
// legacy None rows are attributed via an exact unique local match (#P3).
// The batch resolver issues at most one query per distinct local repo
// rather than one per event row (#P2).
let pairs: Vec<(&str, Option<&str>)> = updates
.iter()
.map(|u| (u.repo.as_str(), u.owner_did.as_deref()))
.collect();
let owner_dids = db
.resolve_ref_update_owner_dids(&pairs)
.await
.map_err(crate::graphql::graphql_db_err)?;
let resolved: Vec<RefUpdateType> = updates
Ok(updates
.into_iter()
.zip(owner_dids)
.map(|(u, owner_did)| RefUpdateType {
.map(|u| RefUpdateType {
repo: u.repo,
ref_name: u.ref_name,
old_sha: u.old_sha,
@@ -99,10 +83,8 @@ impl QueryRoot {
pusher_did: u.pusher_did,
node_did: u.node_did,
timestamp: u.timestamp,
owner_did,
})
.collect();
Ok(resolved)
.collect())
}
async fn tasks(
@@ -110,21 +92,13 @@ impl QueryRoot {
ctx: &Context<'_>,
status: Option<String>,
assignee_did: Option<String>,
#[graphql(
default = 50,
desc = "Max 200; larger requests are clamped to 200 (no error). Negative values clamp to 0."
)]
limit: i64,
#[graphql(default = 50)] limit: i64,
) -> Result<Vec<AgentTaskType>> {
let db = ctx.data_unchecked::<Arc<Db>>();
// Clamp before SQL: a negative LIMIT is a client fault that Postgres
// rejects with 2201W, which would otherwise trip the opaque DB path
// and write an error-level log on every probe (#250 review).
let limit = limit.clamp(0, 200);
let tasks = db
.list_tasks(status.as_deref(), assignee_did.as_deref(), limit)
.await
.map_err(crate::graphql::graphql_db_err)?;
.map_err(|e| async_graphql::Error::new(e.to_string()))?;
Ok(tasks.into_iter().map(AgentTaskType::from).collect())
}
@@ -133,7 +107,7 @@ impl QueryRoot {
let t = db
.get_task(&id)
.await
.map_err(crate::graphql::graphql_db_err)?;
.map_err(|e| async_graphql::Error::new(e.to_string()))?;
Ok(t.map(AgentTaskType::from))
}
}
@@ -188,7 +162,6 @@ mod tests {
cert_id: None,
received_at: Utc::now().to_rfc3339(),
from_peer: "peer1".into(),
owner_did: None,
}
}
@@ -272,10 +245,10 @@ mod tests {
.await
.unwrap();
let schema = schema(db);
let q = r#"{ refUpdates { repo refName ownerDid } }"#;
let q = r#"{ refUpdates { repo refName } }"#;
let resp = anon(&schema, q).await;
assert_eq!(count(&resp), 1);
// The one row returned must be the public repo's with owner_did echoed.
// The one row returned must be the public repo's.
let async_graphql::Value::Object(obj) = &resp.data else {
unreachable!()
};
@@ -289,11 +262,6 @@ mod tests {
row.get("repo").unwrap(),
&async_graphql::Value::from("z6MkOwner/openrepo")
);
assert_eq!(
row.get("ownerDid").unwrap(),
&async_graphql::Value::from(OWNER),
"ownerDid must fall back to the local record's owner for legacy rows"
);
}
// Scenario 4 — alias fail-closed: private repo's row stored full-DID form.
@@ -427,91 +395,4 @@ mod tests {
let q = r#"{ refUpdates { repo } }"#;
assert_eq!(count(&authed(&schema, q, "did:key:z6MkQuar").await), 0);
}
/// #250: anonymous GraphQL query DB failures must not leak sqlx/schema text.
#[sqlx::test]
async fn repos_query_db_error_message_is_opaque(pool: PgPool) {
let db = db(pool.clone()).await;
db.create_repo(&repo("r1", OWNER, "widget", true))
.await
.unwrap();
sqlx::query("ALTER TABLE repos DROP COLUMN is_public")
.execute(&pool)
.await
.unwrap();
let schema = schema(db);
let resp = anon(&schema, "{ repos { name ownerDid } }").await;
assert!(
!resp.errors.is_empty(),
"DB failure must surface as a GraphQL error"
);
for err in &resp.errors {
assert_eq!(
err.message,
crate::graphql::GRAPHQL_DB_ERROR_MESSAGE,
"raw DB detail leaked into GraphQL error: {}",
err.message
);
assert!(
!err.message.contains("is_public") && !err.message.contains("column"),
"schema text leaked: {}",
err.message
);
}
}
/// #250: negative tasks(limit) must not hit Postgres (and must not 500-log).
#[sqlx::test]
async fn tasks_negative_limit_clamped(pool: PgPool) {
let db = db(pool).await;
let schema = schema(db);
let resp = anon(&schema, "{ tasks(limit: -1) { id } }").await;
assert!(
resp.errors.is_empty(),
"negative limit must clamp, not fail: {:?}",
resp.errors
);
assert_eq!(count_tasks(&resp), 0);
}
/// #255: ceiling of the tasks(limit) clamp must be held (schema promises Max 200).
#[sqlx::test]
async fn tasks_limit_ceiling_clamped_to_200(pool: PgPool) {
let db = db(pool).await;
let now = Utc::now().to_rfc3339();
for i in 0..201 {
db.create_task(&crate::db::AgentTask {
id: format!("task-ceil-{i}"),
repo_id: None,
kind: "build".into(),
status: "pending".into(),
delegator_did: OWNER.into(),
assignee_did: None,
capability: "repo:write".into(),
ucan_token: None,
payload: None,
result: None,
created_at: now.clone(),
updated_at: now.clone(),
deadline: None,
})
.await
.unwrap();
}
let schema = schema(db);
let resp = anon(&schema, "{ tasks(limit: 5000) { id } }").await;
assert_eq!(count_tasks(&resp), 200, "limit above 200 must clamp to 200");
}
fn count_tasks(resp: &async_graphql::Response) -> usize {
assert!(resp.errors.is_empty(), "graphql errors: {:?}", resp.errors);
let async_graphql::Value::Object(obj) = &resp.data else {
panic!("data not an object: {:?}", resp.data);
};
let async_graphql::Value::List(rows) = obj.get("tasks").expect("tasks key") else {
panic!("tasks not a list");
};
rows.len()
}
}
@@ -39,7 +39,6 @@ impl SubscriptionRoot {
pusher_did: ev.pusher_did,
node_did: ev.node_did,
timestamp: ev.timestamp,
owner_did: Some(ev.owner_did),
}),
_ => None,
}
-1
View File
@@ -57,7 +57,6 @@ pub struct RefUpdateType {
pub pusher_did: String,
pub node_did: String,
pub timestamp: String,
pub owner_did: Option<String>,
}
#[derive(SimpleObject, Clone)]
File diff suppressed because it is too large. Load diff
+74 -539
View File
@@ -31,7 +31,6 @@ use axum::response::IntoResponse;
use axum::routing::get;
use axum::{Json, Router};
use clap::Parser;
use std::collections::HashSet;
use std::sync::atomic::{AtomicU64, Ordering};
use std::sync::Arc;
use tokio::net::TcpListener;
@@ -65,7 +64,7 @@ struct DbStartupStatus {
/// pool used to derive its size straight from that knob, so raising the push cap
/// silently raised the node's Postgres connection ceiling with no CLI error and no
/// relation to the server's own `max_connections` (#173 F4). The node's total budget is
/// now bounded: `db_max_connections` (default 48) + at most this.
/// now bounded: `db_max_connections` (default 20) + at most this.
const LOCK_POOL_MAX_CONNECTIONS: u32 = 64;
/// Connections the lock pool keeps above the push cap. Covers the three non-push
@@ -101,13 +100,6 @@ async fn main() -> Result<()> {
// bootstrap peers. Operators can opt out via GITLAWB_BOOTSTRAP_DISABLE_SEEDS.
bootstrap::merge_seeds(&mut config);
// Fail fast on config combinations that are individually in-range but jointly
// unsafe — notably a DB pool too small for the concurrent-write cap, which
// would let a push burst starve every other DB path (#174 F1).
config
.validate()
.map_err(|e| anyhow::anyhow!("invalid configuration: {e}"))?;
if !config.public_read {
warn!(
"GITLAWB_PUBLIC_READ=false is reserved; per-repository private-read enforcement is not wired in alpha"
@@ -116,9 +108,6 @@ async fn main() -> Result<()> {
// Load or generate the node's identity keypair
let keypair = load_or_create_keypair(&config)?;
// Sealing key for the legacy-scan continuation tokens, DERIVED from the identity
// just loaded so it is the same key after a restart (see `derive_scan_token_key`).
let scan_token_key = AppState::derive_scan_token_key(&keypair);
let node_did = keypair.did();
// One-time metrics init. Must run before any handler that calls into
@@ -316,14 +305,11 @@ async fn main() -> Result<()> {
None
};
// Repo write locks run on their own pool, never the main query pool: each push
// holds its connection for the whole receive-pack, so a burst of concurrent
// pushes drawing from the main pool would park that many connections for the
// duration of their receive-packs and starve every other query. That holds
// whatever the two pools are sized at, which is why the separation is
// structural rather than a consequence of the defaults; config validate()
// separately requires db_max_connections >= max_concurrent_git_pushes + 8. See
// build_lock_pool for the cancellation semantics (#173).
// Repo write locks run on their own pool, never the main query pool: each
// push holds its connection for the whole receive-pack, and
// db_max_connections (20) is below max_concurrent_git_pushes (32), so sharing
// would starve every other query under a push burst. See build_lock_pool for
// the cancellation semantics (#173).
let lock_pool = git::repo_store::build_lock_pool(
db.pool(),
lock_pool_size(config.max_concurrent_git_pushes),
@@ -424,20 +410,10 @@ async fn main() -> Result<()> {
create_ip_rate_limiter,
push_rate_limiter,
ipfs_max_history_walks: crate::api::ipfs::MAX_HISTORY_WALKS_PER_REQUEST,
// The legacy-probe budget is operator-tunable via GITLAWB_IPFS_MAX_LEGACY_PROBES
// (R5), which is what `ipfs_legacy_probe_budget` reads. NOT
// GITLAWB_IPFS_MAX_REPOS_WALKED, which this comment used to name: that is the
// separate cap on expensive visibility walks, so an operator following the old
// text tuned the walk cap and left this fan-out unchanged. The history-walk
// ceiling above stays constant (a smaller value false-503s a provenanced
// request). Default 256 preserves the shipped behaviour.
// The legacy-probe budget is operator-tunable via GITLAWB_IPFS_MAX_REPOS_WALKED
// (R5); the history-walk ceiling above stays constant (a smaller value false-503s
// a provenanced request). Default 256 preserves the shipped behaviour.
ipfs_max_legacy_probes: AppState::ipfs_legacy_probe_budget(&config),
ipfs_legacy_scan_page_rows: crate::api::ipfs::LEGACY_SCAN_PAGE_ROWS,
// Operator-tunable via GITLAWB_IPFS_MAX_LEGACY_SCAN_ROWS, read through the same
// helper shape as the probe budget so the knob cannot be a silent no-op.
ipfs_max_legacy_scan_rows: AppState::ipfs_legacy_scan_row_budget(&config),
ipfs_max_legacy_scan_rule_bytes: crate::api::ipfs::MAX_LEGACY_SCAN_RULE_BYTES_PER_REQUEST,
ipfs_scan_token_key: Arc::new(scan_token_key),
ipfs_max_served_object_bytes: crate::api::ipfs::MAX_SERVED_OBJECT_BYTES,
push_limiter_trust,
sync_trigger_rate_limiter,
@@ -460,44 +436,27 @@ async fn main() -> Result<()> {
git_encrypt_semaphore: Arc::new(tokio::sync::Semaphore::new(
config.max_concurrent_git_pushes,
)),
// Bounds how many post-push pin loops run concurrently across all repos (#174 F6),
// independent of the per-repo encrypt-task coalescing below. Not a bound on the
// MB-scale object-id lists themselves: parked tasks still hold theirs (see the
// field doc on AppState::pin_semaphore).
pin_semaphore: Arc::new(tokio::sync::Semaphore::new(config.max_concurrent_pin_tasks)),
// Coalesces the DETACHED post-push encryption tasks per repo so a rapid pusher
// cannot grow the outstanding parked-waiter set past one task per repo (#174
// P2-2). No knob: it is a natural cap (one entry per distinct repo), not a
// sized pool.
encrypt_inflight: crate::state::EncryptInflight::new(),
// Per-repo in-process write-lease serializer (#174 U2/F3): supplements the pg
// advisory lock so a disconnected push's still-reaping git group can't be raced
// by a second same-node push. The map is naturally capped (one entry per contended
// repo, freed when unreferenced); the sized knob is how many pushes may PARK on
// one repo, since each parked push holds a fully buffered pack.
repo_write_leases: crate::state::RepoWriteLeases::new(config.repo_lease_max_waiters),
git_read_per_caller: rate_limit::PerCallerConcurrency::with_default_max_keys(
config.max_concurrent_reads_per_caller,
),
// Per-source cap on the receive-pack advertisement, sized to an eighth of the
// write pool (min 1): one resolved client key (rate_limit::client_key) can hold
// at most this many slots in the DEDICATED advert pool (git_push_advert_semaphore,
// disjoint from the write pool), so saturating that pool takes ~8 distinct keys
// (#174). That bounds an IPv4 or single-address caller; a caller controlling many
// addresses (an IPv6 /64 is 2^64 keys) still gets one cap per address, since
// client_key uses the full IP with no prefix folding. Narrowing the keying is a
// deferred design call, not something these caps claim to solve. Sized off the
// write pool only because the advert pool is created at the same size; an advert
// flood cannot touch a write permit.
// write pool (min 1): a single source can hold at most this many write-pool
// slots via the anon advertisement, so saturating the pool takes ~8 distinct
// source IPs, each also rate-limited (#174).
git_push_advert_per_caller: rate_limit::PerCallerConcurrency::with_default_max_keys(
rate_limit::per_source_push_cap(config.max_concurrent_git_pushes),
(config.max_concurrent_git_pushes / 8).max(1),
),
// Per-source cap on the authenticated receive-pack POST, sized like the advert
// cap: one resolved client key can hold at most this many write-pool slots, so
// monopolizing the pool takes ~8 distinct keys (#174 P1-d). Same residual as
// above: keys are full IPs, so a caller with many addresses has many caps.
// cap: one source IP can hold at most this many write-pool slots, so
// monopolizing the pool takes ~8 distinct source IPs, each also rate-limited
// (#174 P1-d).
git_write_per_caller: rate_limit::PerCallerConcurrency::with_default_max_keys(
rate_limit::per_source_push_cap(config.max_concurrent_git_pushes),
(config.max_concurrent_git_pushes / 8).max(1),
),
// Bounds concurrent /ipfs visibility walks — a distinct public cost center, so
// its own pool + per-source sub-cap + per-IP rate limiter, never a git pool
@@ -708,7 +667,7 @@ async fn main() -> Result<()> {
Ok(())
}
/// U4 (#173): spawn the periodic legacy provider-CID repair sweep. Releases before this
/// U4 (#173): spawn the one-shot legacy provider-CID repair sweep. Releases before this
/// version stored the PROVIDER CID (Kubo dag-pb / Pinata CIDv0) in `pinned_cids.cid`,
/// and this version's `/ipfs/{cid}` resolver withholds any row whose stored key is not
/// the raw-content CID. The opportunistic repair on the pin path only fires when a push
@@ -716,9 +675,7 @@ async fn main() -> Result<()> {
/// need a walk. DETACHED, never on the boot path: the caller keeps serving while this
/// runs, and the sweep's own batch bound plus inter-batch delay keep it off the DB's
/// critical path. Its cursor is durable, so a restart mid-walk resumes instead of
/// rewinding. It re-arms after every run, including a failed one, so it never returns
/// and there is no awaited value to log here; the shutdown watcher below is what ends
/// it.
/// rewinding.
///
/// A named function rather than an inline block in `main` so the WIRING has a seam a
/// test can call: that the task is spawned at all, that it reads its batch and delay
@@ -735,10 +692,17 @@ fn spawn_legacy_cid_sweep(state: &AppState, config: &Config) -> tokio::task::Joi
let mut shutdown_rx = state.subscribe_shutdown();
tokio::spawn(async move {
tokio::select! {
_ = ipfs_pin::run_sweep_rearmed(
&repos_dir, &git_bin, git_timeout, batch, delay,
ipfs_pin::SWEEP_REARM_DELAY, &db,
) => {}
stats = ipfs_pin::sweep_legacy_provider_cids(
&repos_dir, &git_bin, git_timeout, batch, delay, &db,
) => {
if stats.repaired > 0 {
tracing::info!(
scanned = stats.scanned,
repaired = stats.repaired,
"legacy provider-CID sweep finished"
);
}
}
// Shutdown mid-walk simply drops the run; the persisted cursor means the
// next boot picks up where this one stopped.
_ = shutdown_rx.changed() => {}
@@ -915,8 +879,8 @@ fn build_degraded_router(node_did: String, db_startup: Arc<DbStartupStatus>) ->
db_startup,
};
// Everything answers 503 with the same body — including /health and
// /ready, so peer readiness probes and uptime monitors correctly see a
// node that cannot serve traffic.
// /ready, so peer pings (which treat any 2xx /health as alive) and
// uptime monitors correctly see a node that cannot serve traffic.
// `/` additionally carries the node identity for probing peers.
Router::new()
.route("/", get(degraded_node_info))
@@ -1048,7 +1012,7 @@ async fn gossip_task(
}
// Reuse the shared no-redirect client for every gossip outbound call (the
// bootstrap announce POST and the periodic peer /ready ping). Peer URLs are
// bootstrap announce POST and the periodic peer /health ping). Peer URLs are
// attacker-influenceable, so a 3xx to a private address must not be followed.
// Do NOT fall back to reqwest::Client::new(): its default follows redirects
// and would reintroduce the SSRF closed here (#93).
@@ -1103,33 +1067,8 @@ async fn gossip_task(
json.get("node_url").and_then(|v| v.as_str()),
) {
if !their_url.is_empty() {
// Unproven, unconditionally: their_did and
// their_url come straight out of the contacted
// peer's JSON response body, with no signature
// and no proof the claimed DID belongs to the
// peer we reached. This path therefore seeds
// new peers and can never repoint an existing
// row. The upsert's result is matched rather
// than discarded, because "bootstrap peer
// added" printed over a refused write is a
// denial rendering as success. Non-fatal to the
// loop either way.
match state
.db
.upsert_peer(
their_did,
their_url,
db::PeerWriteAuthority::Unproven,
)
.await
{
Ok(()) => {
tracing::info!(did = %their_did, url = %their_url, "bootstrap peer added")
}
Err(e) => {
tracing::warn!(did = %their_did, url = %their_url, err = %e, "bootstrap peer announce-back rejected")
}
}
let _ = state.db.upsert_peer(their_did, their_url).await;
tracing::info!(did = %their_did, url = %their_url, "bootstrap peer added");
}
}
}
@@ -1144,8 +1083,6 @@ async fn gossip_task(
// Periodic ping every 5 minutes — exit on shutdown.
let mut interval = tokio::time::interval(std::time::Duration::from_secs(300));
interval.set_missed_tick_behavior(tokio::time::MissedTickBehavior::Delay);
let mut failed_once: HashSet<String> = HashSet::new();
loop {
tokio::select! {
_ = interval.tick() => {
@@ -1153,13 +1090,10 @@ async fn gossip_task(
Ok(p) => p,
Err(_) => continue,
};
gossip_ping_round(
state.db.as_ref(),
client.as_ref(),
&mut failed_once,
peers,
)
.await;
for peer in peers {
let ok = ping_peer_health(&client, &peer.http_url).await;
let _ = state.db.mark_peer_ping(&peer.did, ok).await;
}
}
_ = shutdown_rx.changed() => {
if *shutdown_rx.borrow() {
@@ -1171,93 +1105,6 @@ async fn gossip_task(
}
}
#[cfg(test)]
mod rate_limiter_sweep_tests {
use crate::rate_limit::RateLimiter;
use std::time::Duration;
// Every per-key limiter the router mounts must be swept by the periodic
// task, the `/ipfs` one included: a limiter left out keeps expired keys
// until its map fills and the inline capacity sweep fires. Fails on the
// pre-fix sweeper, which skipped `ipfs_rate_limiter`.
#[tokio::test]
async fn sweep_evicts_expired_keys_from_every_limiter() {
let window = Duration::from_millis(30);
let mut state = crate::test_support::test_state_lazy();
state.rate_limiter = RateLimiter::new(10, window);
state.create_ip_rate_limiter = RateLimiter::new(10, window);
state.push_rate_limiter = RateLimiter::new(10, window);
state.sync_trigger_rate_limiter = RateLimiter::new(10, window);
state.peer_write_rate_limiter = RateLimiter::new(10, window);
state.ipfs_rate_limiter = RateLimiter::new(10, window);
state.ipfs_work_rate_limiter = RateLimiter::new(10, window);
let limiters = |s: &crate::state::AppState| {
[
s.rate_limiter.clone(),
s.create_ip_rate_limiter.clone(),
s.push_rate_limiter.clone(),
s.sync_trigger_rate_limiter.clone(),
s.peer_write_rate_limiter.clone(),
s.ipfs_rate_limiter.clone(),
s.ipfs_work_rate_limiter.clone(),
]
};
for l in limiters(&state) {
assert!(l.check("1.2.3.4").await);
assert_eq!(l.tracked_keys().await, 1);
}
tokio::time::sleep(window * 3).await;
state.sweep_rate_limiters().await;
for (i, l) in limiters(&state).into_iter().enumerate() {
assert_eq!(l.tracked_keys().await, 0, "limiter {i} was not swept");
}
}
}
async fn gossip_ping_round(
db: &Db,
client: &reqwest::Client,
failed_once: &mut HashSet<String>,
peers: Vec<db::PeerRecord>,
) {
{
let current_dids: HashSet<&str> = peers.iter().map(|peer| peer.did.as_str()).collect();
failed_once.retain(|did| current_dids.contains(did.as_str()));
}
for peer in peers {
let ok = ping_peer_readiness(client, &peer.http_url).await;
match peer_ping_db_update(failed_once, &peer.did, ok) {
Some(reachable) => {
if let Err(error) = db.mark_peer_ping(&peer.did, reachable).await {
warn!(did = %peer.did, err = %error, "failed to persist peer readiness");
}
}
None => warn!(
did = %peer.did,
"peer readiness probe failed once; preserving previous federation status"
),
}
}
}
/// Decide whether one readiness sample should update the persisted federation
/// gate. A success is authoritative immediately. A failure must repeat on the
/// next gossip tick before it can mark a peer unreachable, so one transient
/// database hiccup does not hide the peer's repositories for five minutes.
fn peer_ping_db_update(failed_once: &mut HashSet<String>, did: &str, ready: bool) -> Option<bool> {
if ready {
failed_once.remove(did);
Some(true)
} else if failed_once.insert(did.to_owned()) {
None
} else {
Some(false)
}
}
/// Build the shared node HTTP client used for every outbound fan-out (sync
/// trigger, profile/repo fetches, gossip announce + peer pings).
///
@@ -1268,96 +1115,26 @@ fn peer_ping_db_update(failed_once: &mut HashSet<String>, did: &str, ready: bool
/// runs, not a hand-rolled equivalent.
fn build_http_client() -> reqwest::Result<reqwest::Client> {
reqwest::Client::builder()
.timeout(OUTBOUND_HTTP_TIMEOUT)
.timeout(std::time::Duration::from_secs(10))
.redirect(reqwest::redirect::Policy::none())
.build()
}
const OUTBOUND_HTTP_TIMEOUT: std::time::Duration = std::time::Duration::from_secs(10);
/// Ping a peer's DB-aware `/ready` endpoint and report whether it answered 2xx.
///
/// A 404 falls back to `/health` for compatibility with nodes released before
/// `/ready` existed. Other readiness failures, including 503 during a database
/// outage, fail closed and must not use the liveness-only endpoint. The complete
/// `/ready`-then-`/health` probe shares one timeout budget.
/// Ping a peer's `/health` endpoint and report whether it answered 2xx.
///
/// Takes the client by reference so callers supply the shared, no-redirect
/// `state.http_client`. Peer URLs are attacker-influenceable, so a `3xx` to a
/// private address must not be followed. Do NOT call this with a bare
/// `reqwest::Client::new()`: its default follows redirects and would
/// reintroduce the SSRF this guards against (#93).
pub(crate) async fn ping_peer_readiness(client: &reqwest::Client, http_url: &str) -> bool {
ping_peer_readiness_with_timeout(client, http_url, OUTBOUND_HTTP_TIMEOUT).await
}
async fn ping_peer_readiness_with_timeout(
client: &reqwest::Client,
http_url: &str,
timeout: std::time::Duration,
) -> bool {
let base_url = http_url.trim_end_matches('/');
let result = tokio::time::timeout(timeout, async {
let readiness = client.get(format!("{base_url}/ready")).send().await;
match readiness {
Ok(response) if response.status().is_success() => true,
Ok(response) if response.status() == reqwest::StatusCode::NOT_FOUND => {
info!(
peer_url = %http_url,
"peer has no readiness endpoint; falling back to legacy health probe"
);
match client.get(format!("{base_url}/health")).send().await {
Ok(response) if response.status().is_success() => true,
Ok(response) => {
warn!(
peer_url = %http_url,
status = %response.status(),
"legacy peer health probe reported unhealthy"
);
false
}
Err(error) => {
warn!(
peer_url = %http_url,
err = %error,
"legacy peer health probe failed"
);
false
}
}
}
Ok(response) => {
warn!(
peer_url = %http_url,
status = %response.status(),
"peer readiness probe reported unready"
);
false
}
Err(error) => {
warn!(
peer_url = %http_url,
err = %error,
"peer readiness probe failed"
);
false
}
}
})
.await;
match result {
Ok(ready) => ready,
Err(_) => {
warn!(
peer_url = %http_url,
timeout_ms = timeout.as_millis(),
"peer readiness probe timed out"
);
false
}
}
async fn ping_peer_health(client: &reqwest::Client, http_url: &str) -> bool {
let url = format!("{}/health", http_url.trim_end_matches('/'));
client
.get(&url)
.send()
.await
.map(|r| r.status().is_success())
.unwrap_or(false)
}
fn load_or_create_keypair(config: &Config) -> Result<Keypair> {
@@ -1529,130 +1306,20 @@ mod lock_pool_sizing_tests {
#[cfg(test)]
mod gossip_ssrf_tests {
use super::{
gossip_ping_round, peer_ping_db_update, ping_peer_readiness,
ping_peer_readiness_with_timeout,
};
use axum::{http::StatusCode, routing::get, Router};
use sqlx::PgPool;
use std::collections::HashSet;
use std::sync::atomic::{AtomicUsize, Ordering};
use std::sync::Arc;
use std::time::Duration;
use super::ping_peer_health;
// Build the client exactly as production does (super::build_http_client) so
// these tests bind the redirect guarantee to the real shared client the
// node runs. A regression that makes build_http_client follow redirects
// fails ping_peer_readiness_does_not_follow_redirect.
// fails ping_peer_health_does_not_follow_redirect.
fn production_http_client() -> reqwest::Client {
super::build_http_client().expect("failed to build production http client")
}
#[test]
fn peer_ping_requires_two_failures_before_marking_unreachable() {
let mut failed_once = HashSet::new();
let did = "did:key:z6MkPeer";
assert_eq!(
peer_ping_db_update(&mut failed_once, did, false),
None,
"one transient failure must preserve the persisted federation gate"
);
assert_eq!(
peer_ping_db_update(&mut failed_once, did, false),
Some(false),
"a sustained failure must mark the peer unreachable"
);
assert_eq!(
peer_ping_db_update(&mut failed_once, did, true),
Some(true),
"a success must restore reachability immediately"
);
assert_eq!(
peer_ping_db_update(&mut failed_once, did, false),
None,
"a success must reset the consecutive-failure state"
);
}
#[sqlx::test]
async fn gossip_ping_round_requires_two_failures_before_persisting_unreachable(pool: PgPool) {
let mut server = mockito::Server::new_async().await;
let ready = server
.mock("GET", "/ready")
.with_status(503)
.expect(2)
.create_async()
.await;
let state = crate::test_support::test_state(pool.clone()).await;
let did = "did:key:z6MkGossipHysteresis";
let now = chrono::Utc::now().to_rfc3339();
sqlx::query(
"INSERT INTO peers (did, http_url, last_seen, last_ping_ok, announced_at)
VALUES ($1, $2, $3, TRUE, $3)",
)
.bind(did)
.bind(server.url())
.bind(now)
.execute(&pool)
.await
.unwrap();
let mut failed_once = HashSet::from(["did:key:z6MkRemovedPeer".to_owned()]);
gossip_ping_round(
state.db.as_ref(),
&production_http_client(),
&mut failed_once,
state.db.list_peers().await.unwrap(),
)
.await;
assert!(
!failed_once.contains("did:key:z6MkRemovedPeer"),
"failure tracking must prune peers outside the current snapshot"
);
assert!(
failed_once.contains(did),
"the first failed sample must be retained for the next round"
);
assert!(
state
.db
.list_peers()
.await
.unwrap()
.into_iter()
.find(|peer| peer.did == did)
.unwrap()
.last_ping_ok,
"one transient failure must preserve the persisted federation gate"
);
gossip_ping_round(
state.db.as_ref(),
&production_http_client(),
&mut failed_once,
state.db.list_peers().await.unwrap(),
)
.await;
assert!(
!state
.db
.list_peers()
.await
.unwrap()
.into_iter()
.find(|peer| peer.did == did)
.unwrap()
.last_ping_ok,
"two consecutive failed rounds must persist the peer as unreachable"
);
ready.assert_async().await;
}
// A peer answering `/ready` with a 302 toward an internal address must not
// A peer answering `/health` with a 302 toward an internal address must not
// be followed: the redirect target must never be requested (#93).
#[tokio::test]
async fn ping_peer_readiness_does_not_follow_redirect() {
async fn ping_peer_health_does_not_follow_redirect() {
let mut server = mockito::Server::new_async().await;
let internal = server
.mock("GET", "/internal-metadata")
@@ -1660,154 +1327,6 @@ mod gossip_ssrf_tests {
.expect(0)
.create_async()
.await;
let _ready = server
.mock("GET", "/ready")
.with_status(302)
.with_header("location", &format!("{}/internal-metadata", server.url()))
.create_async()
.await;
let ok = ping_peer_readiness(&production_http_client(), &server.url()).await;
assert!(!ok, "a 302 must not count as a healthy peer");
// expect(0) is enforced only at assert time; this fails if the redirect
// was followed to the internal target.
internal.assert_async().await;
}
#[tokio::test]
async fn ping_peer_readiness_reports_success_on_200() {
let mut server = mockito::Server::new_async().await;
let _ready = server
.mock("GET", "/ready")
.with_status(200)
.create_async()
.await;
let ok = ping_peer_readiness(&production_http_client(), &server.url()).await;
assert!(ok, "a 200 /ready must count as a ready peer");
}
#[tokio::test]
async fn ping_peer_readiness_ignores_liveness_only_health() {
let mut server = mockito::Server::new_async().await;
let health = server
.mock("GET", "/health")
.with_status(200)
.expect(0)
.create_async()
.await;
let ready = server
.mock("GET", "/ready")
.with_status(503)
.expect(1)
.create_async()
.await;
let ok = ping_peer_readiness(&production_http_client(), &server.url()).await;
assert!(!ok, "a peer with an unavailable database must not be ready");
health.assert_async().await;
ready.assert_async().await;
}
#[tokio::test]
async fn ping_peer_readiness_falls_back_for_legacy_peer() {
let mut server = mockito::Server::new_async().await;
let ready = server
.mock("GET", "/ready")
.with_status(404)
.expect(1)
.create_async()
.await;
let health = server
.mock("GET", "/health")
.with_status(200)
.expect(1)
.create_async()
.await;
let ok = ping_peer_readiness(&production_http_client(), &server.url()).await;
assert!(
ok,
"a legacy peer with a healthy /health must remain reachable"
);
ready.assert_async().await;
health.assert_async().await;
}
#[tokio::test]
async fn ping_peer_readiness_legacy_fallback_shares_deadline() {
let health_requests = Arc::new(AtomicUsize::new(0));
let health_requests_for_route = Arc::clone(&health_requests);
let app = Router::new()
.route(
"/ready",
get(|| async {
tokio::time::sleep(Duration::from_millis(100)).await;
StatusCode::NOT_FOUND
}),
)
.route(
"/health",
get(move || {
let health_requests = Arc::clone(&health_requests_for_route);
async move {
health_requests.fetch_add(1, Ordering::Relaxed);
tokio::time::sleep(Duration::from_millis(300)).await;
StatusCode::OK
}
}),
);
let listener = tokio::net::TcpListener::bind("127.0.0.1:0")
.await
.expect("failed to bind delayed legacy peer");
let peer_url = format!(
"http://{}",
listener
.local_addr()
.expect("delayed legacy peer has no local address")
);
let server = tokio::spawn(async move {
axum::serve(listener, app)
.await
.expect("delayed legacy peer failed");
});
let ok = ping_peer_readiness_with_timeout(
&production_http_client(),
&peer_url,
// Each response arrives within 350 ms, but the two serial requests
// cannot both finish within one 350 ms probe budget.
Duration::from_millis(350),
)
.await;
assert!(!ok, "the fallback must not start a fresh timeout budget");
assert_eq!(
health_requests.load(Ordering::Relaxed),
1,
"the delayed 404 should reach the legacy fallback"
);
server.abort();
}
#[tokio::test]
async fn ping_peer_readiness_legacy_fallback_does_not_follow_redirect() {
let mut server = mockito::Server::new_async().await;
let internal = server
.mock("GET", "/internal-metadata")
.with_status(200)
.expect(0)
.create_async()
.await;
let _ready = server
.mock("GET", "/ready")
.with_status(404)
.create_async()
.await;
let _health = server
.mock("GET", "/health")
.with_status(302)
@@ -1815,17 +1334,33 @@ mod gossip_ssrf_tests {
.create_async()
.await;
let ok = ping_peer_readiness(&production_http_client(), &server.url()).await;
let ok = ping_peer_health(&production_http_client(), &server.url()).await;
assert!(!ok, "the legacy fallback must not follow redirects");
assert!(!ok, "a 302 must not count as a healthy peer");
// expect(0) is enforced only at assert time; this fails if the redirect
// was followed to the internal target.
internal.assert_async().await;
}
#[tokio::test]
async fn ping_peer_health_reports_success_on_200() {
let mut server = mockito::Server::new_async().await;
let _health = server
.mock("GET", "/health")
.with_status(200)
.create_async()
.await;
let ok = ping_peer_health(&production_http_client(), &server.url()).await;
assert!(ok, "a 200 /health must count as a healthy peer");
}
// A transport error (nothing listening) must map to unhealthy, never a
// spurious healthy — the .unwrap_or(false) arm.
#[tokio::test]
async fn ping_peer_readiness_reports_unready_on_connection_error() {
let ok = ping_peer_readiness(&production_http_client(), "http://127.0.0.1:1").await;
assert!(!ok, "a connection error must count as an unready peer");
async fn ping_peer_health_reports_unhealthy_on_connection_error() {
let ok = ping_peer_health(&production_http_client(), "http://127.0.0.1:1").await;
assert!(!ok, "a connection error must count as an unhealthy peer");
}
}
+9 -47
View File
@@ -57,15 +57,10 @@ static PEERS_CONNECTED: OnceLock<IntGauge> = OnceLock::new();
/// more than once is a silent no-op. MUST be called from `main()` after
/// the node DID is known.
pub fn init(version: &str, node_did: &str) {
// Guard with Once so two concurrent callers cannot both pass an unsynchronized
// `REGISTRY.get()` check and then race on `OnceLock::set(...).expect(...)`,
// panicking the loser (#192 F4). Once runs the body exactly once, so the
// `.expect("set X once")` calls below can never observe an already-set slot.
static INIT: std::sync::Once = std::sync::Once::new();
INIT.call_once(|| init_inner(version, node_did));
}
if REGISTRY.get().is_some() {
return;
}
fn init_inner(version: &str, node_did: &str) {
let registry = Registry::new();
let info = IntGaugeVec::new(
@@ -223,17 +218,6 @@ pub fn record_fetch(repo: &str) {
}
}
/// Test-only: current `gitlawb_fetches_total` value for `repo` (0 if the registry
/// is not initialized). Lets api-layer tests assert the completed-fetch count with
/// a unique label instead of scraping the encoded text.
#[cfg(test)]
pub fn fetch_count_for_test(repo: &str) -> u64 {
FETCHES
.get()
.map(|c| c.with_label_values(&[repo]).get())
.unwrap_or(0)
}
/// Record one HTTP signature check that passed.
#[allow(dead_code)] // wired in a follow-up; helpers are part of the public metrics surface
pub fn record_auth_success(route: &str) {
@@ -251,11 +235,7 @@ pub fn record_auth_failure(route: &str, reason: &str) {
}
}
/// Record one sync_queue item outcome. `status` ∈ {done, failed, skipped,
/// rejected, deferred}. `rejected` is terminal and caused by a guard (a
/// malformed slug, or a path resolving outside repos_dir); `deferred` leaves
/// the row pending for a later pass, so a queue stalled on an operator
/// condition is visible rather than looking idle.
/// Record one sync_queue item outcome. `status` ∈ {done, failed, skipped}.
pub fn record_sync_processed(status: &str) {
if let Some(c) = SYNC_PROCESSED.get() {
c.with_label_values(&[status]).inc();
@@ -302,11 +282,11 @@ pub fn encode() -> Result<String, prometheus::Error> {
mod tests {
use super::*;
// Note: cargo runs tests in a binary in parallel via threads, and several
// tests here call init(). init() is guarded by std::sync::Once, so concurrent
// callers run the body exactly once and never race on the OnceLock setters
// (#192 F4). The encode test below is structured so it's safe to run alongside
// other tests in the same binary regardless of ordering.
// Note: these tests are not run in parallel by default (cargo runs
// tests in a binary in parallel via threads but the OnceLock guard
// means only the first init() call succeeds; subsequent ones are
// no-ops). The encode test below is structured so it's safe to run
// alongside other tests in the same binary.
#[test]
fn encode_after_init_returns_prometheus_text() {
@@ -337,24 +317,6 @@ mod tests {
);
}
/// #192 F4: `init` is idempotent and safe to call repeatedly. The panic that
/// hit concurrent callers (both passing the old unsynchronized `REGISTRY.get()`
/// check, then racing on `OnceLock::set(...).expect(...)`) is fixed by the
/// `Once` guard. That race is inherently non-deterministic to assert in the
/// shared-binary suite — a 32-thread barrier stress-test reproduces it only in
/// isolation (verified: RED before the `Once` fix, GREEN after). This guard
/// covers the deterministic half of the contract: a repeat call is a no-op,
/// never a panic, and the registry stays usable.
#[test]
fn init_is_idempotent_no_panic_on_repeat() {
init("0.0.0-a", "did:key:a");
init("0.0.0-b", "did:key:b");
assert!(
encode().is_ok(),
"registry must stay usable after a repeated init"
);
}
#[test]
fn record_helpers_are_noops_before_init() {
// These don't panic. They also don't show up in encode() output
-70
View File
@@ -39,11 +39,6 @@ pub struct RefUpdateEvent {
pub pusher_did: String,
/// Repository identifier (owner/name)
pub repo: String,
/// Full owner DID — added in #144 for display and storage; not yet
/// wired into the feed gate matcher. Optional for backward compat with
/// older peers that don't include it.
#[serde(default)]
pub owner_did: Option<String>,
/// Git ref that changed (e.g., "refs/heads/main")
pub ref_name: String,
/// SHA before the push (all-zeros for new ref)
@@ -312,7 +307,6 @@ pub async fn start(
node_did: event.node_did.clone(),
pusher_did: event.pusher_did.clone(),
repo: event.repo.clone(),
owner_did: event.owner_did.clone(),
ref_name: event.ref_name.clone(),
old_sha: event.old_sha.clone(),
new_sha: event.new_sha.clone(),
@@ -438,67 +432,3 @@ pub async fn start(
Ok(handle)
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn ref_update_event_round_trip_with_owner_did() {
let event = RefUpdateEvent {
node_did: "did:key:zNode".into(),
pusher_did: "did:key:zPusher".into(),
repo: "zOwner/myrepo".into(),
owner_did: Some("did:key:zOwner".into()),
ref_name: "refs/heads/main".into(),
old_sha: "0000000000000000000000000000000000000000".into(),
new_sha: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa".into(),
timestamp: "2026-07-02T12:00:00Z".into(),
cert_id: None,
cid: None,
};
let json = serde_json::to_value(&event).unwrap();
// owner_did must be present in the serialized output
assert_eq!(json["owner_did"], "did:key:zOwner");
assert_eq!(json["repo"], "zOwner/myrepo");
let deserialized: RefUpdateEvent = serde_json::from_value(json).unwrap();
assert_eq!(deserialized.owner_did, Some("did:key:zOwner".into()));
}
#[test]
fn ref_update_event_backward_compat_no_owner_did() {
let old_json = serde_json::json!({
"node_did": "did:key:zNode",
"pusher_did": "did:key:zPusher",
"repo": "zOwner/myrepo",
"ref_name": "refs/heads/main",
"old_sha": "0000000000000000000000000000000000000000",
"new_sha": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
"timestamp": "2026-07-02T12:00:00Z",
"cert_id": null,
"cid": null
});
let deserialized: RefUpdateEvent = serde_json::from_value(old_json).unwrap();
assert_eq!(deserialized.owner_did, None);
assert_eq!(deserialized.repo, "zOwner/myrepo");
}
#[test]
fn ref_update_event_backward_compat_null_owner_did() {
let with_null = serde_json::json!({
"node_did": "did:key:zNode",
"pusher_did": "did:key:zPusher",
"repo": "zOwner/myrepo",
"owner_did": null,
"ref_name": "refs/heads/main",
"old_sha": "0000000000000000000000000000000000000000",
"new_sha": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
"timestamp": "2026-07-02T12:00:00Z",
"cert_id": null,
"cid": null
});
let deserialized: RefUpdateEvent = serde_json::from_value(with_null).unwrap();
assert_eq!(deserialized.owner_did, None);
}
}
File diff suppressed because it is too large. Load diff
+1 -16
View File
@@ -154,28 +154,13 @@ impl RateLimiter {
}
/// Number of distinct keys currently tracked. Test-only introspection so a
/// cross-module test can assert that a sweep actually evicted expired entries
/// and observe what it reclaimed. There is no production reader.
/// cross-module test can assert that a sweep actually evicted expired entries.
#[cfg(test)]
pub(crate) async fn tracked_keys(&self) -> usize {
self.state.lock().await.len()
}
}
/// Per-source concurrency cap derived from the write-pool size: one resolved client
/// key (see [`client_key`]) may hold at most an eighth of the pool, so saturating it
/// takes ~8 distinct keys. Real for an IPv4 or single-address caller; a caller with a
/// routed IPv6 /64 has 2^64 keys, because `client_key` returns the full address with
/// no prefix folding. Narrowing that keying is a deferred decision, so this cap is a
/// bound per key, not a bound per operator.
///
/// Floored at 1 because the value feeds [`PerCallerConcurrency`], where a cap of 0
/// would shed EVERY receive-pack advertisement and break all pushes. The floor is
/// load-bearing at the minimum write-pool size (1), which integer-divides to 0.
pub(crate) fn per_source_push_cap(max_concurrent_git_pushes: usize) -> usize {
(max_concurrent_git_pushes / 8).max(1)
}
/// A bounded per-caller CONCURRENCY limiter — distinct from [`RateLimiter`], which
/// caps request RATE. Each caller key may hold at most `per_caller` in-flight
/// permits at once; beyond that [`try_acquire`](Self::try_acquire) returns `None`
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
+14 -448
View File
@@ -85,24 +85,21 @@ fn inv22_concurrency_gates_present_and_not_bypassed() {
is reaped on disconnect (the path-scoped half of #174 P1-a)"
);
// U2 / R1: a cancelled or timed-out `GET /ipfs/{cid}` must release admission only
// after the blocking work it admitted has finished, not the instant the handler
// future drops (the /ipfs half of #174 P1-a). The mechanism is the shared
// `Arc<WalkAdmission>`: both walk permits are moved into it once per request and a
// clone rides every `spawn_blocking`, so the permits release when the LAST holder
// drops. Reverting to handler-local permits trips this; the per-site clones are
// bound separately by `inv22_ipfs_walk_admission_reaches_every_blocking_site`.
//
// This previously required the permits to be owned by a detached `tokio::spawn`
// running the whole pipeline. That closed the same bypass but kept an abandoned
// request's full legacy scan running against a held slot, so the merge of #173 and
// #174 kept the Arc and dropped the detached task.
// U2 / R1 (#173 round-10): the `GET /ipfs/{cid}` serve pipeline must run in a
// DETACHED tokio task that OWNS the AdmissionGuard built from BOTH walk permits, so a
// cancelled or timed-out request releases admission only after the spawned (bounded)
// work completes — not the instant the handler future drops (the /ipfs half of #174
// P1-a). Two load-bearing markers: the guard is constructed from the two named
// permits, and the whole pipeline is moved into a `tokio::spawn`. Reverting to
// handler-local permits (dropping the guard/spawn) trips this.
assert!(
ipfs.contains("struct WalkAdmission")
&& ipfs.contains("let admission = std::sync::Arc::new(WalkAdmission {"),
"U2/R1 gate missing: get_by_cid must move both /ipfs admission permits into a \
shared Arc<WalkAdmission> whose clones ride the blocking work, so admission is \
released only once that work completes (the /ipfs half of #174 P1-a)"
ipfs.contains("AdmissionGuard::new(ipfs_walk_permit, ipfs_caller_permit)")
&& ipfs.contains(
"let serve: tokio::task::JoinHandle<Result<Response>> = tokio::spawn(async move {"
),
"U2/R1 gate missing: get_by_cid must move both /ipfs admission permits into an \
AdmissionGuard owned by a detached tokio::spawn task so admission is released \
only after the spawned serve work completes (the /ipfs half of #174 P1-a)"
);
// U2 / KTD2 (#173 round-10): the probe/read children on the /ipfs path must be the
@@ -139,435 +136,4 @@ fn inv22_concurrency_gates_present_and_not_bypassed() {
"U4/P2-2 gate missing: the detached post-push encryption spawn must consult \
encrypt_inflight.try_begin to coalesce per repo (bound the outstanding-task set)"
);
// F5: coalescing must REQUEUE, not shed. The in-flight task pins only its own
// pre-spawn snapshot, so a coalesced push's tip pairs are recorded on its key and
// the task must loop-drain them (`finish_or_take_pending`) before releasing it.
// Removing the drain reverts to the silent loss: a coalesced push's pins and
// recovery copies are absent until an unrelated later push. Scan only the
// production half of the file — the u5 tests in its `mod tests` also name the
// drain call, and matching them would make this check vacuous.
// Split at the TEST MODULE, not at the first `#[cfg(test)]`. api/repos.rs carries
// test-only items (the drain fault seam, the task entry point) ABOVE the production
// code these gates scan for, so splitting on the attribute truncated the production
// half above every line being checked and the gate went vacuously blind.
let repos_production = repos
.split("\nmod tests {")
.next()
.expect("split always yields a first chunk");
assert!(
repos_production.contains("finish_or_take_pending"),
"F5 gate missing: the post-push encryption task must loop-drain coalesced \
pushes via finish_or_take_pending before releasing its repo key"
);
// F4: every post-receive scan helper admits itself to the shared scan pool via
// `crate::state::acquire_scan_permit` BEFORE its spawn_blocking git walk, so a
// push burst cannot accumulate unbounded concurrent scans once the write permit
// is released. Two halves, both load-bearing: the helper body must actually
// acquire the pool (state.rs sits the helper at the end of the file, so the
// definition tail contains no other `acquire_owned` to match vacuously), and
// within each scan helper the first qualified call precedes the first
// `spawn_blocking`. Severing a call site pushes the next occurrence past the
// helper's own `spawn_blocking` (or off the end of the file), turning the
// assertion red; comments name the helper without the `crate::state::` prefix,
// so this targets the real call sites.
let state_rs = src("state.rs");
let helper_def = state_rs
.find("fn acquire_scan_permit")
.expect("F4 gate missing: state.rs no longer defines acquire_scan_permit");
assert!(
state_rs[helper_def..].contains("acquire_owned"),
"F4 gate gutted: acquire_scan_permit must acquire the scan pool via acquire_owned"
);
let push_delta = src("git/push_delta.rs");
for (file_src, file, helper) in [
(&repos, "api/repos.rs", "fn replication_withheld_set"),
(&repos, "api/repos.rs", "fn fail_closed_full_scan_objects"),
(
&push_delta,
"git/push_delta.rs",
"fn resolve_candidates_for_push",
),
] {
let start = file_src
.find(helper)
.unwrap_or_else(|| panic!("{file}: `{helper}` not found"));
let tail = &file_src[start..];
let acquire = tail
.find("crate::state::acquire_scan_permit(")
.unwrap_or_else(|| {
panic!("F4 gate missing: {file} `{helper}` no longer acquires a scan permit")
});
let spawn = tail.find("spawn_blocking").unwrap_or_else(|| {
panic!("{file}: `{helper}` lost its spawn_blocking walk — update this guard")
});
assert!(
acquire < spawn,
"F4 gate bypassed: {file} `{helper}` must acquire its git_encrypt_semaphore \
permit BEFORE dispatching the blocking git scan"
);
}
}
/// F4 (repo_store advisory-unlock cancellation safety): `RepoWriteGuard::release`
/// must await `pg_advisory_unlock` while `self` still owns the pooled connection,
/// and must not mark itself `released` until that await resolves. Either shape,
/// reintroduced, re-opens the mid-unlock cancellation leak: taking the connection
/// early leaves `Drop` with `conn == None`, and setting `released = true` early
/// leaves the `Drop` backstop inert — both strand the session lock on cancellation.
///
/// Scoped to the `release` fn body: the `Drop` impl legitimately takes the
/// connection and unlocks, so a whole-file scan would match it and read as a false
/// pass. Reverting either ordering turns this red (proven load-bearing).
#[test]
fn f4_release_keeps_conn_owned_until_unlock_resolves() {
let repo_store = src("git/repo_store.rs");
let rel_start = repo_store
.find("pub async fn release(mut self")
.expect("F4 gate: repo_store.rs no longer defines RepoWriteGuard::release");
let rel_end = repo_store[rel_start..]
.find("impl Drop for RepoWriteGuard")
.map(|off| rel_start + off)
.expect("F4 gate: release fn / Drop impl markers moved — update this guard");
let release_body = &repo_store[rel_start..rel_end];
let unlock = release_body
.find("pg_advisory_unlock")
.expect("F4 gate: release must still issue pg_advisory_unlock");
let before_unlock = &release_body[..unlock];
// (a) the connection must still be owned by `self` at the unlock await.
assert!(
!before_unlock.contains("self.conn.take()"),
"F4 regression: RepoWriteGuard::release takes self.conn BEFORE awaiting \
pg_advisory_unlock. A cancellation during the unlock await then strands the \
session advisory lock (Drop sees conn == None and skips its backstop). \
Unlock through the still-owned connection instead."
);
// (b) `released` must not be set before the unlock await — the other
// reintroduction shape a single-reorder check on (a) alone is blind to.
assert!(
!before_unlock.contains("released = true"),
"F4 regression: RepoWriteGuard::release sets `released = true` BEFORE awaiting \
pg_advisory_unlock. A cancellation during the await then leaves the Drop \
backstop inert (it early-returns on released). Set released only AFTER the \
unlock await resolves."
);
}
/// F6/KTD-5 (IPFS metadata queries deadline-wrapped): `get_by_cid` acquires
/// the scarce walk permits (RAII, held for the whole request) BEFORE its metadata
/// queries, and the per-repo loop's first budget gate runs only later. So
/// both `list_repos_page_for_scan` and `list_visibility_rules_for_repos` must be clamped to the
/// remaining request budget — otherwise a query blocked in Postgres pins the walk slot
/// for the whole stall, past the budget. This scans the PRODUCTION half of `api/ipfs.rs`
/// (the `mod tests` half names the same calls in its own harness and would make the
/// check vacuous) and asserts each query call is immediately preceded by a
/// `tokio::time::timeout(` wrapper. Removing either wrapper turns this red (proven
/// load-bearing).
#[test]
fn f6_ipfs_metadata_queries_are_deadline_wrapped() {
let ipfs = src("api/ipfs.rs");
// Split at the TEST MODULE, not at the first `#[cfg(test)]`: the handler carries
// in-line `#[cfg(test)]` query counters, so splitting on the attribute cut the
// production half off above the calls this guard exists to check and the scan went
// vacuously quiet (found 0 of each rather than failing on a missing wrapper).
let production = ipfs
.split("\nmod tests {")
.next()
.expect("split always yields a first chunk");
// EVERY occurrence must be wrapped, not just the first. The handler now reaches
// these queries from two places (the provenance fast path, one repo at a time, and
// the legacy scan's preload), and an exact-count check would have to be relaxed
// every time a call site is added, which is how a guard quietly stops covering the
// site that matters. Requiring all of them scales with the code instead.
for call in [
".list_repos_page_for_scan(",
".list_visibility_rules_for_repos(",
".get_repo_by_id(",
".is_repo_quarantined(",
] {
let occurrences = production.matches(call).count();
assert!(
occurrences >= 1,
"F6 guard stale: api/ipfs.rs no longer calls `{call}` — update this guard"
);
for (n, (idx, _)) in production.match_indices(call).enumerate() {
// The wrapper opens a few lines above the call (match tokio::time::timeout(
// ... remaining budget ..., <query> )); a 240-char lookback covers it
// without reaching the previous statement.
let window = &production[idx.saturating_sub(240)..idx];
assert!(
window.contains("tokio::time::timeout("),
"F6 gate missing: occurrence {n} of `{call}` (of {occurrences}) is not \
wrapped in tokio::time::timeout(...) clamped to the remaining request \
budget. An unwrapped await pins the held walk permit for the whole DB \
stall, past GITLAWB_IPFS_REQUEST_BUDGET_SECS."
);
}
}
}
/// #174 F2 / KTD-3: the detached post-receive Pinata replication task must enqueue
/// only the push's `(ref, old, new)` tuples and RE-DERIVE its object set inside the
/// worker once a pin slot frees — it must NOT move the full per-push object list into
/// the closure and hold it across the `pin_semaphore` acquire. Retaining the list makes
/// every parked task (under a slow Pinata backend) hold an MB-scale OID list, so
/// outstanding memory grows O(pushes x object-list). Coalescing/shedding the task is
/// forbidden (its per-ref effects are non-idempotent), so the fix bounds the retained
/// data, not the task count.
///
/// Two load-bearing checks, both against the PRODUCTION half of `api/repos.rs` (the
/// `mod tests` half names the same identifiers in its own harness and would make the
/// scan vacuous): (a) the closure-local `object_list_pinata` binding — the retain form —
/// must be gone; reintroducing `let object_list_pinata = object_list;` turns this red.
/// (b) the re-derivation (`pinata_object_list_for_refs`) must appear AFTER the Pinata
/// pin permit is acquired, so the object list is materialized only inside the pin-bounded
/// section and a parked task holds O(ref tuples).
#[test]
fn f2_pinata_enqueues_refs_not_retained_object_lists() {
let repos = src("api/repos.rs");
// Split at the TEST MODULE, not at the first `#[cfg(test)]`. api/repos.rs carries
// test-only items (the drain fault seam, the task entry point) ABOVE the production
// code these gates scan for, so splitting on the attribute truncated the production
// half above every line being checked and the gate went vacuously blind.
let production = repos
.split("\nmod tests {")
.next()
.expect("split always yields a first chunk");
// (a) the retained-list form must be gone from production.
assert!(
!production.contains("object_list_pinata"),
"F2/KTD-3 regression: the Pinata task retains a full per-push object list \
(`object_list_pinata`) across the pin-permit acquire. Enqueue only the ref \
tuples and re-derive the object set inside the worker (pinata_object_list_for_refs)."
);
// (b) the re-derivation runs AFTER the pin permit is acquired. Anchor on the
// Pinata pin-admission clone so the window is the Pinata task, not the sibling
// IPFS/encrypt spawn (which shares `pin_semaphore` but never re-derives).
let anchor = production
.find("let pin_sem_pinata")
.expect("F2 gate stale: the Pinata pin-admission clone (pin_sem_pinata) moved");
let tail = &production[anchor..];
let acquire = tail
.find(".acquire_owned()")
.expect("F2 gate: the Pinata task no longer acquires a pin permit (acquire_owned)");
let rederive = tail.find("pinata_object_list_for_refs(").expect(
"F2 gate missing: the Pinata task must re-derive its object set via \
pinata_object_list_for_refs; it can no longer move a pre-resolved list into the closure",
);
assert!(
acquire < rederive,
"F2 gate bypassed: the Pinata object-set re-derivation must run AFTER the pin \
permit is acquired, so a parked task never holds the MB-scale object list"
);
// The re-derivation is driven by the push's ref tuples (the enqueued unit), not a
// retained object list — tie "enqueue ref_updates" to the call explicitly.
assert!(
tail[rederive..].starts_with("pinata_object_list_for_refs(")
&& tail[rederive..]
.get(..400)
.map(|w| w.contains("ref_updates_clone"))
.unwrap_or(false),
"F2 gate: pinata_object_list_for_refs must re-derive from the push's ref tuples \
(ref_updates_clone), the small unit the parked task retains"
);
}
/// #174 U2 / F3 (second same-repo writer serialized until a disconnected first push's
/// git group is reaped): `git_receive_pack` must take the per-repo in-process write
/// lease and CARRY it on the write-path `AdmissionGuard` (via `.with_lease`), so the
/// lease rides `KillGroupOnDrop`'s detached reaper and frees only after the group is
/// reaped — supplementing the pg advisory lock, which `RepoWriteGuard::Drop` releases at
/// the disconnect instant. Tying the lease to `RepoWriteGuard` instead (or dropping the
/// `.with_lease` carry) reopens the race; the behavioral RED/GREEN test
/// (`f3_second_push_serialized_until_disconnected_group_reaped`) is the real bar, this
/// is the completeness tripwire. Scanned against the PRODUCTION half of `api/repos.rs`
/// (the `mod tests` half names these identifiers in its own harness).
#[test]
fn f3_second_writer_leased_until_reap() {
let repos = src("api/repos.rs");
let smart_http = src("git/smart_http.rs");
// Split at the TEST MODULE, not at the first `#[cfg(test)]`. api/repos.rs carries
// test-only items (the drain fault seam, the task entry point) ABOVE the production
// code these gates scan for, so splitting on the attribute truncated the production
// half above every line being checked and the gate went vacuously blind.
let repos_production = repos
.split("\nmod tests {")
.next()
.expect("split always yields a first chunk");
// The lease is acquired, then carried by the AdmissionGuard (.with_lease), before
// receive_pack runs the write. Severing any of the three turns this red.
let lease_acquire = repos_production.find("repo_write_leases").expect(
"F3 gate missing: git_receive_pack no longer takes the per-repo write lease \
(state.repo_write_leases)",
);
let with_lease = repos_production.find(".with_lease(").expect(
"F3 gate missing: the write-path AdmissionGuard no longer carries the lease \
(.with_lease). The lease must ride the disconnect reaper via the AdmissionGuard, \
NOT RepoWriteGuard (which drops at the disconnect instant, reopening F3).",
);
let receive = repos_production
.find("smart_http::receive_pack(")
.expect("F3 gate stale: git_receive_pack no longer calls smart_http::receive_pack");
assert!(
lease_acquire < with_lease && with_lease < receive,
"F3 gate bypassed: the write lease must be acquired, then carried by the \
AdmissionGuard (.with_lease), BEFORE receive_pack runs the write"
);
// The AdmissionGuard must actually hold the lease (Option<RepoWriteLease>) via a
// with_lease setter, so it travels into the detached reaper. Removing the field or
// method turns this red.
assert!(
smart_http.contains("_lease: Option<crate::state::RepoWriteLease>")
&& smart_http.contains("pub fn with_lease("),
"F3 gate missing: AdmissionGuard must hold an Option<RepoWriteLease> set via \
with_lease, so the lease rides the guard into KillGroupOnDrop's reaper"
);
}
/// #174 U1 — every blocking walk in the `/ipfs` scan carries the request's walk
/// admission.
///
/// The admission is an `Arc<WalkAdmission>` cloned into each `spawn_blocking`
/// closure, so the global + per-source permits release only when the last holder
/// drops: a client disconnect leaves the abandoned closure holding the slot, and a
/// panicking closure leaves the handler holding it.
///
/// Only the walk site runs under `state.git_bin`, so only it can be pinned by the
/// fake-git harness and mutation-verified dynamically (see
/// `get_by_cid_walk_permit_held_through_blocking_walk`). The probe and the content
/// read deliberately shell to the real `git`, which is why this structural check
/// exists: it binds all three sites, and any blocking site added to this loop
/// later, without reversing that deliberate independence.
///
/// MUTATION (RED): delete any one `Arc::clone(ctx.admission)` binding, or drop the
/// clone from inside its closure, and the count falls below three.
#[test]
fn inv22_ipfs_walk_admission_reaches_every_blocking_site() {
let ipfs = src("api/ipfs.rs");
// The shared owner must exist and be built once per request.
assert!(
ipfs.contains("struct WalkAdmission") && ipfs.contains("Arc::new(WalkAdmission {"),
"U1 gate missing: the /ipfs walk admission must be a shared WalkAdmission, \
not a handler-local permit pair"
);
// Every blocking site in the scan takes its own clone...
// `Arc::clone(ctx.admission)` in the gate, `Arc::clone(&admission)` if a site is
// ever added back in the handler body itself; count both spellings.
let clones = ipfs.matches("Arc::clone(ctx.admission)").count()
+ ipfs.matches("Arc::clone(&admission)").count();
assert!(
clones >= 3,
"U1 gate bypassed: expected an admission clone for each of the three \
/ipfs spawn_blocking sites (probe, walk, read); found {clones}. A blocking \
walk that does not hold the admission lets a disconnect or a panic free the \
slot while its git child is still running."
);
// ...and each clone is actually moved INTO the blocking closure, not merely
// created in the async frame (which would hold nothing across the join).
let held = ipfs.matches("let _admission = ").count();
assert!(
held >= 3,
"U1 gate bypassed: each admission clone must be bound inside its \
spawn_blocking closure so the blocking work owns it; found {held} of 3."
);
// The count of blocking sites is itself the thing being covered: if a fourth
// appears, it needs an admission clone too and this gate must be revisited.
let sites = ipfs.matches("spawn_blocking(move ||").count();
assert_eq!(
sites, 3,
"the /ipfs scan grew or lost a spawn_blocking site ({sites} found, expected 3); \
give any new blocking walk its own Arc::clone(&admission) and update this gate"
);
}
/// #174 U5: the post-receive replication tail is spawned at the DURABILITY BOUNDARY,
/// which is the moment receive-pack returns success, not the end of the handler and
/// not after `guard.release()`.
///
/// The tail owes this push its pins, recovery copy, and announcements. Everything
/// below the spawn stays in the cancellable request future, so anything the tail is
/// spawned after is a window where a client disconnect drops that work while the pack
/// is already durable on disk. `guard.release()` is such a window: on success it
/// awaits the Tigris upload and then the advisory unlock.
///
/// The lower bound matters just as much as the upper one: `release` runs on failure
/// too, so an ungated spawn would fire for a push git rejected, pinning and announcing
/// a half-applied repo. Above `release` the `?` on `receive_result` can no longer be
/// what gates it, so the success check is explicit and this gate binds it: the spawn
/// must sit inside `if push_succeeded`, and `release` must consume the same flag so
/// the two cannot drift apart.
///
/// This is an ordering check rather than a cancellation-race test on purpose: it is
/// the companion to `receive_pack_tail_survives_a_disconnect_during_release`, which
/// drives the actual disconnect through a parked `release`. Same instrument the F3
/// gate above uses.
///
/// MUTATION (RED): move the `tokio::spawn(post_receive_replication_tail` call below
/// `guard.release(` and the ordering assertion fails; take it out of the
/// `if push_succeeded` block and the failed-push assertion fails.
#[test]
fn inv22_replication_tail_spawns_at_the_durability_boundary() {
let repos = src("api/repos.rs");
// Production half only — the tests below name these identifiers too.
// Split at the TEST MODULE, not at the first `#[cfg(test)]`. api/repos.rs carries
// test-only items (the drain fault seam, the task entry point) ABOVE the production
// code these gates scan for, so splitting on the attribute truncated the production
// half above every line being checked and the gate went vacuously blind.
let production = repos
.split("\nmod tests {")
.next()
.expect("split always yields a first chunk");
let success_flag = production
.find("let push_succeeded = receive_result.is_ok();")
.expect(
"U5 gate missing: the tail's success gate must be bound from receive_result.is_ok()",
);
let gate_open = production
.find("if push_succeeded {")
.expect("U5 gate missing: the tail spawn must be gated on the push having succeeded");
let spawn = production
.find("tokio::spawn(post_receive_replication_tail(")
.expect("U5 gate missing: the replication tail must be spawned by git_receive_pack");
let release = production
.find(".release(push_succeeded)")
.expect("U5 gate stale: release must consume the same success flag as the tail gate");
let touch = production
.find("state.db.touch_repo(")
.expect("U5 gate stale: git_receive_pack no longer calls touch_repo");
let webhook = production
.find("webhooks::fire_event(")
.expect("U5 gate stale: git_receive_pack no longer fires push webhooks");
assert!(
success_flag < gate_open && gate_open < spawn,
"U5 gate bypassed: the tail must be spawned inside `if push_succeeded`, or a \
rejected push spawns a tail that pins and announces a half-applied repo"
);
// Still inside that block: no `}` may close it between the gate and the spawn.
assert!(
!production[gate_open + "if push_succeeded {".len()..spawn].contains('}'),
"U5 gate bypassed: the tail spawn left the `if push_succeeded` block, so a \
rejected push now spawns a tail"
);
assert!(
spawn < release && spawn < touch && spawn < webhook,
"U5 gate bypassed: the tail must be spawned BEFORE guard.release, touch_repo \
and the webhook fan-out, so a disconnect in any of those windows cannot drop \
this push's pins, recovery copy, and announcements"
);
}
+2 -3
View File
@@ -1,7 +1,7 @@
[package]
name = "gl"
description = "The gitlawb CLI — identity management, node control, MCP server"
version = "0.7.1" # x-release-please-version
version = "0.5.1" # x-release-please-version
edition.workspace = true
rust-version.workspace = true
license.workspace = true
@@ -11,9 +11,8 @@ name = "gl"
path = "src/main.rs"
[dependencies]
gitlawb-core = { path = "../gitlawb-core", features = ["redirect"] }
gitlawb-core = { path = "../gitlawb-core" }
icaptcha-client = { path = "../icaptcha-client" }
base64 = { workspace = true }
tokio = { workspace = true }
serde = { workspace = true }
serde_json = { workspace = true }
+27 -216
View File
@@ -41,16 +41,6 @@ pub enum CertCmd {
node: String,
#[arg(long)]
dir: Option<PathBuf>,
/// Exit non-zero unless the Ed25519 signature verifies AND the
/// issuing node matches the queried node (or --expect-node)
#[arg(long)]
verify: bool,
/// Expected issuing node DID for --verify. A valid signature alone
/// only proves the cert is internally consistent — signed by whatever
/// key it names — so --verify also anchors the issuer to a DID you
/// trust: this value when given, else the queried node's DID.
#[arg(long, requires = "verify")]
expect_node: Option<String>,
},
}
@@ -62,9 +52,7 @@ pub async fn run(args: CertArgs) -> Result<()> {
id,
node,
dir,
verify,
expect_node,
} => cmd_show(repo, id, node, dir, verify, expect_node).await,
} => cmd_show(repo, id, node, dir).await,
}
}
@@ -126,14 +114,7 @@ async fn cmd_list(repo: String, node: String, dir: Option<PathBuf>) -> Result<()
Ok(())
}
async fn cmd_show(
repo: String,
id: String,
node: String,
dir: Option<PathBuf>,
require_valid: bool,
expect_node: Option<String>,
) -> Result<()> {
async fn cmd_show(repo: String, id: String, node: String, dir: Option<PathBuf>) -> Result<()> {
let (owner, name) = resolve_repo(&repo, &node, dir.as_deref()).await?;
let client = signed_client(&node, dir.as_deref());
@@ -167,124 +148,39 @@ async fn cmd_show(
println!(" Signature: {signature}");
println!();
// Verify the Ed25519 signature: rebuild the exact canonical payload the
// node signed (see gitlawb-node/src/cert.rs::issue_ref_certificate) and
// check it against the public key embedded in the certificate's node DID.
// This proves the cert is internally authentic — signed by the key it
// names; the node-DID comparison below covers *which* node that is.
let repo_id = cert["repo_id"].as_str().unwrap_or("");
let verdict = verify_signature(
repo_id, ref_name, old_sha, new_sha, pusher, node_did, issued_at, signature,
);
// Reconstruct the signing payload and verify
// Fetch the node's current public key to verify
let info: Value = client
.get("/")
.await?
.json()
.await
.context("failed to fetch node info")?;
let current_node_did = info["did"].as_str().unwrap_or("");
println!("Signature verification:");
match &verdict {
Ok(()) => {
println!(
" VALID — Ed25519 signature verified against the key the certificate names ({node_did})"
);
}
Err(reason) => {
println!(" INVALID — {reason}");
}
println!(" Signing payload would be:");
println!(" {{\"repo_id\": ..., \"ref\": \"{ref_name}\", \"old\": \"{old_sha}\",");
println!(" \"new\": \"{new_sha}\", \"pusher\": \"{pusher}\",");
println!(" \"node\": \"{node_did}\", \"ts\": \"{issued_at}\"}}");
println!();
if current_node_did == node_did {
println!(" Node DID matches current node. Signature is an Ed25519/base64url value.");
println!(" To verify offline, use the node's Ed25519 public key derived from:");
println!(" did:key → {node_did}");
} else {
println!(" WARNING: Certificate node DID ({node_did}) does not match");
println!(" current node DID ({current_node_did}).");
println!(" This certificate was issued by a different node.");
}
// Contextual only — the verdict above stands on its own, so a node-info
// hiccup here must not turn a successfully displayed certificate into an
// error exit.
let current_node_did = match client.get("/").await {
Ok(resp) => resp
.json::<Value>()
.await
.ok()
.and_then(|info| info["did"].as_str().map(str::to_string)),
Err(_) => None,
};
match current_node_did.as_deref() {
Some(current) if current == node_did => {
println!(" Issuing node DID matches the node being queried.");
}
Some(current) => {
println!(" WARNING: Certificate node DID ({node_did}) does not match");
println!(" current node DID ({current}).");
println!(" This certificate was issued by a different node.");
}
None => {
println!(" NOTE: could not fetch current node info — skipping node-DID comparison.");
}
}
if require_valid {
if let Err(reason) = verdict {
anyhow::bail!("certificate signature did not verify: {reason}");
}
// A valid signature proves internal consistency only: the payload was
// signed by whatever key the certificate itself names. A hostile
// source can mint a keypair, put its DID in node_did, and self-sign.
// --verify therefore also anchors the issuer to a trusted DID:
// --expect-node when given, else the DID of the node being queried.
let expected = expect_node.as_deref().or(current_node_did.as_deref());
match expected {
Some(expected) if expected == node_did => {}
Some(expected) => anyhow::bail!(
"certificate is signed by {node_did}, but the expected issuer is {expected} — \
a valid signature alone proves internal consistency, not a trusted issuer"
),
None => anyhow::bail!(
"cannot anchor the issuer: node info is unreachable and no --expect-node was given"
),
}
}
println!();
println!(" Signature (base64url): {signature}");
Ok(())
}
/// Rebuild the node's canonical signing payload (field order must match
/// gitlawb-node/src/cert.rs::issue_ref_certificate exactly) and verify the
/// certificate's Ed25519 signature against the key embedded in `node_did`.
#[allow(clippy::too_many_arguments)]
fn verify_signature(
repo_id: &str,
ref_name: &str,
old_sha: &str,
new_sha: &str,
pusher: &str,
node_did: &str,
issued_at: &str,
signature_b64: &str,
) -> std::result::Result<(), String> {
use base64::{engine::general_purpose::URL_SAFE_NO_PAD, Engine};
use std::str::FromStr;
let payload = serde_json::json!({
"repo_id": repo_id,
"ref": ref_name,
"old": old_sha,
"new": new_sha,
"pusher": pusher,
"node": node_did,
"ts": issued_at,
});
let payload_bytes =
serde_json::to_vec(&payload).map_err(|e| format!("could not serialize payload: {e}"))?;
let did =
gitlawb_core::did::Did::from_str(node_did).map_err(|e| format!("bad node DID: {e}"))?;
let verifying_key = did
.to_verifying_key()
.map_err(|e| format!("cannot derive a public key from {node_did}: {e}"))?;
let sig_vec = URL_SAFE_NO_PAD
.decode(signature_b64)
.map_err(|e| format!("signature is not valid base64url: {e}"))?;
let sig_bytes: [u8; 64] = sig_vec
.try_into()
.map_err(|_| "signature is not 64 bytes".to_string())?;
gitlawb_core::identity::verify(&verifying_key, &payload_bytes, &sig_bytes)
.map_err(|_| "Ed25519 signature does not match the signed payload".to_string())
}
async fn resolve_cert_id(client: &NodeClient, owner: &str, name: &str, id: &str) -> Result<String> {
if id.len() >= 36 {
return Ok(id.to_string());
@@ -313,88 +209,3 @@ async fn resolve_cert_id(client: &NodeClient, owner: &str, name: &str, id: &str)
_ => anyhow::bail!("certificate prefix {id} matches multiple certificates"),
}
}
#[cfg(test)]
mod tests {
use super::*;
/// Pins gl's payload reconstruction to the frozen canonical byte form the
/// node signs (default serde_json maps = alphabetically ordered keys). If
/// serialization drifts — a field added, or a preserve_order feature
/// landing anywhere in the workspace (feature unification flips every
/// crate at once) — this literal stops matching and the test fails,
/// instead of every real certificate silently rendering INVALID.
#[test]
fn payload_serialization_matches_frozen_canonical_form() {
let payload = serde_json::json!({
"repo_id": "repo-1",
"ref": "refs/heads/main",
"old": "oldsha",
"new": "newsha",
"pusher": "did:key:z6MkPusher",
"node": "did:key:z6MkNode",
"ts": "2026-07-22T00:00:00+00:00",
});
let frozen = concat!(
r#"{"new":"newsha","node":"did:key:z6MkNode","old":"oldsha","#,
r#""pusher":"did:key:z6MkPusher","ref":"refs/heads/main","#,
r#""repo_id":"repo-1","ts":"2026-07-22T00:00:00+00:00"}"#,
);
assert_eq!(serde_json::to_string(&payload).unwrap(), frozen);
}
/// Signing exactly as the node does must round-trip through
/// verify_signature; any field tampering must fail it.
#[test]
fn verify_signature_round_trip_and_tamper() {
let kp = gitlawb_core::identity::Keypair::generate();
let node_did = kp.did().as_str().to_string();
let payload = serde_json::json!({
"repo_id": "repo-1",
"ref": "refs/heads/main",
"old": "0".repeat(40),
"new": "a".repeat(40),
"pusher": "did:key:z6MkPusher",
"node": node_did,
"ts": "2026-07-22T00:00:00+00:00",
});
let sig = kp.sign_b64(&serde_json::to_vec(&payload).unwrap());
let ok = verify_signature(
"repo-1",
"refs/heads/main",
&"0".repeat(40),
&"a".repeat(40),
"did:key:z6MkPusher",
&node_did,
"2026-07-22T00:00:00+00:00",
&sig,
);
assert!(ok.is_ok(), "expected valid signature, got: {ok:?}");
let tampered = verify_signature(
"repo-1",
"refs/heads/main",
&"0".repeat(40),
&"b".repeat(40), // new_sha changed after signing
"did:key:z6MkPusher",
&node_did,
"2026-07-22T00:00:00+00:00",
&sig,
);
assert!(tampered.is_err(), "tampered payload must not verify");
let garbage = verify_signature(
"repo-1",
"refs/heads/main",
&"0".repeat(40),
&"a".repeat(40),
"did:key:z6MkPusher",
&node_did,
"2026-07-22T00:00:00+00:00",
"not-base64url!!!",
);
assert!(garbage.is_err(), "malformed signature must not verify");
}
}
+8 -259
View File
@@ -12,58 +12,9 @@ use serde::Deserialize;
use std::path::Path;
use std::process::Command;
use crate::http::{sanitize_node_msg, NodeClient};
use crate::http::NodeClient;
use crate::identity::load_keypair_from_dir;
/// Report why one candidate blob could not be recovered, in the register of the
/// recovery loop's other warnings.
///
/// Both halves are gateway-supplied text on their way to a terminal: the oid comes
/// out of a manifest fetched from a caller-chosen Arweave gateway, and a transport
/// error carries the URL the same manifest chose. So the whole line is defanged and
/// length-capped, like every other node/gateway string this crate prints.
///
/// Under `cfg(test)` the line is also mirrored into a per-thread buffer, so a test
/// can assert the loop actually says what it skipped rather than only that it
/// returned nothing.
fn warn_skip(oid: &str, why: &str) {
emit_warning(&format!(
"warning: could not fetch encrypted blob {oid}: {why}; skipping"
));
}
/// The same report for a manifest rather than a blob. The manifest is one step
/// earlier in the same recovery: losing it silently means every blob it would have
/// named is missing with no reason given anywhere.
fn warn_skip_manifest(id: &str, why: &str) {
emit_warning(&format!(
"warning: could not fetch blob manifest {id}: {why}; skipping"
));
}
/// Sanitize a warning and write it, once.
///
/// The write goes to [`warn_sink`], which is stderr in a normal build and the tests'
/// per-thread mirror under `cfg(test)`. That indirection is the point: the mirror
/// used to sit BESIDE an `eprintln!`, so deleting the user-visible write left every
/// assertion on the mirror green and the shipped behaviour could be removed with the
/// tests unchanged. There is one write now, and the tests observe it.
fn emit_warning(line: &str) {
use std::io::Write;
let line = sanitize_node_msg(line);
let _ = writeln!(warn_sink(), "{line}");
}
#[cfg(not(test))]
fn warn_sink() -> impl std::io::Write {
std::io::stderr()
}
#[cfg(test)]
fn warn_sink() -> impl std::io::Write {
tests::WarnMirror
}
#[derive(Args)]
pub struct CloneArgs {
/// Repo to clone: gitlawb://<owner_did>/<name> or <owner>/<name>.
@@ -361,25 +312,10 @@ async fn recover_encrypted_blobs(
.await
{
Ok(r) if r.status().is_success() => r,
// The node path has the same silent exit the gateway path had: a 403, a
// 404, or a dead connection all reached the caller as "blob not
// recoverable" with no reason attached. One unreachable blob still must
// not end the recovery of the rest, so it warns and moves on.
Ok(r) => {
warn_skip(oid, &format!("node returned {}", r.status()));
continue;
}
Err(e) => {
warn_skip(oid, &format!("node request failed: {e}"));
continue;
}
_ => continue,
};
let envelope = match env_resp.bytes().await {
Ok(b) => b,
Err(e) => {
warn_skip(oid, &format!("reading the envelope failed: {e}"));
continue;
}
let Ok(envelope) = env_resp.bytes().await else {
continue;
};
let plaintext = match open_blob(&envelope, keypair) {
Ok(p) => p,
@@ -706,14 +642,7 @@ async fn recover_from_arweave(
for r in refs {
let m = match client.get(format!("{ag}/{}", r.id)).send().await {
Ok(resp) if resp.status().is_success() => resp,
Ok(resp) => {
warn_skip_manifest(&r.id, &format!("gateway returned {}", resp.status()));
continue;
}
Err(e) => {
warn_skip_manifest(&r.id, &format!("gateway request failed: {e}"));
continue;
}
_ => continue,
};
if let Ok(parsed) = m.json::<Manifest>().await {
manifests.push((parsed, r.height));
@@ -755,31 +684,10 @@ async fn recover_from_arweave(
}
let env_resp = match client.get(format!("{ig}/ipfs/{cid}")).send().await {
Ok(r) if r.status().is_success() => r,
// Every other outcome used to leave through a bare `continue`, so a
// gateway that answered 503, 404, or nothing at all was reported to the
// caller as "blob not recoverable" with no reason attached. This is the
// second in-repo client of GET /ipfs/{cid} and it has no resume ladder,
// so saying what happened is all the recourse there is. It still skips to
// the next candidate either way: one unreachable blob must not end the
// recovery of the rest.
Ok(r) => {
warn_skip(&oid, &format!("gateway returned {}", r.status()));
continue;
}
Err(e) => {
warn_skip(&oid, &format!("gateway request failed: {e}"));
continue;
}
_ => continue,
};
// A body that dies part-way through is the same mid-read failure the capped
// read exists to stop rendering as silence, and it sits INSIDE the loop whose
// status arms were already fixed.
let envelope = match env_resp.bytes().await {
Ok(b) => b,
Err(e) => {
warn_skip(&oid, &format!("reading the envelope failed: {e}"));
continue;
}
let Ok(envelope) = env_resp.bytes().await else {
continue;
};
// open_blob succeeds only if this caller is a recipient: this is the
// authorization gate (no node, no DID check needed).
@@ -893,42 +801,9 @@ pub async fn run(args: CloneArgs) -> Result<()> {
#[cfg(test)]
mod tests {
use super::*;
use std::cell::RefCell;
use std::process::Command;
use tempfile::TempDir;
thread_local! {
/// Test-visible copy of the stderr warnings `warn_skip` emits.
/// `#[tokio::test]` runs the future on the test's own thread, so a
/// thread-local is enough.
static WARNINGS: RefCell<String> = const { RefCell::new(String::new()) };
}
/// The `cfg(test)` warning sink. `emit_warning` writes here instead of to
/// stderr, so the assertions below observe the shipped write rather than a copy
/// made beside it: delete that write and they go red.
pub(super) struct WarnMirror;
impl std::io::Write for WarnMirror {
fn write(&mut self, buf: &[u8]) -> std::io::Result<usize> {
let text = String::from_utf8_lossy(buf).into_owned();
WARNINGS.with(|w| w.borrow_mut().push_str(&text));
Ok(buf.len())
}
fn flush(&mut self) -> std::io::Result<()> {
Ok(())
}
}
fn reset_warnings() {
WARNINGS.with(|w| w.borrow_mut().clear());
}
fn warnings() -> String {
WARNINGS.with(|w| w.borrow().clone())
}
fn g(args: &[&str], dir: &Path) {
assert!(Command::new("git")
.args(args)
@@ -1380,132 +1255,6 @@ mod tests {
assert_eq!(a.get("o1").map(String::as_str), Some("cidTS"));
}
/// A gateway that refuses one blob must SAY so, and must not take the rest of the
/// recovery down with it.
///
/// The IPFS fetch used to leave through a bare `_ => continue` on every
/// non-success, so a 503 (the truncation status a tuned node makes more likely),
/// a 404, and a dead connection all reached the caller as the same silent "blob
/// not recoverable". Two withheld blobs here: the gateway answers 503 for the
/// first and serves the second. The recovered path proves the loop continued; the
/// warning proves the skip was reported and names both the object and the status.
#[tokio::test]
async fn recover_from_arweave_reports_a_refusing_gateway_and_continues() {
use gitlawb_core::encrypt::seal_blob;
use gitlawb_core::identity::Keypair;
reset_warnings();
let (td, url) = bare_remote(&[
("public/a.txt", b"pub\n"),
("secret/b.txt", b"SECRET B\n"),
("secret/c.txt", b"SECRET C\n"),
]);
let dest = td.path().join("dest");
let bare = url.strip_prefix("file://").unwrap();
assert!(Command::new("git")
.args(["-C", bare, "config", "uploadpack.allowFilter", "true"])
.status()
.unwrap()
.success());
setup_partial_clone(&dest, &url, &["/secret/**".to_string()], &[], None).unwrap();
let oid_of = |path: &str| {
let out = Command::new("git")
.args([
"-C",
dest.to_str().unwrap(),
"rev-parse",
&format!("HEAD:{path}"),
])
.output()
.unwrap();
String::from_utf8_lossy(&out.stdout).trim().to_string()
};
let refused_oid = oid_of("secret/b.txt");
let served_oid = oid_of("secret/c.txt");
// Origin death, so recovery has to go through the gateways.
std::fs::remove_dir_all(bare).unwrap();
let reader = Keypair::generate();
let envelope = seal_blob(b"SECRET C\n", &[reader.verifying_key()]).unwrap();
let refused_cid = "cidrefused";
let served_cid = "cidserved";
let mut server = mockito::Server::new_async().await;
let _gql = server
.mock("POST", "/graphql")
.with_status(200)
.with_header("content-type", "application/json")
.with_body(r#"{"data":{"transactions":{"edges":[{"node":{"id":"TX1"}}]}}}"#)
.create_async()
.await;
let manifest_body = serde_json::json!({
"timestamp": "2026-06-11T00:00:00Z",
"blobs": [
{ "oid": refused_oid, "cid": refused_cid },
{ "oid": served_oid, "cid": served_cid },
],
})
.to_string();
let _tx = server
.mock("GET", "/TX1")
.with_status(200)
.with_header("content-type", "application/json")
.with_body(manifest_body)
.create_async()
.await;
let refusal = server
.mock("GET", format!("/ipfs/{refused_cid}").as_str())
.with_status(503)
.with_header("content-type", "application/json")
.with_body(r#"{"error":"search_incomplete","message":"scan truncated"}"#)
.expect(1)
.create_async()
.await;
let ok = server
.mock("GET", format!("/ipfs/{served_cid}").as_str())
.with_status(200)
.with_body(envelope)
.expect(1)
.create_async()
.await;
let paths = recover_from_arweave(
&server.url(),
&server.url(),
"alice",
"myrepo",
&dest,
&reader,
)
.await
.unwrap();
refusal.assert_async().await;
ok.assert_async().await;
assert_eq!(
paths,
vec!["secret/c.txt".to_string()],
"a refused candidate must not stop the loop reaching the next one"
);
let warnings = warnings();
assert!(
warnings.contains(&refused_oid),
"the skip must name the object it could not fetch, got: {warnings}"
);
assert!(
warnings.contains("503"),
"the skip must name the gateway's status rather than failing silently, \
got: {warnings}"
);
assert!(
!warnings.contains(&served_oid),
"the blob that was served must not be reported as skipped, got: {warnings}"
);
}
/// Read-path end-to-end over a mocked Arweave + IPFS gateway: discover the
/// manifest via GraphQL, fetch it, fetch the envelope, decrypt with the
/// caller's key, and install the previously-withheld blob.
+11 -238
View File
@@ -16,7 +16,6 @@ use std::path::PathBuf;
use crate::http::NodeClient;
const PUBLIC_NODE: &str = "https://node.gitlawb.com";
const GITHUB_API_BASE: &str = "https://api.github.com";
#[derive(Args)]
pub struct DoctorArgs {
@@ -142,20 +141,18 @@ pub async fn run(args: DoctorArgs) -> Result<()> {
// ── 3. GITLAWB_NODE env var ───────────────────────────────────────────
match std::env::var("GITLAWB_NODE") {
// A loopback host is a legitimate setup (self-hosted node, dev
// harness) — the connectivity check below fails loudly if it is not
// actually reachable, so don't red-flag the configuration itself.
Ok(v) if is_loopback_url(&v) => {
checks.push(Check::pass(
Ok(v) if !v.is_empty() && !v.contains("127.0.0.1") && !v.contains("localhost") => {
checks.push(Check::pass("GITLAWB_NODE", v.to_string()));
}
Ok(v) if v.contains("127.0.0.1") || v.contains("localhost") => {
checks.push(Check::fail(
"GITLAWB_NODE",
format!(
"{v} (local node — intentional for self-hosting/dev; unset to target the public network)"
"set to local address ({v}) — git push/clone will fail against remote nodes"
),
"export GITLAWB_NODE=https://node.gitlawb.com",
));
}
Ok(v) if !v.is_empty() => {
checks.push(Check::pass("GITLAWB_NODE", v.to_string()));
}
_ => {
checks.push(Check::fail(
"GITLAWB_NODE",
@@ -245,15 +242,6 @@ pub async fn run(args: DoctorArgs) -> Result<()> {
));
}
// ── 5b. shell alias shadowing ─────────────────────────────────────────
// oh-my-zsh's default `git` plugin aliases gl='git pull', which silently
// shadows this binary in every interactive zsh — the classic symptom is
// `gl` printing "fatal: not a git repository". Aliases beat PATH, so no
// install method can fix it; the user's rc file has to unalias.
if let Some(check) = check_shell_alias_shadowing(dirs::home_dir()) {
checks.push(check);
}
// ── 6. git ────────────────────────────────────────────────────────────
match std::process::Command::new("git")
.arg("--version")
@@ -275,7 +263,7 @@ pub async fn run(args: DoctorArgs) -> Result<()> {
// ── 7. Version up to date ─────────────────────────────────────────────
let current = env!("CARGO_PKG_VERSION");
checks.push(check_version(current, GITHUB_API_BASE).await);
checks.push(check_version(current).await);
// ── Render ────────────────────────────────────────────────────────────
for check in &checks {
@@ -320,83 +308,6 @@ pub async fn run(args: DoctorArgs) -> Result<()> {
}
/// Check if a binary name exists anywhere on PATH.
/// True when the rc file contains a real `unalias` command naming `gl` —
/// not a comment, and not a longer word like `unalias global`. Ordering
/// relative to oh-my-zsh loading is not modeled (an unalias placed before the
/// plugin loads would be ineffective); acceptable slack for a warning check.
fn rc_unaliases_gl(rc: &str) -> bool {
rc.lines().any(|line| {
let line = line.trim_start();
if line.starts_with('#') {
return false;
}
line.split([';', '&', '|']).any(|cmd| {
// A trailing comment must not count: `unalias foo # gl`
let cmd = cmd.split('#').next().unwrap_or("");
let mut tokens = cmd.split_whitespace();
tokens.next() == Some("unalias") && tokens.any(|t| t == "gl")
})
})
}
/// True when the URL's host is exactly a loopback address. Substring checks
/// misclassify hosts like `localhost.example` or URLs with "localhost" in the
/// path, so parse and compare the actual host.
fn is_loopback_url(value: &str) -> bool {
reqwest::Url::parse(value)
.ok()
.and_then(|u| {
u.host_str()
.map(|h| h == "localhost" || h == "127.0.0.1" || h == "[::1]" || h == "::1")
})
.unwrap_or(false)
}
/// Detect interactive-shell setups where an alias shadows `gl` (aliases beat
/// PATH, so no install method can fix this). Best-effort heuristic over
/// ~/.zshrc: an explicit `alias gl=`, or oh-my-zsh's `git` plugin (which
/// ships gl='git pull'). Returns None when nothing suspicious is found or the
/// rc file already contains an `unalias gl`.
fn check_shell_alias_shadowing(home: Option<PathBuf>) -> Option<Check> {
let home = home?;
let rc = std::fs::read_to_string(home.join(".zshrc")).ok()?;
if rc_unaliases_gl(&rc) {
return None;
}
let explicit_alias = rc.lines().any(|l| l.trim_start().starts_with("alias gl="));
// Single-line `plugins=(git ...)` is the overwhelmingly common form; a
// multi-line plugins array slips past this heuristic, which is acceptable
// for a warning-level check.
let omz_git_plugin = home.join(".oh-my-zsh/plugins/git").exists()
&& rc.lines().any(|l| {
let l = l.trim_start();
l.starts_with("plugins=")
&& l.trim_start_matches("plugins=(")
.trim_end_matches(')')
.split_whitespace()
.any(|p| p == "git")
});
if explicit_alias || omz_git_plugin {
let source = if explicit_alias {
"~/.zshrc defines `alias gl=`"
} else {
"oh-my-zsh's git plugin aliases gl='git pull'"
};
Some(Check::warn(
"shell alias",
format!(
"{source} — interactive shells run that instead of this binary \
(symptom: `gl` prints \"fatal: not a git repository\")"
),
"echo 'unalias gl 2>/dev/null' >> ~/.zshrc && source ~/.zshrc (must come after oh-my-zsh loads)",
))
} else {
None
}
}
fn which_in_path(name: &str) -> bool {
std::env::var_os("PATH")
.map(|paths| {
@@ -408,9 +319,8 @@ fn which_in_path(name: &str) -> bool {
.unwrap_or(false)
}
/// Fetch the latest release tag from Gitlawb/node (the actual release repo — see install.sh's
/// `GITLAWB_RELEASE_REPO` default) and compare to current version.
async fn check_version(current: &'static str, github_api_base: &str) -> Check {
/// Fetch the latest release tag from gitlawb/releases and compare to current version.
async fn check_version(current: &'static str) -> Check {
let client = match reqwest::Client::builder()
.user_agent(format!("gl/{current}"))
.timeout(std::time::Duration::from_secs(5))
@@ -421,9 +331,7 @@ async fn check_version(current: &'static str, github_api_base: &str) -> Check {
};
let resp = match client
.get(format!(
"{github_api_base}/repos/Gitlawb/node/releases/latest"
))
.get("https://api.github.com/repos/gitlawb/releases/releases/latest")
.send()
.await
{
@@ -431,13 +339,6 @@ async fn check_version(current: &'static str, github_api_base: &str) -> Check {
Err(_) => return Check::pass("version", format!("v{current} (offline — could not check)")),
};
if !resp.status().is_success() {
return Check::pass(
"version",
format!("v{current} (GitHub API returned HTTP {})", resp.status()),
);
}
let body: serde_json::Value = match resp.json().await {
Ok(v) => v,
Err(_) => return Check::pass("version", format!("v{current} (invalid response)")),
@@ -481,88 +382,6 @@ fn is_newer(latest: &str, current: &str) -> bool {
mod tests {
use super::*;
fn fake_home(zshrc: Option<&str>, with_omz_git: bool) -> tempfile::TempDir {
let home = tempfile::TempDir::new().unwrap();
if let Some(rc) = zshrc {
std::fs::write(home.path().join(".zshrc"), rc).unwrap();
}
if with_omz_git {
std::fs::create_dir_all(home.path().join(".oh-my-zsh/plugins/git")).unwrap();
}
home
}
#[test]
fn alias_shadowing_flags_omz_git_plugin() {
let home = fake_home(Some("plugins=(git z)\nsource $ZSH/oh-my-zsh.sh\n"), true);
let check = check_shell_alias_shadowing(Some(home.path().to_path_buf()));
assert!(check.is_some(), "omz git plugin without unalias must warn");
}
#[test]
fn alias_shadowing_flags_explicit_alias() {
let home = fake_home(Some("alias gl='git pull'\n"), false);
assert!(check_shell_alias_shadowing(Some(home.path().to_path_buf())).is_some());
}
#[test]
fn alias_shadowing_silent_when_unaliased() {
let home = fake_home(
Some("plugins=(git z)\nsource $ZSH/oh-my-zsh.sh\nunalias gl 2>/dev/null\n"),
true,
);
assert!(check_shell_alias_shadowing(Some(home.path().to_path_buf())).is_none());
}
#[test]
fn alias_shadowing_ignores_fake_unaliases() {
// None of these actually free `gl` — the warning must survive them.
for rc in [
"plugins=(git)\n# unalias gl\n", // commented out
"plugins=(git)\nunalias global\n", // longer word
"plugins=(git)\nunalias foo # gl\n", // gl only in a comment
"plugins=(git)\necho unalias-gl-later\n", // not an unalias command
] {
let home = fake_home(Some(rc), true);
assert!(
check_shell_alias_shadowing(Some(home.path().to_path_buf())).is_some(),
"must still warn for rc: {rc:?}"
);
}
// Real unalias forms that do free it — all must silence the warning.
for rc in [
"plugins=(git)\nunalias gl\n",
"plugins=(git)\nunalias gl 2>/dev/null\n",
"plugins=(git)\ntrue; unalias gl\n",
"plugins=(git)\nunalias glog gl gst\n",
] {
let home = fake_home(Some(rc), true);
assert!(
check_shell_alias_shadowing(Some(home.path().to_path_buf())).is_none(),
"must be silent for rc: {rc:?}"
);
}
}
#[test]
fn loopback_url_detection_is_host_exact() {
assert!(is_loopback_url("http://127.0.0.1:7545"));
assert!(is_loopback_url("http://localhost:7545"));
assert!(is_loopback_url("http://[::1]:7545"));
assert!(!is_loopback_url("https://localhost.example"));
assert!(!is_loopback_url("https://node.gitlawb.com/localhost"));
assert!(!is_loopback_url("not a url"));
}
#[test]
fn alias_shadowing_silent_without_signals() {
let home = fake_home(Some("plugins=(z)\nexport EDITOR=vim\n"), false);
assert!(check_shell_alias_shadowing(Some(home.path().to_path_buf())).is_none());
let no_rc = fake_home(None, false);
assert!(check_shell_alias_shadowing(Some(no_rc.path().to_path_buf())).is_none());
assert!(check_shell_alias_shadowing(None).is_none());
}
#[test]
fn test_is_newer_minor_bump() {
assert!(is_newer("0.2.0", "0.1.0"));
@@ -598,50 +417,4 @@ mod tests {
fn test_which_in_path_missing_binary() {
assert!(!which_in_path("this-binary-does-not-exist-gl-test-12345"));
}
#[tokio::test]
async fn test_check_version_queries_gitlawb_node_releases() {
let mut server = mockito::Server::new_async().await;
let _m = server
.mock("GET", "/repos/Gitlawb/node/releases/latest")
.with_status(200)
.with_header("content-type", "application/json")
.with_body(r#"{"tag_name":"v9.9.9"}"#)
.create_async()
.await;
let check = check_version("0.1.0", &server.url()).await;
assert!(matches!(check.state, CheckState::Warn));
assert!(check.detail.contains("v9.9.9"));
}
#[tokio::test]
async fn test_check_version_up_to_date() {
let mut server = mockito::Server::new_async().await;
let _m = server
.mock("GET", "/repos/Gitlawb/node/releases/latest")
.with_status(200)
.with_header("content-type", "application/json")
.with_body(r#"{"tag_name":"v0.1.0"}"#)
.create_async()
.await;
let check = check_version("0.1.0", &server.url()).await;
assert!(matches!(check.state, CheckState::Ok));
assert!(check.detail.contains("up to date"));
}
#[tokio::test]
async fn test_check_version_http_error() {
let mut server = mockito::Server::new_async().await;
let _m = server
.mock("GET", "/repos/Gitlawb/node/releases/latest")
.with_status(403)
.create_async()
.await;
let check = check_version("0.1.0", &server.url()).await;
assert!(matches!(check.state, CheckState::Ok));
assert!(check.detail.contains("GitHub API returned HTTP 403"));
}
}
+1 -699
View File
@@ -15,47 +15,6 @@ use icaptcha_client::IcaptchaCfg;
/// (absorbs proof expiry / first-seen replay).
const MAX_ICAPTCHA_RETRIES: usize = 2;
/// Total request timeout: from the start of connecting through the end of the
/// response body, so it bounds a slow download and not just a slow handshake.
const TOTAL_REQUEST_TIMEOUT: std::time::Duration = std::time::Duration::from_secs(30);
/// Follow a redirect only when it stays on the origin that issued it AND re-issues the
/// identical request-target, and only for as long as the chain bound allows.
///
/// The decision itself is [`gitlawb_core::redirect::may_follow`], shared with
/// `git-remote-gitlawb` so the two signing clients cannot drift apart on it. Refusal
/// is `stop`, not `error`: the 3xx comes back as an ordinary response and each caller
/// reports it through the status path it already has.
///
/// `Policy::custom` replaces reqwest's built-in limit, so the chain bound is restated
/// here. Same-origin redirects can cycle, and this is what stops a node answering 302
/// to itself from being followed indefinitely. It is not what makes the request
/// finite: `.timeout(...)` on the same builder is a TOTAL request timeout covering the
/// whole chain, so without this bound the worst case is a 30 second spin, not an
/// endless one. The bound is what keeps that spin from costing the node a request per
/// round trip for the full 30 seconds.
///
/// `>` and not `>=`: reqwest pushes the redirecting URL onto `previous` before
/// consulting the policy, so on the first redirect `previous.len()` is already 1, and
/// `>=` would permit `MAX_REDIRECTS - 1` follows. `Policy::limited(max)` refuses at
/// `previous.len() > max`, and matching it is the point of reusing its value.
fn same_origin_redirect(attempt: reqwest::redirect::Attempt<'_>) -> reqwest::redirect::Action {
let Some(previous) = attempt.previous().last() else {
// No previous URL to compare against. Unreachable through reqwest, which
// pushes the redirecting URL before consulting the policy, but the safe
// reading of "cannot prove same-origin" is to refuse.
return attempt.stop();
};
if attempt.previous().len() > gitlawb_core::redirect::MAX_REDIRECTS {
return attempt.stop();
}
if gitlawb_core::redirect::may_follow(previous, attempt.url()) {
attempt.follow()
} else {
attempt.stop()
}
}
pub struct NodeClient {
inner: reqwest::Client,
pub node_url: String,
@@ -64,20 +23,8 @@ pub struct NodeClient {
impl NodeClient {
pub fn new(node_url: impl Into<String>, keypair: Option<Keypair>) -> Self {
Self::with_timeout(node_url, keypair, TOTAL_REQUEST_TIMEOUT)
}
/// `new` with the total request timeout as a parameter, so a test can drive the
/// timeout's behaviour without waiting out the shipped value. `new` supplies the
/// shipped one, which is what makes the scaled-down test cover the real client.
fn with_timeout(
node_url: impl Into<String>,
keypair: Option<Keypair>,
timeout: std::time::Duration,
) -> Self {
let inner = reqwest::Client::builder()
.timeout(timeout)
.redirect(reqwest::redirect::Policy::custom(same_origin_redirect))
.timeout(std::time::Duration::from_secs(30))
.user_agent(format!("gl/{} gitlawb-cli", env!("CARGO_PKG_VERSION")))
.build()
.expect("failed to build HTTP client");
@@ -125,23 +72,6 @@ impl NodeClient {
req.send().await.with_context(|| format!("GET {url}"))
}
/// GET that signs when an identity keypair is present and falls back to an
/// anonymous GET otherwise — for read-visibility endpoints, where a public
/// repo is readable anonymously but a private repo requires the owner/reader
/// to be authenticated. Mirrors the conditional signing of post/put/delete.
pub async fn get_maybe_signed(&self, path: &str) -> Result<reqwest::Response> {
let url = format!("{}{}", self.node_url, path);
let mut req = self.inner.get(&url);
if let Some(kp) = &self.keypair {
let signed = sign_request(kp, "GET", path, b"");
req = req
.header("Content-Digest", signed.content_digest)
.header("Signature-Input", signed.signature_input)
.header("Signature", signed.signature);
}
req.send().await.with_context(|| format!("GET {url}"))
}
/// POST with JSON body + RFC 9421 signing + transparent iCaptcha solve/retry.
pub async fn post(&self, path: &str, body: &[u8]) -> Result<reqwest::Response> {
self.send_signed("POST", path, body).await
@@ -256,79 +186,6 @@ async fn obtain_proof(cfg: IcaptchaCfg) -> Result<String> {
.context("iCaptcha solver task panicked")?
}
/// What a capped body read produced. `text` is the bytes that arrived; the two flags
/// say why the read stopped where it did, which a caller that CLASSIFIES on the body
/// cannot work out from the text alone.
pub(crate) struct CappedBody {
/// The bytes read, lossily decoded.
pub(crate) text: String,
/// The cap cut the body short.
pub(crate) truncated: bool,
/// A chunk read FAILED part-way through, so the body is not merely short, it is
/// unfinished and the node may have had more to say.
pub(crate) read_failed: bool,
}
/// Read at most `cap` bytes of a response body. Bounds the allocation from a
/// hostile or broken node returning a huge error body — the display is capped
/// separately, but the read itself must not be unbounded (INV-6, read half).
///
/// `truncated` reports whether the cap cut the body short. A caller that CLASSIFIES
/// on the body needs it: a cut body fails JSON parse, and a parse failure is
/// indistinguishable from a node that sent no code at all, so without this flag an
/// oversized body silently picks a different arm.
///
/// `read_failed` reports the other way a body can end early. A mid-body read error
/// used to leave through the same exit as a clean end of stream, so a 500 whose body
/// died in transit surfaced as an empty message and the caller was told
/// `node returned 500: ` with nothing after the colon. That is a report of what the
/// node said, and the node never got to say it.
pub(crate) async fn read_body_capped(mut resp: reqwest::Response, cap: usize) -> CappedBody {
let mut buf: Vec<u8> = Vec::new();
let mut truncated = false;
let mut read_failed = false;
while buf.len() < cap {
match resp.chunk().await {
Ok(Some(chunk)) => {
let take = (cap - buf.len()).min(chunk.len());
buf.extend_from_slice(&chunk[..take]);
if take < chunk.len() {
truncated = true;
break; // hit the cap mid-chunk
}
}
Ok(None) => break, // clean end of body
Err(_) => {
read_failed = true;
break;
}
}
}
// A body that lands exactly on the cap may or may not have more behind it;
// report it as cut, since the classification that follows cannot tell either.
if buf.len() >= cap {
truncated = true;
}
CappedBody {
text: String::from_utf8_lossy(&buf).into_owned(),
truncated,
read_failed,
}
}
/// Strip terminal-dangerous characters from (and cap the length of) a
/// node-supplied error string before surfacing it. The node a caller talks to
/// could be hostile and embed escape sequences in its error body; those must not
/// reach the terminal verbatim (INV-6). We drop the C0/C1 control bytes (which
/// defangs ANSI/OSC escapes) AND the Unicode bidi/format controls (which
/// `char::is_control` does not cover — they can reorder the displayed line).
pub(crate) fn sanitize_node_msg(s: &str) -> String {
s.chars()
.filter(|c| !c.is_control() && !gitlawb_core::sanitize::is_bidi_format(*c))
.take(200)
.collect()
}
#[cfg(test)]
mod tests {
use super::*;
@@ -713,559 +570,4 @@ mod tests {
ic.challenge.assert();
ic.answer.assert();
}
// ── redirect policy ─────────────────────────────────────────────────
/// The signed headers must not survive a redirect off the node's origin.
///
/// reqwest strips only `Authorization`, `Cookie`, `Proxy-Authorization` and
/// `WWW-Authenticate` across hosts, so `Signature` and `Signature-Input` used to
/// ride a 302 straight to whatever origin the node named. The signature binds
/// `@method`, `@path` and `content-digest` and nothing about the authority, so the
/// receiving host holds a credential that reads path-scoped objects as the caller
/// at any node until the clock-skew window closes.
///
/// Two mockito servers are two ports on one host, which is exactly the boundary
/// this policy draws (and the one reqwest's own header stripping draws). The
/// second server answers everything and expects nothing: a followed redirect
/// fails the expectation whether or not the signature came with it. MUTATION
/// (RED): drop the `.redirect(...)` line and the second server is hit.
#[tokio::test]
async fn cross_origin_redirect_is_not_followed() {
let mut elsewhere = Server::new_async().await;
let never = elsewhere
.mock("GET", mockito::Matcher::Any)
.with_status(200)
.with_body("bytes from the redirect target")
.expect(0)
.create_async()
.await;
let signature_seen = elsewhere
.mock("GET", mockito::Matcher::Any)
.match_header("signature", mockito::Matcher::Any)
.with_status(200)
.expect(0)
.create_async()
.await;
let mut node = Server::new_async().await;
let bounce = node
.mock("GET", "/api/v1/thing")
.with_status(302)
.with_header("location", &format!("{}/api/v1/thing", elsewhere.url()))
.expect(1)
.create_async()
.await;
let client = NodeClient::new(node.url(), Some(test_keypair()));
let resp = client.get_signed("/api/v1/thing").await.unwrap();
assert_eq!(
resp.status(),
302,
"a refused redirect stops rather than errors, so the caller sees the 3xx \
and reports it through the status path it already has"
);
let body = resp.text().await.unwrap();
assert!(
!body.contains("bytes from the redirect target"),
"the redirect target's bytes must never reach the caller, got: {body}"
);
bounce.assert_async().await;
never.assert_async().await;
signature_seen.assert_async().await;
}
/// A node redirecting to itself is same-origin, so the origin predicate follows it
/// every time and only the chain bound ends the loop. Deleting the bound left the
/// whole suite green, because nothing here had ever built a cycle.
///
/// It has a second job now. A redirect back to the identical path and query is the
/// only same-origin hop the predicate still follows, so the eleven hits below are
/// also this crate's executed proof that such a hop IS followed. The old positive
/// fixture drove a trailing-slash rewrite, which the request-target rule refuses,
/// and an http-to-https upgrade cannot be mocked over mockito's plain http.
///
/// The route answers 301 pointing back at itself. Bounded, the handler is hit
/// once for the original request plus `MAX_REDIRECTS` follows and the call returns
/// the 301 (a refused redirect stops rather than errors). Unbounded, it runs until
/// the client's total request timeout cuts it off, which is a 30 second spin at the
/// shipped value and a request per round trip for the node.
///
/// MUTATION (RED): delete the `previous().len()` check.
#[tokio::test]
async fn a_self_redirect_stops_at_the_chain_bound() {
let mut node = Server::new_async().await;
let hits = std::sync::Arc::new(std::sync::atomic::AtomicUsize::new(0));
let h = hits.clone();
let loop_route = node
.mock("GET", "/api/v1/loop")
.with_status(301)
.with_header("location", "/api/v1/loop")
.with_body_from_request(move |_req| {
h.fetch_add(1, std::sync::atomic::Ordering::SeqCst);
Vec::new()
})
.expect_at_least(1)
.create_async()
.await;
let client = NodeClient::with_timeout(
node.url(),
Some(test_keypair()),
std::time::Duration::from_secs(5),
);
let resp = client
.get_signed("/api/v1/loop")
.await
.expect("the bound must end the chain, not the timeout");
assert_eq!(
resp.status(),
301,
"the chain ends by refusing the next hop, so the last 3xx is what comes back"
);
assert_eq!(
hits.load(std::sync::atomic::Ordering::SeqCst),
gitlawb_core::redirect::MAX_REDIRECTS + 1,
"one original request plus MAX_REDIRECTS follows, matching what \
Policy::limited(MAX_REDIRECTS) would have permitted"
);
loop_route.assert_async().await;
}
/// A same-origin hop that REWRITES the request-target is refused, even though the
/// origin never changes.
///
/// The signature binds `@path` as the literal path-and-query the client signed, and
/// the node rebuilds it from the URI it received. A `/api/v1/thing` to
/// `/api/v1/thing/` bounce therefore presents a signature over a target the node
/// never saw, and every signed read behind such a proxy 401s. Refusing the hop
/// hands the caller the 3xx that names what happened instead.
///
/// The target mock expects zero hits and is asserted: mockito only checks
/// `.expect(N)` when `.assert()` runs, so an unbound or unasserted mock passes
/// vacuously.
#[tokio::test]
async fn same_origin_path_changing_redirect_is_refused() {
let mut node = Server::new_async().await;
let bounce = node
.mock("GET", "/api/v1/thing")
.with_status(301)
.with_header("location", "/api/v1/thing/")
.expect(1)
.create_async()
.await;
let target = node
.mock("GET", "/api/v1/thing/")
.with_status(200)
.with_body("bytes from the rewritten target")
.expect(0)
.create_async()
.await;
let client = NodeClient::new(node.url(), Some(test_keypair()));
let resp = client.get_signed("/api/v1/thing").await.unwrap();
assert_eq!(
resp.status(),
301,
"a refused redirect stops rather than errors, so the caller sees the 3xx \
and reports it through the status path it already has"
);
let body = resp.text().await.unwrap();
assert!(
!body.contains("bytes from the rewritten target"),
"the rewritten target's bytes must never reach the caller, got: {body}"
);
bounce.assert_async().await;
target.assert_async().await;
}
// ── the node's own verification, run over what the client actually sent ──
/// What the verifying mock made of a request that reached it.
///
/// The empty slot (`None`) is its own state and means the mock was never hit at
/// all, which is what the refusal test asserts. It must stay distinguishable from
/// [`Verdict::WrongIdentity`], because "nobody verified anything" and "something
/// verified against the wrong key" are opposite findings.
///
/// The payloads are read through `Debug` in the assertion messages and nowhere
/// else, which the dead-code pass does not count; they carry the detail that makes
/// a failure legible, so they stay.
#[derive(Debug)]
#[allow(dead_code)]
enum Verdict {
/// The chain accepted the signature AND the key it resolved is the test's DID.
Accepted,
/// The chain refused it. Carries the error so a failure reads as the actual
/// rejection rather than a bare hit count.
Rejected(String),
/// The chain accepted a signature made by somebody else. A key resolved from
/// the parsed `key_id` is read out of the artifact under verification, so an
/// accept on it alone proves consistency, never authenticity.
WrongIdentity { expected: String, got: String },
}
/// The node's `require_signature` verification, over a request this crate did not
/// build: parse the headers, recompute the content-digest from the body, rebuild
/// the signing string over `@method`/`@path`/`content-digest`, Ed25519-verify.
/// Returns the DID the signature resolved to, so a caller can pin the identity.
///
/// A hand-copy of its twin in `crates/git-remote-gitlawb/src/main.rs`, which gl
/// cannot import (`git-remote-gitlawb` is a binary crate and this is its test
/// module). Keep the two textually identical apart from the mockito seam around
/// them, so an edit to one is visibly an edit to both.
///
/// The production verifier both copies mirror is `crate::auth::require_signature` in
/// `crates/gitlawb-node/src/auth/mod.rs`. This is a re-implementation, not a call, so
/// an edit to that middleware has to land here too: otherwise the copies drift and
/// this test keeps passing against a rule the node has stopped applying.
///
/// It asserts internally, which is deliberate but constrains its callers: inside
/// `with_body_from_request` those assertions fire on the server thread and reach
/// the client as a transport error, not as a recorded verdict. So the identity
/// check lives in the caller as a [`Verdict`] variant, never as an assert in here.
fn node_verifies(
method: &str,
path_and_query: &str,
body: &[u8],
sig_input: &str,
sig_header: &str,
content_digest: &str,
) -> anyhow::Result<String> {
use gitlawb_core::http_sig::{build_signing_string, compute_content_digest, HttpSignature};
use gitlawb_core::identity::verify;
use std::collections::HashMap;
let sig = HttpSignature::parse(sig_input, sig_header)?;
sig.check_created()?;
assert!(
sig.missing_components().is_empty(),
"signature must cover all required components"
);
assert_eq!(sig.alg, "ed25519");
assert_eq!(
content_digest,
compute_content_digest(body),
"content-digest must match the body"
);
let vk = sig.key_id.to_verifying_key()?;
let mut values = HashMap::new();
values.insert("@method".to_string(), method.to_uppercase());
values.insert("@path".to_string(), path_and_query.to_string());
values.insert("content-digest".to_string(), content_digest.to_string());
let sig_params_value = sig_input.strip_prefix("sig1=").unwrap_or(sig_input);
let components: Vec<&str> = sig.components.iter().map(String::as_str).collect();
let signing_string = build_signing_string(&components, sig_params_value, &values)?;
let sig_array: [u8; 64] = sig.signature_bytes.as_slice().try_into()?;
verify(&vk, signing_string.as_bytes(), &sig_array)?;
Ok(sig.key_id.to_string())
}
/// Pull a header value off a received mockito request, or explain which one the
/// client failed to send.
fn received_header(req: &mockito::Request, name: &str) -> String {
req.header(name)
.first()
.unwrap_or_else(|| panic!("the client sent no {name} header"))
.to_str()
.unwrap()
.to_string()
}
/// Run [`node_verifies`] over a GET that arrived at the mock and record what the
/// node would have made of it, pinned to `expected_did`.
fn record_get_verdict(
req: &mockito::Request,
expected_did: &str,
slot: &std::sync::Arc<std::sync::Mutex<Option<Verdict>>>,
) {
let verdict = match node_verifies(
"GET",
req.path_and_query(),
b"",
&received_header(req, "signature-input"),
&received_header(req, "signature"),
&received_header(req, "content-digest"),
) {
Ok(did) if did == expected_did => Verdict::Accepted,
Ok(did) => Verdict::WrongIdentity {
expected: expected_did.to_string(),
got: did,
},
Err(e) => Verdict::Rejected(e.to_string()),
};
*slot.lock().unwrap() = Some(verdict);
}
/// The finding's repro, now a guard: a rewritten same-origin target must never
/// receive the signature, and the proof is the node's own verification, not a hit
/// count.
///
/// The target mock runs the full `require_signature` chain over the request it
/// receives. Post-fix the hop is refused, so the slot stays empty. Pre-fix the hop
/// is followed and the slot records the Ed25519 rejection of a signature made over
/// `/api/v1/thing` and presented at `/api/v1/thing/`, which is the 401 an operator
/// behind such a proxy actually sees. The verdict is asserted first, so a failure
/// speaks about verification rather than about reachability.
///
/// Its paired positive control is
/// `a_direct_signed_get_verifies_under_the_node_verifier`: without it, an empty
/// slot would be satisfied just as well by a harness that can never record
/// anything.
#[tokio::test]
async fn a_rewritten_target_never_receives_the_signature() {
let kp = test_keypair();
let expected_did = kp.did().to_string();
let slot = std::sync::Arc::new(std::sync::Mutex::new(None::<Verdict>));
let mut node = Server::new_async().await;
let bounce = node
.mock("GET", "/api/v1/thing")
.with_status(301)
.with_header("location", "/api/v1/thing/")
.expect(1)
.create_async()
.await;
let recorder = slot.clone();
let did_for_target = expected_did.clone();
let target = node
.mock("GET", "/api/v1/thing/")
.with_status(200)
.with_body_from_request(move |req| {
record_get_verdict(req, &did_for_target, &recorder);
Vec::new()
})
.expect(0)
.create_async()
.await;
let client = NodeClient::new(node.url(), Some(kp));
let resp = client.get_signed("/api/v1/thing").await.unwrap();
let verdict = slot.lock().unwrap().take();
assert!(
verdict.is_none(),
"the node's own verifier must never see this request: the signature covers \
/api/v1/thing and the rewritten target is /api/v1/thing/, so what arrives \
there is a stale request-target; recorded verdict: {verdict:?}"
);
assert_eq!(
resp.status(),
301,
"the caller sees the 3xx, not the rewritten target's answer"
);
bounce.assert_async().await;
target.assert_async().await;
}
/// The positive control for the test above, and the proof that the client signs
/// the query it sends.
///
/// A direct signed GET, no redirect anywhere, through the same verifying mock. The
/// verdict must be `Accepted`, which is what makes the refusal test's empty slot
/// attributable to the refusal rather than to a harness that cannot record. The
/// path carries a query, so a client that signed the bare path would land here as
/// `Rejected`.
#[tokio::test]
async fn a_direct_signed_get_verifies_under_the_node_verifier() {
let kp = test_keypair();
let expected_did = kp.did().to_string();
let slot = std::sync::Arc::new(std::sync::Mutex::new(None::<Verdict>));
let mut node = Server::new_async().await;
let recorder = slot.clone();
let did_for_target = expected_did.clone();
let route = node
.mock("GET", "/api/v1/thing?x=1")
.with_status(200)
.with_body_from_request(move |req| {
record_get_verdict(req, &did_for_target, &recorder);
Vec::new()
})
.expect(1)
.create_async()
.await;
let client = NodeClient::new(node.url(), Some(kp));
let resp = client.get_signed("/api/v1/thing?x=1").await.unwrap();
assert_eq!(resp.status(), 200);
let verdict = slot.lock().unwrap().take();
assert!(
matches!(verdict, Some(Verdict::Accepted)),
"a direct signed GET must verify under the node's own chain and resolve to \
{expected_did}, or the refusal test's empty slot proves nothing; recorded \
verdict: {verdict:?}"
);
route.assert_async().await;
}
// ── read_body_capped ────────────────────────────────────────────────
/// A body whose read FAILS mid-stream must be distinguishable from a body that
/// ended. Both used to leave through the same `_ => break`, so a 500 whose body
/// died in transit produced an empty message and the caller was told
/// `node returned 500: ` with nothing after the colon.
///
/// The fixture is a raw listener that promises 64 bytes in `Content-Length`,
/// writes 5, and closes. mockito cannot express that: it always completes the
/// response it advertises. MUTATION (RED): restore the single `_ => break` arm
/// (or hard-code `read_failed: false`) and the flag reads false.
#[tokio::test]
async fn read_body_capped_flags_a_mid_body_read_failure() {
let addr = spawn_short_body_listener().await;
let resp = reqwest::get(format!("http://{addr}/truncated"))
.await
.expect("headers arrive before the body is cut");
let body = read_body_capped(resp, 8192).await;
assert!(
body.read_failed,
"a body cut off mid-stream must be reported as a failed read, not as a \
body that ended: got {:?}",
body.text
);
assert!(
!body.truncated,
"the cap did not cut this one; 5 bytes are nowhere near 8 KiB"
);
}
/// The must-not half: a body that ends cleanly must NOT be flagged, or the flag
/// means nothing and every terminal starts claiming the node went quiet.
#[tokio::test]
async fn read_body_capped_does_not_flag_a_clean_body() {
let mut server = Server::new_async().await;
let _m = server
.mock("GET", "/ok")
.with_status(500)
.with_body("node said this")
.create_async()
.await;
let resp = reqwest::get(format!("{}/ok", server.url())).await.unwrap();
let body = read_body_capped(resp, 8192).await;
assert_eq!(body.text, "node said this");
assert!(!body.read_failed, "a complete body is not a failed read");
assert!(!body.truncated, "a complete body is not a truncated one");
}
/// Answer one request with headers promising more body than gets written, then
/// close the connection. Returns the listener's address.
async fn spawn_short_body_listener() -> std::net::SocketAddr {
use tokio::io::{AsyncReadExt, AsyncWriteExt};
let listener = tokio::net::TcpListener::bind("127.0.0.1:0").await.unwrap();
let addr = listener.local_addr().unwrap();
tokio::spawn(async move {
let (mut sock, _) = listener.accept().await.unwrap();
let mut scratch = [0u8; 1024];
let _ = sock.read(&mut scratch).await;
let _ = sock
.write_all(b"HTTP/1.1 500 Internal Server Error\r\nContent-Length: 64\r\n\r\nshort")
.await;
let _ = sock.flush().await;
// Drop closes the socket with 59 of the promised bytes never sent.
});
addr
}
// ── total request timeout ───────────────────────────────────────────
/// The client's timeout is a TOTAL request timeout, so it bounds a download and
/// not just the handshake. `gl ipfs get`'s documentation leans on exactly that:
/// the wall-clock deadline covers the search and deliberately stops at the
/// response headers, and this timeout is the only thing left bounding the body.
///
/// Driven at 250ms through `with_timeout`, the seam `new` itself calls with
/// `TOTAL_REQUEST_TIMEOUT`, because a test at the shipped 30s has no place in this
/// suite. What that costs is the value; what it proves is the SHAPE, which is the
/// part in doubt: that the deadline keeps running once the headers have landed. A
/// timeout that covered only the handshake would let this request hang until the
/// listener gives up.
#[tokio::test]
async fn total_timeout_cuts_off_a_body_that_outruns_it() {
use tokio::io::{AsyncReadExt, AsyncWriteExt};
let listener = tokio::net::TcpListener::bind("127.0.0.1:0").await.unwrap();
let addr = listener.local_addr().unwrap();
tokio::spawn(async move {
let (mut sock, _) = listener.accept().await.unwrap();
let mut scratch = [0u8; 1024];
let _ = sock.read(&mut scratch).await;
// Headers land immediately, then the body stalls indefinitely.
let _ = sock
.write_all(b"HTTP/1.1 200 OK\r\nContent-Length: 1024\r\n\r\nfirst")
.await;
let _ = sock.flush().await;
tokio::time::sleep(std::time::Duration::from_secs(30)).await;
});
let client = NodeClient::with_timeout(
format!("http://{addr}"),
None,
std::time::Duration::from_millis(250),
);
let started = std::time::Instant::now();
let resp = client.get("/slow").await.expect("headers arrive promptly");
assert_eq!(
resp.status(),
200,
"the stall is in the body, not the status"
);
let err = resp
.bytes()
.await
.expect_err("a body still arriving past the total timeout must be cut off");
let elapsed = started.elapsed();
assert!(
err.is_timeout(),
"the body read must end in a timeout, got: {err}"
);
assert!(
elapsed < std::time::Duration::from_secs(5),
"the timeout must fire on its own schedule, not wait out the listener; \
took {elapsed:?}"
);
}
#[test]
fn shipped_client_uses_the_documented_total_timeout() {
// The scaled-down test above proves the shape at 250ms. This pins the value
// `new` actually ships, so the two together cover the documented behaviour.
assert_eq!(TOTAL_REQUEST_TIMEOUT, std::time::Duration::from_secs(30));
}
#[test]
fn sanitize_strips_controls_bidi_and_caps_length() {
// C0 (ESC/BEL) and the Cf bidi override (U+202E) are both removed; the
// printable text survives. (Note: a stripped ESC leaves any following
// "[31m" as inert literal text — that is the point, so the input here
// avoids that residue to keep the expectation unambiguous.)
let out = sanitize_node_msg("a\u{1b}\u{07}b\u{202e}c");
assert!(
!out.chars().any(|c| c.is_control()),
"control char leaked: {out:?}"
);
assert!(
!out.contains('\u{202e}'),
"RLO bidi override leaked: {out:?}"
);
assert_eq!(out, "abc");
// Length is capped at 200 chars regardless of input size.
let long = "x".repeat(250);
assert_eq!(sanitize_node_msg(&long).chars().count(), 200);
}
#[test]
fn sanitize_preserves_legitimate_and_rtl_text() {
// Must not over-strip: a plain word, a genuine RTL SCRIPT letter (Arabic
// U+0627, category Lo — NOT a format char), and ZWJ (U+200D, a legitimate
// Cf char, e.g. emoji sequences) all survive. Guards the shared predicate
// against being widened into a blanket Cf stripper.
let out = sanitize_node_msg("ok \u{0627}\u{200D}b");
assert_eq!(out, "ok \u{0627}\u{200D}b");
}
}
+4 -52
View File
@@ -38,23 +38,13 @@ pub async fn run(args: InitArgs) -> Result<()> {
let git_dir = cwd.join(".git");
if !git_dir.exists() {
println!("Initializing git repository...");
// -b main: the push flow targets `main`; without it a fresh repo uses
// the user's init.defaultBranch (often `master`). Older git (<2.28)
// lacks the flag, so fall back to a plain init.
let status = std::process::Command::new("git")
.args(["init", "-b", "main"])
.args(["init"])
.current_dir(&cwd)
.status()
.context("failed to run git init")?;
if !status.success() {
let status = std::process::Command::new("git")
.args(["init"])
.current_dir(&cwd)
.status()
.context("failed to run git init")?;
if !status.success() {
anyhow::bail!("git init failed");
}
anyhow::bail!("git init failed");
}
} else {
println!("Git repository detected.");
@@ -175,47 +165,9 @@ pub async fn run(args: InitArgs) -> Result<()> {
}
}
// The hint must match the repo's actual state: with no commits yet, a bare
// `git push` fails with "src refspec ... does not match any" — exactly the
// trap a zero-to-push command must not set.
let has_commits = std::process::Command::new("git")
.args(["rev-parse", "--verify", "HEAD"])
.current_dir(&cwd)
.stdout(std::process::Stdio::null())
.stderr(std::process::Stdio::null())
.status()
.map(|s| s.success())
.unwrap_or(false);
// symbolic-ref succeeds on a branch (including an unborn one) and fails
// on a detached HEAD — where guessing "main" could push the wrong ref.
let branch = std::process::Command::new("git")
.args(["symbolic-ref", "--short", "HEAD"])
.current_dir(&cwd)
.stderr(std::process::Stdio::null())
.output()
.ok()
.filter(|o| o.status.success())
.and_then(|o| String::from_utf8(o.stdout).ok())
.map(|s| s.trim().to_string())
.filter(|s| !s.is_empty());
println!();
match (branch, has_commits) {
(Some(branch), true) => {
println!("Ready! Push with:");
println!(" git push gitlawb {branch}");
}
(Some(branch), false) => {
println!("Ready! Nothing is committed yet — commit, then push:");
println!(" git add -A && git commit -m \"initial commit\"");
println!(" git push gitlawb {branch}");
}
(None, _) => {
println!("Ready! HEAD is detached — create or switch to a branch, then push:");
println!(" git switch -c main");
println!(" git push gitlawb main");
}
}
println!("Ready! Push with:");
println!(" git push gitlawb main");
Ok(())
}
+34 -1931
View File
File diff suppressed because it is too large. Load diff
+6 -207
View File
@@ -932,33 +932,13 @@ async fn call_tool(
"webhook_list" => {
let repo = args["repo"].as_str().context("missing 'repo'")?;
// Owner-gated route: an explicit `owner` arg wins, otherwise default to
// the signing keypair's short DID (NOT the node DID). The request is
// signed by this keypair, so the path owner must match it or the node
// returns 403/404.
let owner = if let Some(o) = args.get("owner").and_then(|v| v.as_str()) {
o.to_string()
} else {
let kp = keypair
.as_ref()
.context("no identity found — run `gl identity new` first")?;
let did = kp.did().to_string();
did.split(':').next_back().unwrap_or(&did).to_string()
};
// Owner-gated route: must be signed (get_signed), not a plain get().
let resp = client
.get_signed(&format!("/api/v1/repos/{owner}/{repo}/hooks"))
let owner = resolve_owner(&args, &client).await?;
let resp: Value = client
.get(&format!("/api/v1/repos/{owner}/{repo}/hooks"))
.await?
.json()
.await?;
// Check the HTTP status before deserializing: a 401/403/404 JSON error
// body (missing identity, wrong owner, private/deleted repo) must fail
// the tool call, not be returned as a successful result.
let status = resp.status();
let body: Value = resp.json().await?;
if !status.is_success() {
let msg = body["message"].as_str().unwrap_or("unknown error");
anyhow::bail!("webhook_list failed ({status}): {msg}");
}
Ok(serde_json::to_string_pretty(&body)?)
Ok(serde_json::to_string_pretty(&resp)?)
}
"webhook_delete" => {
@@ -1845,187 +1825,6 @@ mod tests {
assert_eq!(parsed["body"], "looks good");
}
#[tokio::test]
async fn test_webhook_list_via_mcp_signs_the_request() {
let mut server = mockito::Server::new_async().await;
let dir = tempfile::TempDir::new().unwrap();
let kp = gitlawb_core::identity::Keypair::generate();
std::fs::write(
dir.path().join("identity.pem"),
kp.to_pem().unwrap().as_bytes(),
)
.unwrap();
// /hooks is owner-gated, so the MCP webhook_list tool must send a SIGNED
// request (get_signed). Requiring the header catches a regression to get().
// Passing `owner` in args makes resolve_owner skip the node-root lookup.
let _m = server
.mock("GET", mockito::Matcher::Regex(r"/hooks$".to_string()))
.match_header("signature", mockito::Matcher::Any)
.match_header("signature-input", mockito::Matcher::Any)
.with_status(200)
.with_header("content-type", "application/json")
.with_body(r#"{"webhooks":[],"count":0}"#)
.create_async()
.await;
let result = call_tool(
"webhook_list",
json!({"owner": "alice", "repo": "myrepo"}),
&server.url(),
Some(dir.path()),
)
.await
.unwrap();
assert!(result.contains("webhooks"), "got: {result}");
}
#[tokio::test]
async fn test_webhook_list_default_owner_is_keypair_not_node_did() {
// When `owner` is omitted, the owner segment must come from the signing
// keypair's short DID, NOT the node root DID. The "/" mock returns a
// different DID; the /hooks mock only matches the keypair's short DID,
// so a regression to the node DID would leave the request unmatched.
let mut server = mockito::Server::new_async().await;
let dir = tempfile::TempDir::new().unwrap();
let kp = gitlawb_core::identity::Keypair::generate();
std::fs::write(
dir.path().join("identity.pem"),
kp.to_pem().unwrap().as_bytes(),
)
.unwrap();
let _root = server
.mock("GET", "/")
.with_status(200)
.with_header("content-type", "application/json")
.with_body(r#"{"did":"did:key:z6MkNodeRootDid"}"#)
.create_async()
.await;
let did = kp.did().to_string();
let short = did.split(':').next_back().unwrap_or(&did).to_string();
let _m = server
.mock(
"GET",
mockito::Matcher::Regex(format!(r"/api/v1/repos/{short}/myrepo/hooks$")),
)
.match_header("signature", mockito::Matcher::Any)
.with_status(200)
.with_header("content-type", "application/json")
.with_body(r#"{"webhooks":[],"count":0}"#)
.expect(1)
.create_async()
.await;
let result = call_tool(
"webhook_list",
json!({"repo": "myrepo"}),
&server.url(),
Some(dir.path()),
)
.await
.unwrap();
assert!(result.contains("webhooks"), "got: {result}");
_m.assert_async().await;
}
#[tokio::test]
async fn test_webhook_list_explicit_owner_overrides_keypair() {
// An explicit `owner` arg must win over the keypair default and the node
// DID. The path must use the supplied owner; "/" must not even be hit.
let mut server = mockito::Server::new_async().await;
let dir = tempfile::TempDir::new().unwrap();
let kp = gitlawb_core::identity::Keypair::generate();
std::fs::write(
dir.path().join("identity.pem"),
kp.to_pem().unwrap().as_bytes(),
)
.unwrap();
let _m = server
.mock(
"GET",
mockito::Matcher::Regex(r"/api/v1/repos/someoneelse/myrepo/hooks$".to_string()),
)
.match_header("signature", mockito::Matcher::Any)
.with_status(200)
.with_header("content-type", "application/json")
.with_body(r#"{"webhooks":[],"count":0}"#)
.expect(1)
.create_async()
.await;
let result = call_tool(
"webhook_list",
json!({"repo": "myrepo", "owner": "someoneelse"}),
&server.url(),
Some(dir.path()),
)
.await
.unwrap();
assert!(result.contains("webhooks"), "got: {result}");
_m.assert_async().await;
}
#[tokio::test]
async fn test_webhook_list_non_success_status_errors() {
// A 403 (or any non-2xx) JSON error body for a missing identity, wrong
// owner, or private/deleted repo must fail the tool call, NOT be returned
// as a successful result. Mirrors the `gl webhook list` status check.
let mut server = mockito::Server::new_async().await;
let dir = tempfile::TempDir::new().unwrap();
let kp = gitlawb_core::identity::Keypair::generate();
std::fs::write(
dir.path().join("identity.pem"),
kp.to_pem().unwrap().as_bytes(),
)
.unwrap();
let _m = server
.mock("GET", mockito::Matcher::Regex(r"/hooks$".to_string()))
.match_header("signature", mockito::Matcher::Any)
.with_status(403)
.with_header("content-type", "application/json")
.with_body(r#"{"message":"only the repo owner can perform this action"}"#)
.create_async()
.await;
let err = call_tool(
"webhook_list",
json!({"owner": "alice", "repo": "myrepo"}),
&server.url(),
Some(dir.path()),
)
.await
.expect_err("non-2xx must error, not return the error body as success");
let msg = err.to_string();
assert!(msg.contains("webhook_list failed (403"), "got: {msg}");
assert!(
msg.contains("only the repo owner"),
"must surface the node message, got: {msg}"
);
}
#[tokio::test]
async fn test_webhook_list_without_keypair_errors() {
// No identity loaded and no explicit owner → the tool must error rather
// than fall back to the node DID and issue an unsigned, mis-scoped GET.
let server = mockito::Server::new_async().await;
let dir = tempfile::TempDir::new().unwrap(); // empty: no identity.pem
let err = call_tool(
"webhook_list",
json!({"repo": "myrepo"}),
&server.url(),
Some(dir.path()),
)
.await
.expect_err("must error without an identity");
assert!(err.to_string().contains("no identity found"), "got: {err}");
}
#[test]
fn test_tool_count_is_42() {
let tools = tool_definitions();
+12 -406
View File
@@ -7,7 +7,7 @@ use clap::{Args, Subcommand};
use serde_json::Value;
use std::path::PathBuf;
use crate::http::{read_body_capped, sanitize_node_msg, NodeClient};
use crate::http::NodeClient;
use crate::identity::load_keypair_from_dir;
#[derive(Args)]
@@ -97,74 +97,6 @@ async fn cmd_list(node: String) -> Result<()> {
Ok(())
}
/// The warning text for a local peer-add reply, or `None` when the node
/// accepted it.
///
/// Split out so the refusal path is assertable. The node now answers 403 when
/// an unproven caller tries to repoint an existing peer's `http_url`, so a
/// reply that arrived and refused must not reach the success line.
fn local_add_refusal(status: reqwest::StatusCode, body: &Value) -> Option<String> {
if status.is_success() {
return None;
}
// The message is the node's, not ours, so it is defanged before it reaches
// the terminal. Only the message: the status and the prose are ours.
let msg = sanitize_node_msg(body["message"].as_str().unwrap_or("unknown error"));
Some(format!(
"warning: local peer list not updated ({status}): {msg}"
))
}
/// The error text for a remote announce reply, or `None` when the peer accepted
/// it.
///
/// Split out so the ordering is assertable: the status has to reach the caller
/// even when the body does not parse. That case is reachable rather than
/// theoretical, because the read above is capped, so a peer answering with a
/// body past the cap leaves truncated JSON behind, and `peer_url` is whatever
/// the caller passed, so a non-JSON error page is just as easy. Parsing first
/// and failing on the parse would swallow the status the user needs.
fn remote_announce_failure(status: reqwest::StatusCode, raw: &str) -> Option<String> {
if status.is_success() {
return None;
}
// Falls back to the body itself when there is no `message` field, which is
// what a truncated or non-JSON reply leaves. Sanitized either way: it is the
// peer's text and it is headed for the terminal.
let parsed: Value = serde_json::from_str(raw).unwrap_or(Value::Null);
let msg = sanitize_node_msg(parsed["message"].as_str().unwrap_or(raw));
Some(format!("announce failed ({status}): {msg}"))
}
/// Which line the local peer-add prints, and whether it is a warning.
///
/// Split out because the SELECTION is what the fix is: `local_add_refusal` was
/// already unit-tested six ways, and replacing this whole match with an
/// unconditional "Added to local peer list." still left 312 tests green. The
/// helper was pinned; the wiring was not, which is the shape
/// `unit-test-on-helper-does-not-prove-handler-wiring` names. With the choice
/// extracted, the call site is a two-line dispatch carrying no logic.
fn local_add_report(status: reqwest::StatusCode, body: &Value) -> (bool, String) {
match local_add_refusal(status, body) {
Some(warning) => (true, warning),
None => (false, " Added to local peer list.".to_string()),
}
}
/// The block printed after a peer accepts our announce. Split out so the
/// defanging is assertable: the DID and URL are the remote peer's own strings,
/// and `peer_url` is caller-chosen, so this is the least trusted body the
/// command handles. Only the display is sanitized; the values sent on to our
/// local node stay verbatim, since that is data rather than terminal output and
/// the node runs its own validation on them.
fn announced_peer_summary(their_did: &str, their_url: &str, peer_count: u64) -> String {
format!(
"Announced to peer node:\n DID: {}\n URL: {}\n Their peers: {peer_count}\n",
sanitize_node_msg(their_did),
sanitize_node_msg(their_url),
)
}
async fn cmd_add(peer_url: String, node: String, dir: Option<PathBuf>) -> Result<()> {
let keypair = load_keypair_from_dir(dir.as_deref())?;
let my_did = keypair.did().to_string();
@@ -195,26 +127,21 @@ async fn cmd_add(peer_url: String, node: String, dir: Option<PathBuf>) -> Result
.await
.context("failed to connect to peer")?;
let status = resp.status();
// `peer_url` is fully caller-chosen, so this body is the least trusted one
// in the command: bound the read, and defang the message before it reaches
// the terminal through the error return. An announce reply is a DID, a URL
// and a count, so 8 KiB is well past what the shape needs.
let raw = read_body_capped(resp, 8 * 1024).await.text;
let result: Value = resp.json().await.context("invalid JSON response")?;
if let Some(failure) = remote_announce_failure(status, &raw) {
anyhow::bail!("{failure}");
if !status.is_success() {
let msg = result["message"].as_str().unwrap_or("unknown error");
anyhow::bail!("announce failed ({status}): {msg}");
}
let result: Value = serde_json::from_str(&raw).context("invalid JSON response")?;
let their_did = result["node_did"].as_str().unwrap_or("?");
let their_url = result["node_url"].as_str().unwrap_or("?");
let peer_count = result["peer_count"].as_u64().unwrap_or(0);
print!(
"{}",
announced_peer_summary(their_did, their_url, peer_count)
);
println!("Announced to peer node:");
println!(" DID: {their_did}");
println!(" URL: {their_url}");
println!(" Their peers: {peer_count}");
// Also add their info to our local node's peer list
// (the peer's /announce response includes their did + url)
@@ -223,31 +150,9 @@ async fn cmd_add(peer_url: String, node: String, dir: Option<PathBuf>) -> Result
"did": their_did,
"http_url": their_url,
}))?;
// This requires the local node to be running, so a transport failure
// stays a warning rather than failing the command. A reply that
// ARRIVED and refused is different: the announce route answers 403
// when an unproven caller tries to repoint an existing peer, and
// printing the success line over that reports a completed add that
// never happened. Same status handling as the remote announce above,
// except best effort.
match local_client.post("/api/v1/peers/announce", &add_body).await {
Ok(resp) => {
let status = resp.status();
// Same bound as the remote announce above: a compromised local
// node (or a MITM on plain http) must not force an unbounded
// read. A body that does not parse stays `Null`, which still
// routes a non-success status to the warning.
let raw = read_body_capped(resp, 8 * 1024).await.text;
let result: Value = serde_json::from_str(&raw).unwrap_or(Value::Null);
let (is_warning, line) = local_add_report(status, &result);
if is_warning {
eprintln!("{line}");
} else {
println!("{line}");
}
}
Err(e) => eprintln!("warning: local peer list not updated: {e}"),
}
// This requires the local node to be running; ignore errors here
let _ = local_client.post("/api/v1/peers/announce", &add_body).await;
println!(" Added to local peer list.");
}
Ok(())
@@ -305,302 +210,3 @@ async fn cmd_resolve(did: String, node: String) -> Result<()> {
}
Ok(())
}
#[cfg(test)]
mod tests {
use super::{
announced_peer_summary, cmd_add, local_add_refusal, local_add_report,
remote_announce_failure,
};
use reqwest::StatusCode;
use serde_json::json;
/// The node's 403 for repointing an existing peer must surface, not print
/// as a completed add. Without the status check the command prints "Added
/// to local peer list." over a refusal.
///
/// The fixture body is the real wire shape, not an invented one: the
/// message is the Display of `PeerWriteDenied::UnprovenRepoint`, declared
/// in `crates/gitlawb-node/src/db/mod.rs`. The executable detector for a
/// wording change is the node-side pin
/// `announce_unsigned_repoint_is_a_403_denial` in
/// `crates/gitlawb-node/src/api/peers.rs`, which asserts this same
/// substring against a response driven through the real router, so
/// rewording the Display turns that test red and the grep for the old
/// phrase lands here.
///
/// The assertion bites rather than echoing its own input because the
/// substring is reachable only through the helper's `body["message"]`
/// extraction; the fixture's `error` field carries a different string, so
/// reading `body["error"]` instead fails this test.
#[test]
fn a_refused_local_add_warns_with_the_node_reason() {
let warning = local_add_refusal(
StatusCode::FORBIDDEN,
&json!({
"error": "forbidden",
"message": "unproven announce cannot change an existing peer's http_url: did:key:z6MkuMqUm4i228K9qXidJ57zqSWAcQLgrcbMxB8RKVLuqitj"
}),
)
.expect("a refusal must not render as success");
assert!(warning.contains("403"), "must name the status: {warning}");
assert!(
warning.contains("unproven announce cannot change an existing peer"),
"must carry the node's own reason: {warning}"
);
}
/// A refusal with no message body still warns rather than passing silently.
#[test]
fn a_refusal_without_a_message_still_warns() {
let warning = local_add_refusal(StatusCode::BAD_REQUEST, &serde_json::Value::Null)
.expect("a refusal must not render as success");
assert!(warning.contains("400"), "must name the status: {warning}");
}
/// The accepted case stays quiet, so the success line still prints.
#[test]
fn an_accepted_local_add_produces_no_warning() {
assert!(local_add_refusal(StatusCode::OK, &json!({ "peer_count": 3 })).is_none());
}
/// The warning is printed straight to a terminal, so a node-supplied message
/// carrying an OSC title-rewrite plus a C0 escape must reach it defanged.
#[test]
fn a_refusal_message_cannot_carry_terminal_escapes() {
let warning = local_add_refusal(
StatusCode::FORBIDDEN,
&json!({ "message": "\u{1b}]2;pwned\u{7}legit-looking text" }),
)
.expect("a refusal must not render as success");
assert!(
!warning.chars().any(|c| c.is_control()),
"control char leaked to the terminal: {warning:?}"
);
assert!(
warning.contains("legit-looking text"),
"printable text dropped: {warning:?}"
);
}
/// Bidi format chars are not `char::is_control`, yet they reorder the
/// displayed line, so a refusal must not carry them either.
#[test]
fn a_refusal_message_cannot_carry_bidi_overrides() {
let warning = local_add_refusal(
StatusCode::FORBIDDEN,
&json!({ "message": "peer \u{202e}refused\u{202c} here" }),
)
.expect("a refusal must not render as success");
assert!(
!warning.chars().any(gitlawb_core::sanitize::is_bidi_format),
"bidi format char leaked to the terminal: {warning:?}"
);
}
/// A hostile node must not be able to dump an arbitrary amount of text into
/// the user's scrollback through the warning line.
#[test]
fn a_refusal_message_is_length_bounded() {
let warning = local_add_refusal(
StatusCode::FORBIDDEN,
&json!({ "message": "x".repeat(5000) }),
)
.expect("a refusal must not render as success");
assert!(
warning.chars().count() < 500,
"warning not bounded: {} chars",
warning.chars().count()
);
}
/// The status is the part the user can act on, so it must survive a body
/// that does not parse. The read is capped, so a peer answering past the cap
/// leaves exactly this: a valid status and truncated JSON.
#[test]
fn a_remote_announce_failure_keeps_the_status_when_the_body_does_not_parse() {
let truncated = r#"{"error":"service_unavailable","message":"backend is do"#;
let failure = remote_announce_failure(StatusCode::SERVICE_UNAVAILABLE, truncated)
.expect("a non-success status must produce a failure");
assert!(
failure.contains("503"),
"the status must survive an unparseable body: {failure:?}"
);
}
/// A parseable error body still reports the peer's own message.
#[test]
fn a_remote_announce_failure_carries_the_peers_message() {
let failure = remote_announce_failure(
StatusCode::FORBIDDEN,
r#"{"error":"forbidden","message":"unproven announce cannot change an existing peer"}"#,
)
.expect("a non-success status must produce a failure");
assert!(
failure.contains("403")
&& failure.contains("unproven announce cannot change an existing peer"),
"status and reason must both survive: {failure:?}"
);
}
/// The fallback path prints the raw body, so it is defanged like every other
/// peer-supplied string headed for the terminal.
#[test]
fn a_remote_announce_failure_defangs_an_unparseable_body() {
let failure = remote_announce_failure(
StatusCode::BAD_GATEWAY,
"\u{1b}]2;pwned\u{7}<html>gateway error</html>",
)
.expect("a non-success status must produce a failure");
assert!(
!failure.chars().any(|c| c.is_control()),
"control char leaked to the terminal: {failure:?}"
);
assert!(failure.contains("502"), "status dropped: {failure:?}");
assert!(
failure.contains("gateway error"),
"the body is all the user gets when there is no message field: {failure:?}"
);
}
/// The unit tests above bind the helper. This one binds the CALL SITE: the
/// failure check has to run before the parse, or a body past the read cap
/// turns a 503 into "invalid JSON response" and the status the user needs is
/// gone. Reordering the two statements in `cmd_add` compiles and passes every
/// other test in this file, so only driving the command catches it.
#[tokio::test]
async fn a_peer_failure_past_the_read_cap_still_reports_its_status() {
let mut peer = mockito::Server::new_async().await;
let mut local = mockito::Server::new_async().await;
let dir = tempfile::TempDir::new().unwrap();
let kp = gitlawb_core::identity::Keypair::generate();
std::fs::write(
dir.path().join("identity.pem"),
kp.to_pem().unwrap().as_bytes(),
)
.unwrap();
let _local_info = local
.mock("GET", "/")
.with_status(200)
.with_header("content-type", "application/json")
.with_body(r#"{"public_url":"https://me.example"}"#)
.create_async()
.await;
// Valid JSON, but past the 8 KiB read cap, so what survives the read is
// truncated and unparseable. This is the shape the cap itself creates.
let oversized = format!(r#"{{"message":"{}"}}"#, "x".repeat(16 * 1024));
let _peer_refusal = peer
.mock("POST", "/api/v1/peers/announce")
.with_status(503)
.with_header("content-type", "application/json")
.with_body(oversized)
.create_async()
.await;
let err = cmd_add(peer.url(), local.url(), Some(dir.path().to_path_buf()))
.await
.expect_err("a 503 from the peer must fail the command");
let rendered = format!("{err:#}");
assert!(
rendered.contains("503"),
"the status must survive a body past the read cap: {rendered:?}"
);
assert!(
!rendered.contains("invalid JSON response"),
"the parse failure must not stand in for the status: {rendered:?}"
);
}
/// An accepted announce produces no error, so the caller proceeds to parse.
#[test]
fn an_accepted_remote_announce_produces_no_failure() {
assert!(remote_announce_failure(StatusCode::OK, r#"{"peer_count":3}"#).is_none());
}
/// The selection, not the helper. A refusal must never yield the success
/// line, and the success line must never carry the warning prefix. This is
/// what stayed green when the entire match was replaced with an
/// unconditional "Added to local peer list."
#[test]
fn a_local_add_refusal_never_selects_the_success_line() {
let (is_warning, line) = local_add_report(
StatusCode::FORBIDDEN,
&json!({ "message": "unproven announce cannot change an existing peer's http_url: did:key:zAbc" }),
);
assert!(is_warning, "a 403 must select the warning stream: {line:?}");
assert!(
!line.contains("Added to local peer list"),
"a refusal rendered as the success line: {line:?}"
);
assert!(
line.contains("unproven announce cannot change an existing peer"),
"the node's reason must reach the user: {line:?}"
);
}
/// The other direction, so the fix cannot be satisfied by always warning.
#[test]
fn an_accepted_local_add_selects_the_success_line() {
let (is_warning, line) = local_add_report(StatusCode::OK, &json!({ "peer_count": 3 }));
assert!(!is_warning, "a 200 must not select the warning stream");
assert!(
line.contains("Added to local peer list"),
"the success line must still print: {line:?}"
);
}
/// The accepted-announce block prints the remote peer's own DID and URL, so
/// a hostile peer answering 200 must not get escapes onto the terminal that
/// way either. The refusal path is not the only sink in this command.
#[test]
fn an_accepted_announce_summary_cannot_carry_terminal_escapes() {
let summary = announced_peer_summary(
"did:key:z6Mk\u{1b}]2;pwned\u{7}abc",
"https://peer.example\u{202e}moc.live//:sptth",
3,
);
assert!(
!summary.chars().any(|c| c.is_control() && c != '\n'),
"control char leaked to the terminal: {summary:?}"
);
assert!(
!summary.chars().any(gitlawb_core::sanitize::is_bidi_format),
"bidi format char leaked to the terminal: {summary:?}"
);
assert!(
summary.contains("did:key:z6Mk") && summary.contains("Their peers: 3"),
"printable content dropped: {summary:?}"
);
}
/// The counterweight to the stripping tests: a legitimate message with RTL
/// script letters and a ZWJ must survive intact, so the sanitizer is not
/// proven by over-stripping.
#[test]
fn a_legitimate_refusal_message_survives_intact() {
let warning = local_add_refusal(
StatusCode::FORBIDDEN,
&json!({ "message": "peer refused \u{0627}\u{200D}b" }),
)
.expect("a refusal must not render as success");
assert!(
warning.contains("peer refused \u{0627}\u{200D}b"),
"legitimate text mangled: {warning:?}"
);
}
}
+2 -9
View File
@@ -154,12 +154,10 @@ async fn cmd_remove(
async fn cmd_list(repo: String, node: String, dir: Option<PathBuf>) -> Result<()> {
let (owner, name) = resolve_owner_repo(&repo, &node, dir.as_deref()).await?;
// Read-visibility-gated on the node: public repos list anonymously, private
// repos need the owner's signature. Sign when an identity is available.
let client = NodeClient::new(&node, load_keypair_from_dir(dir.as_deref()).ok());
let client = NodeClient::new(&node, None);
let resp = client
.get_maybe_signed(&format!("/api/v1/repos/{owner}/{name}/branches/protected"))
.get(&format!("/api/v1/repos/{owner}/{name}/branches/protected"))
.await
.context("failed to connect to node")?;
@@ -309,9 +307,6 @@ mod tests {
"GET",
mockito::Matcher::Regex(r"branches/protected".to_string()),
)
// An identity is supplied, so get_maybe_signed must sign the request.
.match_header("signature", mockito::Matcher::Any)
.match_header("signature-input", mockito::Matcher::Any)
.with_status(200)
.with_header("content-type", "application/json")
.with_body(r#"{"protected_branches":[],"count":0}"#)
@@ -343,8 +338,6 @@ mod tests {
"GET",
mockito::Matcher::Regex(r"branches/protected".to_string()),
)
.match_header("signature", mockito::Matcher::Any)
.match_header("signature-input", mockito::Matcher::Any)
.with_status(200)
.with_header("content-type", "application/json")
.with_body(r#"{"protected_branches":["main","release"],"count":2}"#)
+34 -431
View File
@@ -146,8 +146,6 @@ pub enum RepoCmd {
repo: String,
#[arg(long, default_value = "https://node.gitlawb.com", env = "GITLAWB_NODE")]
node: String,
#[arg(long)]
dir: Option<PathBuf>,
},
}
@@ -205,7 +203,7 @@ pub async fn run(args: RepoArgs) -> Result<()> {
RepoCmd::ReplicaUnregister { repo, node, dir } => {
cmd_replica_unregister(repo, node, dir).await
}
RepoCmd::Replicas { repo, node, dir } => cmd_replicas(repo, node, dir).await,
RepoCmd::Replicas { repo, node } => cmd_replicas(repo, node).await,
}
}
@@ -331,10 +329,7 @@ async fn cmd_clone(name: String, node: String, dir: Option<PathBuf>) -> Result<(
}
async fn cmd_info(repo: String, node: String, dir: Option<PathBuf>) -> Result<()> {
// Sign when an identity is available so the read-visibility-gated replica
// sub-fetch below resolves for a private repo's owner (public repos still
// work anonymously).
let client = NodeClient::new(&node, load_keypair_from_dir(dir.as_deref()).ok());
let client = NodeClient::new(&node, None);
let (owner, name) = if repo.contains('/') {
let (o, n) = repo.split_once('/').unwrap();
@@ -344,10 +339,7 @@ async fn cmd_info(repo: String, node: String, dir: Option<PathBuf>) -> Result<()
(short, repo)
};
let resp = client
.get_maybe_signed(&format!("/api/v1/repos/{owner}/{name}"))
.await
.context("failed to connect to node")?;
let resp = client.get(&format!("/api/v1/repos/{owner}/{name}")).await?;
// A non-existent (or unreadable/quarantined) repo is a real 404 from the
// node — surface it plainly instead of printing a stub card with `?` fields
// and a placeholder owner DID.
@@ -387,7 +379,7 @@ async fn cmd_info(repo: String, node: String, dir: Option<PathBuf>) -> Result<()
// Replica count — failure to fetch is non-fatal (older nodes don't expose this).
if let Ok(resp) = client
.get_maybe_signed(&format!("/api/v1/repos/{owner}/{name}/replicas"))
.get(&format!("/api/v1/repos/{owner}/{name}/replicas"))
.await
{
if resp.status().is_success() {
@@ -467,17 +459,15 @@ async fn cmd_replica_unregister(repo: String, node: String, dir: Option<PathBuf>
Ok(())
}
async fn cmd_replicas(repo: String, node: String, dir: Option<PathBuf>) -> Result<()> {
async fn cmd_replicas(repo: String, node: String) -> Result<()> {
let (owner, name) = repo
.split_once('/')
.map(|(o, n)| (o.to_string(), n.to_string()))
.context("use owner/repo format")?;
// Read-visibility-gated: public repos list anonymously, private repos need
// the owner's signature. Sign when an identity is available.
let client = NodeClient::new(&node, load_keypair_from_dir(dir.as_deref()).ok());
let client = NodeClient::new(&node, None);
let resp = client
.get_maybe_signed(&format!("/api/v1/repos/{owner}/{name}/replicas"))
.get(&format!("/api/v1/repos/{owner}/{name}/replicas"))
.await
.context("failed to connect to node")?;
let status = resp.status();
@@ -595,7 +585,7 @@ async fn cmd_fork(
}
/// Returns (owner, repo_name) — if repo contains '/', splits on it; otherwise uses caller's DID.
pub(crate) async fn resolve_owner_repo_pair(
async fn resolve_owner_repo_pair(
repo: &str,
node: &str,
dir: Option<&std::path::Path>,
@@ -668,23 +658,16 @@ async fn cmd_label_remove(
async fn cmd_label_list(repo: String, node: String, dir: Option<PathBuf>) -> Result<()> {
let (owner, name) = resolve_owner_repo_pair(&repo, &node, dir.as_deref()).await?;
// Read-visibility-gated like the sibling read surfaces (replicas, protected
// branches): sign when an identity is available so a private-repo owner can
// read their own labels, while public repos stay anonymously listable.
let client = NodeClient::new(&node, load_keypair_from_dir(dir.as_deref()).ok());
let resp = client
.get_maybe_signed(&format!("/api/v1/repos/{owner}/{name}/labels"))
let resp: Value = client
.get_authed(&format!("/api/v1/repos/{owner}/{name}/labels"))
.await?
.json()
.await
.context("failed to connect to node")?;
let status = resp.status();
let body: Value = resp.json().await.unwrap_or_default();
if !status.is_success() {
let msg = body["message"].as_str().unwrap_or("unknown error");
anyhow::bail!("list labels failed ({status}): {msg}");
}
.context("invalid JSON")?;
let labels = body["labels"].as_array().cloned().unwrap_or_default();
let labels = resp["labels"].as_array().cloned().unwrap_or_default();
if labels.is_empty() {
println!("No labels on {owner}/{name}");
} else {
@@ -696,39 +679,17 @@ async fn cmd_label_list(repo: String, node: String, dir: Option<PathBuf>) -> Res
Ok(())
}
/// Parse the protected-branch list from the node's `GET /branches/protected`
/// response. The endpoint returns `{"protected_branches": [...], "count": N}`
/// (see `crates/gitlawb-node/src/api/protect.rs`); each entry may be a plain
/// branch-name string or an object carrying a `name` field.
fn parse_protected_branches(val: &Value) -> Vec<String> {
val["protected_branches"]
.as_array()
.map(|arr| {
arr.iter()
.filter_map(|b| {
b.as_str()
.map(String::from)
.or_else(|| b["name"].as_str().map(String::from))
})
.collect()
})
.unwrap_or_default()
}
async fn cmd_owner(repo: String, node: String, dir: Option<PathBuf>, json_out: bool) -> Result<()> {
let keypair = load_keypair_from_dir(dir.as_deref())?;
let my_did = keypair.did().to_string();
let my_short = my_did.split(':').next_back().unwrap_or(&my_did).to_string();
let (owner, name) = resolve_owner_repo_pair(&repo, &node, dir.as_deref()).await?;
// Sign with the loaded identity so the read-visibility-gated protected-branch
// fetch below works on the owner's own private repos.
let client = NodeClient::new(&node, Some(keypair));
let client = NodeClient::new(&node, None);
// Fetch repo info (get_repo is read-visibility-gated; sign so the owner can
// inspect their own private repo).
// Fetch repo info
let resp = client
.get_maybe_signed(&format!("/api/v1/repos/{owner}/{name}"))
.get(&format!("/api/v1/repos/{owner}/{name}"))
.await
.context("failed to connect to node")?;
if !resp.status().is_success() {
@@ -743,14 +704,25 @@ async fn cmd_owner(repo: String, node: String, dir: Option<PathBuf>, json_out: b
.to_string();
let is_owner = my_did == owner_did || my_short == owner_short;
// Fetch protected branches (read-visibility-gated; signed via the client).
// Fetch protected branches
let protected: Vec<String> = match client
.get_maybe_signed(&format!("/api/v1/repos/{owner}/{name}/branches/protected"))
.get(&format!("/api/v1/repos/{owner}/{name}/branches/protected"))
.await
{
Ok(resp) if resp.status().is_success() => {
let val: Value = resp.json().await.unwrap_or_default();
parse_protected_branches(&val)
val.as_array()
.or_else(|| val["branches"].as_array())
.map(|arr| {
arr.iter()
.filter_map(|b| {
b.as_str()
.map(String::from)
.or_else(|| b["name"].as_str().map(String::from))
})
.collect()
})
.unwrap_or_default()
}
_ => Vec::new(),
};
@@ -788,20 +760,6 @@ mod tests {
use super::*;
use tempfile::TempDir;
#[test]
fn parse_protected_branches_reads_node_shape() {
use serde_json::json;
assert_eq!(
parse_protected_branches(&json!({"protected_branches":["main","release"],"count":2})),
vec!["main".to_string(), "release".to_string()]
);
assert!(parse_protected_branches(&json!({"count":0})).is_empty());
assert_eq!(
parse_protected_branches(&json!({"protected_branches":[{"name":"main"}]})),
vec!["main".to_string()]
);
}
fn write_identity(dir: &TempDir) {
let kp = gitlawb_core::identity::Keypair::generate();
let pem = kp.to_pem().unwrap();
@@ -1139,10 +1097,6 @@ mod tests {
"GET",
mockito::Matcher::Regex(r"^/api/v1/repos/[^/]+/myrepo/labels$".to_string()),
)
// Identity is loaded, so get_maybe_signed must sign — requiring the
// header guards against a regression back to bare get().
.match_header("signature", mockito::Matcher::Any)
.match_header("signature-input", mockito::Matcher::Any)
.with_status(200)
.with_header("content-type", "application/json")
.with_body(r#"{"labels":[]}"#)
@@ -1169,10 +1123,6 @@ mod tests {
"GET",
mockito::Matcher::Regex(r"^/api/v1/repos/[^/]+/myrepo/labels$".to_string()),
)
// Identity is loaded, so get_maybe_signed must sign — requiring the
// header guards against a regression back to bare get().
.match_header("signature", mockito::Matcher::Any)
.match_header("signature-input", mockito::Matcher::Any)
.with_status(200)
.with_header("content-type", "application/json")
.with_body(r#"{"labels":["language:rust","topic:defi"]}"#)
@@ -1188,97 +1138,6 @@ mod tests {
.unwrap();
}
#[tokio::test]
async fn test_cmd_label_list_anonymous_when_no_identity() {
// Empty dir → no identity → get_maybe_signed must fall back to an
// anonymous GET (the public-repo path). Assert NO signature header.
let dir = TempDir::new().unwrap();
let mut server = mockito::Server::new_async().await;
let _m = server
.mock("GET", mockito::Matcher::Regex(r"/labels$".to_string()))
.match_header("signature", mockito::Matcher::Missing)
.with_status(200)
.with_header("content-type", "application/json")
.with_body(r#"{"labels":[]}"#)
.create_async()
.await;
cmd_label_list(
"owner/pubrepo".to_string(),
server.url(),
Some(dir.path().to_path_buf()),
)
.await
.unwrap();
}
#[tokio::test]
async fn test_cmd_label_list_404_errors() {
// A non-reader of a private repo gets 404 from the now-gated endpoint.
// The client must surface that as an error, not print "No labels" and
// exit 0 (the error body has no `labels` key).
let dir = TempDir::new().unwrap();
write_identity(&dir);
let mut server = mockito::Server::new_async().await;
let _m = server
.mock(
"GET",
mockito::Matcher::Regex(r"^/api/v1/repos/[^/]+/myrepo/labels$".to_string()),
)
.with_status(404)
.with_header("content-type", "application/json")
.with_body(r#"{"message":"repo not found"}"#)
.create_async()
.await;
let err = cmd_label_list(
"myrepo".to_string(),
server.url(),
Some(dir.path().to_path_buf()),
)
.await
.unwrap_err();
assert!(
err.to_string().contains("list labels failed (404")
&& err.to_string().contains("repo not found"),
"got: {err}"
);
}
#[tokio::test]
async fn test_cmd_label_list_non_json_error_surfaces_status() {
// A non-2xx response with a non-JSON body (e.g. a proxy 502) must still
// surface the HTTP status, not collapse to a bare JSON-parse error.
let dir = TempDir::new().unwrap();
write_identity(&dir);
let mut server = mockito::Server::new_async().await;
let _m = server
.mock(
"GET",
mockito::Matcher::Regex(r"^/api/v1/repos/[^/]+/myrepo/labels$".to_string()),
)
.with_status(502)
.with_header("content-type", "text/html")
.with_body("<html>502 Bad Gateway</html>")
.create_async()
.await;
let err = cmd_label_list(
"myrepo".to_string(),
server.url(),
Some(dir.path().to_path_buf()),
)
.await
.unwrap_err();
assert!(
err.to_string().contains("list labels failed (502"),
"got: {err}"
);
}
#[tokio::test]
async fn test_cmd_owner_is_owner() {
let dir = TempDir::new().unwrap();
@@ -1289,15 +1148,11 @@ mod tests {
let short = did.split(':').next_back().unwrap().to_string();
let mut server = mockito::Server::new_async().await;
// Both fetches are read-visibility-gated; with an identity loaded they
// must be signed. Requiring the header guards a regression to get().
let _repo = server
.mock(
"GET",
mockito::Matcher::Regex(format!(r"^/api/v1/repos/{short}/myrepo$")),
)
.match_header("signature", mockito::Matcher::Any)
.match_header("signature-input", mockito::Matcher::Any)
.with_status(200)
.with_header("content-type", "application/json")
.with_body(format!(
@@ -1312,11 +1167,9 @@ mod tests {
r"^/api/v1/repos/{short}/myrepo/branches/protected$"
)),
)
.match_header("signature", mockito::Matcher::Any)
.match_header("signature-input", mockito::Matcher::Any)
.with_status(200)
.with_header("content-type", "application/json")
.with_body(r#"{"protected_branches":["main"],"count":1}"#)
.with_body(r#"["main"]"#)
.create_async()
.await;
@@ -1357,7 +1210,7 @@ mod tests {
)
.with_status(200)
.with_header("content-type", "application/json")
.with_body(r#"{"protected_branches":["main","release"],"count":2}"#)
.with_body(r#"["main","release"]"#)
.create_async()
.await;
@@ -1398,7 +1251,7 @@ mod tests {
)
.with_status(200)
.with_header("content-type", "application/json")
.with_body(r#"{"protected_branches":["main"],"count":1}"#)
.with_body(r#"["main"]"#)
.create_async()
.await;
@@ -1439,254 +1292,4 @@ mod tests {
.unwrap_err();
assert!(err.to_string().contains("not found"), "got: {err}");
}
#[tokio::test]
async fn test_cmd_replicas_signs_when_identity_present() {
let dir = TempDir::new().unwrap();
write_identity(&dir);
let mut server = mockito::Server::new_async().await;
// Read-visibility-gated: with an identity the request must be signed.
let _m = server
.mock("GET", mockito::Matcher::Regex(r"/replicas$".to_string()))
.match_header("signature", mockito::Matcher::Any)
.match_header("signature-input", mockito::Matcher::Any)
.with_status(200)
.with_header("content-type", "application/json")
.with_body(r#"{"repo":"owner/myrepo","replica_count":0,"replicas":[]}"#)
.create_async()
.await;
cmd_replicas(
"owner/myrepo".to_string(),
server.url(),
Some(dir.path().to_path_buf()),
)
.await
.unwrap();
}
#[tokio::test]
async fn test_cmd_replicas_anonymous_when_no_identity() {
// Empty dir → no identity → get_maybe_signed must fall back to an
// anonymous GET (the public-repo path). Assert NO signature header.
let dir = TempDir::new().unwrap();
let mut server = mockito::Server::new_async().await;
let _m = server
.mock("GET", mockito::Matcher::Regex(r"/replicas$".to_string()))
.match_header("signature", mockito::Matcher::Missing)
.with_status(200)
.with_header("content-type", "application/json")
.with_body(r#"{"repo":"owner/pubrepo","replica_count":0,"replicas":[]}"#)
.create_async()
.await;
cmd_replicas(
"owner/pubrepo".to_string(),
server.url(),
Some(dir.path().to_path_buf()),
)
.await
.unwrap();
}
#[tokio::test]
async fn test_cmd_info_signs_repo_and_replica_fetches() {
let dir = TempDir::new().unwrap();
write_identity(&dir);
let mut server = mockito::Server::new_async().await;
// Both the primary repo fetch and the replica sub-fetch are
// read-visibility-gated; with an identity loaded both must be signed.
let _repo = server
.mock(
"GET",
mockito::Matcher::Regex(r"^/api/v1/repos/owner/myrepo$".to_string()),
)
.match_header("signature", mockito::Matcher::Any)
.match_header("signature-input", mockito::Matcher::Any)
.with_status(200)
.with_header("content-type", "application/json")
.with_body(
r#"{"id":"r1","name":"myrepo","owner_did":"did:key:zOwner","is_public":false,"default_branch":"main"}"#,
)
.create_async()
.await;
let _replicas = server
.mock("GET", mockito::Matcher::Regex(r"/replicas$".to_string()))
.match_header("signature", mockito::Matcher::Any)
.match_header("signature-input", mockito::Matcher::Any)
.with_status(200)
.with_header("content-type", "application/json")
.with_body(r#"{"replica_count":0,"replicas":[]}"#)
.create_async()
.await;
cmd_info(
"owner/myrepo".to_string(),
server.url(),
Some(dir.path().to_path_buf()),
)
.await
.unwrap();
}
#[tokio::test]
async fn test_cmd_info_404_surfaces_error_not_fake_repo() {
// A non-reader of a private repo gets a JSON 404 from the now-gated
// GET /api/v1/repos/{owner}/{name}. The command must surface that as an
// error, not parse the error body and print a fabricated repo summary
// (owner / "?" / public=true / main) before exiting 0.
let dir = TempDir::new().unwrap();
write_identity(&dir);
let mut server = mockito::Server::new_async().await;
let _repo = server
.mock(
"GET",
mockito::Matcher::Regex(r"^/api/v1/repos/owner/myrepo$".to_string()),
)
.with_status(404)
.with_header("content-type", "application/json")
.with_body(r#"{"message":"repo not found"}"#)
.create_async()
.await;
let err = cmd_info(
"owner/myrepo".to_string(),
server.url(),
Some(dir.path().to_path_buf()),
)
.await
.unwrap_err();
// The merged cmd_info maps a 404 to a generic "not found" (it does not
// echo the server's message body on 404), which still surfaces the error
// rather than fabricating a repo card, and avoids leaking any
// server-provided detail on a private-repo denial.
assert!(
err.to_string()
.contains("repository 'owner/myrepo' not found"),
"got: {err}"
);
}
#[tokio::test]
async fn test_cmd_info_non_json_error_surfaces_status() {
// A non-2xx response with a non-JSON body (e.g. a proxy 502) must still
// surface the HTTP status, not collapse to a bare JSON-parse error.
let dir = TempDir::new().unwrap();
write_identity(&dir);
let mut server = mockito::Server::new_async().await;
let _repo = server
.mock(
"GET",
mockito::Matcher::Regex(r"^/api/v1/repos/owner/myrepo$".to_string()),
)
.with_status(502)
.with_header("content-type", "text/html")
.with_body("<html>502 Bad Gateway</html>")
.create_async()
.await;
let err = cmd_info(
"owner/myrepo".to_string(),
server.url(),
Some(dir.path().to_path_buf()),
)
.await
.unwrap_err();
assert!(
err.to_string().contains("repo info failed (502"),
"got: {err}"
);
}
}
// Drives the three deny/negative arms that the merge reconciliation changed but
// that no existing test executed (they were only reasoned): resolve_owner_did
// with no identity, cmd_info on an authenticated-but-denied 403, and
// get_maybe_signed with no keypair. Each asserts the must-not case.
#[cfg(test)]
mod vet_merge_deny_arms {
use super::*;
use tempfile::TempDir;
fn write_identity(dir: &TempDir) {
let kp = gitlawb_core::identity::Keypair::generate();
let pem = kp.to_pem().unwrap();
std::fs::write(dir.path().join("identity.pem"), pem.as_bytes()).unwrap();
}
// No local identity must hard-error, never fall back to the node's own DID.
// (Load-bearing: #113's pre-merge resolve_owner_did returned Ok via a node
// fetch here; a reintroduced fallback makes this go red.)
#[tokio::test]
async fn resolve_owner_did_no_identity_errors_not_fallback() {
let empty = TempDir::new().unwrap(); // no identity.pem written
let err = resolve_owner_did("http://unused.invalid", Some(empty.path()))
.await
.unwrap_err();
assert!(
err.to_string()
.contains("no local identity to resolve the repo owner"),
"got: {err}"
);
}
// An authenticated caller DENIED with a 403 must surface an error, never
// render a repo card (deny-as-success is the INV-8 client-contract break).
#[tokio::test]
async fn cmd_info_403_denied_surfaces_error_not_card() {
let dir = TempDir::new().unwrap();
write_identity(&dir);
let mut server = mockito::Server::new_async().await;
let _m = server
.mock(
"GET",
mockito::Matcher::Regex(r"^/api/v1/repos/owner/myrepo$".to_string()),
)
.with_status(403)
.with_header("content-type", "application/json")
.with_body(r#"{"message":"forbidden"}"#)
.create_async()
.await;
let err = cmd_info(
"owner/myrepo".to_string(),
server.url(),
Some(dir.path().to_path_buf()),
)
.await
.unwrap_err();
assert!(
err.to_string().contains("repo info failed (403"),
"got: {err}"
);
}
// No keypair must send an unsigned GET; the mock only matches when the
// Signature header is absent, so a wrongly-signed request fails to match.
#[tokio::test]
async fn get_maybe_signed_no_keypair_sends_unsigned() {
let mut server = mockito::Server::new_async().await;
let _m = server
.mock("GET", "/api/v1/repos/owner/pub")
.match_header("signature", mockito::Matcher::Missing)
.with_status(200)
.with_header("content-type", "application/json")
.with_body("{}")
.create_async()
.await;
let client = NodeClient::new(server.url(), None);
let resp = client
.get_maybe_signed("/api/v1/repos/owner/pub")
.await
.unwrap();
assert!(
resp.status().is_success(),
"unsigned GET should match the Missing-signature mock; got {}",
resp.status()
);
}
}
+13 -52
View File
@@ -59,7 +59,7 @@ pub async fn run(args: StatusArgs) -> Result<()> {
println!(" repo {short_did}/{repo}");
}
None => {
println!(" repo (not in a gitlawb repo — no gitlawb:// remote)");
println!(" repo (not in a gitlawb repo — no gitlawb:// origin)");
}
}
@@ -143,68 +143,29 @@ fn trust_bar(score: f64) -> String {
format!("{}{}", "█".repeat(filled), "░".repeat(empty))
}
/// Find a `gitlawb://<did>/<repo>` remote in the current repo.
///
/// Checks `origin` first (clones), then `gitlawb` (the name `gl init` adds),
/// then any other remote with a gitlawb:// URL.
/// Parse `gitlawb://<did>/<repo>` from the git origin remote.
fn detect_gitlawb_remote() -> Option<(String, String)> {
let out = std::process::Command::new("git")
.args(["remote"])
.args(["remote", "get-url", "origin"])
.stderr(std::process::Stdio::null())
.output()
.ok()?;
if !out.status.success() {
return None;
}
let names: Vec<String> = String::from_utf8(out.stdout)
.ok()?
.lines()
.map(str::trim)
.filter(|l| !l.is_empty())
.map(str::to_string)
.collect();
let preferred = ["origin", "gitlawb"];
let ordered = preferred
.iter()
.filter(|p| names.iter().any(|n| n == *p))
.map(|p| p.to_string())
.chain(
names
.iter()
.filter(|n| !preferred.contains(&n.as_str()))
.cloned(),
);
for name in ordered {
// A remote can carry several URLs, and the gitlawb:// one may be a
// push URL behind an https fetch URL — check every fetch and push URL.
for extra in [&["--all"][..], &["--push", "--all"][..]] {
let mut args = vec!["remote", "get-url"];
args.extend_from_slice(extra);
args.push(&name);
let Ok(out) = std::process::Command::new("git")
.args(&args)
.stderr(std::process::Stdio::null())
.output()
else {
continue;
};
if !out.status.success() {
continue;
}
let Ok(urls) = String::from_utf8(out.stdout) else {
continue;
};
if let Some(parsed) = urls.lines().find_map(parse_gitlawb_url) {
return Some(parsed);
}
}
let url = String::from_utf8(out.stdout).ok()?;
let rest = url.trim().strip_prefix("gitlawb://")?;
let slash = rest.rfind('/')?;
let did = rest[..slash].to_string();
let repo = rest[slash + 1..].to_string();
if did.is_empty() || repo.is_empty() {
return None;
}
None
Some((did, repo))
}
/// Parse a gitlawb:// URL string into (did, repo).
/// Parse a gitlawb:// URL string into (did, repo) — extracted for testing.
#[cfg(test)]
fn parse_gitlawb_url(url: &str) -> Option<(String, String)> {
let rest = url.trim().strip_prefix("gitlawb://")?;
let slash = rest.rfind('/')?;
+69 -3
View File
@@ -2,7 +2,7 @@ use anyhow::{Context, Result};
use clap::{Args, Subcommand};
use std::path::PathBuf;
use crate::http::{read_body_capped, sanitize_node_msg, NodeClient};
use crate::http::NodeClient;
use crate::identity::load_keypair_from_dir;
#[derive(Args)]
@@ -46,7 +46,7 @@ pub async fn run(args: SyncArgs) -> Result<()> {
if !status.is_success() {
// Bound the read: a hostile or broken node must not force an
// unbounded allocation just to surface a denial (INV-6, read half).
let raw = read_body_capped(resp, 8 * 1024).await.text;
let raw = read_body_capped(resp, 8 * 1024).await;
let msg = serde_json::from_str::<serde_json::Value>(&raw)
.ok()
.and_then(|v| {
@@ -104,6 +104,51 @@ fn trigger_counts(resp: &serde_json::Value) -> (u64, u64) {
)
}
/// Read at most `cap` bytes of a response body. Bounds the allocation from a
/// hostile or broken node returning a huge error body — the display is capped
/// separately, but the read itself must not be unbounded (INV-6, read half).
async fn read_body_capped(mut resp: reqwest::Response, cap: usize) -> String {
let mut buf: Vec<u8> = Vec::new();
while buf.len() < cap {
match resp.chunk().await {
Ok(Some(chunk)) => {
let take = (cap - buf.len()).min(chunk.len());
buf.extend_from_slice(&chunk[..take]);
if take < chunk.len() {
break; // hit the cap mid-chunk
}
}
_ => break, // end of body or read error — return what we have
}
}
String::from_utf8_lossy(&buf).into_owned()
}
/// Strip terminal-dangerous characters from (and cap the length of) a
/// node-supplied error string before surfacing it. The node a caller talks to
/// could be hostile and embed escape sequences in its error body; those must not
/// reach the terminal verbatim (INV-6). We drop the C0/C1 control bytes (which
/// defangs ANSI/OSC escapes) AND the Unicode bidi/format controls (which
/// `char::is_control` does not cover — they can reorder the displayed line).
fn sanitize_node_msg(s: &str) -> String {
s.chars()
.filter(|c| !c.is_control() && !is_bidi_format(*c))
.take(200)
.collect()
}
/// Unicode bidirectional and directional-isolate format characters (category
/// `Cf`). These are not `char::is_control()` (that is category `Cc` only), but a
/// right-to-left override or isolate can visually reorder a terminal line to
/// spoof the error text, so they are stripped alongside the control bytes.
fn is_bidi_format(c: char) -> bool {
matches!(c,
'\u{200E}' | '\u{200F}' | '\u{061C}' // LRM, RLM, ALM
| '\u{202A}'..='\u{202E}' // LRE, RLE, PDF, LRO, RLO
| '\u{2066}'..='\u{2069}' // LRI, RLI, FSI, PDI
)
}
#[cfg(test)]
mod tests {
use super::*;
@@ -227,6 +272,27 @@ mod tests {
run(args).await.unwrap();
}
#[test]
fn sanitize_strips_controls_bidi_and_caps_length() {
// C0 (ESC/BEL) and the Cf bidi override (U+202E) are both removed; the
// printable text survives. (Note: a stripped ESC leaves any following
// "[31m" as inert literal text — that is the point, so the input here
// avoids that residue to keep the expectation unambiguous.)
let out = sanitize_node_msg("a\u{1b}\u{07}b\u{202e}c");
assert!(
!out.chars().any(|c| c.is_control()),
"control char leaked: {out:?}"
);
assert!(
!out.contains('\u{202e}'),
"RLO bidi override leaked: {out:?}"
);
assert_eq!(out, "abc");
// Length is capped at 200 chars regardless of input size.
let long = "x".repeat(250);
assert_eq!(sanitize_node_msg(&long).chars().count(), 200);
}
#[tokio::test]
async fn trigger_handles_oversized_error_body_without_unbounded_output() {
// A hostile/broken node returns a 2 MB error body. The command must still
@@ -261,7 +327,7 @@ mod tests {
.create_async()
.await;
let resp = reqwest::get(format!("{}/big", server.url())).await.unwrap();
let out = read_body_capped(resp, 8192).await.text;
let out = read_body_capped(resp, 8192).await;
assert!(out.len() <= 8192, "read not bounded: {} bytes", out.len());
assert!(!out.is_empty(), "expected some body");
}
+26 -169
View File
@@ -42,8 +42,6 @@ pub enum WebhookCmd {
repo: String,
#[arg(long, default_value = "https://node.gitlawb.com", env = "GITLAWB_NODE")]
node: String,
#[arg(long)]
dir: Option<PathBuf>,
},
/// Delete a webhook
Delete {
@@ -68,7 +66,7 @@ pub async fn run(args: WebhookArgs) -> Result<()> {
node,
dir,
} => cmd_create(repo, url, events, secret, node, dir).await,
WebhookCmd::List { repo, node, dir } => cmd_list(repo, node, dir).await,
WebhookCmd::List { repo, node } => cmd_list(repo, node).await,
WebhookCmd::Delete {
repo,
id,
@@ -78,6 +76,15 @@ pub async fn run(args: WebhookArgs) -> Result<()> {
}
}
async fn resolve_owner(client: &NodeClient) -> Result<String> {
let info: Value = client.get("/").await?.json().await?;
let did = info["did"]
.as_str()
.context("node missing DID")?
.to_string();
Ok(did.split(':').next_back().unwrap_or(&did).to_string())
}
async fn cmd_create(
repo: String,
url: String,
@@ -87,8 +94,8 @@ async fn cmd_create(
dir: Option<PathBuf>,
) -> Result<()> {
let keypair = load_keypair_from_dir(dir.as_deref())?;
let (owner, name) = crate::repo::resolve_owner_repo_pair(&repo, &node, dir.as_deref()).await?;
let client = NodeClient::new(&node, Some(keypair));
let owner = resolve_owner(&client).await?;
let event_list: Vec<&str> = events.split(',').map(str::trim).collect();
@@ -99,7 +106,7 @@ async fn cmd_create(
}))?;
let resp = client
.post(&format!("/api/v1/repos/{owner}/{name}/hooks"), &payload)
.post(&format!("/api/v1/repos/{owner}/{repo}/hooks"), &payload)
.await
.context("failed to connect to node")?;
let status = resp.status();
@@ -136,31 +143,18 @@ async fn cmd_create(
Ok(())
}
async fn cmd_list(repo: String, node: String, dir: Option<PathBuf>) -> Result<()> {
// The node owner-gates GET /hooks (callback URLs are owner-secret), so the
// list request must be signed — anonymous callers get 401.
let keypair = load_keypair_from_dir(dir.as_deref())?;
let (owner, name) = crate::repo::resolve_owner_repo_pair(&repo, &node, dir.as_deref()).await?;
let client = NodeClient::new(&node, Some(keypair));
async fn cmd_list(repo: String, node: String) -> Result<()> {
let client = NodeClient::new(&node, None);
let owner = resolve_owner(&client).await?;
// get_signed (not get) attaches the RFC 9421 signature — plain get() never
// signs, and the node owner-gates this route, so an unsigned GET 401s.
let resp = client
.get_signed(&format!("/api/v1/repos/{owner}/{name}/hooks"))
.await?;
let status = resp.status();
let body: Value = resp.json().await.context("invalid JSON")?;
let resp: Value = client
.get(&format!("/api/v1/repos/{owner}/{repo}/hooks"))
.await?
.json()
.await
.context("invalid JSON")?;
if !status.is_success() {
let msg = body["message"].as_str().unwrap_or("unknown error");
anyhow::bail!("list webhooks failed ({status}): {msg}");
}
let hooks = body
.get("webhooks")
.and_then(Value::as_array)
.cloned()
.context("node response missing webhooks array")?;
let hooks = resp["webhooks"].as_array().cloned().unwrap_or_default();
if hooks.is_empty() {
println!("No webhooks for {repo}");
return Ok(());
@@ -191,13 +185,13 @@ async fn cmd_list(repo: String, node: String, dir: Option<PathBuf>) -> Result<()
async fn cmd_delete(repo: String, id: String, node: String, dir: Option<PathBuf>) -> Result<()> {
let keypair = load_keypair_from_dir(dir.as_deref())?;
let (owner, name) = crate::repo::resolve_owner_repo_pair(&repo, &node, dir.as_deref()).await?;
let client = NodeClient::new(&node, Some(keypair));
let owner = resolve_owner(&client).await?;
let payload = serde_json::to_vec(&serde_json::json!({}))?;
let resp = client
.delete(
&format!("/api/v1/repos/{owner}/{name}/hooks/{id}"),
&format!("/api/v1/repos/{owner}/{repo}/hooks/{id}"),
&payload,
)
.await
@@ -343,170 +337,33 @@ mod tests {
#[tokio::test]
async fn test_list_webhooks_empty() {
let mut server = mockito::Server::new_async().await;
let (dir, _kp) = tmp_identity();
let _root = mock_root(&mut server).await;
let _m = server
.mock("GET", mockito::Matcher::Regex(r"/hooks$".to_string()))
// The route is owner-gated, so the list request must be signed.
// Requiring the header here is what catches a regression to plain get().
.match_header("signature", mockito::Matcher::Any)
.match_header("signature-input", mockito::Matcher::Any)
.with_status(200)
.with_header("content-type", "application/json")
.with_body(r#"{"webhooks":[]}"#)
.create_async()
.await;
cmd_list(
"my-repo".to_string(),
server.url(),
Some(dir.path().to_path_buf()),
)
.await
.unwrap();
cmd_list("my-repo".to_string(), server.url()).await.unwrap();
}
#[tokio::test]
async fn test_list_webhooks_with_items() {
let mut server = mockito::Server::new_async().await;
let (dir, _kp) = tmp_identity();
let _root = mock_root(&mut server).await;
let _m = server
.mock("GET", mockito::Matcher::Regex(r"/hooks$".to_string()))
.match_header("signature", mockito::Matcher::Any)
.match_header("signature-input", mockito::Matcher::Any)
.with_status(200)
.with_header("content-type", "application/json")
.with_body(r#"{"webhooks":[{"id":"h1","url":"https://a.com","events":["push"],"active":true},{"id":"h2","url":"https://b.com","events":["*"],"active":false}]}"#)
.create_async()
.await;
cmd_list(
"my-repo".to_string(),
server.url(),
Some(dir.path().to_path_buf()),
)
.await
.unwrap();
}
#[tokio::test]
async fn test_list_owner_comes_from_keypair_not_node_did() {
// The owner segment must be derived from the caller's keypair (loaded
// from --dir), NOT the node root DID. mock_root returns a different DID;
// if the code regressed to using it, this path mock would not match.
let mut server = mockito::Server::new_async().await;
let (dir, kp) = tmp_identity();
let _root = mock_root(&mut server).await;
let did = kp.did().to_string();
let short = did.split(':').next_back().unwrap_or(&did).to_string();
let _m = server
.mock(
"GET",
mockito::Matcher::Regex(format!(r"/api/v1/repos/{short}/my-repo/hooks$")),
)
.with_status(200)
.with_header("content-type", "application/json")
.with_body(r#"{"webhooks":[]}"#)
.expect(1)
.create_async()
.await;
cmd_list(
"my-repo".to_string(),
server.url(),
Some(dir.path().to_path_buf()),
)
.await
.unwrap();
_m.assert_async().await;
}
#[tokio::test]
async fn test_list_honors_owner_repo_arg() {
// An explicit `owner/repo` argument is split and used verbatim.
let mut server = mockito::Server::new_async().await;
let (dir, _kp) = tmp_identity();
let _root = mock_root(&mut server).await;
let _m = server
.mock(
"GET",
mockito::Matcher::Regex(r"/api/v1/repos/someoneelse/their-repo/hooks$".to_string()),
)
.with_status(200)
.with_header("content-type", "application/json")
.with_body(r#"{"webhooks":[]}"#)
.expect(1)
.create_async()
.await;
cmd_list(
"someoneelse/their-repo".to_string(),
server.url(),
Some(dir.path().to_path_buf()),
)
.await
.unwrap();
_m.assert_async().await;
}
#[tokio::test]
async fn test_list_webhooks_non_success_status_errors() {
// A 403 (or any non-2xx) JSON body has no `webhooks` field; cmd_list must
// surface the failure, not print "No webhooks" and exit 0.
let mut server = mockito::Server::new_async().await;
let (dir, _kp) = tmp_identity();
let _root = mock_root(&mut server).await;
let _m = server
.mock("GET", mockito::Matcher::Regex(r"/hooks$".to_string()))
.with_status(403)
.with_header("content-type", "application/json")
.with_body(r#"{"message":"not the owner"}"#)
.create_async()
.await;
let err = cmd_list(
"my-repo".to_string(),
server.url(),
Some(dir.path().to_path_buf()),
)
.await
.unwrap_err();
assert!(err.to_string().contains("not the owner"));
}
#[tokio::test]
async fn test_list_webhooks_2xx_missing_array_errors() {
// A 2xx body without a `webhooks` array signals a node/API contract
// regression. cmd_list must error rather than print "No webhooks".
let mut server = mockito::Server::new_async().await;
let (dir, _kp) = tmp_identity();
let _root = mock_root(&mut server).await;
let _m = server
.mock("GET", mockito::Matcher::Regex(r"/hooks$".to_string()))
.match_header("signature", mockito::Matcher::Any)
.match_header("signature-input", mockito::Matcher::Any)
.with_status(200)
.with_header("content-type", "application/json")
.with_body(r#"{"ok":true}"#)
.create_async()
.await;
let err = cmd_list(
"my-repo".to_string(),
server.url(),
Some(dir.path().to_path_buf()),
)
.await
.unwrap_err();
assert!(err.to_string().contains("missing webhooks array"));
cmd_list("my-repo".to_string(), server.url()).await.unwrap();
}
// ── delete ───────────────────────────────────────────────────────
+14 -334
View File
@@ -1,11 +1,11 @@
//! `gl whoami` — print current identity and optional node registration info.
use anyhow::{bail, Result};
use anyhow::Result;
use clap::Args;
use serde_json::{json, Value};
use std::path::PathBuf;
use crate::http::{read_body_capped, sanitize_node_msg, NodeClient};
use crate::http::NodeClient;
use crate::identity::load_keypair_from_dir;
#[derive(Args)]
@@ -22,10 +22,6 @@ pub struct WhoamiArgs {
}
pub async fn run(args: WhoamiArgs) -> Result<()> {
run_to_writer(args, &mut std::io::stdout()).await
}
pub(crate) async fn run_to_writer(args: WhoamiArgs, w: &mut impl std::io::Write) -> Result<()> {
let keypair = load_keypair_from_dir(args.dir.as_deref())?;
let did = keypair.did().to_string();
let short = did.split(':').next_back().unwrap_or(&did).to_string();
@@ -45,7 +41,7 @@ pub(crate) async fn run_to_writer(args: WhoamiArgs, w: &mut impl std::io::Write)
if let Some(caps) = info["capabilities"].as_array() {
capabilities = caps
.iter()
.filter_map(|c| c.as_str().map(sanitize_node_msg))
.filter_map(|c| c.as_str().map(String::from))
.collect();
}
// Try to get repo count
@@ -55,35 +51,11 @@ pub(crate) async fn run_to_writer(args: WhoamiArgs, w: &mut impl std::io::Write)
}
}
}
Ok(resp) if resp.status().as_u16() == 404 => {
Ok(_) => {
registered = Some(false);
}
Ok(resp) => {
let status = resp.status();
let raw = read_body_capped(resp, 8 * 1024).await.text;
let msg = serde_json::from_str::<Value>(&raw)
.ok()
.and_then(|v| {
let non_empty = |m: Option<&Value>| {
m.and_then(|m| m.as_str())
.map(String::from)
.filter(|s| !s.is_empty())
};
non_empty(v.get("message")).or_else(|| non_empty(v.get("error")))
})
.unwrap_or(raw);
bail!(
"agent lookup failed ({status}): {}",
sanitize_node_msg(&msg)
);
}
Err(e) => {
let detail: String = e
.chain()
.map(|e| sanitize_node_msg(&e.to_string()))
.collect::<Vec<_>>()
.join(": ");
bail!("agent lookup failed: {detail}");
Err(_) => {
registered = Some(false);
}
}
}
@@ -105,21 +77,21 @@ pub(crate) async fn run_to_writer(args: WhoamiArgs, w: &mut impl std::io::Write)
if let Some(rc) = repo_count {
out["repos"] = json!(rc);
}
writeln!(w, "{}", serde_json::to_string_pretty(&out)?)?;
println!("{}", serde_json::to_string_pretty(&out)?);
} else {
writeln!(w, "DID: {did}")?;
writeln!(w, "Short: {short}")?;
println!("DID: {did}");
println!("Short: {short}");
if let Some(reg) = registered {
writeln!(w, "Registered: {}", if reg { "yes" } else { "no" })?;
println!("Registered: {}", if reg { "yes" } else { "no" });
}
if let Some(ts) = trust_score {
writeln!(w, "Trust: {ts:.2}")?;
println!("Trust: {ts:.2}");
}
if !capabilities.is_empty() {
writeln!(w, "Caps: {}", capabilities.join(", "))?;
println!("Caps: {}", capabilities.join(", "));
}
if let Some(rc) = repo_count {
writeln!(w, "Repos: {rc}")?;
println!("Repos: {rc}");
}
}
Ok(())
@@ -220,236 +192,7 @@ mod tests {
node: Some(server.url()),
json: false,
};
let mut out = Vec::new();
run_to_writer(args, &mut out).await.unwrap();
let out = String::from_utf8(out).unwrap();
assert!(out.contains("Registered: no"), "unexpected output: {out}");
let args = WhoamiArgs {
dir: Some(dir.path().to_path_buf()),
node: Some(server.url()),
json: true,
};
let mut out = Vec::new();
run_to_writer(args, &mut out).await.unwrap();
let out = String::from_utf8(out).unwrap();
assert!(
out.contains("\"registered\": false"),
"unexpected output: {out}"
);
}
#[tokio::test]
async fn test_whoami_with_node_forbidden() {
let dir = TempDir::new().unwrap();
let kp = gitlawb_core::identity::Keypair::generate();
let pem = kp.to_pem().unwrap();
std::fs::write(dir.path().join("identity.pem"), pem.as_bytes()).unwrap();
let did = kp.did().to_string();
let mut server = mockito::Server::new_async().await;
let _agent = server
.mock("GET", format!("/api/v1/agents/{did}").as_str())
.with_status(403)
.with_header("content-type", "application/json")
.with_body(r#"{"message":"forbidden"}"#)
.create_async()
.await;
let args = WhoamiArgs {
dir: Some(dir.path().to_path_buf()),
node: Some(server.url()),
json: false,
};
let err = run(args).await.unwrap_err();
let msg = format!("{err:?}");
assert!(msg.contains("403"), "expected 403 error, got: {msg}");
assert!(
msg.contains("forbidden"),
"expected 'forbidden' in error, got: {msg}"
);
}
#[tokio::test]
async fn test_whoami_with_node_server_error() {
let dir = TempDir::new().unwrap();
let kp = gitlawb_core::identity::Keypair::generate();
let pem = kp.to_pem().unwrap();
std::fs::write(dir.path().join("identity.pem"), pem.as_bytes()).unwrap();
let did = kp.did().to_string();
let mut server = mockito::Server::new_async().await;
let _agent = server
.mock("GET", format!("/api/v1/agents/{did}").as_str())
.with_status(500)
.with_header("content-type", "application/json")
.with_body(r#"{"message":"internal error"}"#)
.create_async()
.await;
let args = WhoamiArgs {
dir: Some(dir.path().to_path_buf()),
node: Some(server.url()),
json: false,
};
let err = run(args).await.unwrap_err();
let msg = format!("{err:?}");
assert!(msg.contains("500"), "expected 500 error, got: {msg}");
assert!(
msg.contains("internal error"),
"expected 'internal error' in error, got: {msg}"
);
}
#[tokio::test]
async fn test_whoami_server_error_unusable_message_uses_error() {
let dir = TempDir::new().unwrap();
let kp = gitlawb_core::identity::Keypair::generate();
let pem = kp.to_pem().unwrap();
std::fs::write(dir.path().join("identity.pem"), pem.as_bytes()).unwrap();
let did = kp.did().to_string();
let mut server = mockito::Server::new_async().await;
let _agent = server
.mock("GET", format!("/api/v1/agents/{did}").as_str())
.with_status(500)
.with_header("content-type", "application/json")
.with_body(r#"{"message":"","error":"boom"}"#)
.create_async()
.await;
let args = WhoamiArgs {
dir: Some(dir.path().to_path_buf()),
node: Some(server.url()),
json: false,
};
let err = run(args).await.unwrap_err();
let msg = format!("{err:?}");
assert!(
msg.contains("boom"),
"expected 'boom' from error field, got: {msg}"
);
}
#[tokio::test]
async fn test_whoami_with_node_transport_error() {
let dir = TempDir::new().unwrap();
let kp = gitlawb_core::identity::Keypair::generate();
let pem = kp.to_pem().unwrap();
std::fs::write(dir.path().join("identity.pem"), pem.as_bytes()).unwrap();
let listener = std::net::TcpListener::bind("127.0.0.1:0").unwrap();
let node = format!("http://{}", listener.local_addr().unwrap());
drop(listener);
let args = WhoamiArgs {
dir: Some(dir.path().to_path_buf()),
node: Some(node),
json: false,
};
let err = run(args).await.unwrap_err();
let msg = format!("{err:?}");
assert!(
msg.contains("agent lookup failed"),
"expected transport error, got: {msg}"
);
}
#[tokio::test]
async fn test_whoami_transport_sanitizes_control_chars_in_node_url() {
let dir = TempDir::new().unwrap();
let kp = gitlawb_core::identity::Keypair::generate();
let pem = kp.to_pem().unwrap();
std::fs::write(dir.path().join("identity.pem"), pem.as_bytes()).unwrap();
let listener = std::net::TcpListener::bind("127.0.0.1:0").unwrap();
let port = listener.local_addr().unwrap().port();
drop(listener);
let node = format!("http://127.0.0.1:{port}/\u{1b}]0;PWNED\u{7} \u{202e}gnitirw");
let args = WhoamiArgs {
dir: Some(dir.path().to_path_buf()),
node: Some(node),
json: false,
};
let err = run(args).await.unwrap_err();
let msg = format!("{err:?}");
assert!(
msg.contains("agent lookup failed"),
"expected transport error, got: {msg}"
);
assert!(!msg.contains('\u{1b}'), "ESC leaked into output: {msg:?}");
assert!(!msg.contains('\u{7}'), "BEL leaked into output: {msg:?}");
assert!(!msg.contains('\u{202e}'), "RLO leaked into output: {msg:?}");
}
#[tokio::test]
async fn test_whoami_server_error_body_display_bounded() {
let dir = TempDir::new().unwrap();
let kp = gitlawb_core::identity::Keypair::generate();
let pem = kp.to_pem().unwrap();
std::fs::write(dir.path().join("identity.pem"), pem.as_bytes()).unwrap();
let did = kp.did().to_string();
let mut server = mockito::Server::new_async().await;
let _agent = server
.mock("GET", format!("/api/v1/agents/{did}").as_str())
.with_status(502)
.with_header("content-type", "application/json")
.with_body("x".repeat(100_000))
.create_async()
.await;
let args = WhoamiArgs {
dir: Some(dir.path().to_path_buf()),
node: Some(server.url()),
json: false,
};
let err = run(args).await.unwrap_err();
let msg = format!("{err:?}");
assert!(
msg.contains("502"),
"expected 502 error with bounded body, got: {msg}"
);
let display = format!("{err}");
assert!(
display.len() < 1000,
"error message too long ({} bytes) — body was not capped",
display.len()
);
}
#[tokio::test]
async fn test_whoami_server_error_sanitizes_controls() {
let dir = TempDir::new().unwrap();
let kp = gitlawb_core::identity::Keypair::generate();
let pem = kp.to_pem().unwrap();
std::fs::write(dir.path().join("identity.pem"), pem.as_bytes()).unwrap();
let did = kp.did().to_string();
let mut server = mockito::Server::new_async().await;
let _agent = server
.mock("GET", format!("/api/v1/agents/{did}").as_str())
.with_status(500)
.with_header("content-type", "application/json")
.with_body("{\"message\":\"\\u001b[31mowned\\u0007\\u202eevil\"}")
.create_async()
.await;
let args = WhoamiArgs {
dir: Some(dir.path().to_path_buf()),
node: Some(server.url()),
json: false,
};
let err = run(args).await.unwrap_err();
let msg = format!("{err:?}");
assert!(
msg.contains("owned"),
"expected sanitized error body, got: {msg}"
);
assert!(!msg.contains('\u{1b}'), "ESC control char leaked: {msg}");
assert!(!msg.contains('\u{07}'), "BEL control char leaked: {msg}");
assert!(!msg.contains('\u{202e}'), "RTL override leaked: {msg}");
run(args).await.unwrap();
}
#[tokio::test]
@@ -487,67 +230,4 @@ mod tests {
};
run(args).await.unwrap();
}
#[tokio::test]
async fn test_whoami_sanitizes_hostile_capabilities() {
let dir = TempDir::new().unwrap();
let kp = gitlawb_core::identity::Keypair::generate();
let pem = kp.to_pem().unwrap();
std::fs::write(dir.path().join("identity.pem"), pem.as_bytes()).unwrap();
let did = kp.did().to_string();
let short = did.split(':').next_back().unwrap().to_string();
let mut server = mockito::Server::new_async().await;
let _agent = server
.mock("GET", format!("/api/v1/agents/{did}").as_str())
.with_status(200)
.with_header("content-type", "application/json")
.with_body(
"{\"trust_score\":0.5,\"capabilities\":[\"\\u001b]0;PWNED\\u0007repo:write\",\"\\u202egnitirw-tfel\"]}",
)
.create_async()
.await;
let _repos = server
.mock(
"GET",
mockito::Matcher::Regex(format!(r"^/api/v1/repos\?owner={short}")),
)
.with_status(200)
.with_header("content-type", "application/json")
.with_body("[]")
.create_async()
.await;
// Human mode: no control bytes or bidi overrides reach the terminal
let mut buf = Vec::new();
let args = WhoamiArgs {
dir: Some(dir.path().to_path_buf()),
node: Some(server.url()),
json: false,
};
run_to_writer(args, &mut buf).await.unwrap();
let out = String::from_utf8(buf).unwrap();
assert!(!out.contains('\u{1b}'), "ESC leaked in human mode: {out:?}");
assert!(!out.contains('\u{07}'), "BEL leaked in human mode: {out:?}");
assert!(
!out.contains('\u{202e}'),
"RLO leaked in human mode: {out:?}"
);
assert!(out.contains("repo:write"), "benign text missing: {out:?}");
assert!(out.contains("tfel"), "reversed text missing: {out:?}");
// JSON mode: serde escapes C0 but passes bidi — ensure no U+202E
let mut buf = Vec::new();
let args = WhoamiArgs {
dir: Some(dir.path().to_path_buf()),
node: Some(server.url()),
json: true,
};
run_to_writer(args, &mut buf).await.unwrap();
let out = String::from_utf8(buf).unwrap();
assert!(
!out.contains('\u{202e}'),
"RLO leaked in JSON mode: {out:?}"
);
}
}
-3
View File
@@ -16,6 +16,3 @@ anyhow = { workspace = true }
thiserror = { workspace = true }
tracing = { workspace = true }
sha2 = { workspace = true }
[dev-dependencies]
mockito = "1"
-257
View File
@@ -64,12 +64,6 @@ fn sanitize_excerpt(s: &str) -> String {
}
continue;
}
if gitlawb_core::sanitize::is_bidi_format(ch) {
// Drop Cf bidi/format controls (INV-6, #183) without setting
// pending_space: they are not whitespace, so stripping leaves no
// phantom gap.
continue;
}
if ch.is_whitespace() {
pending_space = true;
continue;
@@ -427,34 +421,6 @@ mod tests {
assert!(clean.contains("owned") && clean.contains("done"));
}
#[test]
fn sanitize_strips_bidi_format_controls() {
// The Cc-only test above is vacuous for Cf: it passes whether or not bidi
// controls survive, which is why this gap shipped (#183). Here every Cf
// bidi class is stripped — override (RLO), embedding + pop (LRE/PDF),
// isolate (LRI/PDI), marks (LRM/ALM) — asserted per code point, and the
// visible words JOIN with no phantom space (a bidi char is dropped like a
// control byte, but unlike a newline/tab it is not whitespace). The leading
// bidi char (index 0) also exercises the empty-out path.
let out = sanitize_excerpt("\u{202e}err\u{202a}\u{202c}\u{2066}\u{2069}\u{200e}\u{061c}ok");
for c in [
'\u{202e}', '\u{202a}', '\u{202c}', '\u{2066}', '\u{2069}', '\u{200e}', '\u{061c}',
] {
assert!(!out.contains(c), "bidi U+{:04X} leaked: {out:?}", c as u32);
}
assert_eq!(out, "errok");
}
#[test]
fn sanitize_preserves_legitimate_and_rtl_text() {
// Must not over-strip: plain text, a genuine RTL SCRIPT letter (Arabic
// U+0627, category Lo), and ZWJ (U+200D, a legitimate Cf char) survive.
assert_eq!(
sanitize_excerpt("ok \u{0627}\u{200D}b"),
"ok \u{0627}\u{200D}b"
);
}
#[test]
fn sanitize_collapses_whitespace_and_caps_length() {
let collapsed = sanitize_excerpt("a\n\n\t b");
@@ -463,18 +429,6 @@ mod tests {
assert_eq!(sanitize_excerpt(&long).chars().count(), MAX_ERR_CHARS);
}
#[test]
fn sanitize_bidi_chars_do_not_consume_the_length_cap() {
// A dropped bidi char must not count toward MAX_ERR_CHARS: it is dropped
// before `kept` increments, so interleaving bidi controls cannot truncate
// the visible output early. MAX_ERR_CHARS + 100 visible 'x', each trailed by
// a bidi override, must still yield exactly MAX_ERR_CHARS visible 'x' with
// no phantom spaces. (Would regress to fewer 'x' if bidi consumed budget, or
// to spaces if the drop branch wrongly set pending_space.)
let body = "x\u{202e}".repeat(MAX_ERR_CHARS + 100);
assert_eq!(sanitize_excerpt(&body), "x".repeat(MAX_ERR_CHARS));
}
// ── P2: never talk to (or hand a key to) an untrusted node-chosen URL ──
#[test]
@@ -550,215 +504,4 @@ mod tests {
);
assert!(key_trusted, "key stays trusted with operator origin");
}
#[test]
fn new_withholds_key_from_untrusted_origin_but_attaches_for_operator() {
let guard = ICAPTCHA_ENV_LOCK.lock().unwrap();
let prev_key = std::env::var_os("GITLAWB_ICAPTCHA_API_KEY");
let prev_op = std::env::var_os("GITLAWB_ICAPTCHA_URL");
std::env::set_var("GITLAWB_ICAPTCHA_API_KEY", "secret-bearer");
// Arm 1: No operator configured, node advertises an attacker origin.
// The key has no trusted destination; the advert is rejected.
std::env::remove_var("GITLAWB_ICAPTCHA_URL");
let untrusted =
IcaptchaCfg::new("did:key:zTEST", Some("https://evil.example".into()), None);
// Arm 1b: No operator configured, no URL advertised. The fallback goes
// to the default public origin, and with no operator to trust, the key
// must not ride.
let no_operator_no_advert = IcaptchaCfg::new("did:key:zTEST", None, None);
// Arm 2: Operator origin configured and advertised — key attached.
std::env::set_var("GITLAWB_ICAPTCHA_URL", "https://icap.mynode.example");
let operator_match = IcaptchaCfg::new(
"did:key:zTEST",
Some("https://icap.mynode.example/v1".into()),
None,
);
// Arm 3: Operator configured, default host advertised. The default host
// is allowlisted and selected, but its origin differs from the
// operator's, so key must not ride.
let default_advert = IcaptchaCfg::new(
"did:key:zTEST",
Some("https://icaptcha.gitlawb.com/v2".into()),
None,
);
// Arm 4: Operator configured, no URL advertised (the node sent only
// x-icaptcha-level, not x-icaptcha-url). resolve_solver_url falls back
// to the operator origin and marks the key trusted. This is the common
// API-key deployment path — a regression that additionally required
// url.is_some() for key attachment would leave all current tests green
// but make these fallback challenge/answer requests omit Authorization
// and fail against the protected operator service.
let operator_fallback = IcaptchaCfg::new("did:key:zTEST", None, Some(3));
// Arm 5: Operator configured, node advertises an attacker origin.
// The advert is rejected and the solver falls back to the operator's
// own origin, which keeps the key trusted — this is the exfiltration
// scenario the test name describes (withhold key from *untrusted*
// origin, not from the operator's fallback).
let operator_hostile_advert =
IcaptchaCfg::new("did:key:zTEST", Some("https://evil.example".into()), None);
// Arm 6: Operator configured, same host but different port.
// origin_key includes the port, so :8443 must not match the
// operator origin at the default :443 — the advert is rejected
// and the solver falls back to the operator origin.
let operator_diff_port = IcaptchaCfg::new(
"did:key:zTEST",
Some("https://icap.mynode.example:8443/evil".into()),
None,
);
// Restore env and release the lock BEFORE asserting, so a failing
// assertion can't poison the shared lock or cascade into a sibling.
match prev_key {
Some(v) => std::env::set_var("GITLAWB_ICAPTCHA_API_KEY", v),
None => std::env::remove_var("GITLAWB_ICAPTCHA_API_KEY"),
}
match prev_op {
Some(v) => std::env::set_var("GITLAWB_ICAPTCHA_URL", v),
None => std::env::remove_var("GITLAWB_ICAPTCHA_URL"),
}
drop(guard);
let mut failures: Vec<String> = Vec::new();
if untrusted.api_key.is_some() {
failures.push(format!(
"arm 1 (no operator, attacker advert): expected api_key=None, got {:?}",
untrusted.api_key
));
}
if untrusted.url != DEFAULT_URL {
failures.push(format!(
"arm 1 (no operator, attacker advert): expected url={DEFAULT_URL}, got {}",
untrusted.url
));
}
if no_operator_no_advert.api_key.is_some() {
failures.push(format!(
"arm 1b (no operator, no advert): expected api_key=None, got {:?}",
no_operator_no_advert.api_key
));
}
if no_operator_no_advert.url != DEFAULT_URL {
failures.push(format!(
"arm 1b (no operator, no advert): expected url={DEFAULT_URL}, got {}",
no_operator_no_advert.url
));
}
if operator_match.api_key.as_deref() != Some("secret-bearer") {
failures.push(format!(
"arm 2 (operator match): expected api_key=Some(\"secret-bearer\"), got {:?}",
operator_match.api_key
));
}
if operator_match.url != "https://icap.mynode.example/v1" {
failures.push(format!(
"arm 2 (operator match): expected url=https://icap.mynode.example/v1, got {}",
operator_match.url
));
}
if default_advert.api_key.is_some() {
failures.push(format!(
"arm 3 (operator + default advert): expected api_key=None, got {:?}",
default_advert.api_key
));
}
if default_advert.url != "https://icaptcha.gitlawb.com/v2" {
failures.push(format!(
"arm 3 (operator + default advert): expected url=https://icaptcha.gitlawb.com/v2, got {}",
default_advert.url
));
}
if operator_fallback.api_key.as_deref() != Some("secret-bearer") {
failures.push(format!(
"arm 4 (operator fallback, no advert): expected api_key=Some(\"secret-bearer\"), got {:?}",
operator_fallback.api_key
));
}
if operator_fallback.url != "https://icap.mynode.example" {
failures.push(format!(
"arm 4 (operator fallback, no advert): expected url=https://icap.mynode.example, got {}",
operator_fallback.url
));
}
if operator_hostile_advert.api_key.as_deref() != Some("secret-bearer") {
failures.push(format!(
"arm 5 (operator + hostile advert): expected api_key=Some(\"secret-bearer\"), got {:?}",
operator_hostile_advert.api_key
));
}
if operator_hostile_advert.url != "https://icap.mynode.example" {
failures.push(format!(
"arm 5 (operator + hostile advert): expected url=https://icap.mynode.example, got {}",
operator_hostile_advert.url
));
}
if operator_diff_port.api_key.as_deref() != Some("secret-bearer") {
failures.push(format!(
"arm 6 (operator + different port advert): expected api_key=Some(\"secret-bearer\"), got {:?}",
operator_diff_port.api_key
));
}
if operator_diff_port.url != "https://icap.mynode.example" {
failures.push(format!(
"arm 6 (operator + different port advert): expected url=https://icap.mynode.example, got {}",
operator_diff_port.url
));
}
assert!(failures.is_empty(), "\n{}", failures.join("\n"));
}
#[test]
fn new_normalizes_empty_api_key_to_none_on_trusted_origin() {
let guard = ICAPTCHA_ENV_LOCK.lock().unwrap();
let prev_key = std::env::var_os("GITLAWB_ICAPTCHA_API_KEY");
let prev_op = std::env::var_os("GITLAWB_ICAPTCHA_URL");
// Operator origin configured and resolved as trusted (no advert), so
// api_key reaches the env read and the emptiness filter is the only
// thing standing between a blank var and an empty bearer token.
std::env::set_var("GITLAWB_ICAPTCHA_URL", "https://icap.mynode.example");
// "Operator set the var but left it blank" must stay keyless: a
// Some("") would attach `Authorization: Bearer ` (empty token) to the
// challenge and answer requests instead of omitting the header.
std::env::set_var("GITLAWB_ICAPTCHA_API_KEY", "");
let empty = IcaptchaCfg::new("did:key:zTEST", None, None);
// Contrast arm: a real key on the same trusted path must survive the
// filter, so the filter can't regress to dropping every key.
std::env::set_var("GITLAWB_ICAPTCHA_API_KEY", "secret-bearer");
let non_empty = IcaptchaCfg::new("did:key:zTEST", None, None);
// Restore env and release the lock BEFORE asserting, so a failing
// assertion can't poison the shared lock or cascade into a sibling.
match prev_key {
Some(v) => std::env::set_var("GITLAWB_ICAPTCHA_API_KEY", v),
None => std::env::remove_var("GITLAWB_ICAPTCHA_API_KEY"),
}
match prev_op {
Some(v) => std::env::set_var("GITLAWB_ICAPTCHA_URL", v),
None => std::env::remove_var("GITLAWB_ICAPTCHA_URL"),
}
drop(guard);
assert_eq!(
empty.api_key, None,
"empty GITLAWB_ICAPTCHA_API_KEY must normalize to None"
);
assert_eq!(non_empty.api_key.as_deref(), Some("secret-bearer"));
}
}
@@ -1,146 +0,0 @@
//! Tripwire: a committed negative case that locks the blackhole mechanism.
//!
//! The guard against non-loopback egress works because `obtain_proof` builds a
//! stock `reqwest::blocking::Client` that inherits the process's proxy env vars.
//! If the builder later gains `.no_proxy()` or a custom connector, the guard
//! disarms silently and every existing test stays green (no live call is made
//! during the normal mock-consumed run because the mock is on loopback and
//! `NO_PROXY` covers it — the blackhole isn't even exercised).
//!
//! This test asserts that a destination the blackhole's `NO_PROXY` does NOT
//! cover fails *while the blackhole is armed*. The destination is a local HTTP
//! server on `[::1]` (IPv6 loopback), which `NO_PROXY` (`127.0.0.1, localhost`)
//! does not list. With the blackhole active the proxy intercepts the
//! connection and blocks it; without the blackhole the request would reach the
//! server directly and succeed, turning this test RED.
//!
//! A positive control (disarmed blackhole) runs first, proving the fixture is
//! valid and `[::1]` is reachable.
//!
//! Design constraints (see issue #211):
//! - An unresolvable host would keep the request failing even when the guard
//! is disarmed (DNS failure masquerading as the blackhole), so we must use
//! a reachable address.
//! - A real external host would make a live network call the moment the guard
//! disarms, and on any runner that cannot reach that host the connect error
//! again masquerades as the blackhole. A local loopback address avoids both
//! problems — it is always reachable when unblocked and never reaches the
//! real network.
//! - The address must NOT be covered by `NO_PROXY`, yet must exist on every
//! platform. An arbitrary `127/8` alias is not portable: macOS only binds
//! `127.0.0.1` by default. `[::1]` (IPv6 loopback) is present by default
//! on Linux, macOS, and Windows, and `NO_PROXY` only lists `127.0.0.1` and
//! `localhost`, so the blackhole still intercepts it.
//! - This depends on hyper-util's proxy matcher having no implicit loopback
//! bypass — a host is only proxied-around when literally listed in
//! `NO_PROXY` (verified in the pinned hyper-util; recheck on upgrade).
use std::io::{Read, Write};
use std::net::TcpListener;
use std::thread;
use icaptcha_client::{obtain_proof, Challenge, IcaptchaCfg};
mod support;
/// A minimal HTTP server that responds to iCaptcha challenge and answer
/// requests on the IPv6 loopback address NOT covered by NO_PROXY.
///
/// When the blackhole is working, the proxy should intercept these requests
/// and `obtain_proof` should fail. When the blackhole is disarmed, the
/// requests reach this server and the flow succeeds.
fn serve_icaptcha(listener: TcpListener) {
thread::spawn(move || {
for stream in listener.incoming() {
let Ok(mut stream) = stream else { continue };
// Read until the first \n (end of request line) so TCP
// segmentation cannot split the path out of a single read.
let mut buf = [0u8; 4096];
let mut n = 0usize;
while n < buf.len() {
match stream.read(&mut buf[n..]) {
Ok(0) | Err(_) => break,
Ok(read) => {
n += read;
if buf[..n].contains(&b'\n') {
break;
}
}
}
}
if n == 0 {
continue;
}
let request = String::from_utf8_lossy(&buf[..n]);
let body = if request.contains("/v1/answer") {
r#"{"status":"passed","proof":"PROOF-TRIP"}"#
} else {
r#"{"challengeId":"c1","type":"anagram","difficulty":1,"prompt":"listen","token":"tok-1"}"#
};
let response = format!(
"HTTP/1.1 200 OK\r\nContent-Type: application/json\r\nContent-Length: {}\r\nConnection: close\r\n\r\n{body}",
body.len(),
);
let _ = stream.write_all(response.as_bytes());
}
});
}
fn make_cfg(url: &str) -> IcaptchaCfg {
IcaptchaCfg {
url: url.to_string(),
did: "did:key:zTEST".to_string(),
level: 1,
api_key: None,
}
}
#[test]
fn obtain_proof_blackhole_tripwire() {
// Bind to [::1], IPv6 loopback, which NO_PROXY (127.0.0.1, localhost)
// does not cover. Unlike an arbitrary 127/8 alias, IPv6 loopback is
// present by default on Linux, macOS, and Windows, so the fixture is
// portable across platforms. Fail loudly if it cannot be bound: a
// silent skip would report this tripwire as green while exercising
// none of the negative control, recreating the very gap it exists to
// close.
let listener = TcpListener::bind(("::1", 0)).expect(
"cannot bind [::1] for the blackhole tripwire. IPv6 loopback (::1) \
is present by default on Linux, macOS, and Windows. This test fails \
rather than silently skipping so a platform without it cannot count \
the negative control as passed.",
);
let port = listener.local_addr().unwrap().port();
let url = format!("http://[::1]:{port}");
serve_icaptcha(listener);
// ── Positive control: disarmed blackhole → obtain_proof succeeds ──────
let _disarmed_guard = support::disarm_proxy_env();
let solve: &dyn Fn(&Challenge) -> Option<String> = &|_c| Some("silent".to_string());
let cfg = make_cfg(&url);
let proof = obtain_proof(&cfg, Some(solve))
.expect("positive control: obtain_proof should succeed when the proxy is disarmed");
assert_eq!(
proof, "PROOF-TRIP",
"positive control: unexpected proof value"
);
drop(_disarmed_guard);
// ── Negative control: armed blackhole → obtain_proof fails with connect error ──
let _armed_guard = support::arm_blackhole("http://127.0.0.1:1");
let result = obtain_proof(&cfg, Some(solve));
let err = result.expect_err(
"blackhole tripwire: obtain_proof succeeded against [::1], \
meaning the proxy blackhole did not intercept the request; \
the no-live-call guard is disarmed",
);
assert!(
err.chain().any(|c| c
.downcast_ref::<reqwest::Error>()
.is_some_and(|e| e.is_connect())),
"expected a connect/proxy error, got: {err:#}",
);
}
@@ -1,199 +0,0 @@
//! INV-19 guard: the icaptcha flow must actually call the service (consuming the
//! mock), not short-circuit or make a live call. If a future change let
//! `obtain_proof` skip the network (e.g. a stray `cfg(test)` relaxation, which
//! INV-19/INV-20 warn is inert across crates), the `.assert()` calls below go
//! RED because the mocked endpoints were never hit.
//!
//! The mock-consumed assertions alone only prove the mock was hit; they cannot
//! see a request that ALSO leaks the DID / answer / API key to `DEFAULT_URL` or
//! any other origin (a different host the mock never observes). To make the
//! "no live call" half of the guard load-bearing, the test blackholes every
//! non-loopback destination via a proxy observer: `NO_PROXY` lets the loopback
//! mock through while every proxy variable routes any other host through a TCP
//! listener that counts connections. Zero observed connections proves no leak
//! reached the network on the exercised path.
//!
//! The observer design counts every accepted connection regardless of whether
//! the caller propagates the transport error, so it catches leaks that a
//! closed-port blackhole (which only surfaces propagating errors) would miss.
//! To close the timing race inherent in a detached accept loop, the count is
//! read only after a bounded drain: the test opens a flush connection and
//! blocks until the accept thread has accepted *that specific connection*
//! (tagged by its source port), not merely until any next counter tick.
//! Because `accept()` hands out connections FIFO, every leak connection that
//! completed its TCP handshake before the flush is counted before the flush —
//! the assertion therefore covers all egress in flight when `obtain_proof`
//! returned. A leak that starts its handshake after the drain is outside the
//! window; that is the irreducible limit of observing a detached accept loop,
//! and it is the remaining, well-defined gap this guard accepts.
//!
//! SCOPE: The observer only witnesses leaks issued on the path this test drives
//! (a single passing round, 200 on both endpoints, no PoW, no Continue/Failed,
//! no non-2xx response). A leak living on an error or retry branch makes no
//! connection during the run and the zero-count assertion stays green. The
//! guarantee covers only the exercised happy path and proxy-honoring transports;
//! a future `.no_proxy()` or raw `TcpStream` bypasses the observer entirely.
//!
//! `ALL_PROXY` alone is NOT enough: reqwest honors the scheme-specific
//! `HTTPS_PROXY` / `https_proxy` (and the http equivalents) OVER `ALL_PROXY`, so
//! a runner with a working `HTTPS_PROXY` in its environment would route an https
//! leak to `DEFAULT_URL` (which is https) through that proxy while the loopback
//! mock is still consumed and this test stayed green. So we blackhole the
//! scheme-specific variables too.
use std::net::{TcpListener, TcpStream};
use std::sync::{Arc, Condvar, Mutex};
use icaptcha_client::{obtain_proof, Challenge, IcaptchaCfg};
mod support;
/// A TCP listener that counts every accepted connection.
///
/// Acts as the proxy target: any leaked connection to a non-loopback
/// destination will be routed through this listener and counted. Every
/// accepted connection is recorded by its source port so the test's own
/// flush connection ([`ProxyObserver::drain`]) can be recognised and
/// excluded from the leak count.
struct ProxyObserver {
_listener: Arc<TcpListener>,
port: u16,
/// Source ports of every accepted connection, in accept order.
accepted: Arc<Mutex<Vec<u16>>>,
/// Source ports of the test's own flush connections (never counted).
flush_ports: Arc<Mutex<Vec<u16>>>,
/// Signalled after each accept so `drain` can wait on the flush.
ack: Arc<Condvar>,
}
impl ProxyObserver {
fn bind() -> Self {
let listener = Arc::new(TcpListener::bind("127.0.0.1:0").expect("bind proxy observer"));
let port = listener.local_addr().unwrap().port();
let accepted = Arc::new(Mutex::new(Vec::new()));
let flush_ports = Arc::new(Mutex::new(Vec::new()));
let ack = Arc::new(Condvar::new());
let a = Arc::clone(&accepted);
let c = Arc::clone(&ack);
let l = Arc::clone(&listener);
std::thread::spawn(move || {
for stream in l.incoming() {
let Ok(stream) = stream else { continue };
a.lock()
.unwrap()
.push(stream.peer_addr().map(|a| a.port()).unwrap_or(0));
c.notify_all();
}
});
ProxyObserver {
_listener: listener,
port,
accepted,
flush_ports,
ack,
}
}
fn port(&self) -> u16 {
self.port
}
/// Number of accepted connections that are not the test's own flush
/// connections — i.e. the leak count.
fn count(&self) -> usize {
let flush = self.flush_ports.lock().unwrap();
let accepted = self.accepted.lock().unwrap();
accepted.iter().filter(|p| !flush.contains(p)).count()
}
/// Bounded acknowledgement barrier for the accept loop.
///
/// Opens a flush connection and blocks until the accept thread has
/// accepted *that connection* (identified by its source port), bounded by
/// a timeout. Because `accept()` hands out connections FIFO, once the
/// flush is accepted every connection that completed its handshake before
/// it has been accepted and recorded too — so counting on afterwards
/// cannot miss a leak that was already in flight. The flush connection
/// itself is never counted.
///
/// A counter-tick barrier would be unsound here: sampling "current + 1"
/// and waiting for any increment lets a leak accepted after the sample
/// satisfy the wait, and the flush is then miscounted as that leak.
fn drain(&self) {
let flush =
TcpStream::connect(("127.0.0.1", self.port)).expect("observer flush connection");
let flush_port = flush.local_addr().unwrap().port();
self.flush_ports.lock().unwrap().push(flush_port);
let mut accepted = self.accepted.lock().unwrap();
while !accepted.contains(&flush_port) {
let (guard, timed_out) = self
.ack
.wait_timeout(accepted, std::time::Duration::from_secs(5))
.unwrap();
accepted = guard;
assert!(
!timed_out.timed_out(),
"proxy observer drain timed out; accept loop stalled"
);
}
}
}
#[test]
fn obtain_proof_consumes_the_mocked_service_and_makes_no_live_call() {
let observer = ProxyObserver::bind();
let _guard = support::arm_blackhole(&format!("http://127.0.0.1:{}", observer.port()));
let mut server = mockito::Server::new();
let challenge = server
.mock("POST", "/v1/challenge")
.with_status(200)
.with_header("content-type", "application/json")
.with_body(
r#"{"challengeId":"c1","type":"anagram","difficulty":1,"prompt":"listen","token":"tok-1"}"#,
)
.expect(1)
.create();
let answer = server
.mock("POST", "/v1/answer")
.with_status(200)
.with_header("content-type", "application/json")
.with_body(r#"{"status":"passed","proof":"PROOF-XYZ"}"#)
.expect(1)
.create();
let cfg = IcaptchaCfg {
url: server.url(),
did: "did:key:zTEST".to_string(),
level: 1,
api_key: None,
};
let solve = |_c: &Challenge| Some("silent".to_string());
let solver: &dyn Fn(&Challenge) -> Option<String> = &solve;
let count_before = observer.count();
let proof = obtain_proof(&cfg, Some(solver))
.expect("obtain_proof should complete against the mocked service");
assert_eq!(proof, "PROOF-XYZ");
challenge.assert();
answer.assert();
// Drain the accept loop before asserting: the flush connection is not
// counted, and because accept() is FIFO every leak connection that
// completed its handshake before the flush is recorded before the flush
// is — so the count after drain reflects all egress in flight during the
// call.
observer.drain();
let leak_count = observer.count() - count_before;
assert_eq!(
leak_count, 0,
"proxy observer caught leaked connections; \
the blackhole failed to contain egress"
);
}
@@ -1,93 +0,0 @@
//! Shared helpers for the icaptcha-client integration tests.
//!
//! CONCURRENCY CONSTRAINT: [`arm_blackhole`] and [`disarm_proxy_env`] mutate
//! process-wide proxy env vars and restore them on drop. Each test binary
//! that consumes this module currently has exactly one `#[test]`, so there is
//! no concurrent use. `cargo test` runs tests within a binary in parallel by
//! default, so the NEXT test added to either file MUST be serialized against
//! these helpers (e.g. a shared `Mutex` in this module, or the `serial_test`
//! crate) or it will race on the proxy vars.
use std::ffi::OsString;
const PROXY_VARS: &[&str] = &[
"NO_PROXY",
"no_proxy",
"ALL_PROXY",
"all_proxy",
"HTTPS_PROXY",
"https_proxy",
"HTTP_PROXY",
"http_proxy",
"REQUEST_METHOD",
];
/// Captures the current values of a set of env vars and restores them on drop.
pub struct EnvSnapshot(Vec<(String, Option<OsString>)>);
impl EnvSnapshot {
pub fn capture(vars: &[&str]) -> Self {
EnvSnapshot(
vars.iter()
.map(|v| (v.to_string(), std::env::var_os(v)))
.collect(),
)
}
}
impl Drop for EnvSnapshot {
fn drop(&mut self) {
for (var, val) in &self.0 {
match val {
Some(v) => std::env::set_var(var, v),
None => std::env::remove_var(var),
}
}
}
}
/// Guard that restores env vars captured at construction time.
pub struct EnvGuard {
_prev: EnvSnapshot,
}
/// Override every proxy-related env var so that non-loopback destinations are
/// forced through `proxy_url`, with NO_PROXY covering loopback. Also clears
/// REQUEST_METHOD, which causes hyper-util to ignore all proxy variables when
/// set. Restores prior values on drop.
///
/// Not safe to call from concurrent tests (see module docs).
pub fn arm_blackhole(proxy_url: &str) -> EnvGuard {
let prev = EnvSnapshot::capture(PROXY_VARS);
std::env::set_var("NO_PROXY", "127.0.0.1,localhost");
std::env::set_var("no_proxy", "127.0.0.1,localhost");
for var in [
"ALL_PROXY",
"all_proxy",
"HTTPS_PROXY",
"https_proxy",
"HTTP_PROXY",
"http_proxy",
] {
std::env::set_var(var, proxy_url);
}
std::env::remove_var("REQUEST_METHOD");
EnvGuard { _prev: prev }
}
/// Clear every proxy-related env var so connections go direct. Restores
/// prior values on drop.
///
/// Not safe to call from concurrent tests (see module docs).
#[allow(dead_code)]
pub fn disarm_proxy_env() -> EnvGuard {
let prev = EnvSnapshot::capture(PROXY_VARS);
for var in PROXY_VARS {
std::env::remove_var(var);
}
EnvGuard { _prev: prev }
}
-4
View File
@@ -44,10 +44,6 @@ services:
GITLAWB_BOOTSTRAP_PEERS: ${GITLAWB_BOOTSTRAP_PEERS:-}
GITLAWB_BOOTSTRAP_DISABLE_SEEDS: ${GITLAWB_BOOTSTRAP_DISABLE_SEEDS:-false}
GITLAWB_REQUIRE_SIGNED_PEER_WRITES: ${GITLAWB_REQUIRE_SIGNED_PEER_WRITES:-false}
# Owner-only push. The default matches the node's own, so the container path
# behaves like a source build; forwarding it is what makes the documented
# rolling-upgrade opt-out reachable for an operator running compose.
GITLAWB_ENFORCE_OWNER_PUSH: ${GITLAWB_ENFORCE_OWNER_PUSH:-true}
GITLAWB_P2P_BOOTSTRAP: ${GITLAWB_P2P_BOOTSTRAP:-}
GITLAWB_PUBLIC_READ: ${GITLAWB_PUBLIC_READ:-true}
GITLAWB_MAX_PACK_BYTES: ${GITLAWB_MAX_PACK_BYTES:-2147483648}
+2 -2
View File
@@ -38,8 +38,8 @@ Owner focus: node reliability.
Owner focus: protocol/auth.
- Close default-open repo write authorization and wire trusted UCAN delegation into repository permissions.
- Close the remaining visibility gaps: restrict or document task, pin, and anchor metadata routes, and publish an irreversible-publication policy.
- Implement repo write authorization: repo owner checks, protected branches, UCAN capability checks, and clear delegation semantics.
- Implement private-read enforcement or remove private-repo affordances until it exists.
- Add UCAN revocation or blocklisting, with an emergency compromised-key runbook.
- Add mutation-aware GraphQL auth before GraphQL becomes a public write API surface.
- Harden peer registration and outbound peer calls against SSRF and peer-list poisoning.
+1 -6
View File
@@ -3,11 +3,6 @@
Date: 2026-05-28
Repo state inspected: `main` tracking `origin/main`, starting at `b12c6bc feat: per-DID rate limiting on creation endpoints (10/hour) (#13)`.
> **Historical snapshot.** This audit describes the repository at the commit above. Its
> statements about incomplete UCAN chain validation and absent per-repository read
> enforcement have been superseded; see [`SECURITY.md`](../SECURITY.md) for the current
> security posture.
## Commands run
```sh
@@ -93,7 +88,7 @@ Fixed or staged in this pass:
Live-network blockers to prioritize:
- GraphQL POST is still open for compatibility; GraphQL mutations should get mutation-aware auth before it becomes a public write API surface.
- Push authorization is still not capability-complete. A valid DID signature is authentication, not authorization. Owner checks are now enforced on every branch, protected or not (`GITLAWB_ENFORCE_OWNER_PUSH`, on by default); what remains is that a UCAN `git/push` capability is not yet honored, so a delegated or CI key cannot push.
- Push authorization is still not capability-complete. A valid DID signature is authentication, not authorization; unprotected repo branches do not yet enforce owner/UCAN capability checks.
- UCAN chain validation is incomplete and UCAN revocation/blocklisting is not implemented as an operator feature.
- Private repository reads are not enforced. `is_public` and `GITLAWB_PUBLIC_READ` exist, but per-repository private-read behavior is not wired.
- Peer URLs are self-asserted by DIDs. Signatures prove control of the DID key when present, not ownership/safety of the announced URL.
+20 -35
View File
@@ -141,43 +141,31 @@ During the cooldown your node still earns rewards if it keeps heartbeating.
---
## Owner-only push
## Hardening: owner-only push
The node requires the authenticated pusher to be the repo owner on **every**
branch. A push whose authenticated DID is not the repo owner is rejected with
HTTP 403 before any ref update is applied. The owner is matched in both the full
`did:key:z6Mk…` form and its bare `z6Mk…` suffix.
By default the node authenticates every `git-receive-pack` push (a valid RFC 9421
`did:key` signature) but does **not** check that the pusher owns the repo, except
on branches that are explicitly protected. Because `did:key` is self-certifying,
any party can generate a key, derive its DID, sign, and push to an unprotected
branch — authentication is not authorization.
This is on by default, and the default is the point. The node authenticates every
`git-receive-pack` push with a valid RFC 9421 `did:key` signature, but `did:key`
is self-certifying: any party can generate a key, derive its DID and sign.
Authentication is not authorization, so without this gate every signed caller can
push to every repository — private ones included — on any branch that is not
explicitly protected.
### Turning it off
To require the authenticated pusher to be the repo owner on **every** branch, set:
```bash
GITLAWB_ENFORCE_OWNER_PUSH=false
GITLAWB_ENFORCE_OWNER_PUSH=true
```
Both the environment variable and `--enforce-owner-push false` disable it; the
bare `--enforce-owner-push` flag still means `true`.
Do this only for a rolling upgrade whose pushers are not yet the repo owner, and
treat it as temporary: while it is off, your node accepts a push to any repository
from anyone who can generate a keypair.
### Caution: delegated and CI keys are non-owners
Push authorization is owner-only today. A UCAN `git/push` capability is verified
but **not yet honored for authorization**, so a delegated key cannot push while
this gate is on, even when it holds a valid capability for the repo.
If your automation pushes under its own DID rather than the owner's, it will start
getting 403s. Either have it push as the owner, or set
`GITLAWB_ENFORCE_OWNER_PUSH=false` until scoped collaborator / UCAN-delegated push
rights land — that work is what removes this trade-off.
- **Default `false`** — preserves current behavior so live nodes are unaffected by
an upgrade. Turn it on once you're ready for owner-only writes.
- **When `true`** — a push whose authenticated DID is not the repo owner is
rejected (HTTP 403) before any ref update is applied. The owner is matched in
both the full `did:key:z6Mk…` form and its bare `z6Mk…` suffix.
- **Caution: this blocks every non-owner pusher, including your own delegated and
CI agents.** Push authorization is owner-only today — a UCAN `git/push`
capability is verified but not yet honored for authorization, so delegated keys
cannot push while this is on. Don't enable it until every identity that pushes
to your repos is the owner, or you'll lock out your own automation. Scoped
collaborator / UCAN-delegated push rights are a planned follow-up.
---
@@ -189,8 +177,7 @@ rights land — that work is what removes this trade-off.
| Missed heartbeats | After 3 days without one, you're excluded from rewards until you beat again |
| Operator key | Dedicated wallet, small ETH balance, not your main treasury |
| Monitoring | Watch `lastHeartbeat` on-chain; alert if > 22h since last beat |
| Public URL | Must resolve and serve `/health` for liveness and DB-aware `/ready` for peer readiness |
| Rolling upgrade (advisory-lock algorithm) | The SHA-256 advisory-lock key swap (replacing `DefaultHasher`) re-keys every repo in the first release that carries it. During a shared-Postgres rolling upgrade, old nodes compute a different `i64` key than new nodes for the same repo, so PostgreSQL treats them as independent locks and cross-machine write-exclusion is lost. **Drain in-flight writes** or cut over through a **single node** before bringing new-binary nodes online. |
| Public URL | Must resolve and serve `/health` — peers will ping it |
---
@@ -200,8 +187,6 @@ rights land — that work is what removes this trade-off.
**Node refuses to start with "strict-mode operator check failed"** — either `gl node register` first, or unset `GITLAWB_OPERATOR_STRICT_MODE`.
**Node refuses to start with "GITLAWB_DB_MAX_CONNECTIONS (20) must be at least max_concurrent_git_pushes (32) + 8 headroom"**: raise `GITLAWB_DB_MAX_CONNECTIONS` to at least the push cap plus 8 (40 with the default cap of 32; 48 is the shipped default and the recommended value), or lower `GITLAWB_MAX_CONCURRENT_GIT_PUSHES`. This bites a node upgraded in place that still sets the old pool size of 20. The check is deliberate: each concurrent push pins one pooled connection for its whole receive-pack, so a pool that does not clear the push cap lets a burst of slow pushes starve every other database path.
**Rewards are 0 after a week** — run `gl node onchain-status`. If `currentlyActive: false`, check your heartbeat loop (node logs for `operator heartbeat sent`).
**Want to rotate operator wallet** — requires unstake → re-register with new wallet. No in-place rotation in v1.
+11 -60
View File
@@ -91,39 +91,13 @@ User-data only runs at first boot, so upgrades go through SSM:
$(terraform output -raw upgrade_command)
```
This reinstalls the rendered `/opt/gitlawb/compose.yaml`, then runs
`docker compose pull && docker compose up -d` on the instance.
**Run `terraform apply` before the upgrade command.** The command is an SSM
document managed by Terraform, and it embeds the compose file rendered from your
current variables. An instance that has not had a fresh `apply` still holds the
previous document and will reinstall the previous compose file.
This runs `docker compose pull && docker compose up -d` on the instance.
- With `image_tag = "latest"` (default) that picks up the newest release.
- With a **pinned tag**, set `image_tag` in `terraform.tfvars` and `apply` — the
rendered compose carries it, so the upgrade installs the right version.
- **`compose.yaml` is Terraform-owned and is overwritten on every upgrade.**
Per-instance settings belong in `/opt/gitlawb/.env`, which is never touched.
- **The upgrade reconciles the service set, not just the image.** The rendering is
conditional — `postgres` exists only when `db_host` is empty, `caddy` only when
`domain_name` is set — and the command runs `--remove-orphans`. So setting
`use_rds`, or clearing `domain_name`, and then upgrading will **stop and remove**
the local postgres or the TLS terminator. That is the intended outcome of those
variables, but it happens on the upgrade rather than on `apply`. Data survives
either way: both bind-mount under `/mnt/data`.
### Why the upgrade rewrites the compose file
`aws_instance` deliberately ignores `user_data` drift, so an instance keeps the
compose file it was created with. Compose passes only the variables named in a
service's `environment:` block into the container, so **a node setting added to
the template never reaches an existing instance** — writing it to
`/opt/gitlawb/.env` and restarting leaves the node on its built-in default, with
no error to indicate it. `pull && up -d` alone cannot fix that, because the file
on disk is the authoritative one.
Reinstalling the rendering first makes the upgrade a real migration. It is
idempotent: on an already-current instance it writes identical bytes.
- With a **pinned tag**, first edit the tag in `/opt/gitlawb/compose.yaml` on
the instance (via SSM session), then run the upgrade command — and keep
`image_tag` in terraform.tfvars in sync so a future instance replacement
boots the same version.
Replace the instance itself (OS/AMI/instance-type changes) with
`terraform apply -replace=aws_instance.node` — the data volume reattaches and
@@ -131,24 +105,12 @@ Replace the instance itself (OS/AMI/instance-type changes) with
## Changing configuration
`terraform apply` alone never changes a running instance: user-data runs once and
the instance ignores `user_data` drift. How a change rolls out depends on which
of the two files carries it.
**Rendered into `compose.yaml`** — `image_tag`, `gitlawb_port`, `metrics_port`,
`domain_name`, `db_host`, the `icaptcha_*` values. `apply`, then run
`upgrade_command`: it installs the new rendering and restarts.
```sh
terraform apply
$(terraform output -raw upgrade_command)
```
**Written into `/opt/gitlawb/.env` at first boot** — `public_url`,
`bootstrap_peers`, `auto_sync`, `max_pack_bytes`, and the integration secrets.
Editing these in `terraform.tfvars` does **not** reach a running instance, because
nothing rewrites `.env` after the first boot. Either edit `/opt/gitlawb/.env` on
the instance and `docker compose up -d`, or replace it:
User-data only runs at first boot, and the instance ignores `user_data` drift
(`ignore_changes`), so editing terraform.tfvars values that feed the bootstrap
(`bootstrap_peers`, `public_url`, integrations, `image_tag`) does **not**
affect a running instance on `terraform apply`. To roll out such changes,
either edit `/opt/gitlawb/.env` on the instance (SSM session, then
`docker compose up -d`), or replace the instance:
```sh
terraform apply -replace=aws_instance.node
@@ -156,17 +118,6 @@ terraform apply -replace=aws_instance.node
The data volume reattaches; repos, postgres data, and the identity key survive.
**A node setting the installed `compose.yaml` does not already name** —
`GITLAWB_ENFORCE_OWNER_PUSH` on an instance created before it was added, for
example. Editing `.env` is **not enough**: compose only passes through the
variables its `environment:` block lists, so the value is read from `.env` and
then dropped. Run `apply` plus `upgrade_command` first, which installs a compose
file that names the key; after that, `.env` governs it like any other.
That last case is the one worth remembering: `.env` reaches the container only for
keys the *installed* compose interpolates, and the upgrade is what makes a newly
added key one of them.
## Remote state (optional)
Local state is the default. To move state to S3: create a versioned bucket,
-3
View File
@@ -72,9 +72,6 @@ services:
GITLAWB_BOOTSTRAP_PEERS: $${GITLAWB_BOOTSTRAP_PEERS}
GITLAWB_AUTO_SYNC: $${GITLAWB_AUTO_SYNC}
GITLAWB_MAX_PACK_BYTES: $${GITLAWB_MAX_PACK_BYTES}
# Owner-only push. Matches the node's own default; forwarding it is what makes
# the documented rolling-upgrade opt-out reachable on the container path.
GITLAWB_ENFORCE_OWNER_PUSH: $${GITLAWB_ENFORCE_OWNER_PUSH:-true}
# On-chain PoS (optional — empty unless set in terraform.tfvars)
GITLAWB_CHAIN_RPC_URL: $${GITLAWB_CHAIN_RPC_URL}
GITLAWB_CONTRACT_NODE_STAKING: $${GITLAWB_CONTRACT_NODE_STAKING}
+7 -62
View File
@@ -350,16 +350,10 @@ resource "aws_instance" "node" {
# ami: new AL2023 releases shouldn't churn the instance; replace
# deliberately for OS upgrades.
# user_data: only runs at first boot, so re-rendering it on a live
# instance is a pointless stop/start. How a config change reaches a running
# instance depends on which file carries it:
# - rendered into compose.yaml (image_tag, ports, domain_name, db_host,
# icaptcha_*): `terraform apply` then `upgrade_command`, which
# reinstalls the rendering and restarts.
# - written into /opt/gitlawb/.env at first boot (public_url,
# bootstrap_peers, integrations): nothing rewrites .env afterwards, so
# these need an edit on the instance or
# `terraform apply -replace=aws_instance.node`.
# See README "Changing configuration".
# instance is a pointless stop/start. Config changes that feed user-data
# (bootstrap peers, integrations, image tag) require a deliberate
# `terraform apply -replace=aws_instance.node` — see README "Changing
# configuration".
ignore_changes = [ami, user_data]
}
}
@@ -443,62 +437,13 @@ resource "aws_ssm_document" "upgrade" {
content = jsonencode({
schemaVersion = "2.2"
description = "Reinstall the rendered compose file, pull the latest image, and restart"
description = "Pull the latest gitlawb node image and restart the compose stack"
mainSteps = [{
action = "aws:runShellScript"
name = "upgrade"
inputs = {
# Reinstall the CURRENT rendering before restarting.
#
# `aws_instance` ignores user_data drift, so an instance created before a
# template change keeps the compose file it was born with. Compose passes
# only the variables named in its `environment:` block into the container,
# so a node setting added to the template never reaches an existing
# instance — the operator writes it to /opt/gitlawb/.env, restarts, and the
# node silently keeps the built-in default. `docker compose pull && up -d`
# cannot fix that: the authoritative file is the one on disk.
#
# Idempotent: writing the same bytes and restarting is a no-op. This does
# overwrite hand-edits to compose.yaml, which is intended — the file is
# Terraform-owned; per-instance settings belong in /opt/gitlawb/.env.
#
# One shell step, under flock, writing through mktemp. Two overlapping
# executions — a double run, a retry while the first is still pulling, two
# operators — otherwise share one fixed temp path: the second truncates the
# file the first is about to install, and once the first renames, the
# second's open descriptor keeps writing into the live compose.yaml inode.
# Both outcomes stay valid YAML, so the `up -d --remove-orphans` on the next
# line accepts a blended or truncated service set and deletes whatever fell
# off — on a live node, the local postgres or the TLS terminator. The lock
# also covers pull and up, so a second run cannot restart against a file the
# first is mid-install.
runCommand = [<<-UPGRADE
set -euo pipefail
install -d -m 0755 /opt/gitlawb
exec 9>/opt/gitlawb/.upgrade.lock
if ! flock -n 9; then
echo "another upgrade is already running on this instance" >&2
exit 1
fi
tmp="$(mktemp /opt/gitlawb/compose.yaml.XXXXXX)"
trap 'rm -f "$tmp"' EXIT
# base64 rather than a nested heredoc: the rendered compose contains
# `$${VAR}` passthroughs that must reach the file unexpanded, and a
# heredoc inside an indented Terraform heredoc depends on the dedent
# landing the delimiter at column 0. Decoding a single line has neither
# failure mode, and no delimiter can collide with the content.
echo '${base64encode(local.compose_yaml)}' | base64 -d > "$tmp"
chmod 0644 "$tmp"
mv "$tmp" /opt/gitlawb/compose.yaml
trap - EXIT
cd /opt/gitlawb
docker compose pull
docker compose up -d --remove-orphans
docker image prune -f
UPGRADE
runCommand = [
"cd /opt/gitlawb && docker compose pull && docker compose up -d --remove-orphans && docker image prune -f"
]
}
}]
-65
View File
@@ -1,65 +0,0 @@
# fly.toml app configuration file generated for gitlawb-node-2 on 2026-07-22T09:47:18+08:00
#
# See https://fly.io/docs/reference/configuration/ for information about how to use this file.
#
app = 'gitlawb-node-2'
primary_region = 'sjc'
[build]
dockerfile = '../../Dockerfile'
[env]
GITLAWB_AUTO_SYNC = 'true'
GITLAWB_BOOTSTRAP_PEERS = 'https://node.gitlawb.com,https://node3.gitlawb.com'
GITLAWB_HOST = '0.0.0.0'
GITLAWB_KEY = '/data/keys/identity.pem'
GITLAWB_MAX_PACK_BYTES = '524288000'
GITLAWB_P2P_PORT = '7546'
GITLAWB_PORT = '7545'
GITLAWB_PUBLIC_URL = 'https://node2.gitlawb.com'
GITLAWB_REPOS_DIR = '/data/repos'
ICAPTCHA_MODE = 'enforce'
ICAPTCHA_PUBKEY = 'xjyPNqIbvc9U-kwXW6u9mDqRJ7E2UUMOaJdUWhpEXq8'
ICAPTCHA_REQUIRED_LEVEL = '3'
[[mounts]]
source = 'gitlawb_data'
destination = '/data'
[http_service]
internal_port = 7545
force_https = true
auto_stop_machines = 'off'
auto_start_machines = true
min_machines_running = 1
[http_service.concurrency]
type = 'connections'
hard_limit = 500
soft_limit = 400
[http_service.http_options]
idle_timeout = 120
[[http_service.checks]]
interval = '15s'
timeout = '5s'
grace_period = '30s'
method = 'GET'
path = '/ready'
[[services]]
protocol = 'udp'
internal_port = 7546
[[services.ports]]
port = 7546
[[vm]]
size = 'shared-cpu-1x'
memory = '1024'
[[metrics]]
port = 9091
path = '/metrics'
-65
View File
@@ -1,65 +0,0 @@
# fly.toml app configuration file generated for gitlawb-node-3 on 2026-07-22T09:47:19+08:00
#
# See https://fly.io/docs/reference/configuration/ for information about how to use this file.
#
app = 'gitlawb-node-3'
primary_region = 'nrt'
[build]
dockerfile = '../../Dockerfile'
[env]
GITLAWB_AUTO_SYNC = 'true'
GITLAWB_BOOTSTRAP_PEERS = 'https://node.gitlawb.com,https://node2.gitlawb.com'
GITLAWB_HOST = '0.0.0.0'
GITLAWB_KEY = '/data/keys/identity.pem'
GITLAWB_MAX_PACK_BYTES = '524288000'
GITLAWB_P2P_PORT = '7546'
GITLAWB_PORT = '7545'
GITLAWB_PUBLIC_URL = 'https://node3.gitlawb.com'
GITLAWB_REPOS_DIR = '/data/repos'
ICAPTCHA_MODE = 'enforce'
ICAPTCHA_PUBKEY = 'xjyPNqIbvc9U-kwXW6u9mDqRJ7E2UUMOaJdUWhpEXq8'
ICAPTCHA_REQUIRED_LEVEL = '3'
[[mounts]]
source = 'gitlawb_data'
destination = '/data'
[http_service]
internal_port = 7545
force_https = true
auto_stop_machines = 'off'
auto_start_machines = true
min_machines_running = 1
[http_service.concurrency]
type = 'connections'
hard_limit = 500
soft_limit = 400
[http_service.http_options]
idle_timeout = 120
[[http_service.checks]]
interval = '15s'
timeout = '5s'
grace_period = '30s'
method = 'GET'
path = '/ready'
[[services]]
protocol = 'udp'
internal_port = 7546
[[services.ports]]
port = 7546
[[vm]]
size = 'shared-cpu-1x'
memory = '1024'
[[metrics]]
port = 9091
path = '/metrics'
-14
View File
@@ -193,19 +193,5 @@ else
echo " gl quickstart"
fi
# oh-my-zsh's default git plugin aliases gl='git pull', which silently shadows
# the gl binary in every interactive zsh (aliases beat PATH). Detect the common
# setup and say so now, at install time, instead of letting the first `gl`
# print git's baffling "fatal: not a git repository".
if [ -f "$HOME/.zshrc" ] && [ -d "$HOME/.oh-my-zsh/plugins/git" ] \
&& grep -Eq '^[[:space:]]*plugins=\(([^)]*[[:space:]])?git([[:space:]][^)]*)?\)' "$HOME/.zshrc" \
&& ! grep -Eq '^([^#]*[[:space:];&|])?unalias[[:space:]]+([^;&|#]*[[:space:]])?gl([[:space:]]|$)' "$HOME/.zshrc"; then
echo ""
echo "NOTE: oh-my-zsh's git plugin aliases gl='git pull', which will shadow"
echo "the gl command in interactive shells. To use gl by name, run:"
echo ""
echo " echo 'unalias gl 2>/dev/null' >> ~/.zshrc && source ~/.zshrc"
fi
echo ""
echo "Docs: https://docs.gitlawb.com"
@@ -19,11 +19,6 @@ class Config {
var operatorPrivateKey: String = ""
var tigrisBucket: String = ""
var autoSync: Bool = false
/// Owner-only push. Persisted as a setting rather than read from a hand-edited
/// `.env`, because `writeEnvFile()` regenerates that file on every start and
/// would discard the edit. Matches the node's own default; set false only for a
/// rolling upgrade whose pushers are not yet the repo owner.
var enforceOwnerPush: Bool = true
var repoPathString: String = ""
// MARK: - Paths
@@ -81,7 +76,6 @@ class Config {
"operatorPrivateKey": operatorPrivateKey,
"tigrisBucket": tigrisBucket,
"autoSync": autoSync,
"enforceOwnerPush": enforceOwnerPush,
"repoPath": repoPathString,
]
@@ -111,7 +105,6 @@ class Config {
operatorPrivateKey = dict["operatorPrivateKey"] as? String ?? operatorPrivateKey
tigrisBucket = dict["tigrisBucket"] as? String ?? tigrisBucket
autoSync = dict["autoSync"] as? Bool ?? autoSync
enforceOwnerPush = dict["enforceOwnerPush"] as? Bool ?? enforceOwnerPush
repoPathString = dict["repoPath"] as? String ?? repoPathString
}
@@ -128,7 +121,6 @@ class Config {
lines.append("GITLAWB_PORT=\(httpPort)")
lines.append("GITLAWB_P2P_PORT=\(p2pPort)")
lines.append("GITLAWB_AUTO_SYNC=\(autoSync)")
lines.append("GITLAWB_ENFORCE_OWNER_PUSH=\(enforceOwnerPush)")
if !chainRpcURL.isEmpty {
lines.append("GITLAWB_CHAIN_RPC_URL=\(chainRpcURL)")
Loaded 100 of 109 files, more files were not shown because too many files have changed in this diff. Show more