Compare commits

..
Author SHA1 Message Date
beardthelion 370a364b0f test(node): cover the sweep's boot wiring and prove an interrupted push never reaches the upload
Two claims from earlier commits were argued rather than executed. Both are now
observed.

The legacy-CID sweep's spawn sat inline in main() where no test could reach it,
so the batch size, the inter-batch delay, the detachment and the shutdown select
were all reasoned. The block moves verbatim into `spawn_legacy_cid_sweep`, which
a test calls directly. It seeds rows whose keys are already canonical, so the
cost gate skips each and the persisted cursor reads out walk progress cleanly,
then asserts the cursor stops at the configured batch boundary, the call returns
without blocking, the task is still alive inside its delay, and shutdown ends it.
Never spawning, reading the batch or delay from the wrong knob, or removing the
shutdown branch each turns it red. What remains reasoned is the single call line
inside main(), down from a twenty-six line block.

The claim that a disconnect-interrupted push never uploads to Tigris was
structural: the tests build the store with no Tigris client, so a counter inside
the client arm could never move and would have proved nothing. The counter now
sits at the upload decision, before the client is consulted, and the disconnect
test asserts zero while the success test asserts exactly one. That positive
control is what makes the zero meaningful, and it is itself load-bearing:
releasing the success path with failure semantics drops it to zero, and adding a
Drop impl that uploads raises the disconnect count to one.

The counter observes reaching the upload site, not an S3 request. TigrisClient
wraps a concrete SDK client with no injectable endpoint, and plumbing one through
would be out of proportion to the risk here.
2026-07-27 21:44:31 -05:00
beardthelion 03aee267e6 fix(node): shed 503 from the issue and PR write paths when the lock pool is exhausted
The lock-pool work gave `acquire_write` a distinct exhaustion error and mapped it
to 503 with Retry-After, but only on the push handler. The three other callers,
both issue writes and the PR merge, still turned it into a generic git error and
returned 500. Those are exactly the callers that hold no concurrency permit, so
they are the likeliest to meet an exhausted pool in the first place.

All three now go through the push handler's helper rather than repeating the
mapping, so the log level and the message cannot drift between the four sites.
Each has a test that occupies the single connection of a one-slot pool with a
guard on a different repo, so the failure is pool capacity rather than
advisory-lock contention, and asserts status 503 plus retry-after through
IntoResponse. Reverting the mapping at one site turns only that site's test red.

Worth knowing for review, and left alone deliberately: close_issue takes the
write guard before its author check, so this shed is reachable by any
authenticated caller whether or not they could close the issue. That ordering
predates this change and reordering the handler is not this commit's business,
but it does mean the 503 discloses that the repo's write pool is busy.
2026-07-27 21:14:55 -05:00
beardthelion c519d54fc1 fix(node): keep the incompleteness marker until a source is really recorded, and let the sweep retry
Four follow-ups from an adversarial pass over the previous commits.

The marker clear ran unconditionally inside `record_pin_source`'s transaction,
but its INSERT is `ON CONFLICT DO NOTHING` guarded by the cap, so it records
nothing when the repo is already a source. That is the common case, not a rare
one: the pin path calls `record_pin_source` for every already-pinned object, and
on a requeue pass that is the whole-repo enumeration. So a re-push from the repo
that was already recorded cleared a marker set by a different repo's failure and
the public copy went back to 404ing. The clear is now gated on the insert
actually adding a row. A genuine record from a third repo still clears the
marker, which needs a per-(oid, repo) table to close and is documented rather
than papered over.

The sweep advanced its cursor before every skip, which stops a wedge but made a
transient skip permanent: once the walk reached the end of the table the cursor
parked there and no later boot read another row. On a Tigris-backed node the
common skip is "repo is not on local disk", so the sweep would repair whatever
happened to be warm at boot and then never run again, and the new advertise
filter turned those rows from "advertised but 404" into "never advertised, never
repaired". Repairs now report whether a skip was retryable or terminal, and a run
that skipped anything retryable rewinds the cursor once, after the walk, so the
next run re-walks. A permanently unrepairable row is terminal and does not
trigger a rewind, so this cannot become a hot loop.

The sweep also reached its repo paths through the raw join helper, skipping the
three-layer traversal barrier `RepoStore::local_path` applies. A test with a repo
row named `../../escapee` showed it reading and repairing bytes from outside
`repos_dir`. That barrier is now extracted and shared, so the two callers cannot
drift. And the repair's synchronous `git cat-file` moved under `spawn_blocking`,
matching the existing treatment of the withheld walk, so a wedged read no longer
parks a worker thread for the full service timeout.

Finally the resolver's two source-set queries ran ahead of the work-budget peek
that exists to shed a spent caller before doing work. The peek now runs first. A
throttled caller whose set turns out complete gets 429 where it used to get 404,
which is the honest answer since its search never ran, and it removes an oracle
that let a throttled caller tell a complete source set from an incomplete one.
2026-07-27 18:29:01 -05:00
beardthelion 1c82060c2b fix(node): stop the lock pool from being occupied by spinners and racing the reaper
Three regressions from the previous four commits, each demonstrated before being
fixed.

The advisory-lock retry loop checked its connection out before spinning, so a
caller that lost the lock held a pool connection through all sixty one-second
attempts. The old pool-per-query shape returned it between iterations, so a
spinner occupied nothing. That matters more than the pool sizing suggested:
`acquire_write` has three non-push callers (two in issues, one in pulls) that
hold no concurrency permit, and the issue routes carry no rate limit, so any
self-minted DID could fire enough concurrent closes at one repo to strand the
pool and fail authenticated pushes on every repo. The checkout now happens per
attempt and a losing attempt returns the connection before sleeping, so only the
winner retains one. Lock-pool exhaustion also carries its own error now and sheds
503 with Retry-After instead of surfacing as a 500.

Holding the lock on one connection then meant a client disconnect released it
early: the handler future dies mid-receive-pack, the guard drops without
`release`, and `after_release` frees the lock while the detached reaper is still
giving the process group its SIGTERM grace. A second writer was admitted 90ms
after the drop, well inside that window, which is the invariant smart_http
already documents. The write guard now rides the AdmissionGuard into the reaper,
the same seam the admission permits already use, so the lock outlives the group.
The success path still reclaims the guard and releases it synchronously, and a
dropped guard performs no Tigris upload, so an interrupted push cannot publish a
half-applied repo.

The requeue loop's exit on an exhausted re-read discarded a push that coalesced
during the retry window: `requeue_or_release` had cleared the dirty bit without
marking the guard released, so `Drop` removed the key and no pass was ever
attempted. It now falls through with nothing to replicate and lets the tail's
atomic check-and-clear decide. A lap only happens when a push actually coalesced
and each pays a full bounded re-read, so a sustained outage still terminates,
which is tested with ten thousand injected faults under a watchdog rather than
argued.

Also clamps the lock pool's derived size, which previously followed
max_concurrent_git_pushes up to its million-connection ceiling, and documents the
node's total connection budget on that knob.
2026-07-27 17:47:56 -05:00
beardthelion 73429c7a56 fix(node): sweep legacy provider-CID pins instead of waiting for a re-push
`repair_legacy_provider_cid` had exactly one trigger: the already-pinned skip
branch inside `pin_new_objects`, which a push only reaches when it re-carries the
object. Normal git negotiation omits objects the node already has, so on an
upgraded node that push generally never arrives. Meanwhile `list_pinned_cids`
kept advertising the stored provider CID that this branch's `/ipfs/{cid}`
deliberately withholds, so the node handed out CIDs it would then refuse to
serve, and previously pinned data stayed unresolvable.

Two halves, both needed. A detached background sweep walks `pinned_cids` in
bounded batches with an inter-batch delay, resolves each row's repo from its
recorded provenance, and reuses the existing repair (and its cost gate, so a row
already keyed on a raw CIDv1 reads no bytes). Migration v16 adds a single-row
cursor table so a restart mid-walk resumes rather than rewinding; the cursor
advances on every row read, before any skip, so an unrepairable row cannot wedge
the walk. A row whose bytes are gone, whose provenance is NULL, or whose repo is
not on local disk is left exactly as it is. And `list_pinned_cids` now omits rows
whose key is not a raw CIDv1, so the window before the sweep catches up
advertises nothing the resolver withholds.

The sweep resolves paths directly rather than through `repo_store.acquire`, which
would pull cold repos back from Tigris and turn a repair pass into a bulk
restore. Defaults are deliberately quiet: 64 rows per batch, 60s between batches.

Ten tests cover the repair, the bytes-gone and unrepairable arms, the cost gate,
the batch bound, cursor resumption, both states of one row through the advertise
filter, and the degenerate empty and no-legacy-rows tables. Removing the filter,
the batch bound, the cursor advance, the bytes-gone early return, or the
inter-batch sleep each turns a test red.

One pre-existing test used placeholder resolver keys that the new filter drops;
its values are now real raw CIDv1s with every assertion unchanged.
2026-07-27 15:53:26 -05:00
beardthelion fccdfc1820 fix(node): record a failed pin-source write so the resolver keeps its fallback
The resolver treated a non-empty, below-cap pin-source set as proof that every
source had been recorded, and skipped the bounded scan on that basis. But
`record_pin_source` is best effort at every call site: the ipfs_pin sites were
warn-only once their retries exhausted, and both pinata sites had no retry at
all. An object first pinned from a private repo and later pushed from a public
one whose source write failed keeps a set naming only the private source, so
`GET /ipfs/{cid}` returned 404 for an object the public repo would have served.

Migration v15 adds `pinned_cids.pin_sources_incomplete`, NOT NULL DEFAULT FALSE
so every existing row reads as complete. A record that fails outright sets it; a
later successful record clears it, inside the same transaction as the insert so
the two paths cannot drift. `needs_scan` now ORs the marker in, short-circuiting
after the existing empty and at-cap checks so it costs nothing on the serve path.
Pinata's two bare calls now share `retry_db_record` with ipfs_pin. The ipfs_pin
first-pin path additionally records the CID and its source in one transaction;
Pinata's remains two retried calls.

INV-10 is preserved: a complete set still tail-404s with no preload, asserted on
the existing preload counter. The fallback is not an authorization bypass, and
the test for that asserts the scan actually ran before checking the denial, so
the denial cannot pass vacuously. Dropping the marker term from the gate, forcing
it true, or removing the clear each turns a test red.

The marker is per-object rather than per-(object, repo): a successful record from
one repo clears a marker set by another's failure. That is the conservative
direction, since the marker only ever adds the fallback back.
2026-07-27 14:40:03 -05:00
beardthelion 8e6e2ad0fe fix(node): retry the requeue re-read so a coalesced push is not silently dropped
`requeue_or_release` clears the dirty flag atomically with the decision to loop,
which is correct and stays as it is. The problem was what the pass did next: a
failed `get_repo_by_id` fell into an Err arm that yielded no rules, and
`list_visibility_rules(...).ok()` collapsed its own error the same way. With no
rules, `replication_withheld_set` returns `(false, None)`, the pass skipped its
work entirely, and the loop exited having consumed the dirty bit that stood for
the coalesced push. There is no reconciliation sweep to re-derive it, so that
push's objects were never pinned and its withheld blobs never sealed.

`requeue_refresh_state` now separates the three cases the old code collapsed.
`Ok(Some)` proceeds. `Ok(None)` means the repo really is gone and releases
without touching the retry budget. `Err` from either read is transient and is
retried with doubling backoff up to three attempts. A rules-read error is no
longer indistinguishable from a repo that has no rules.

Exhausting the retries still drops the pass, which is the pre-existing residual,
but it now logs at error level with the repo id and attempt count instead of
disappearing behind a warning. Six tests cover the retried-then-lands path, the
bounded-exhaustion path, repo-gone, the rules-read arm, freshness of the applied
rules, and the coalescing property that had to survive. Reverting the Err arm,
restoring the `.ok()`, raising the bound, or downgrading the error log each turns
a test red.
2026-07-27 13:52:44 -05:00
beardthelion 9a70d75d13 fix(node): own the repo write lock on one connection so cancellation cannot leak it
`acquire_write` took `pg_try_advisory_lock` through the shared pool, then awaited
the Tigris exists/download path before constructing `RepoWriteGuard`. Two bugs
fell out of that. The `tokio::time::timeout` this branch added around the call
can drop the future inside that window, after the lock is held and before the
only unlock (`RepoWriteGuard::release`) exists, wedging every later push to the
repo. And because a session advisory lock belongs to the connection that took
it, running the lock and the unlock through the pool lets them land on different
connections: the unlock returns false and leaks, while a competing acquire that
draws the holding connection re-enters the lock and two writers run against the
same repo concurrently.

`RepoStore` now owns a dedicated lock pool built with an `after_release` hook
that runs `pg_advisory_unlock_all()`. `acquire_write` checks out one connection,
takes the lock on it, and moves it into the guard, which unlocks on that same
connection. sqlx's `PoolConnection::drop` spawns `return_to_pool`, which runs the
hook, so a connection dropped by cancellation still clears its locks; the unlock
is asynchronous with respect to the drop, which the tests account for. The pool
is separate from the main query pool because a push holds its connection for the
whole receive-pack and `db_max_connections` (20) sits below
`max_concurrent_git_pushes` (32).

Seven tests cover the cancellation window, the same-session release, and the
serialization the pool-mismatch bug was breaking. Both the cancellation
regression and the concurrent-writer case were observed red before the fix, and
removing the `after_release` hook or moving the unlock back to the pool turns
them red again.
2026-07-27 13:22:45 -05:00
t 62a99270a9 fix(node): bound the post-push pin and repair reads so a wedge cannot pin the coalescing key
A cross-model adversarial pass found that run_post_push_replication holds the
per-repo coalescing key until requeue_or_release, but pin_new_objects reached it
only after two unbounded git reads: the U7 legacy-repair read and the
pre-existing pin read, both plain Command::output with no teardown. A wedged
cat-file on a stuck backend hung the task forever, so the key was held until
process death and later pushes only marked it dirty without spawning a
replacement (the same class this PR closed on the /ipfs serve path).

Add read_object_bounded (the bounded twins under one shared deadline) and use it
at both sites; a timeout skips that object and lets the task proceed to
requeue_or_release. Adds a wedge-reap regression proven load-bearing two ways.
2026-07-20 21:06:10 -05:00
t fba3c6fda3 fix(review): share one deadline across the /ipfs size+content reads
Code review found each candidate's size and content reads were each granted a
full git_service_timeout, so a single served candidate could hold the /ipfs
walk permit for 2x the timeout. Share one deadline across both stages (mirrors
build_filtered_pack) so the read holds admission for one timeout total, and
correct the admission-bound comment: the worst case is O(candidates x timeout)
bounded by the walk concurrency cap, not a single deadline.
2026-07-20 20:27:35 -05:00
t 45e269b2ae fix(node): opportunistically repair legacy provider-CID pins
Before this PR the pin path stored the provider CID (Kubo dag-pb / Pinata) in
pinned_cids.cid, which the new resolver recomputes-and-404s, while
/api/v1/ipfs/pins still advertises the stale value. Repair a legacy row in the
already-pinned skip branch: gate on a cheap stored-CID codec check (only a
non-raw-codec CID reads bytes, so raw rows keep the DB-only skip cost),
recompute the raw CID, and rewrite it while stashing the old value in a new v14
legacy_provider_cid column (distinct from pinata_cid, which gates the Pinata
pin-skip). Rows whose bytes are gone stay withheld. Documents that the deferred
one-shot sweep, not this opportunistic path, fully retires the window.
2026-07-20 20:11:42 -05:00
t b774c435a6 fix(node): retry pin-source and pinned-CID recording on transient errors
The three warn-only record sites in the detached post-push task dropped a
transient DB failure silently, leaving a permanently incomplete pin-source set
that makes the resolver 404 a valid public copy. Wrap them in a bounded retry
(inserts are idempotent via ON CONFLICT DO NOTHING); on exhaustion the warn
still fires so behavior degrades to today's. Documents the residual crash/outage
window that only a reconciliation sweep retires.
2026-07-20 19:50:23 -05:00
t 0a1618f7fb fix(node): split /ipfs work accounting off the route rate limiter
The route middleware charged ipfs_rate_limiter once per request while the
legacy scan and provenance walk charged the same bucket per probe/walk, so a
one-probe request cost two tokens and GITLAWB_IPFS_RATE_LIMIT=1 admitted the
request at the route then 429'd it at the pre-scan peek.

Add a bounded ipfs_work_rate_limiter, move the in-scan, per-walk, and pre-scan
charges onto it, and leave the route limiter as the pure once-per-request
brake. The work-budget capacity derives from the route limit with a floor of
one full legacy-probe budget per window (no new operator knob), so a
default-config deep search never self-throttles. Adds it to the periodic sweep,
updates all three AppState construction sites, and rewrites the contradictory
limiter comments.
2026-07-20 19:41:57 -05:00
t c2fce67bae fix(node): wire GITLAWB_IPFS_MAX_REPOS_WALKED to the legacy-probe budget
The knob was parsed and documented as the per-request /ipfs fan-out cap but
never read in production: AppState seeded the legacy-probe budget from a fixed
256 constant, so setting the knob did nothing and an operator could still incur
256 acquire/probe operations per request.

Seed ipfs_max_legacy_probes from the knob (default raised to 256 to preserve
shipped behavior); leave the history-walk ceiling on its MAX_PIN_SOURCES+1
constant so a provenanced request is never truncated into a false 503. Updates
the README and .env.example to the shipped behavior and adds cap-honored and
plumbing tests.
2026-07-20 19:17:12 -05:00
t 3351e09c08 fix(node): requeue coalesced pushes instead of dropping them
Per-repo coalescing dropped every push that arrived while a repo's encryption
task was in flight, and no reconciliation ever processed it, so a withheld
blob added by the coalesced push had its recovery copy permanently absent. The
drop also silently skipped the coalesced push's local-IPFS pin work.

EncryptInflight becomes a dirty-flag map; the detached task loops, and the
atomic check-and-clear sits at the task tail, unconditional on the
has_path_scoped_rule gate and walk success (a public or rules-free repo would
otherwise exit before it ran). Each requeue re-reads repo state fresh and
re-enumerates the pin half through the fail-closed full scan, never bare
list_all_objects, so a coalesced rule change is honored and no withheld or
dangling object leaks. At-most-one-task-per-repo is preserved.
2026-07-20 19:04:55 -05:00
t 757edf560c fix(node): own /ipfs admission for the life of the walk
get_by_cid held the global walk permit and the per-source permit as handler
locals, so dropping the request future released both while the spawn_blocking
probe, visibility walk, and object read kept running; a cancel-spam or timeout
client could exceed MAX_CONCURRENT_IPFS_WALKS and the per-source cap, and the
bare cat-file probe/read children had no teardown at all.

Move the gated serve pipeline into a detached task that owns an AdmissionGuard,
awaited by the handler, so admission releases only after the bounded work is
gone; back the /ipfs probe and read with run_bounded_git twins for
process-group teardown. Adds cancel-mid-walk, timeout-reap, and cancel-spam
regressions plus two load-bearing INV-22 rows. Also reflows one U1 test line to
satisfy cargo fmt.
2026-07-20 18:34:09 -05:00
t 86828046ea fix(node): hold read admission through the filtered-pack reaper
The path-scoped upload-pack branch kept the read and per-caller permits as
handler locals and called upload_pack_excluding with no AdmissionGuard, so a
cancelled filtered clone released admission while KillGroupOnDrop was still
reaping the git group, bypassing the read and per-caller concurrency caps.

drive_git_child now returns the disarmed guard on success so one guard rides
both build_filtered_pack stages (rev-list then pack-objects); the handler builds
the guard from the permits as the plain path does. Adds a disconnect regression
and an INV-22 gate row, both proven load-bearing.
2026-07-20 17:54:49 -05:00
t d26cf6dd1d fix(node): close the /ipfs pin-source griefing hole via a bounded scan fallback
An attacker who pins a public object from MAX_PIN_SOURCES repos before the
legitimate public source registers fills pin_repo_sources (record_pin_source is
first-N-wins and drops later sources silently); the buried public source is then
never reached because a non-empty provenance set suppressed the legacy scan, so
anon GET /ipfs/{cid} 404s forever for a public object (re-breaks F1).

U1: db::pin_sources_at_cap reports whether pin_repo_sources is at MAX_PIN_SOURCES
for an oid (the only observable signal that a servable source may have been
dropped, since the write cap never overshoots).

U2: get_by_cid falls back to the bounded legacy scan on a provenance miss when the
set is empty OR at_cap. The scan gates every repo through the real per-caller gate,
so it finds the buried public copy. A complete (non-full) set still fast-404s, so
ordinary denials never fan out to O(repos) (INV-10/F3); the fallback honors the
is_throttled peek.

Regression ipfs_cid_buried_public_source_still_serves_via_scan_fallback: 404 before,
200 after; pin_sources_at_cap_flips_at_max covers the boundary.
2026-07-17 10:15:56 -05:00
t 94ca744269 fix(node): align ipfs_pin return with pinata; add Retry-After to SearchIncomplete
- ipfs_pin::pin_new_objects returns the provider Hash (not the raw resolver key),
  matching the pinata twin's contract; the DB cid stays the raw content CID. The
  return is logging-only here, so this is drift-avoidance, not a functional fix.
- AppError::SearchIncomplete now carries Retry-After: 1 like Overloaded, so both
  retryable 503s from the /ipfs handler advertise the retry hint consistently.
2026-07-17 10:15:56 -05:00
t 2eea354188 fix(node): tie the /ipfs walk ceiling to MAX_PIN_SOURCES + 1
A provenanced object has a bounded source set (first-pinner + up to
MAX_PIN_SOURCES additional). With the walk ceiling at 16 == MAX_PIN_SOURCES, an
authorizing public source sorting after 16 path-scoped denials was never reached:
the ceiling truncated the search and returned a false 503 for a readable object.
Raise the ceiling to MAX_PIN_SOURCES + 1 so the whole bounded provenance set is
always tried; the legacy scan stays bounded by MAX_LEGACY_PROBES_PER_REQUEST.
2026-07-17 10:15:56 -05:00
t c767658b7b fix(node): bound the /ipfs cat-file probes under the walk permit; backfill pinata provenance
P2-E: the object-type probe and the F6 size+read ran bare git cat-file inside
spawn_blocking while the /ipfs walk permit was held, so a wedged cat-file could
pin the global walk slot for the request's life. Bound both under
git_service_timeout_secs; on timeout free the slot (truncated -> retryable 503)
and skip the repo.

P2-D: the pinata already-pinned branch now backfills NULL first-pinner
provenance in lockstep with the ipfs_pin skip branch (consistency; the object
was already resolvable via the pin_repo_sources union).
2026-07-17 10:15:56 -05:00
t 8afc9eaf97 fix(node): close grok P1 findings on the #173/#174 IPFS integration
P1-A: gate_and_serve held the /ipfs walk permit across a bare repo_store.acquire;
wrap it in git_acquire_timeout_secs (mirroring #174's own handler) so a cold/hung
Tigris acquire cannot pin the global walk slot. On expiry skip the repo and mark
the search truncated (retryable 503, not a false 404).

P1-B: the local-IPFS pin path recorded Kubo's provider Hash as pinned_cids.cid;
for objects above the block size that is a dag-pb root that does not hash the raw
content, so GET /ipfs/{cid} listed then 404'd them under the F2 integrity check.
Record the locally-computed raw-content CID, mirroring the pinata twin.
2026-07-17 10:15:56 -05:00
t 433c8915bf feat(node): integrate the #173 IPFS CID tree-gate onto the #174 walk-concurrency base
Rebased onto #174 (fix/served-git-concurrency-cap). #173's incremental history
cannot replay commit-by-commit because #174 rewrote the same handler files in
parallel; this single commit carries the fully-integrated tree (identical to the
verified merge result), with the follow-up fixes as separate commits on top.

Brings in #173: GET /ipfs/{cid} CID->oid resolution via pinned_cids, per-caller
path-scoped blob/tree/commit-tag gating (#135/#173 F1-F6), and the pin-source
provenance table. Adapts #173's tree/commit-tag walks to #174's run_bounded_git
so the held /ipfs walk-concurrency permit is duration-safe (F5).
2026-07-17 10:15:56 -05:00
t 413d6cf0bc fix(node): reject unsupported git service before the read slot; document the new admission knobs (#174)
P2-1: git_info_refs treated every service other than git-receive-pack as a read op
and did the DB/visibility/Tigris work under a read permit, validating the service
string only downstream in smart_http. So an unauthenticated `?service=anything` to
a public repo consumed a read slot + storage work before rejection. Validate the
service is exactly git-upload-pack or git-receive-pack immediately after parsing,
returning 400 before the pre-DB shed and the permit. Handler-layer regression:
with the read pool exhausted, `?service=git-explode` returns 400 (validated first),
not 503 — removing the validation makes it 503 (RED).

U5/INV-24 docs: document the new knobs from U2/U3 in .env.example and the README
env-var table — GITLAWB_GIT_ACQUIRE_TIMEOUT_SECS, GITLAWB_MAX_CONCURRENT_IPFS_WALKS,
GITLAWB_IPFS_WALK_PER_SOURCE, GITLAWB_IPFS_MAX_REPOS_WALKED, GITLAWB_IPFS_RATE_LIMIT.
Each is already enforced on the path it names (U2/U3 tests), so the docs match the
implemented routing.
2026-07-15 13:16:40 -05:00
t 394768711e fix(node): bound the post-push encryption task set by per-repo coalescing without dropping recovery copies (#174)
After a successful path-scoped push the handler spawns a DETACHED task that runs
pin_new_objects then withheld_recipients_gated, which acquire_owned().await's on
git_encrypt_semaphore — it DEFERS (blocks) when the pool is full. The semaphore
caps active walks but nothing capped how many detached tasks were spawned and
PARKED on that await: N rapid path-scoped pushes spawned N tasks, each holding
cloned object lists/rules/paths/keys — an unbounded parked-waiter set (P2-2).

2a54c15 deliberately chose defer-not-shed because dropping the walk loses the
withheld-blob recovery copy and there is no reconciliation sweep to rebuild it.
So bound the OUTSTANDING-TASK set by per-repo coalescing rather than shedding:
EncryptInflight (a bounded Arc<Mutex<HashSet<repo_id>>>) + try_begin — before the
spawn, if a task for the repo is already in-flight skip the duplicate (the pending
walk covers the newer objects); otherwise spawn and move an RAII
EncryptInflightGuard into the task that removes the key on drop (completion, error,
or panic-unwind). This bounds the set to <=1 pending task per repo and never drops
work; withheld_recipients_gated's defer is unchanged.

The second detached post-push spawn (Pinata replication, also runs pin_new_objects)
is scoped out with rationale: it parks on no semaphore (no unbounded waiter set)
and does per-push per-ref work (branch->CID, gossip, subscription, Arweave,
peer-notify) keyed to this push's ref_updates — coalescing it would DROP a later
push's announcements, a correctness regression.

Regressions (mutation-verified RED->GREEN): the outstanding set is bounded to 1
per repo under saturation (never-coalesce -> 32 tasks RED); a coalesced repo's key
is released when its task ends so it is reprocessed, never permanently skipped
(guard-drop no-op -> repo locked out, recovery copy lost forever, RED — the exact
durability regression the fix prevents); distinct repos each admit; cold-set first
push admits; plus an inv22_gates structural tripwire that fails if the handler gate
is removed.
2026-07-15 13:10:04 -05:00
t f424ccbbcd fix(node): gate the /ipfs walk with bounded concurrency admission + per-source cap + route rate limit (#174)
GET /ipfs/{cid} (get_by_cid, publicly reachable via optional_signature) ran
allowed_blob_set_for_caller_bounded in spawn_blocking inside a per-repo loop with
NO global/per-source admission and NO route rate limit — the surface had zero
concurrency or rate control. A permissionless caller could fan out concurrent
full-history git walks (each up to the walk timeout) exhausting blocking-pool
threads and PIDs outside every served-git pool (P1-3).

Acquire a global git_ipfs_walk permit (try_acquire_owned, shed 503) plus a
per-source sub-cap (with_default_max_keys, reject-before-insert, keyed via
client_key/TrustedProxy so XFF can't spoof it) ONCE after CID validation, held as
handler locals across the whole request — including every spawn_blocking walk, so
the slot reflects real blocking-thread occupancy (a tokio timeout cannot cancel
spawn_blocking). None key -> global pool only. Cap repos walked per request so one
request can't serialize N full-history walks. Attach rate_limit_by_ip +
IpRateLimiter to the /ipfs route (mirrors the git/create/peer routers; the
extension is required or the middleware is a silent no-op). Visibility/authz
semantics unchanged.

New knobs: GITLAWB_MAX_CONCURRENT_IPFS_WALKS (32), GITLAWB_IPFS_WALK_PER_SOURCE
(4), GITLAWB_IPFS_MAX_REPOS_WALKED (64), GITLAWB_IPFS_RATE_LIMIT (600/hr, 0
disables) — all range-validated and enforced on-path.

Handler-layer regressions (all mutation-verified RED->GREEN): shed-at-capacity
503 (delete acquire -> falls through), per-source cap sheds same source / admits
another, None-key arm sheds on the global pool, map self-bounds reject-before-insert,
the walk permit is held while the spawn_blocking walk runs (available_permits()==0;
dropping it before the loop -> RED), repos-walked cap (remove break -> 2 walks),
and the route IP rate limit actually fires 429 (drop the Extension -> RED no-op).
2026-07-15 12:42:55 -05:00
t 8d17038e5b fix(node): bound the served-git storage-acquisition phase with a release-on-expiry deadline (#174)
The served-git admission permit is acquired, then RepoStore::acquire*/advisory-lock
runs before the bounded git runner starts — with no local deadline. acquire/acquire_fresh
await Tigris HEAD/GET, and acquire_write's advisory-lock loop awaits a per-iteration
pg_try_advisory_lock that can block indefinitely on a hung Postgres pool. Since
GITLAWB_GIT_SERVICE_TIMEOUT_SECS only starts once git spawns, a stalled backend pins
the permit and drains the pool until every later request 503s (P1-2).

Wrap each git-path acquire (git_info_refs read/advert, git_upload_pack, git_receive_pack)
and the /ipfs per-repo acquire loop in tokio::time::timeout(git_acquire_timeout_secs, ..).
On expiry the git paths return AppError::Overloaded (503 + Retry-After); the permit is a
handler-local at that point (moved into the AdmissionGuard only after acquire) so the early
return frees the slot. The /ipfs per-repo acquire keeps its fail-closed `continue` on
timeout (never serves an un-acquired repo). No repo_store.rs signature change: the outer
timeout cancels a mid-sleep/mid-fetch_one future, so no in-loop deadline is needed.

New knob GITLAWB_GIT_ACQUIRE_TIMEOUT_SECS (default 30, range 1..), modeled on
git_service_timeout_secs, kept separate because acquisition and git execution are distinct
cost centers.

Handler-layer regression receive_pack_acquire_deadline_sheds_and_releases_permit: holds the
same pg advisory lock acquire_write derives on a second connection so the real loop must
retry, drives git-receive-pack with the deadline at 2s and write pool size 1, and asserts
503 at ~2s + the permit recovers + a follow-up admits once the lock frees. Removing the
timeout wrapper turns it RED (blocks ~31s to the test ceiling on the ~59s advisory loop).
2026-07-15 12:00:02 -05:00
t 72e899dae0 fix(node): hold served-git admission until the process group is reaped on the plain spawn paths (#174)
On the plain (non-path-scoped) info_refs / upload-pack / receive-pack paths the
global + per-source admission permits were handler-locals that dropped the
instant the handler future dropped on a client disconnect, while KillGroupOnDrop
reaps the git process group in a DETACHED task (SIGTERM -> grace -> SIGKILL ->
reap). So a disconnect-spammer admitted replacements while prior groups were
still alive, defeating the process cap and the per-source cap during teardown.
be0cdd6 already fixed this for the path-scoped upload-pack walk; this closes the
residual plain paths (P1-a).

Introduce AdmissionGuard, a move-only holder of the permits, threaded through
info_refs/upload_pack/receive_pack/run_git_service into drive_git_child's
KillGroupOnDrop. On disconnect the guard moves into the detached reaper and drops
only after the group is ESRCH-confirmed reaped; on success/timeout disarm()
returns it for the earliest provably-free drop. The rev-list/pack-objects
visibility-walk callers hold no admission and pass None.

Thread git_bin through upload_pack/receive_pack (they hardcoded "git") so the
plain POST path is drivable by an injected fake git, matching the existing
injectable seam on info_refs/run_git_service/upload_pack_excluding.

Handler-layer regression upload_pack_plain_permit_held_through_group_reap_after_disconnect:
isolates the global read pool (size 1, per-source + rate limiter permissive), a
SIGTERM-ignoring descendant keeps the group alive across the reap window, and
asserts available_permits()==0 held on disconnect + a cross-source replacement
sheds 503, then frees after ESRCH. Reverting the guard-into-reaper move (release
immediately) turns it RED at the held-through-reap assertion. Plus a None-key arm.
2026-07-15 12:00:02 -05:00
t 5749df6e67 test(node): INV-22 completeness guard for the served-git concurrency surface (#174)
A source-scan tripwire that fails when a new site reintroduces the concurrency-cap
class PR #174 fixed, or when one of the five gates (U1-U5) is removed. INV-22: a
permit held per op recovers only if every path is duration-bounded and reaps the
group before releasing admission, and every detached git task carries admission.

Lives in tests/ (a separate crate) on purpose — a guard scanning the file it lives
in would match its own identifier literals and pass vacuously; scanning src/ from
here keeps every check load-bearing. Checks: run_bounded_git confirms child exit via
child_terminated_without_reaping (U1); KillGroupOnDrop launches the reaper via
Handle::try_current (U2); git_receive_pack acquires state.git_write_per_caller (U4);
the encryption walk runs through withheld_recipients_gated over git_encrypt_semaphore
(U5); and the bounded recipients walk is invoked exactly once — only inside the gated
helper — so a new spawn_blocking bypass (count > 1) fails (P1-e non-bypass).

Proven load-bearing: a synthetic second call site turns the count tripwire RED, and
each named gate maps to a fix whose own regression already goes RED on revert.
2026-07-14 20:33:01 -05:00
t 2a54c159e6 fix(node): bound concurrent post-push encryption walks with an admission pool (#174)
After a successful path-scoped push the handler released its write permit and
then ran a DETACHED tokio task whose spawn_blocking(withheld_blob_recipients_bounded)
performed another full-history git walk under no admission. N fast completed
pushes spawned N concurrent full-history walks past GITLAWB_MAX_CONCURRENT_GIT_PUSHES
(which bounds only the in-handler phase), and the per-child bounding from 71389e6
caps each walk's duration but not their count (P1-e).

Add git_encrypt_semaphore, a pool of its own (sized from max_concurrent_git_pushes,
no new knob — Q1) so a long background walk never holds a foreground write slot and
a handler holding a write permit can't self-deadlock. Route the walk through a new
withheld_recipients_gated helper that acquires the pool before the walk and DEFERS
(blocks) when full rather than shedding — dropping the walk would lose the
withheld-blob recovery copy, so durability stays fail-closed.

Regression: encrypt_walk_defers_when_pool_exhausted drives the gated helper with the
pool exhausted and asserts the walk blocks and does not run its rev-list, then runs
once a permit frees. RED before (the walk runs regardless of the pool), GREEN after.
The detached push task calls this exact helper.
2026-07-14 20:28:52 -05:00
t 12643573f6 fix(node): cap concurrent receive-pack pushes per source IP (#174)
The authenticated git-receive-pack POST acquired only the global write semaphore
— no per-source sub-cap, unlike the read and anon-advert pools. Owner enforcement
defaults off, so one host minting disposable did:key identities could open
max_concurrent_git_pushes slow POSTs from a single source IP and 503 every other
source's push; the 600/hour push limiter bounds arrival rate, not in-flight
concurrency (P1-d).

Add git_write_per_caller (a PerCallerConcurrency sized like the advert cap,
max_concurrent_git_pushes/8), and acquire it before the global write permit,
keyed on the resolved source IP via read_caller_key — never the signed DID (a DID
farm defeats a DID key). This required adding the PeerAddr + HeaderMap extractors
to git_receive_pack, which it lacked (without them the key is None and the cap is
inert). Neutralized the shared acquire helper's log wording now that it serves
both read and write paths.

Regression: receive_pack_per_source_write_cap_sheds_capped_source_not_others — a
source at its write sub-cap sheds Overloaded/503 (proving the extractors resolve a
key), while a different source is not shed. RED before (the capped source proceeds
past admission to a git error instead of shedding), GREEN after; the existing
receive-pack advert cap test still passes.
2026-07-14 20:20:49 -05:00
t be0cdd6a9e fix(node): hold upload-pack admission through the walk on disconnect (#174)
On the path-scoped upload-pack path the per-source and global read permits were
handler locals, so a client disconnect dropped the handler future and released
both permits while the awaited spawn_blocking withheld-blob walk kept running
(spawn_blocking can't be cancelled). A permitless caller could disconnect-spam to
exceed the read cap while real git work continued (P1-b).

Move both permits into the blocking task and return them out: on success they
flow back so the serve phase keeps them; on a dropped future the returned tuple
(with the permits) is discarded only when the blocking task completes, so
admission tracks the walk's real duration.

Regression: upload_pack_permit_held_through_walk_after_disconnect drives the
handler to mid-walk (fake git hangs on rev-list), disconnects, and asserts the
global read slot stays held (available_permits == 0). RED before (the slot frees
to 1 the instant the future drops), GREEN after. The test roots the repo store at
its own TempDir so it is isolated per run.
2026-07-14 20:09:40 -05:00
t ac59bd7e01 fix(node): reap the served-git process group on disconnect, not just SIGTERM (#174)
On a client disconnect KillGroupOnDrop sent a lone SIGTERM with no escalation or
reap, so a group member that ignores SIGTERM (a wedged pack-objects, a malicious
helper) survived while the handler released its concurrency permit — letting
disconnect-spam accumulate live git processes and defeat the load shed (P1-c).
The timeout path already ran the full teardown.

Make the disconnect teardown as strong as the timeout path: the guard now owns
the child and, on drop, launches a detached reaper that runs the same
TERM -> grace -> SIGKILL -> reap-the-group sequence (reap_group_on_timeout).
Owning the tokio Child in the reaper keeps it the sole reaper of the leader, so
it never races tokio's own orphan reaper (a raw waitpid(-pgid) would).

Regression: run_git_service_sigkills_a_sigterm_ignoring_child_on_disconnect, RED
before (the descendant survives the lone SIGTERM), GREEN after; all existing
teardown/timeout/disarm tests still pass (the three hand-built-guard tests were
updated to the owned-child shape).

Residual: capacity (the handler's semaphore permit) still releases at disconnect
rather than strictly after the reaper confirms the group dead, so accumulation is
now bounded to ~one teardown window (<=4s) rather than eliminated; gating the
permit on the reap needs threading it through the serve handlers and is left for
a follow-up (it also touches the receive-pack permit lifetime U4 changes).
2026-07-14 19:50:41 -05:00
t 3cc01890fb fix(node): keep the visibility-walk watchdog armed until the child is reaped (#174)
A group member (or the leader itself) that closes stdout before the deadline made
the stdout drain return EOF early, which stood the watchdog down via done_tx
before it could ever fire; child.wait() then blocked on the still-live child,
pinning the walk thread and its read/write permit past
GITLAWB_GIT_SERVICE_TIMEOUT_SECS. This is distinct from the descendant case
already covered: there the leader sleeps until the deadline so the watchdog does
time out; here the drain-EOF races ahead of it.

Stand the watchdog down only once the child has actually terminated, detected
without reaping (waitid + WNOWAIT) so the leader's pid stays unreaped and its
pgid un-recycled until the watchdog finishes its teardown and is joined. Past the
deadline the watchdog still owns the full SIGTERM -> grace -> SIGKILL sequence.

Regression: run_bounded_git_reaps_a_leader_that_closes_stdout_then_hangs, RED
before (blocks on child.wait past the recv budget), GREEN after; the three
existing teardown tests still pass (no recycle-hazard regression).
2026-07-14 19:27:12 -05:00
t a030eefb70 test(node): retry the ETXTBSY exec race in build_filtered_pack timeout tests (#174)
build_filtered_pack_times_out_a_hung_rev_list and _a_hung_pack_objects
spawn a freshly-written fake git directly, bypassing the fake_git_run_with_pids
retry the rest of the module uses. Under cargo test fork-storm load a concurrent
worker forks while the fake's write fd is still open, so execve returns ETXTBSY
and the raw io::Error surfaces where the test asserts GitServiceTimeout, failing
test (stable) intermittently.

Route both tests through build_filtered_pack_or_exec_race_retry, which retries
ONLY errno 26 (ETXTBSY) and keeps the outer 10s watchdog per attempt, so a
genuinely-missing async timeout bound still trips the watchdog and a wrong error
type still fails at the assertion. Production paths unchanged.
2026-07-13 16:01:49 -05:00
t cf97405bc1 test(node): execution-cover the delta-path push-scan timeout wiring (#174)
The finding-2 wiring test covered only list_all_objects (the full-scan fallback).
Add delta_path_exec_fns_time_out_on_a_hung_git covering the common push path:
peeled_object_type (cat-file), rev_list_delta (rev-list), and list_all_objects_with_type
(cat-file, which all_blob_oids delegates to). Each must return within the watchdog
budget on a hung git; reverting any to a bare Command::output() blocks past the recv
budget (RED-verified).
2026-07-13 10:55:57 -05:00
t 4410b834df docs(node): correct concurrency + timeout docs to match the final git routing (#174)
jatmn P2 (INV-24): the config help, .env.example, and README described routing the
handler no longer does.

- The read pool serves upload-pack and the UPLOAD-PACK info/refs advertisement only;
  the anon receive-pack info/refs advertisement draws from its own dedicated pool
  (sized like the write pool but disjoint), not the read pool and not the write pool.
- The write pool is the authenticated push POST only.
- The per-caller read cap is keyed on the resolved source IP, never the DID: a
  signature does not move a caller off it (the docs claimed a per-DID budget, which
  would tell a NATed authenticated client it escapes the shared cap when it does not).
- git_service_timeout_secs now also bounds the push-side pin-candidate discovery
  (rev-list / cat-file), reaped via process-group teardown, alongside the info/refs
  advertisements and the withheld-blob walk.
2026-07-13 10:46:24 -05:00
t 9412aa9c5b fix(node): bound and reap push-candidate git children under the write permit (#174)
Route push_delta's candidate-discovery git children (cat-file / rev-list) through the
bounded run_bounded_git (deadline + process-group SIGKILL teardown) instead of bare
Command::output(). A hung scan no longer pins the git-receive-pack write permit past
the deadline or a client disconnect. Threads git_bin + a shared per-scan deadline
(git_service_timeout_secs) through resolve_candidates_for_push and all_blob_oids.
2026-07-13 10:41:31 -05:00
t 7a6bd359eb fix(node): SIGKILL a SIGTERM-ignoring member of the visibility-walk process group (#174)
jatmn/CodeRabbit P1: the walk watchdog stood down as soon as the group LEADER was
reaped. A leader that exits cleanly on SIGTERM while a background member ignores
SIGTERM and closes its inherited pipes let the main drain unblock, the leader be
reaped, and the watchdog skip its SIGKILL escalation — so the handler returned while
the descendant kept running. jatmn's suggested "continue until ESRCH" could not work:
the main thread reaping the leader is what frees the pgid, so continuing to kill(-pgid)
after that races a recycled group.

Restructure the coordination instead: the main thread now defers reaping the leader
until the watchdog thread returns. The watchdog always escalates through an
unconditional group SIGKILL (no leader-reap short-circuit), and because the leader is
still unreaped while every kill(-pgid) fires, the pgid cannot have been recycled. The
grace before SIGKILL is a fixed budget (only paid on the exceptional timeout path).

Regression run_bounded_git_sigkills_a_sigterm_ignoring_descendant_after_leader_exits:
a leader that exits 0 on SIGTERM spawns an sh -c descendant (its own $$, not a ( )
subshell's parent pid) that ignores SIGTERM and closes its pipes; the descendant must
be dead when the runner returns. RED on the old teardown (survives), GREEN now. The
existing leader-ignores-SIGTERM and hung-walk teardown tests still pass.
2026-07-13 10:11:41 -05:00
t 473924dcfe fix(node): compile the bounded visibility git runner on non-Unix targets (#174)
jatmn P1: run_bounded_git was gated #[cfg(unix)], but its callers
(assert_all_refs_are_commits, blob_paths) and the module are compiled on all
targets, so a non-Unix build (the continue-on-error Windows release target) could
not resolve the function.

Add a #[cfg(not(unix))] twin with the same signature and result semantics. It bounds
a single child (no process-group teardown, which is Unix-only): threads feed stdin
and drain stderr, the main thread drains stdout, and a watchdog thread kills the
child at the deadline. The child is shared with the watchdog behind a mutex the main
thread does not hold while draining, so the watchdog can always acquire it to kill,
and killing closes stdout to unblock the drain. Best-effort (reaps only the direct
child), which is why the hardened group-aware path stays Unix-only.

Verified the twin type-checks by compiling its (portable-std) body on the native
target; the full non-Unix link is CI's Windows job.
2026-07-13 09:49:33 -05:00
t f19f2813a2 fix(node): give the receive-pack advert its own pool so it can't shed authed pushes (#174)
jatmn P1: the anon-reachable GET info/refs?service=git-receive-pack drew from the
same git_write_semaphore as the authenticated git-receive-pack POST. The per-source
advert cap reserves no global slots, so ~8 sources (each within the per-IP push rate
limit) could hold all 32 write permits across acquire_fresh + the advertisement, and
the next authenticated push then 503s at admission. That is an anonymous caller
shedding an authenticated push across the auth boundary (INV-10).

Add a dedicated git_push_advert_semaphore, disjoint from the write pool. Both the
pre-DB early-shed peek and the held acquire for the receive-pack advertisement now
draw from it; the write pool is left exclusively for the POST. An advert flood can at
worst exhaust the advert pool (each source still capped by git_push_advert_per_caller
and the per-IP push rate limiter); the authenticated push pool is untouched.

Test receive_pack_advertisement_draws_from_dedicated_advert_pool proves at the handler
layer that the advert sheds when its own pool is saturated AND survives when the write
pool is saturated (the reservation). Both flip under revert, verified independently for
the pre-DB peek and the held acquire.
2026-07-13 09:44:53 -05:00
t 79b9ee5c89 fix(node): shed the served-git pool before the DB and cap per-source before it (#174)
CodeRabbit review: the held git_permit was acquired at the top of git_info_refs
and git_upload_pack, before the per-source cap, so one source could occupy the
global pool during the DB/visibility window before its sub-cap rejected the
excess. Move the held git_permit to after acquire_read_caller_permit (still
before acquire_fresh/git, so INV-10's bound on the fresh Tigris acquire holds),
and add a cheap pre-DB early shed (available_permits() == 0 -> 503, holds no
permit) so the #62 shed-before-DB property is preserved without holding a permit
across the DB work.

The receive-pack POST is unchanged (owner-only; its top-of-handler git_permit is
deliberate for acquire_write bounding).

Tests: the two #62 shed-before-DB tests now exercise the explicit early check
(load-bearing: disable it and they fall through to the DB); three new
*_per_source_cap_sheds_with_global_capacity tests prove the per-source sub-cap
sheds a capped source even with global capacity free (load-bearing on the caller
cap). Full workspace green.
2026-07-12 14:40:59 -05:00
t 454e59ba53 docs(node): the withheld-walk is bounded by the service timeout, not a fixed const (#174)
Follow-up to threading the configured timeout into the walk: the walk now honors
GITLAWB_GIT_SERVICE_TIMEOUT_SECS on both the serve and replication paths, so the
README no longer says a fixed internal deadline.
2026-07-12 02:12:37 -05:00
t c35452ed41 test(node): vet the withheld-walk bound end to end; use the configured timeout (#174)
Close the reasoned-not-run gaps from the code review by making the walk's git
binary and timeout injectable, then driving the missing branches with a real
handler and a fake git instead of reasoning about them.

- Add state.git_bin and *_bounded variants of the walk entry points taking
  (git_bin, timeout); the served handlers (upload-pack serve, receive-pack
  replication and full-scan and encrypt-pin, and the ipfs gate) now pass the
  operator-configured GITLAWB_GIT_SERVICE_TIMEOUT_SECS, so the whole walk is bounded
  by the same budget as the other served-git ops rather than a fixed constant. The
  git_bin-less wrappers stay for the real-git classification tests.

Newly vetted by execution (not reasoning):
- receive-pack replication path is bounded: replication_withheld_set with an injected
  hung git returns within the budget and fails closed, so it cannot pin the write
  permit git_receive_pack holds across it.
- a hung withheld-blob walk on the upload-pack POST returns 504 (real handler, real
  repo on disk, injected hung git), proving the GitServiceTimeout -> git_service_app_error
  wiring end to end.
- the watchdog status-gate: a child that exits successfully is not reported as a
  timeout even when the watchdog fired (mutation-checked: drop the guard -> RED).
- SIGKILL escalation: a SIGTERM-ignoring child is still reaped via SIGKILL and the
  group is gone; a truly uninterruptible D-state child (unreapable by any signal) is
  the documented residual, matching the async teardown.
- the advertisement per-source cap sizing never derives 0.

Full gitlawb-node suite 515 green.
2026-07-12 02:12:08 -05:00
t e750c3042d fix(review): cap the receive-pack advertisement per source so anon cannot starve the write pool (#174)
U2 moved the receive-pack info/refs advertisement onto git_write_semaphore to keep
an anonymous read flood from starving the push handshake. But the advertisement is
anon-reachable on public repos and holds its write permit across the slow
acquire_fresh Tigris download, and the only per-source brake on it was the push
RATE limiter, not a concurrency cap. So a multi-source flood of receive-pack
advertisements could hold the write pool's slots across those downloads and shed
authenticated pushes (both the advertisement and the owner-gated git-receive-pack
POST draw from the same pool). U2 thus introduced the first anonymous consumer of
the write pool the state doc promised anon could never reach; the plan's residual
note (no worse than the POST) was wrong, because the POST is owner-gated and the
advertisement is not.

Add git_push_advert_per_caller, a per-source concurrency sub-cap on the receive-pack
advertisement keyed on the resolved source IP (the same PerCallerConcurrency
mechanism U1 uses for reads), sized to an eighth of the write pool so a single
source holds at most that share and saturating the pool takes many distinct source
IPs, each also braked by the per-IP push rate limiter. The upload-pack advertisement
keeps its read-pool per-caller cap; the owner-gated POST is unchanged. Correct the
state doc for git_write_semaphore accordingly.

Handler-layer proof: a source at its receive-pack advertisement cap sheds 503 (RED
before the acquisition, 500-not-503), while a different source and the upload-pack
advertisement are unaffected. Full suite 510 green.
2026-07-12 00:56:24 -05:00
t 87a7e40513 fix(review): close the withheld-walk watchdog reused-pgid race (#174)
Code review found run_bounded_git's watchdog could return a spurious 504 and
signal a recycled process group. The watchdog runs off a wall clock on its own
thread; done_tx.send() only fires after child.wait() reaps the leader, so a walk
that finished within microseconds of the deadline took the watchdog's Timeout
branch, discarded a fully-captured successful result, and returned GitServiceTimeout
(a 504 for a walk that actually completed). Worse, the Timeout branch SIGTERMed
-pgid unconditionally after the leader was reaped, so a recycled pgid could be
signalled, the exact hazard smart_http guards via disarm-after-wait.

Set a reaped AtomicBool the instant the main thread reaps the child; the watchdog
checks it before every kill and stands down if the leader is already reaped. Gate
the timeout verdict on !status.success(), so a child that exited on its own is never
reported as a timeout even if the watchdog fired late. Add the survived-SIGKILL warn
smart_http's reap already emits, for operator visibility on a wedged (D-state) git.

The hung-walk test stays green (a killed child exits by signal, not success, so it
still surfaces GitServiceTimeout and reaps the group) and the 32 real-git
classification tests stay green (a fast walk is never spuriously killed).
2026-07-12 00:36:42 -05:00
t ab0ea240fc docs(node): correct the git-service-timeout coverage note (#174)
GITLAWB_GIT_SERVICE_TIMEOUT_SECS bounds the info/refs advertisement too:
smart_http::info_refs drives it through drive_git_child under this timeout, with a
passing test proving the 504. The old note claimed it does not. It also claimed the
withheld-blob path is unbounded; after the blob_paths seam bound (this PR) the walk
is bounded and reaped, by a fixed internal deadline rather than this env var, so the
line now states that precisely instead of overclaiming this setting covers it.
2026-07-12 00:20:07 -05:00
t 71389e621a fix(node): bound the withheld-blob walk at the shared blob_paths seam (#174)
The withheld-blob classification walk (blob_paths) fanned out blocking git children
with no deadline and no process-group teardown: git for-each-ref, git cat-file, git
rev-list, a git ls-tree per commit, and an uncounted git rev-parse (via
store::head_commit). A hung or pathologically slow child pinned the caller's
served-git permit for the whole hang, and on client disconnect the spawn_blocking
task and its git children ran on, orphaned. blob_paths is the shared core of five
callers: the upload-pack serve path (holds a read permit) AND, inside
git_receive_pack, the post-push replication and encrypt-then-pin walks (hold the
write permit U2 reserves for pushes). So the same unbounded walk could pin either
pool, and leaving the write-side twin unbounded would have made U2's reservation a
claim that does not match behavior.

Bound every git child at the blob_paths spawn seam on the blocking side: each child
runs in its own process group with a watchdog thread that SIGTERMs (then SIGKILLs)
the group on one shared deadline spanning the whole walk, and retains admission
until the group is reaped. This is the blocking-side counterpart of
smart_http::drive_git_child (spawn_blocking cannot be cancelled by an async
timeout). blob_paths stays sync, so all five callers keep their signatures and the
32 classification tests are unchanged; because every caller funnels through
blob_paths, one seam bounds both the serve and replication paths. The previously
unbounded store::head_commit child becomes a bounded git rev-parse inside the walk.
A walk that hits its deadline carries GitServiceTimeout, which the serve handler now
maps to 504 rather than a generic 500.

Proof: a fake git that hangs on rev-list makes blob_paths return GitServiceTimeout
within the watchdog budget (not block on the child) and the recorded process-group
leader is reaped, not orphaned; neutralizing the watchdog kill makes it hang past
the budget (RED). The 32 real-git classification tests stay green through the
refactor, including detached-HEAD, non-standard-ref, and deleted-in-history cases.
2026-07-12 00:20:07 -05:00
t e444bc0ae5 fix(node): draw the receive-pack advertisement from the write pool (#174)
git_info_refs acquired git_read_semaphore for BOTH services, so the push handshake
(GET /info/refs?service=git-receive-pack) competed in the global read pool. An
anonymous clone flood could exhaust that pool and shed a legitimate push with 503
during its required advertisement phase, before it ever reached git_write_semaphore
on the POST. The write pool exists precisely so anonymous reads cannot shed an
authenticated push, but only the POST drew from it.

Select the pool by service: the receive-pack advertisement (phase one of a push)
now draws from git_write_semaphore, like the git-receive-pack POST, so a saturated
read pool cannot starve it. The per-IP push_rate_limiter that already brakes the
advertisement stays as the anti-flood control, and the advertisement stays
reader-visible with no new auth requirement. Because the receive-pack branch is now
a write-path op, it no longer consumes a read per-caller slot.

Handler-layer proofs: with the read pool at zero the receive-pack advertisement
survives while the upload-pack advertisement sheds; with the write pool at zero the
receive-pack advertisement sheds while upload-pack is unaffected; and a receive-pack
advertisement from an IP whose read per-caller budget is full still gets through
(mutation-checked, RED when the skip is neutralized).
2026-07-11 23:54:21 -05:00
t 4983b854d1 fix(node): key the per-caller read cap on source IP, not the signed DID (#174)
read_caller_key returned the authenticated DID when a caller signed, dropping the
source-IP key. Public read routes accept any valid did:key via optional_signature
with no admission step, so one host could mint N disposable DIDs and hold
max_concurrent_reads_per_caller slots under each, multiplying its budget N-fold and
filling the global read pool, while the same host unsigned was capped on its IP.

Key the read sub-cap on the resolved source IP for every caller, signed or not,
mirroring the push path's IpRateLimiter which already throttles on source IP for
this exact DID-farm reason. Drops the now-unused caller_did parameter at both call
sites (git_info_refs, git_upload_pack).

Inverts info_refs_per_caller_cap_keys_on_did_not_ip into
info_refs_per_caller_cap_keys_on_ip_not_did: fill one source IP's slot, then two
requests signed under different DIDs from that same IP both shed 503 (farm
defeated), while a signed request from a different IP keeps its own budget. RED on
the DID-keyed tree, GREEN after.
2026-07-11 23:44:13 -05:00
t 362ee3462e fix(node,git): bound the rev-list stage under drive_git_child so a hung enumeration can't pin a git permit (#174) 2026-07-11 13:02:24 -05:00
t 3b1fa54b73 docs(config): add GITLAWB_MAX_CONCURRENT_GIT_OPS to .env.example (#174)
The read-pool knob was referenced by the push and per-caller entries'
comments but had no example line of its own, so operators couldn't
discover it from the template. Add it with the config default (128).
2026-07-10 16:05:29 -05:00
t 5069cd13ca fix(node,concurrency): resolve #174 code-review findings
Review of the served-git concurrency cap found no P0/P1; these are the
verified P2 follow-ups.

config: the max_concurrent_git_ops doc overclaimed that "every capped path
is duration-bounded." The rev-list object enumeration in the withheld-blob
path still runs in an uncancellable spawn_blocking, so a stuck rev-list can
hold its slot until git exits. Scope the guarantee to the streaming stages
and name the residual. Also tighten the fairness claim: the receive-pack
advertisement shares the read pool (a shed advertisement is a cheap retryable
GET); only the push POST is on the isolated write pool.

api/repos: add info_refs_per_caller_cap_keys_on_did_not_ip, the missing
handler proof that a signed caller is keyed by its DID, not its source IP.
Filling the DID slot sheds a request from a free IP; collapsing read_caller_key
to its IP arm turns the assertion green-not-503 (mutation-verified RED).

api/repos: extract acquire_read_caller_permit so both read handlers share one
shed path instead of a duplicated match block.

rate_limit: recover from a poisoned PerCallerConcurrency mutex instead of
panicking. The critical section is pure counter arithmetic and cannot poison
the lock, but a panic there would brick the limiter for every caller.

505 tests pass; clippy -D warnings and fmt clean.

Part of #174.
2026-07-10 13:45:38 -05:00
t 6336657efc docs(node,config): reconcile the concurrency-cap comments with the fix (#62)
The max_concurrent_git_ops and git_service_timeout_secs doc-comments (and .env.example) described the info/refs and withheld-blob paths as unbounded follow-up gaps. Both are now closed (#174): the comments reflect the read/write pool split, the per-caller sub-cap, and that every capped path is duration-bounded with process-group teardown. Verified the pattern-doc pre-ship checklist: every git_permit / write-permit / per-caller site holds only a timeout+teardown git path, and all three size knobs are range(1..=1_048_576).

Closes the #174 work. No behavior change.
2026-07-10 13:20:18 -05:00
t 57c6e01c26 fix(node,git): bound and reap the withheld-blob pack build (#62)
The filtered-pack path ran the whole rev-list + pack-objects build inside a spawn_blocking, so an outer tokio timeout could not cancel the blocking thread and a client disconnect orphaned the git child while the permit freed (jatmn P1b, the second gap path). Split it: rev-list enumeration stays blocking off the runtime (rev_list_keep), but the streaming pack-objects stage now runs under the shared drive_git_child on the async side, so it is duration-bounded (GitServiceTimeout -> 504) and its process group is reaped on disconnect. build_filtered_pack becomes async and takes a git_bin seam + timeout; upload_pack_excluding threads the git_service_timeout through.

A watchdog-bounded test proves a hung pack-objects times out (rev-list fast, pack-objects hangs). The refactor's happy path is covered by the existing filtered-pack correctness and real-git partial-clone/fetch tests, all still green; disconnect/group-teardown is the shared drive_git_child code proven by the run_git_service tests. 504 tests pass.

Part of #174.
2026-07-10 13:16:41 -05:00
t 2c264e08e3 fix(node,git): bound and reap the info/refs advertisement (#62)
info_refs ran a bare Command::output() with no timeout and no process-group teardown, so a hung git pinned its concurrency slot indefinitely and a client disconnect orphaned the child (jatmn P1b). Extract the timeout + process_group(0) + KillGroupOnDrop core from run_git_service into a shared drive_git_child, and route info_refs through it with an injectable git_bin. A hung advertisement now aborts with GitServiceTimeout (mapped to 504); disconnect reaps the group.

run_git_service's teardown tests all pass through the shared core (proving the group teardown info_refs inherits), the real-git filter tests cover the advertisement happy path, and a new watchdog-bounded test proves a hung advertisement times out. 503 tests pass.

Part of #174.
2026-07-10 13:09:34 -05:00
t fc4c206c3a fix(node,git): per-caller concurrency sub-cap on the read pool (#62)
Adds PerCallerConcurrency, a bounded-keyed in-flight limiter (distinct from the request-rate RateLimiter) so no single caller monopolizes the served-git read pool. Each caller (per-DID when signed via optional_signature, else per-source-IP via client_key) may hold at most max_concurrent_reads_per_caller concurrent reads; over that it sheds 503. The key map is self-bounding (a key is dropped when its in-flight count hits zero) with a reject-before-insert max_keys backstop so a key farm can't grow it (INV-15). Applied in git_upload_pack and both info/refs advertisements, acquired after the visibility gate so a denied request never consumes a slot (KTD7).

Primitive unit-tested (cap + self-bounding + reject-before-insert) and mutation-checked. Handler-layer SC2: same-caller sheds while a different caller passes, proven on BOTH git_info_refs and git_upload_pack with independent mutation probes; plus a None-key bypass test. Per-source-IP keying is trust-config dependent, documented on the config knob. 502 tests pass.

Part of #174.
2026-07-10 12:50:46 -05:00
t 2434dde42f fix(node,git): isolate authed pushes in a dedicated write pool (#62)
git-receive-pack now draws from a separate git_write_semaphore (max_concurrent_git_pushes) instead of the shared pool, so a flood of anonymous reads can no longer shed an authenticated push at admission (jatmn P1a). The shared field is renamed git_read_semaphore and continues to gate upload-pack and both info/refs advertisements. The write permit stays above acquire_write so it precedes the Tigris fresh-acquire (INV-10).

Handler-layer tests: write-pool shed (503), and a cross-boundary proof that an exhausted read pool does NOT shed a push; both mutation-checked (routing receive-pack back to the read pool flips each RED). 497 tests pass.

Part of #174.
2026-07-10 12:29:30 -05:00
t db4de844ad feat(node,config): add write-pool and per-caller read-cap knobs (#62)
Add max_concurrent_git_pushes (default 32) and max_concurrent_reads_per_caller (default 16), both clap range(1..=1_048_576) so an oversized value is a clean CLI error, not a Semaphore::new boot panic. The per-caller knob documents that per-source-IP keying is only as granular as GITLAWB_TRUSTED_PROXY. Wiring lands in the following commits; these are the config surface for the #174 concurrency-fairness fix.

Resolves jatmn P1a/P1b groundwork on #174.
2026-07-10 12:18:07 -05:00
t 65ba1374af feat(node,git): cap concurrent served git ops with a 503 load-shed (#62)
PR3 of the #62 served-git hardening stack (timeout #165 and teardown
wiring #150 are merged). A bounded semaphore caps how many upload-pack /
receive-pack / info-refs operations run at once; past the cap a request
is shed with a clean 503 + Retry-After before spawning another git
subprocess, instead of exhausting the PID/thread table. A permit is
acquired at the top of each of the three handlers and held for the whole
op, releasing on return.

The cap is a portable backstop: the compose pids_limit is absent on Fly,
whose 500-connection cap is a different axis. Size --max-concurrent-git-ops
(GITLAWB_MAX_CONCURRENT_GIT_OPS, default 128) below the process budget.
Range 1..=1_048_576 so 0 (shed everything) and an oversized value that
would panic tokio's Semaphore at boot are clean CLI errors.

Known gap, tracked separately: info/refs and the withheld-blob
(upload_pack_excluding) path are not duration-bounded and do not reap
their git child on client disconnect, so a hung git on those two paths
holds its slot until it exits and live git can briefly exceed the cap.
The main pack path (run_git_service) tears its group down on drop.

Tests: Overloaded maps to 503 + Retry-After; the config knob defaults
and rejects out-of-range; git_permit sheds at capacity and releases; and
each of the three endpoints sheds with 503 when the semaphore is
exhausted (load-bearing: drop the permit line and the endpoint test goes
red).
2026-07-10 12:10:30 -05:00
176 changed files with 5992 additions and 96380 deletions

No files matched your search

+15 -19
View File
@@ -8,9 +8,24 @@
# The audit job stays hard-fail: only the advisories explicitly listed below are
# suppressed, so any NEW advisory still breaks CI. Each entry below has no
# available upstream fix; everything fixable is fixed in Cargo.lock instead.
#
# Tracking: the hickory ignores are reachable accepted risk. A scheduled re-check
# (.github/workflows/audit-schedule.yml) runs without these ignores and fails if
# the lockfile moves off the pinned vulnerable versions while the ignores remain.
[advisories]
ignore = [
# hickory-proto 0.25.2 (pulled by libp2p-dns 0.44 -> hickory-resolver 0.25).
# REACHABLE: the node wraps its QUIC transport in the system DNS resolver
# (crates/gitlawb-node/src/p2p/mod.rs:238), so a malicious DNS response can
# trigger these DoS paths. No fix is available: the latest libp2p-dns on
# crates.io is 0.44 and pins hickory 0.25; the fix requires hickory >=0.26.1.
# REMOVE both once libp2p-dns ships a release using hickory 0.26 (verified
# 2026-06-21: cargo update -p hickory-proto locks 0 packages; still 0.25.2).
# Tracking: https://github.com/Gitlawb/node/issues/76
"RUSTSEC-2026-0118", # hickory NSEC3 closest-encloser unbounded loop
"RUSTSEC-2026-0119", # hickory O(n^2) name-compression CPU exhaustion
# rsa 0.9.10 (Marvin timing sidechannel). Present in Cargo.lock because sqlx
# resolves sqlx-mysql unconditionally, and sqlx-mysql depends on rsa. cargo
# audit scans the lockfile, so it flags rsa even though it is NOT linked into
@@ -20,23 +35,4 @@ ignore = [
# ever built with the `mysql` feature, or any other consumer of rsa enters
# the build, at which point it becomes a real reachable advisory.
"RUSTSEC-2023-0071", # rsa Marvin attack (no fix; not linked in our build)
# crypto-bigint 0.5.5 (secret-dependent branches in operations meant
# to be constant-time). REACHABLE accepted risk, on two paths:
# - alloy-signer -> elliptic-curve 0.13 -> crypto-bigint 0.5: the k256
# PrivateKeySigner that signs transactions in `gl name`
# (crates/gl/src/name.rs) and the node operator
# (crates/gitlawb-node/src/operator.rs).
# - aws-sigv4 (`sigv4a` feature, crates/gitlawb-node/Cargo.toml) ->
# crypto-bigint 0.5 (directly and via p256 0.13): SigV4a key
# derivation, used only for requests to endpoints that require SigV4a
# (e.g. S3 Multi-Region Access Points).
# Exploiting it needs local timing observation of those signing operations.
# The fix is crypto-bigint >=0.7.6, and no compatible dependent uses it:
# every alloy-signer release (through 2.5.0) requires elliptic-curve ^0.13,
# and aws-sigv4 1.6.4 is the first release on crypto-bigint 0.7 (1.5.x -
# 1.6.3 still require ^0.5.4), but it needs Rust 1.94.1, above our 1.91 MSRV
# (verified 2026-11-03). REMOVE only when Cargo.lock has no crypto-bigint
# below 0.7.6: `grep -A1 'name = "crypto-bigint"' Cargo.lock`.
"RUSTSEC-2026-0387", # crypto-bigint timing variability (no compatible fix)
]
+12 -156
View File
@@ -1,41 +1,12 @@
# Twigpine node — environment variables
# gitlawb node — environment variables
# Copy to .env and fill in your values.
# All variables are optional unless marked REQUIRED.
# ── Node identity ─────────────────────────────────────────────────────────
# Path to the node's Ed25519 keypair PEM file. Must name a key file, not a
# directory: a trailing slash, a final `.` or `..`, and `..` traversal are
# refused so the path cannot retarget a parent. A `~/` value must stay under
# home after expansion. Bare `identity.pem` is still legal.
# Path to the node's Ed25519 keypair PEM file.
# Generate with: gl identity new
GITLAWB_KEY=/data/keys/identity.pem
# Path to the node's persistent libp2p identity key file. Must include a
# directory, because the node will not keep its p2p identity key in the working
# directory. A value that cannot name a securable key file (a bare filename,
# `..` traversal, a trailing slash, the filesystem root, or a `~/` path that
# escapes home) is refused before the node binds, so the process exits.
# Anything wrong with the storage itself (a symlinked or non-directory parent,
# an unsafe cwd or ancestor, an unreadable or malformed key) leaves p2p off
# while HTTP keeps serving, logged at error as p2p_identity_key_load_failed and
# counted by the gitlawb_p2p_identity_key_load_failed metric. /health still
# reports healthy in that state, so alert on the event or the metric. Fixing
# the storage and restarting restores the same PeerId.
# On Unix it is created 0600 inside a 0700 directory. A key directory that
# grants access beyond its owner is tightened to 0700 on start; one closed too
# far is refused with a chmod 700 remedy rather than widened. On other
# platforms no permissions are enforced.
# If the node logs that it tightened a key directory that was GROUP OR WORLD
# WRITABLE, treat the key that was sitting there as possibly exposed: delete
# it so a fresh identity is generated on the next start. An ordinary 0755
# directory is tightened too, but others could not read a 0600 key or replace
# its directory entry, so do not rotate for that case. A directory that was
# merely closed too far is refused, not tightened, and no key in it was ever
# exposed.
# Keep it on a persistent volume so the PeerId survives redeploys.
# Default: ~/.gitlawb/p2p.key
#GITLAWB_P2P_KEY=/data/keys/p2p.key
# Publicly reachable URL of this node (used in peer announcements)
GITLAWB_PUBLIC_URL=https://your-node.example.com
@@ -54,25 +25,7 @@ DATABASE_URL=postgresql://gitlawb:changeme@localhost:5432/gitlawb
# Maximum connections in the PostgreSQL pool. A cap, not a floor —
# connections open lazily. Size against the DB server's max_connections,
# remembering admin tooling opens its own pool.
GITLAWB_DB_MAX_CONNECTIONS=48
# Maximum connections in the DEDICATED advisory-lock pool, separate from the
# pool above. Every in-flight repo write pins one connection here for its whole
# duration, so this is a hard ceiling on simultaneous writes node-wide: size it
# to expected peak concurrent writers, not small. Keeping it separate is what
# stops a push burst from starving ordinary request handlers. Budget
# (GITLAWB_DB_MAX_CONNECTIONS + this) per node against the server's
# max_connections, times node count, plus admin tooling.
GITLAWB_DB_LOCK_POOL_MAX_CONNECTIONS=40
# Upper bound, in seconds, on any object-storage transfer that runs while a
# per-repo write lock is HELD (the archive download inside acquire_write and the
# upload inside release). These were free before the lock's connection was
# pinned to the guard; now an unbounded stall holds a lock-pool slot, and enough
# stalls deny every write on the node. The bound applies PER SPAN and there are
# two (the acquire-side refresh, which covers the existence check and download
# together, and the release-side upload), so worst-case slot occupancy is about
# twice this value plus the git work between them. Read it together with the pool
# size above and with GITLAWB_GIT_SERVICE_TIMEOUT_SECS.
GITLAWB_LOCK_HELD_TRANSFER_TIMEOUT_SECS=300
GITLAWB_DB_MAX_CONNECTIONS=20
# Seconds a request waits for a pool connection before failing with 503.
GITLAWB_DB_ACQUIRE_TIMEOUT_SECS=5
# Upper bound on each startup connect+migrate attempt, in seconds. Keep it
@@ -132,32 +85,23 @@ GITLAWB_P2P_PORT=7546
# GITLAWB_BOOTSTRAP_DISABLE_SEEDS=true for isolated dev/test networks.
GITLAWB_BOOTSTRAP_PEERS=
GITLAWB_BOOTSTRAP_DISABLE_SEEDS=false
# Require a peer write to prove its DID on every transport: RFC 9421 signatures
# on the announce/sync routes, and an Ed25519 payload signature on gossip
# ref-update events. Keep false during rolling upgrades so existing live nodes
# can still communicate. Upgrade every gossip-publishing peer to a build that
# signs events before setting this true, or their updates are dropped on
# arrival with no error visible on their side.
# Require signatures on peer announce/sync write routes. Keep false during
# rolling upgrades so existing live nodes can still communicate.
GITLAWB_REQUIRE_SIGNED_PEER_WRITES=false
# Require the authenticated pusher to be the repo owner on git-receive-pack.
# A valid did:key signature is authentication, not authorization: anyone can
# mint a key, derive its DID and sign, so with this off every signed caller may
# push to every repository, private ones included. On by default.
#
# Set to false only for a rolling upgrade whose pushers are not yet the repo
# owner. Note that delegated and CI keys count as non-owners: UCAN git/push is
# verified but not yet honored for authorization, so they cannot push while this
# is on. See docs/RUN-A-NODE.md.
GITLAWB_ENFORCE_OWNER_PUSH=true
# sign as their own DID. When true, pushes from a non-owner DID are rejected.
# Keep false until the repo owner is ready for owner-only writes.
GITLAWB_ENFORCE_OWNER_PUSH=false
# Comma-separated libp2p multiaddrs.
# Example: /ip4/1.2.3.4/udp/7546/quic-v1/p2p/12D3KooW...
GITLAWB_P2P_BOOTSTRAP=
# ── Access control ────────────────────────────────────────────────────────
# Reserved and currently inert. Repo reads are gated per repository by
# is_public and path-scoped visibility rules; this flag changes nothing.
# Reserved for private-read mode. Public/private repo read enforcement is not
# wired in the current live release; do not rely on this for private repositories.
GITLAWB_PUBLIC_READ=true
# Maximum git smart-HTTP pack request size, in bytes.
@@ -168,13 +112,7 @@ GITLAWB_MAX_PACK_BYTES=2147483648
# worker and, on push, the repo write lock. Also bounds both info/refs
# advertisements, the withheld-blob pack build, and the push-side candidate
# discovery (rev-list / cat-file), all reaped via process-group teardown (#174).
# On the path-scoped upload-pack path the withheld-blob classification walk and
# the pack serve share ONE deadline, so this value bounds their COMBINED duration
# rather than giving each stage a full budget: a walk that consumes it leaves the
# serve nothing and the clone gets a 504. Serving large path-scoped repos may
# need a higher value here than when each stage was budgeted separately.
# Must be 1..=3153600000 (100 years): the node derives deadlines from this value,
# and a larger one cannot be represented. Default 600.
# Default 600.
GITLAWB_GIT_SERVICE_TIMEOUT_SECS=600
# Max seconds the storage-ACQUISITION phase of a served git op may run before the
@@ -203,25 +141,6 @@ GITLAWB_MAX_CONCURRENT_GIT_OPS=128
# Retry-After. Default 32.
GITLAWB_MAX_CONCURRENT_GIT_PUSHES=32
# Max pushes PARKED at once waiting for the per-repo write lease of the SAME repo.
# A parked push holds its already-buffered pack body in memory, so this bounds that
# memory for a hot repo. Past the cap the newest push sheds a clean 503 +
# Retry-After ("repo is busy with another push") instead of joining the queue; a
# push to a DIFFERENT repo is never affected. The lease holder is not counted.
# Raising GITLAWB_MAX_CONCURRENT_GIT_PUSHES does not raise this; set it explicitly.
# Default 8.
GITLAWB_REPO_LEASE_MAX_WAITERS=8
# Max concurrent post-push pin loops (IPFS + Pinata pin_new_objects) across all
# repos. The per-repo encrypt-task set caps the pin-task COUNT; this caps how many
# of those loops RUN CONCURRENTLY, and so how much MB-scale object-id list memory is
# held WHILE BEING PINNED. It does not bound retained memory in general: on the local
# IPFS path the caller builds its list before this pool is entered, so a loop parked
# waiting for a permit still holds its full list, and how many park is capped only
# per repo. Do not size memory from this knob alone. A loop DEFERS (waits) when the
# pool is full, never drops a pin. Default 8.
GITLAWB_MAX_CONCURRENT_PIN_TASKS=8
# Max concurrent read ops (upload-pack + the upload-pack info/refs advertisement)
# a single caller may hold, so one caller cannot monopolize the read pool. Keyed
# on the resolved SOURCE IP, never the DID: a signature does not move a caller off
@@ -232,24 +151,6 @@ GITLAWB_MAX_CONCURRENT_PIN_TASKS=8
# Default 16.
GITLAWB_MAX_CONCURRENT_READS_PER_CALLER=16
# Two further per-source concurrency caps exist on the PUSH side but have NO
# environment variable of their own: both are DERIVED as
# GITLAWB_MAX_CONCURRENT_GIT_PUSHES / 8, with a floor of 1. They are not settable
# independently, so raising the push pool raises both proportionally, and the
# default push pool of 32 gives each a cap of 4.
# * The anon receive-pack info/refs advertisement cap: one source IP may hold at
# most this many slots in the dedicated advert pool, so saturating that pool
# takes ~8 distinct source IPs.
# * The authenticated receive-pack POST cap: one source IP may hold at most this
# many WRITE-pool slots. This one is load-bearing for push availability, since
# it is acquired before the global write permit: without it, one host minting
# disposable did:key identities could open enough slow pushes to monopolize the
# write pool and 503 every other source (the push rate limiter caps arrival
# rate, not in-flight concurrency, and owner enforcement can be turned off for
# a rolling upgrade).
# Keyed on the resolved source IP, never the DID, so a DID farm does not defeat
# them; keying granularity follows GITLAWB_TRUSTED_PROXY like the read cap above.
# ── Push rate limiting (git-receive-pack flood brake) ─────────────────────
# Max receive-pack requests (info/refs advertisement + push POST) per client
# IP per hour. 0 disables. Default 600.
@@ -270,52 +171,7 @@ GITLAWB_IPFS_WALK_PER_SOURCE=4
# Max legacy (NULL-provenance) repos probed per single /ipfs request, bounding the
# scan-fallback fan-out (git cat-file per candidate repo) for an anonymous caller. A
# truncated scan sheds a retryable 503, never a false 404. Default 256.
GITLAWB_IPFS_MAX_LEGACY_PROBES=256
# Max repo ROWS one /ipfs request's legacy scan may read from the database. The probe
# ceiling above only counts once a probe runs, and quarantined or private repos are
# denied before that, so without this an all-denying inventory paged the whole repo
# table for one anonymous request. A truncated scan sheds a retryable 503 carrying a
# sealed `continuation` token; echo it as ?scan= to resume, so a holder buried past the
# ceiling is reachable in ceil(repos / ceiling) + 1 requests. Raising this also raises
# the per-caller /ipfs work allowance, since each page is charged to it. Lowering it
# sharpens a coarse oracle: laddering to the end reveals the node's total repo count to
# within one ceiling. Default 2048.
GITLAWB_IPFS_MAX_LEGACY_SCAN_ROWS=2048
# Max EXPENSIVE path-scope visibility walks per single /ipfs request (only a
# blob in a path-scoped repo costs a full-history walk). Over-cap repos are
# skipped without a verdict and the scan continues; if the object is then found
# nowhere the request sheds a retryable 503 instead of a false 404. The effective
# cap is the tighter of this value and the node's internal per-request ceiling of
# 17, so values above 17 have no effect; lower values do tighten it. Default 64.
GITLAWB_IPFS_MAX_REPOS_WALKED=64
# Ceiling on repos one /ipfs request may VISIT past the visibility gate. Each
# visit costs a repo acquire — on a Tigris cache miss a full archive download,
# so this is also the worst-case object-store fetch count per request — plus a
# cat-file probe. On exhaustion the scan stops and sheds a retryable 503.
# Default 1024.
GITLAWB_IPFS_MAX_REPO_VISITS=1024
# Absolute wall-clock budget for one admitted /ipfs request's acquire+walk
# lifetime (all stages of the whole scan). Per-stage clamps bound the acquire
# and walk stages to the remaining budget, and no stage starts once it is
# exhausted (the scan then sheds a retryable 503). The object-type probe and
# content-read cat-file subprocesses are budget-checked before starting AND each
# run under their own deadline (the lesser of GITLAWB_GIT_SERVICE_TIMEOUT_SECS and
# the remaining budget), reaped via process-group teardown, so a hung cat-file
# cannot hold the request's walk slot past it. One hang path is still unbounded:
# the probe's object-store readability check is a plain filesystem sweep with
# nothing to reap, so a wedged filesystem can hold the slot past the deadline.
# Must be 1..=3153600000 (100 years): the node derives an Instant deadline from
# this value, and a larger one cannot be represented. Default 600.
GITLAWB_IPFS_REQUEST_BUDGET_SECS=600
# Shorter budget (seconds) for the pre-walk CID resolve: the lookup that maps a
# requested CID to its git oid(s), which runs while the scarce walk admission is
# already held. A well-formed CID with no pin row does no probe and no walk work,
# so without this a stalled lookup could hold a walk slot for the whole request
# budget while nothing walked. The effective deadline is the lesser of this and
# the remaining request budget; walk and probe work stay on the request budget,
# so a slow but progressing scan is never shed by it.
# Must be 1..=3153600000 (100 years). Default 10.
GITLAWB_IPFS_RESOLVE_BUDGET_SECS=10
GITLAWB_IPFS_MAX_REPOS_WALKED=256
# Max /ipfs/{cid} requests per client IP per hour (route flood brake, distinct
# from the concurrency caps above). 0 disables. Default 600.
GITLAWB_IPFS_RATE_LIMIT=600
+1 -1
View File
@@ -1,5 +1,5 @@
name: Bug report
description: Report a defect in a Twigpine crate or node behavior
description: Report a defect in a Gitlawb crate or node behavior
labels: ["bug"]
body:
- type: markdown
+2 -2
View File
@@ -1,8 +1,8 @@
blank_issues_enabled: false
contact_links:
- name: Security vulnerability
url: https://github.com/Twigpine/node/blob/main/SECURITY.md
url: https://github.com/Gitlawb/node/blob/main/SECURITY.md
about: Please report security issues privately per our security policy, not as a public issue.
- name: Questions & discussion
url: https://github.com/Twigpine/node/discussions
url: https://github.com/Gitlawb/node/discussions
about: For usage questions and open-ended discussion.
+1 -1
View File
@@ -1,5 +1,5 @@
<!--
Thanks for contributing to Twigpine. Keep PRs focused on one change.
Thanks for contributing to Gitlawb. Keep PRs focused on one change.
Protocol-level changes (identity, signatures, wire format) should start
as an issue before code — see CONTRIBUTING.md.
-->
-121
View File
@@ -1,121 +0,0 @@
# Tracks main's CI state as a labeled issue: opens (or updates) a ci-main-red
# issue when a push-to-main run of PR Checks ends red, and closes it when a
# subsequent run on main's tip goes green.
#
# Why: two individually-green PRs can break main in combination when the
# second merges on checks that ran against a stale base (semantic merge
# race — #113 + #145 broke test compilation, found only via #229). A red main
# is otherwise silent, and release-please PRs inherit the breakage.
#
# The alert reflects main's CURRENT STATE, not raw CI events: runs whose
# commit is no longer main's tip are ignored (a re-run of an old red run
# after a fix has landed must not cry wolf), and a green run on the tip
# closes any open alert.
name: Main red alert
on:
workflow_run:
workflows: ["PR Checks"]
types: [completed]
branches: [main]
permissions:
issues: write
contents: read # getBranch tip check
# Serialize runs: concurrent completions would race the label/issue lookups
# into duplicates (or 422 on createLabel). queue: max keeps every pending
# completion instead of letting a later (possibly green-skipped) arrival
# displace a pending red alert under burst merges; it is only incompatible
# with cancel-in-progress: true, which we don't use.
concurrency:
group: main-red-alert
cancel-in-progress: false
queue: max
jobs:
track:
name: Track main CI state
# timed_out (the test job runs 45 min against Postgres) and
# startup_failure (malformed workflow) are red too — only cancelled is
# excluded as a deliberate human abort. success runs are needed to close
# the alert once main recovers.
if: >-
github.event.workflow_run.event == 'push' &&
contains(fromJSON('["failure", "timed_out", "startup_failure", "success"]'), github.event.workflow_run.conclusion)
runs-on: ubuntu-latest
timeout-minutes: 5
steps:
- name: Reconcile the broken-main issue with main's current state
uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8.0.0
with:
script: |
const run = context.payload.workflow_run;
const label = 'ci-main-red';
// Freshness gate: only the run for main's current tip may open or
// close anything. Re-runs of superseded commits are events about
// the past, not statements about main's state.
const { data: branch } = await github.rest.repos.getBranch({
...context.repo, branch: 'main',
});
if (branch.commit.sha !== run.head_sha) {
core.info(`run ${run.id} is for ${run.head_sha}, main is at ${branch.commit.sha} — ignoring stale run`);
return;
}
// listForRepo returns PRs carrying the label too — those must
// never be mistaken for the tracking issue.
const open = (await github.rest.issues.listForRepo({
...context.repo, state: 'open', labels: label, per_page: 100,
})).data.filter(item => !item.pull_request);
if (run.conclusion === 'success') {
const body = `Recovered: [PR Checks](${run.html_url}) is green on main at \`${run.head_sha}\`.`;
for (const issue of open) {
await github.rest.issues.createComment({
...context.repo, issue_number: issue.number, body,
});
await github.rest.issues.update({
...context.repo, issue_number: issue.number, state: 'closed', state_reason: 'completed',
});
}
return;
}
const headline = run.head_commit?.message?.split('\n')[0] ?? '';
const body = [
`[PR Checks run](${run.html_url}) concluded \`${run.conclusion}\` on main at \`${run.head_sha}\`.`,
'',
`> ${headline}`,
'',
'Likely a semantic merge race: each merged PR was green on a stale base.',
'Fix main before merging anything else — release-please PRs inherit this breakage.',
'',
"This issue closes automatically when a run on main's tip goes green.",
].join('\n');
try {
await github.rest.issues.getLabel({ ...context.repo, name: label });
} catch (e) {
if (e.status !== 404) throw e;
await github.rest.issues.createLabel({
...context.repo,
name: label,
color: 'b60205',
description: 'CI is failing on main',
});
}
if (open.length > 0) {
await github.rest.issues.createComment({
...context.repo, issue_number: open[0].number, body,
});
} else {
await github.rest.issues.create({
...context.repo,
title: 'CI is red on main',
body,
labels: [label],
});
}
+4 -136
View File
@@ -2,17 +2,6 @@ name: PR Checks
on:
pull_request:
# Run in the merge queue so a PR is re-validated against the true merged
# result before it lands. Guards against semantic merge races where two
# individually-green PRs break main in combination (#113 + #145 → #229).
merge_group:
# Release-branch heads are created and pushed by GITHUB_TOKEN (release-please
# and the lock-sync job), and GitHub's recursion guard means those pushes
# trigger no workflow runs. workflow_dispatch is the documented carve-out that
# GITHUB_TOKEN CAN trigger, so the release flow dispatches this workflow at
# the release branch to attach check runs to its head. Read-only workflow
# (contents: read), so the dispatch surface adds no privilege.
workflow_dispatch:
push:
branches:
- main
@@ -53,36 +42,7 @@ jobs:
run: cargo fmt --all -- --check
- name: cargo clippy
run: cargo clippy --locked --workspace --all-targets -- -D warnings
release-tag-validation:
name: release tag validation
runs-on: ubuntu-latest
timeout-minutes: 5
steps:
- name: Check out repository
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
persist-credentials: false
- name: Test release tag resolver
run: scripts/test-resolve-release-tag.sh
triage-detector:
name: triage detector matrix
runs-on: ubuntu-latest
timeout-minutes: 5
steps:
- name: Check out repository
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
persist-credentials: false
# pr-triage.yml runs on pull_request_target and never checks out PR
# code, so its inline detector can only be exercised here, where the
# proposed workflow body IS the checked-out one.
- name: Test inline-test detector
run: node scripts/test-pr-triage-detect.mjs
run: cargo clippy --workspace --all-targets -- -D warnings
test:
name: test (${{ matrix.toolchain }})
@@ -130,7 +90,7 @@ jobs:
- name: cargo test
env:
DATABASE_URL: postgres://postgres:postgres@localhost:5432/gitlawb_test
run: cargo test --locked --workspace
run: cargo test --workspace
build-release:
name: build --release
@@ -153,7 +113,7 @@ jobs:
key: release
- name: cargo build --release
run: cargo build --locked --release --workspace
run: cargo build --release --workspace
audit:
name: cargo audit
@@ -235,7 +195,7 @@ jobs:
key: msrv
- name: cargo check on MSRV
run: cargo check --locked --workspace --all-targets
run: cargo check --workspace --all-targets
docker-build:
name: Docker build smoke test
@@ -252,95 +212,3 @@ jobs:
- name: Smoke test --version
run: docker run --rm gitlawb-node:ci-test --version
test-windows:
name: test (windows, non-blocking)
runs-on: windows-latest
timeout-minutes: 45
# Non-blocking, mirroring the informational `beta` lane above. We ship `gl`
# and `git-remote-gitlawb` to Windows users (release.yml) with zero Windows
# test coverage, so run their suites here to make Windows-only regressions
# VISIBLE on the PR without gating merge (#228). These two are the client
# CLIs and depend on no database, so no Postgres service is needed (service
# containers do not run on windows-latest regardless). `git` is preinstalled
# on windows-latest, which the real_git_fetch harness relies on.
continue-on-error: true
steps:
- name: Check out repository
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
persist-credentials: false
- name: Set up Rust toolchain
uses: dtolnay/rust-toolchain@b3b07ba8b418998c39fb20f53e8b695cdcc8de1b # stable
with:
toolchain: stable
- name: Cache cargo
uses: Swatinem/rust-cache@98c8021b550208e191a6a3145459bfc9fb29c4c0 # v2.8.0
with:
key: windows
- name: cargo test (shipped Windows crates)
run: cargo test --locked -p gl -p git-remote-gitlawb
# gitlawb-core is embedded by every consumer (gl, git-remote-gitlawb, the node
# daemon), so it must stay lean. This gate fails if gitlawb-core's normal
# (non-dev, non-build) dependency tree gains any crate not on the allowlist in
# ci/gitlawb-core-allowed-deps.txt. Exhaustive-by-construction: a new heavy
# dependency reds CI whether or not anyone thought to ban it, which a denylist
# cannot do. Regen instructions live in the allowlist header.
core-deps-purity:
name: gitlawb-core dependency purity
runs-on: ubuntu-latest
timeout-minutes: 15
steps:
- name: Check out repository
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
persist-credentials: false
- name: Set up Rust toolchain
uses: dtolnay/rust-toolchain@b3b07ba8b418998c39fb20f53e8b695cdcc8de1b # stable
with:
toolchain: stable
- name: Cache cargo
uses: Swatinem/rust-cache@98c8021b550208e191a6a3145459bfc9fb29c4c0 # v2.8.0
with:
key: core-deps-purity
- name: Check gitlawb-core dependency allowlist
run: bash scripts/check-gitlawb-core-deps.sh
# release-please only rewrites a crate's version when the line carries the
# `# x-release-please-version` annotation AND the manifest is listed under
# extra-files in release-please-config.json. Miss either half and the crate
# is skipped silently at release time, freezing at its creation version:
# icaptcha-client was added as a workspace member after the config was last
# written and sat at 0.4.0 through four releases before anyone noticed.
# Exhaustive-by-construction, like the allowlist gate above: the required set
# is derived from `cargo metadata`, so a newly added crate reds CI whether or
# not its author knew this gate existed.
release-versioning:
name: release versioning coverage
runs-on: ubuntu-latest
timeout-minutes: 15
steps:
- name: Check out repository
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
persist-credentials: false
- name: Set up Rust toolchain
uses: dtolnay/rust-toolchain@b3b07ba8b418998c39fb20f53e8b695cdcc8de1b # stable
with:
toolchain: stable
- name: Cache cargo
uses: Swatinem/rust-cache@98c8021b550208e191a6a3145459bfc9fb29c4c0 # v2.8.0
with:
key: release-versioning
- name: Check every workspace crate is release-managed
run: bash scripts/check-release-versioning.sh
+5 -223
View File
@@ -24,7 +24,7 @@ jobs:
timeout-minutes: 10
steps:
- name: Label and guide
uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8.0.0
uses: actions/github-script@f28e40c7f34bde8b3046d885e986cb6290c5673b # v7
with:
script: |
const pr = context.payload.pull_request;
@@ -70,227 +70,9 @@ jobs:
const changedRust = names.some(n => n.startsWith("crates/") && n.endsWith(".rs"));
// A test can live in a tests/ dir, a *_test.rs file, or (the common Rust
// pattern) an inline #[test] / #[cfg(test)] block added inside the source file.
// Async tests use a namespaced attribute (#[tokio::test], #[async_std::test],
// possibly with args like #[tokio::test(flavor = ...)]) rather than the bare
// #[test]. Match only on added lines, with the attribute at the start of the
// line after horizontal whitespace, covering bare, cfg(test), namespaced,
// optional leading ::, and raw-identifier (r#test) forms. No head-file fetch
// or lexer: needs-tests is advisory, not a merge gate.
//
// The final path component may also carry `test` as an underscore-delimited
// segment, so third-party harnesses land without enumerating them one by one
// (#[test_case(1)], #[wasm_bindgen_test]). It is deliberately a segment and
// not a substring: a substring would also match #[contest] / #[latest], and a
// false positive here SUPPRESSES needs-tests rather than adding it. This
// heuristic has to fail loud, because a wrongly labeled PR gets corrected by
// the author while a wrongly cleared one is invisible. That trade costs the
// no-underscore harness names (#[rstest]), which stay unmatched on purpose.
//
// The segment classes are [A-Za-z0-9]+ rather than \w+ on purpose, so that `_`
// is only ever the literal delimiter. \w+ contains `_`, which makes the repeat
// ambiguous and backtracks exponentially: on this workflow's pull_request_target
// trigger f.patch is fork-controlled, so `#[test` followed by a long `_a` run is
// a permissionless stall of the triage job.
// Rust allows whitespace, line breaks, and comments between the
// attribute path and ] or (. A flat regex cannot decide that
// boundary: `//` runs to end of line, and block comments NEST, which
// is beyond regular languages — every enumeration of comment
// spellings so far has either dropped a legal separator (regressing
// real tests into needs-tests) or let unrelated patch records
// complete each other (clearing needs-tests from a testless PR, the
// invisible direction). So the closer is found by a token-aware
// scan instead: it consumes horizontal whitespace, `//`-to-newline,
// and depth-counted `/* */` runs, and accepts only when the first
// real token after the attribute path is `]` or `(`.
//
// Association is positional: the scan starts on the attribute's own
// added line and may continue ONLY through the immediately following
// added lines (a context line, hunk header, or removal breaks the
// chain). Unrelated `]`/`(` lines elsewhere in the patch can never
// complete a path they do not adjoin. The continuation is bounded;
// past the bound the scan gives up and reports NO inline test, which
// fails toward the loud label rather than the silent clearance.
//
// The detector body is fenced by the TRIAGE_DETECTOR markers so
// scripts/test-pr-triage-detect.mjs can extract and run it against
// the committed case matrix. Keep the fenced block self-contained.
// TRIAGE_DETECTOR_BEGIN
const TEST_ATTR_PATH =
"(?:cfg[ \\t]*\\([ \\t]*test[ \\t]*\\)|(?:::[ \\t]*)?(?:[\\w-]+[ \\t]*::[ \\t]*)*(?:r#)?(?:[A-Za-z0-9]+_)*test(?:_[A-Za-z0-9]+)*)";
// Anchored to the start of one added line; group 1 is whatever
// follows the path on that line, handed to the separator scan.
const TEST_ATTR_LINE = new RegExp(
"^[ \\t]*#\\[[ \\t]*" + TEST_ATTR_PATH + "([\\s\\S]*)$"
);
// An attribute split across more added lines than this is not a
// spelling anyone writes; refusing to scan further keeps the walk
// linear in the patch and fails toward the loud label.
const MAX_ATTR_CONTINUATION_LINES = 16;
// segs[0] is the remainder of the attribute's own line; each later
// entry is the text of one immediately following added line. True
// only when the first non-separator token across them is ] or (.
// Single forward pass, no backtracking: fork-controlled input.
function attrCloserFollows(segs) {
let depth = 0; // open block-comment nesting carried across lines
for (const seg of segs) {
let i = 0;
while (i < seg.length) {
if (depth > 0) {
if (seg.startsWith("/*", i)) { depth += 1; i += 2; }
else if (seg.startsWith("*/", i)) { depth -= 1; i += 2; }
else { i += 1; }
continue;
}
const c = seg[i];
if (c === " " || c === "\t") { i += 1; continue; }
if (seg.startsWith("//", i)) break; // comment to end of line
if (seg.startsWith("/*", i)) { depth = 1; i += 2; continue; }
return c === "]" || c === "(";
}
// Line exhausted inside separators; the newline is itself a
// separator, so continue on the next added line.
}
return false; // out of adjoined lines (or bound hit): stay loud
}
// Before the candidate scan, a pre-pass walks added patch lines
// tracking Rust lexical state: raw strings (r#*", br#*", cr#*"),
// block comments (/* */ with nesting), regular strings ("...",
// which can span newlines), and line comments (//). An added line
// that starts inside any of these is skipped as a candidate,
// because a #[test_case] appearing there is lexical non-code, not
// a real test attribute. State is only tracked across consecutive
// added lines and reset at any break (context line, removal,
// hunk header): context lines carry a partial view of the file,
// so a " or r#" or /* there might be inside a string that opened
// before the hunk's visible window, and treating it as an opener
// would mark every later added line as non-code (a false negative
// on a real #[test]). The tradeoff is that a non-code region
// opened by a context line is invisible: an added #[test] inside
// an existing raw string or block comment is scanned as code and
// may match (a false positive, silent direction). This is accepted
// because (1) the old detector had the same behavior, (2) the
// pattern is rare, and (3) needs-tests is advisory, not a merge
// gate. Scanning context lines to close this gap would re-introduce
// the false negative, which affects the far more common case of a
// context line carrying a stray " from a string that opened before
// the visible window. Single forward pass, O(n) in patch length.
function patchAddsInlineTest(patch) {
const lines = patch.split("\n");
const nonCodeStart = new Set();
let rawHashes = -1; // -1 = not in raw string; >=0 = # count
let blockDepth = 0;
let inString = false;
for (let i = 0; i < lines.length; i++) {
const line = lines[i];
if (!line) continue;
const prefix = line[0];
// Only scan added lines. Context and removal lines carry
// a partial view of the file: a " or r#" or /* on a context
// line might be inside a string that opened before the
// hunk's visible window, and treating it as an opener would
// mark every later added line as non-code (a false negative
// on a real #[test]). So lexical state is only tracked across
// consecutive added lines and reset at any break. A non-code
// region opened by a context line is invisible; the candidate
// is still scanned, which fails toward the loud label.
if (prefix !== "+") {
rawHashes = -1; blockDepth = 0; inString = false;
continue;
}
const text = line.slice(1);
if (rawHashes >= 0 || blockDepth > 0 || inString)
nonCodeStart.add(i);
let j = 0;
while (j < text.length) {
if (rawHashes >= 0) {
if (text[j] === '"') {
let h = 1;
while (h <= rawHashes && text[j + h] === "#") h++;
if (h > rawHashes) { rawHashes = -1; j += h; continue; }
}
j += 1; continue;
}
if (blockDepth > 0) {
if (text.startsWith("/*", j)) { blockDepth += 1; j += 2; }
else if (text.startsWith("*/", j)) { blockDepth -= 1; j += 2; }
else { j += 1; }
continue;
}
if (inString) {
if (text[j] === "\\") { j += 2; continue; }
if (text[j] === '"') { inString = false; j += 1; continue; }
j += 1; continue;
}
// Recognize Rust char and byte-char literals ('X', '\X',
// '\u{...}', b'X') so their contents never open string or
// comment state. A char literal has a closing ' within a
// bounded distance; a lifetime or label ('a, 'static,
// 'label:) does not, so it falls through as a regular char.
// This is not a skip-to-next-' rule: the lookahead is 2-4
// chars, or bounded to 20 for \u escapes (at most 6 hex
// digits), never unbounded.
if (text[j] === "'") {
let k = j + 1;
if (text[k] === "\\") {
k += 1;
if (text[k] === "u" && text[k + 1] === "{") {
const uBound = k + 20;
while (k < uBound && text[k] && text[k] !== "}") k++;
if (text[k] === "}") k++;
} else {
k += 1;
}
} else {
k += 1;
}
if (text[k] === "'") { j = k + 1; continue; }
// lifetime or label: just skip the apostrophe
}
if (text[j] === '"') { inString = true; j += 1; continue; }
if (text.startsWith("//", j)) break;
if (text.startsWith("/*", j)) { blockDepth = 1; j += 2; continue; }
const prev = j > 0 ? text[j - 1] : "";
const isIdent = /[A-Za-z0-9_]/.test(prev);
if (!isIdent && text[j] === "r") {
let k = j + 1, h = 0;
while (text[k + h] === "#") h++;
if (text[k + h] === '"') {
rawHashes = h; j = k + h + 1; continue;
}
}
if (!isIdent && (text[j] === "b" || text[j] === "c") && text[j + 1] === "r") {
let k = j + 2, h = 0;
while (text[k + h] === "#") h++;
if (text[k + h] === '"') {
rawHashes = h; j = k + h + 1; continue;
}
}
j += 1;
}
}
for (let i = 0; i < lines.length; i++) {
if (lines[i][0] !== "+") continue;
if (nonCodeStart.has(i)) continue;
const m = TEST_ATTR_LINE.exec(lines[i].slice(1));
if (m === null) continue;
const segs = [m[1]];
for (
let j = i + 1;
j < lines.length &&
lines[j][0] === "+" &&
segs.length <= MAX_ATTR_CONTINUATION_LINES;
j++
) {
segs.push(lines[j].slice(1));
}
if (attrCloserFollows(segs)) return true;
}
return false;
}
// TRIAGE_DETECTOR_END
const addsInlineTest = files.some(
f => f.filename.endsWith(".rs") && !!f.patch && patchAddsInlineTest(f.patch)
);
const addsInlineTest = files.some(f =>
f.filename.endsWith(".rs") && f.patch &&
/^\+.*#\[(test\]|cfg\(test\))/m.test(f.patch));
const touchedTests =
names.some(n => n.includes("/tests/") || n.endsWith("_test.rs")) || addsInlineTest;
if (changedRust && !touchedTests) want.add("needs-tests");
@@ -336,7 +118,7 @@ jobs:
"",
lines,
"",
"See [CONTRIBUTING.md](https://github.com/Twigpine/node/blob/main/CONTRIBUTING.md). Update the PR and these notes will clear automatically.",
"See [CONTRIBUTING.md](https://github.com/Gitlawb/node/blob/main/CONTRIBUTING.md). Update the PR and these notes will clear automatically.",
].join("\n");
await github.rest.issues.createComment({ owner, repo, issue_number: prNumber, body: comment });
+89 -593
View File
@@ -4,31 +4,9 @@ on:
push:
branches:
- main
# Manual backfill: republish an already-cut release to a channel that
# missed it. Fill in only the channel(s) you want — an empty input leaves
# that channel's jobs skipped. npm reuses the existing release's binary
# assets; docker rebuilds the image from the tag and re-tags the manifest.
workflow_dispatch:
inputs:
npm_backfill_tag:
description: "Existing release tag to publish to npm (e.g. v0.6.0)"
required: false
type: string
docker_backfill_tag:
description: "Existing release tag to publish to ghcr (e.g. v0.6.0)"
required: false
type: string
concurrency:
# Not keyed on the ref: two runs racing `imagetools create` on the same
# moving tags could interleave, so runs of this workflow serialize outright.
# GitHub keeps at most one running and one pending run per group; a newer
# queued run replaces the pending one, and cancel-in-progress: false
# protects only the run already executing. So a backfill dispatched while a
# release is running, followed by a push to main, is replaced silently: the
# remedy is to re-dispatch it once the running release completes, which the
# push-only gate on the moving-tag step makes safe in any ordering.
group: release
group: release-${{ github.ref }}
cancel-in-progress: false
# Least privilege by default; every job opts in to exactly what it needs.
@@ -36,21 +14,16 @@ permissions: {}
jobs:
release-please:
# Skipped on manual dispatch: a backfill must never create or advance a
# release PR as a side effect (npm-publish tolerates the skipped need via
# !cancelled(), and every other job gates on release_created == 'true').
if: ${{ github.repository == 'Twigpine/node' && github.event_name != 'workflow_dispatch' }}
if: ${{ github.repository == 'Gitlawb/node' }}
name: Release Please
runs-on: ubuntu-latest
permissions:
contents: write
pull-requests: write
environment: release
outputs:
release_created: ${{ steps.release.outputs.release_created }}
tag_name: ${{ steps.release.outputs.tag_name }}
version: ${{ steps.release.outputs.version }}
pr: ${{ steps.release.outputs.pr }}
steps:
- name: Run release-please
id: release
@@ -58,202 +31,24 @@ jobs:
with:
token: ${{ secrets.GITHUB_TOKEN }}
# The release PR bumps the workspace crate versions in the Cargo.tomls but
# release-please cannot update Cargo.lock, so without this job every release
# merge ships a tag whose lockfile disagrees with its manifests (a --locked
# build from the tag fails, and pr-checks builds --locked). Sync the lock on
# the release branch so the tagged tree is internally consistent.
#
# Runs on every push-triggered release run, not only when release-please
# reports the PR: the `pr` output is set only on runs that CREATE or UPDATE
# the release PR, so gating on it would leave an already-open release PR
# (opened before this job existed, or untouched by a non-releasable push)
# permanently without the sync and without check runs. The resolve step
# discovers any open release-please PR itself and no-ops when there is none.
sync-release-lock:
name: Sync Cargo.lock on the release PR
needs: release-please
if: ${{ !cancelled() && github.repository == 'Twigpine/node' && github.event_name != 'workflow_dispatch' }}
runs-on: ubuntu-latest
permissions:
contents: write
pull-requests: read
# actions: write lets the final step dispatch pr-checks on the release
# branch. Note this grant is repo-wide dispatch rights (GitHub cannot
# scope it to one workflow); acceptable here because reaching this job's
# steps already requires write to main. Needed because every push to the
# release branch (release-please's and the lock sync below) is made with
# GITHUB_TOKEN, which triggers no workflow runs, so without an explicit
# dispatch the release PR's head carries no check runs and the --locked
# gate never validates it.
actions: write
environment: release
steps:
# Prefer the fresh `pr` output when this run just created/updated the
# release PR; otherwise fall back to querying for an open release-please
# branch (prefix match: the branch carries `--components--` suffixes).
- name: Resolve the open release PR branch
id: resolve
env:
GH_TOKEN: ${{ github.token }}
OWNER: ${{ github.repository_owner }}
BRANCH_FROM_OUTPUT: ${{ needs.release-please.outputs.pr && fromJSON(needs.release-please.outputs.pr).headBranchName || '' }}
run: |
branch="$BRANCH_FROM_OUTPUT"
if [ -z "$branch" ]; then
# The branch name alone is not a trust signal. `gh pr list` includes
# fork PRs, and a fork's headRefName is bare (no owner prefix), so a
# prefix match by itself lets any fork nominate the branch this job
# checks out, commits to, and dispatches checks on. Require the head
# repo to be THIS repository and the author to be the release-please
# app, so neither a fork nor a same-repo human branch named
# `release-please--*` can be selected. Then take the highest PR
# number: `[0]` depended on gh's default ordering, which is not a
# documented guarantee, and picking a stable one keeps a second
# (component) release PR from making the choice flap between runs.
branch=$(gh pr list --repo "$GITHUB_REPOSITORY" --state open \
--json headRefName,headRepositoryOwner,number,author \
--jq '[ .[]
| select(.headRepositoryOwner.login == env.OWNER)
| select(.author.login == "app/github-actions")
| select(.headRefName | startswith("release-please--"))
] | max_by(.number).headRefName // empty')
fi
if [ -z "$branch" ]; then
echo "no open release PR; nothing to sync"
fi
echo "branch=$branch" >> "$GITHUB_OUTPUT"
- name: Checkout release branch
if: ${{ steps.resolve.outputs.branch != '' }}
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
ref: ${{ steps.resolve.outputs.branch }}
# Resolve on a declared toolchain instead of whatever Rust the runner
# image happens to ship, so the lock this job writes comes off the same
# stable channel the other cargo jobs use. Not a claim of identical
# resolution: `stable` still moves between runs, and the MSRV gate
# checks this same lock on 1.91 (pr-checks.yml). It only removes the
# runner image as an undeclared input.
- name: Install Rust toolchain
if: ${{ steps.resolve.outputs.branch != '' }}
uses: dtolnay/rust-toolchain@b3b07ba8b418998c39fb20f53e8b695cdcc8de1b # stable
with:
toolchain: stable
- name: Sync workspace crate versions into Cargo.lock
if: ${{ steps.resolve.outputs.branch != '' }}
run: cargo update --workspace
- name: Commit and push when the lock changed
if: ${{ steps.resolve.outputs.branch != '' }}
run: |
if git diff --quiet Cargo.lock; then
echo "Cargo.lock already in sync"
exit 0
fi
git config user.name "github-actions[bot]"
git config user.email "41898282+github-actions[bot]@users.noreply.github.com"
git add Cargo.lock
git commit -m "chore: sync Cargo.lock with the release version bump"
git push
# Every push to the release branch is made with GITHUB_TOKEN (release-
# please's own pushes and the lock sync above), and events created by
# GITHUB_TOKEN start no workflow runs. workflow_dispatch is the documented
# exception GITHUB_TOKEN may trigger, so dispatch pr-checks at the release
# branch to attach check runs to its current head. Unconditional (also on
# the already-in-sync path) because the branch's ORIGINAL head has the
# same no-runs problem. Loud on failure by design: a silent fallback
# would recreate the checkless-head problem, and a job rerun is safe
# (sync no-ops, dispatch retries). Loop-safe: pr-checks is read-only and
# dispatches nothing. Runs after the push ack, so the dispatched ref tip
# is the sync commit. Merge-queue note: this validates the branch tip;
# the enforced merge queue's merge_group run still re-validates the true
# merged result before landing.
- name: Dispatch PR checks on the release branch
if: ${{ steps.resolve.outputs.branch != '' }}
env:
GH_TOKEN: ${{ github.token }}
BRANCH: ${{ steps.resolve.outputs.branch }}
run: |
# Every dispatch failure fails the job, including the 422 a release
# branch cut before pr-checks gained its workflow_dispatch trigger
# returns ("does not have a workflow_dispatch trigger", reported
# against the dispatched ref's copy of the file). An earlier revision
# tolerated exactly that 422 on the theory that release-please would
# rewrite the branch and heal it, but a rewrite only happens when a
# later releasable commit lands: an open release PR nobody touches is
# never rewritten. Tolerating it therefore left the lock-sync commit
# this job had just pushed sitting on the release head with no
# --locked validation, which is the precise state this workflow exists
# to prevent. The tolerance is also no longer needed: once this lands,
# every release branch is cut from a main that already carries the
# trigger, so the only branch that could 422 is one open at merge time
# (there is none), and a job rerun after a manual rebase is safe (the
# sync no-ops, the dispatch retries).
gh api "repos/${GITHUB_REPOSITORY}/actions/workflows/pr-checks.yml/dispatches" \
-f "ref=$BRANCH"
# Each architecture builds NATIVELY (amd64 on ubuntu-latest, arm64 on
# ubuntu-24.04-arm) and pushes by digest; docker-manifest below stitches the
# digests into the tagged multi-arch image. No QEMU: emulating the arm64
# Rust release build wedged the v0.6.0 run for 3+ hours and, via the
# workflow concurrency group, blocked every queued release run behind it.
docker:
name: Build & Push Docker Image (${{ matrix.arch }})
name: Build & Push Docker Image
needs: release-please
# Runs on a fresh release, or on manual dispatch for an existing tag
# (release-please is skipped on dispatch, hence !cancelled()).
if: >-
${{ !cancelled() && (
needs.release-please.outputs.release_created == 'true' ||
(github.event_name == 'workflow_dispatch' && inputs.docker_backfill_tag != '')
) }}
strategy:
fail-fast: false
matrix:
include:
- arch: amd64
os: ubuntu-latest
platform: linux/amd64
- arch: arm64
os: ubuntu-24.04-arm
platform: linux/arm64
runs-on: ${{ matrix.os }}
if: ${{ needs.release-please.outputs.release_created == 'true' }}
runs-on: ubuntu-latest
permissions:
contents: read
packages: write
environment: release
steps:
- name: Check out workflow scripts
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
persist-credentials: false
- name: Resolve release tag
id: rel
env:
DISPATCH_TAG: ${{ inputs.docker_backfill_tag }}
RELEASE_TAG: ${{ needs.release-please.outputs.tag_name }}
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
set -euo pipefail
scripts/resolve-release-tag.sh "${DISPATCH_TAG:-$RELEASE_TAG}"
# ghcr requires a lowercase repository path, and unlike metadata-action,
# buildx's `--output name=` does no lowercasing — a mixed-case owner
# makes the digest push fail with "invalid reference format".
echo "image=ghcr.io/${GITHUB_REPOSITORY,,}" >> "$GITHUB_OUTPUT"
- name: Checkout release tag
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
# Pin the commit the resolver verified; a tag ref re-resolved here
# could have been moved between resolve and checkout.
ref: ${{ steps.rel.outputs.tag_commit || format('refs/tags/{0}', steps.rel.outputs.tag) }}
ref: ${{ needs.release-please.outputs.tag_name }}
persist-credentials: false
- name: Set up QEMU
uses: docker/setup-qemu-action@29109295f81e9208d7d86ff1c6c12d2833863392 # v3.6.0
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@b5ca514318bd6ebac0fb2aedd5d36ec1b5c232a2 # v3.10.0
@@ -264,192 +59,45 @@ jobs:
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Extract metadata
id: meta
uses: docker/metadata-action@902fa8ec7d6ecbf8d84d538b9b233a880e428804 # v5.7.0
with:
images: ghcr.io/${{ github.repository }}
tags: |
type=semver,pattern={{version}},value=${{ needs.release-please.outputs.version }}
type=semver,pattern={{major}}.{{minor}},value=${{ needs.release-please.outputs.version }}
type=raw,value=latest
- name: Build and load locally (smoke)
uses: docker/build-push-action@14487ce63c7a62a4a324b0bfb37086795e31c6c1 # v6.16.0
with:
context: .
platforms: ${{ matrix.platform }}
load: true
tags: gitlawb-node:smoke
cache-from: type=gha,scope=docker-${{ matrix.arch }}
cache-from: type=gha
# Native runner on both arches, so the built image can always execute.
# Assert the released version so a stale artifact fails here, not in prod.
- name: Smoke test
env:
VERSION: ${{ steps.rel.outputs.version }}
run: |
set -euo pipefail
out="$(docker run --rm gitlawb-node:smoke --version)"
echo "$out"
grep -qF "$VERSION" <<<"$out" || {
echo "::error::image --version did not report expected version $VERSION (got: $out)"
exit 1
}
run: docker run --rm gitlawb-node:smoke --version
# Push by digest only — tags are applied once by docker-manifest so a
# half-finished matrix can never publish a partially-tagged image.
# provenance:false keeps each push a plain single-arch manifest, which is
# what imagetools create expects to merge.
- name: Build and push by digest
id: push
- name: Build and push (multi-arch)
uses: docker/build-push-action@14487ce63c7a62a4a324b0bfb37086795e31c6c1 # v6.16.0
with:
context: .
platforms: ${{ matrix.platform }}
provenance: false
outputs: type=image,name=${{ steps.rel.outputs.image }},push-by-digest=true,name-canonical=true,push=true
cache-from: type=gha,scope=docker-${{ matrix.arch }}
cache-to: type=gha,mode=max,scope=docker-${{ matrix.arch }}
- name: Export digest
env:
DIGEST: ${{ steps.push.outputs.digest }}
run: |
set -euo pipefail
mkdir -p /tmp/digests
touch "/tmp/digests/${DIGEST#sha256:}"
- name: Upload digest
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
with:
name: docker-digest-${{ matrix.arch }}
path: /tmp/digests/*
if-no-files-found: error
retention-days: 1
docker-manifest:
name: Publish Docker manifest
needs: [release-please, docker]
if: ${{ !cancelled() && needs.docker.result == 'success' }}
runs-on: ubuntu-latest
permissions:
contents: read
packages: write
environment: release
steps:
- name: Check out workflow scripts
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
persist-credentials: false
- name: Resolve release tag
id: rel
env:
DISPATCH_TAG: ${{ inputs.docker_backfill_tag }}
RELEASE_TAG: ${{ needs.release-please.outputs.tag_name }}
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
set -euo pipefail
scripts/resolve-release-tag.sh "${DISPATCH_TAG:-$RELEASE_TAG}"
# ghcr requires a lowercase repository path, and unlike metadata-action,
# buildx's `--output name=` does no lowercasing, so a mixed-case owner
# makes the digest push fail with "invalid reference format".
echo "image=ghcr.io/${GITHUB_REPOSITORY,,}" >> "$GITHUB_OUTPUT"
- name: Download digests
uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4.3.0
with:
pattern: docker-digest-*
path: /tmp/digests
merge-multiple: true
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@b5ca514318bd6ebac0fb2aedd5d36ec1b5c232a2 # v3.10.0
- name: Log in to GitHub Container Registry
uses: docker/login-action@74a5d142397b4f367a81961eba4e8cd7edddf772 # v3.4.0
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Create and push multi-arch manifest
id: manifest
env:
VERSION: ${{ steps.rel.outputs.version }}
IMAGE: ${{ steps.rel.outputs.image }}
run: |
set -euo pipefail
# The docker matrix pushes one digest per arch leg; anything else is
# a broken set, not a smaller multi-arch image.
count="$(find /tmp/digests -maxdepth 1 -type f | wc -l)"
if [ "$count" -ne 2 ]; then
echo "::error::expected 2 arch digests in /tmp/digests, found $count"
exit 1
fi
digests=""
for f in /tmp/digests/*; do
d="$(basename "$f")"
case "$d" in
*[!0-9a-f]*)
echo "::error::digest filename is not lowercase sha256 hex: $d"
exit 1
;;
esac
if [ "${#d}" -ne 64 ]; then
echo "::error::digest is not 64 hex chars: $d"
exit 1
fi
digests="$digests $IMAGE@sha256:$d"
done
# The immutable tag always publishes. The moving tags are applied by
# the gated step below, which never runs on workflow_dispatch.
# shellcheck disable=SC2086
docker buildx imagetools create -t "$IMAGE:$VERSION" $digests
docker buildx imagetools inspect "$IMAGE:$VERSION"
echo "digests=${digests# }" >> "$GITHUB_OUTPUT"
- name: Move floating tags
if: ${{ github.event_name == 'push' && github.ref == 'refs/heads/main' }}
env:
VERSION: ${{ steps.rel.outputs.version }}
IMAGE: ${{ steps.rel.outputs.image }}
DIGESTS: ${{ steps.manifest.outputs.digests }}
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
set -euo pipefail
MAJOR_MINOR="${VERSION%.*}"
# :latest and :X.Y track the newest release. Only a push to main
# carries a release-please-cut version, and release-please versions
# are monotonic, so on this path the tags only ever advance. A
# workflow_dispatch backfill republishes an existing tag and must
# never move a pointer, so this step skips on dispatch. After a
# backfill that is genuinely newest in its minor line, advance :X.Y
# by hand: docker buildx imagetools create -t "$IMAGE:X.Y" <digests>.
# The event gate alone does not bound a stale run: re-running an
# older push run replays its stored needs outputs under the
# original event and ref, and would move the pointers backward to
# that run's version. Floor against live state instead: this run's
# release already exists by now, so VERSION must be the newest
# published release or the run is stale and the tags stay put. The
# releases list is ordered by creation, not semver, so read every
# page and take the semver maximum, not the first page's.
tags="$(gh api --paginate "repos/$GITHUB_REPOSITORY/releases?per_page=100" \
-q '.[] | select(.draft == false) | .tag_name')" || {
echo "::error::could not list releases to bound the moving tags"
exit 1
}
floor="$(printf '%s\n' "$tags" \
| { grep -E '^v[0-9]+\.[0-9]+\.[0-9]+$' || true; } \
| sed 's/^v//' | sort -V | tail -n 1)"
if [ -n "$floor" ] && [ "$floor" != "$VERSION" ]; then
echo "::error::release v$floor is newer than this run's v$VERSION; refusing to move the moving tags backward (stale re-run?)"
exit 1
fi
# shellcheck disable=SC2086
docker buildx imagetools create -t "$IMAGE:latest" -t "$IMAGE:$MAJOR_MINOR" $DIGESTS
platforms: linux/amd64,linux/arm64
push: true
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
cache-from: type=gha
cache-to: type=gha,mode=max
- name: Release summary
env:
VERSION: ${{ steps.rel.outputs.version }}
TAG: ${{ steps.rel.outputs.tag }}
run: |
{
echo "## Released $TAG"
echo "## Released ${{ needs.release-please.outputs.tag_name }}"
echo
echo "- Image: \`ghcr.io/${GITHUB_REPOSITORY,,}:$VERSION\` (linux/amd64 + linux/arm64)"
echo "- GitHub: https://github.com/$GITHUB_REPOSITORY/releases/tag/$TAG"
echo "- Image: \`ghcr.io/${{ github.repository }}:${{ needs.release-please.outputs.version }}\`"
echo "- GitHub: https://github.com/${{ github.repository }}/releases/tag/${{ needs.release-please.outputs.tag_name }}"
} >> "$GITHUB_STEP_SUMMARY"
release-binaries:
@@ -462,7 +110,6 @@ jobs:
continue-on-error: ${{ startsWith(matrix.target, 'x86_64-pc-windows') }}
permissions:
contents: write
environment: release
strategy:
fail-fast: false
matrix:
@@ -482,27 +129,10 @@ jobs:
- target: x86_64-pc-windows-msvc
os: windows-latest
steps:
- name: Check out workflow scripts
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
persist-credentials: false
- name: Resolve release tag
id: rel
env:
RELEASE_TAG: ${{ needs.release-please.outputs.tag_name }}
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
shell: bash # matrix includes windows-latest, where run: defaults to pwsh
run: |
set -euo pipefail
scripts/resolve-release-tag.sh "$RELEASE_TAG"
- name: Checkout release tag
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
# Pin the commit the resolver verified; a tag ref re-resolved here
# could have been moved between resolve and checkout.
ref: ${{ steps.rel.outputs.tag_commit || format('refs/tags/{0}', steps.rel.outputs.tag) }}
ref: ${{ needs.release-please.outputs.tag_name }}
persist-credentials: false
- name: Set up Rust toolchain
@@ -539,7 +169,7 @@ jobs:
set -euo pipefail
args=""
for b in $BINS; do args="$args -p $b"; done
cargo build --release --locked --target ${{ matrix.target }} $args
cargo build --release --target ${{ matrix.target }} $args
# Run each packaged binary's --version so a broken release artifact fails the
# build instead of shipping. Every target builds on a matching-arch runner,
@@ -548,7 +178,7 @@ jobs:
shell: bash
run: |
set -euo pipefail
VERSION="${{ steps.rel.outputs.version }}"
VERSION="${{ needs.release-please.outputs.version }}"
BIN_DIR="target/${{ matrix.target }}/release"
EXE=""
case "${{ matrix.target }}" in *windows*) EXE=".exe" ;; esac
@@ -568,7 +198,7 @@ jobs:
run: |
set -euo pipefail
TARGET="${{ matrix.target }}"
NAME="gitlawb-node-${{ steps.rel.outputs.version }}-${TARGET}"
NAME="gitlawb-node-${{ needs.release-please.outputs.version }}-${TARGET}"
BIN_DIR="target/${TARGET}/release"
EXE=""
case "$TARGET" in *windows*) EXE=".exe" ;; esac
@@ -609,7 +239,7 @@ jobs:
- name: Attach to release
uses: softprops/action-gh-release@72f2c25fcb47643c292f7107632f7a47c1df5cd8 # v2.3.2
with:
tag_name: ${{ steps.rel.outputs.tag }}
tag_name: ${{ needs.release-please.outputs.tag_name }}
files: |
dist/*.tar.gz
dist/*.zip
@@ -618,76 +248,37 @@ jobs:
npm-publish:
name: Publish to npm
needs: [release-please, release-binaries]
# Runs on a fresh release, or on manual dispatch for an existing tag
# (release-binaries is skipped on dispatch, hence !cancelled()). Publishing
# auth is npm Trusted Publishing (GitHub OIDC) — no NPM_TOKEN. Each
# @gitlawb package must have this repo + workflow (release.yml) configured
# as its trusted publisher on npmjs.com, scoped to the release environment
# so a dispatched copy of this file on another ref cannot mint a token, or
# publish fails loudly here —
# deliberately loud: the silent secret-guard skip is how 0.4.x–0.6.0
# never reached npm.
if: >-
${{ !cancelled() && (
(needs.release-please.outputs.release_created == 'true' && needs.release-binaries.result == 'success') ||
(github.event_name == 'workflow_dispatch' && inputs.npm_backfill_tag != '')
) }}
if: ${{ needs.release-please.outputs.release_created == 'true' }}
runs-on: ubuntu-latest
permissions:
contents: read
id-token: write # npm trusted publishing (OIDC) + provenance
environment: release
id-token: write # npm provenance
steps:
- name: Check out workflow scripts
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
persist-credentials: false
- name: Resolve release tag
id: rel
- name: Guard on secret
id: guard
env:
DISPATCH_TAG: ${{ inputs.npm_backfill_tag }}
RELEASE_TAG: ${{ needs.release-please.outputs.tag_name }}
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
NPM_TOKEN: ${{ secrets.NPM_TOKEN }}
run: |
set -euo pipefail
scripts/resolve-release-tag.sh "${DISPATCH_TAG:-$RELEASE_TAG}"
- name: Checkout release tag
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
# Pin the commit the resolver verified; a tag ref re-resolved here
# could have been moved between resolve and checkout.
ref: ${{ steps.rel.outputs.tag_commit || format('refs/tags/{0}', steps.rel.outputs.tag) }}
persist-credentials: false
# npm >= 11.5.1 performs the OIDC token exchange automatically when the
# package has a trusted publisher configured; older npm silently falls
# back to (absent) token auth and fails confusingly.
- name: Set up Node 24 + OIDC-capable npm
uses: actions/setup-node@a0853c24544627f65ddf259abe73b1d18a591444 # v5.0.0
with:
node-version: "24"
- name: Ensure npm supports trusted publishing
run: |
set -euo pipefail
npm install -g [email protected]
# Assert the pin took, so an accidental unpin or a drift off 11.19.1
# fails loudly here instead of silently running a different npm on
# the OIDC path.
installed="$(npm --version)"
echo "npm installed: $installed"
if [ "$installed" != "11.19.1" ]; then
echo "::error::npm is not the pinned 11.19.1 (got: $installed)"
exit 1
if [ -z "${NPM_TOKEN:-}" ]; then
echo "::warning::NPM_TOKEN is not set — skipping npm publish."
echo "enabled=false" >> "$GITHUB_OUTPUT"
else
echo "enabled=true" >> "$GITHUB_OUTPUT"
fi
- name: Checkout release tag
if: ${{ steps.guard.outputs.enabled == 'true' }}
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
ref: ${{ needs.release-please.outputs.tag_name }}
persist-credentials: false
- name: Lay in release binaries
if: ${{ steps.guard.outputs.enabled == 'true' }}
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
VERSION: ${{ steps.rel.outputs.version }}
TAG: ${{ steps.rel.outputs.tag }}
ASSETS: ${{ steps.rel.outputs.assets }}
VERSION: ${{ needs.release-please.outputs.version }}
TAG: ${{ needs.release-please.outputs.tag_name }}
run: |
set -euo pipefail
# npm platform package -> Rust target triple (unix only; Windows is not
@@ -704,18 +295,8 @@ jobs:
target="${entry#*:}"
archive="gitlawb-node-${VERSION}-${target}.tar.gz"
echo "==> $pkg <- $archive"
# Download by the asset id the resolver captured and uploader-
# checked, not by name: release assets are mutable, and an id can
# only ever point at the exact blob captured at resolve time.
# The map keys are base64-encoded asset names.
want="$(printf '%s' "$archive" | base64 -w0)"
asset_id="$(printf '%s\n' "$ASSETS" | awk -v n="$want" '$1 == n {print $2; exit}')"
if [ -z "$asset_id" ]; then
echo "::error::release $TAG has no asset $archive captured at resolve time"
exit 1
fi
gh api "repos/$GITHUB_REPOSITORY/releases/assets/$asset_id" \
-H 'Accept: application/octet-stream' > "_dl/$archive"
gh release download "$TAG" --repo "$GITHUB_REPOSITORY" \
--pattern "$archive" --dir _dl --clobber
tar -xzf "_dl/$archive" -C _dl
src="_dl/gitlawb-node-${VERSION}-${target}"
cp "$src/gl" "npm/packages/$pkg/gl"
@@ -724,8 +305,9 @@ jobs:
done
- name: Set versions
if: ${{ steps.guard.outputs.enabled == 'true' }}
env:
VERSION: ${{ steps.rel.outputs.version }}
VERSION: ${{ needs.release-please.outputs.version }}
run: |
set -euo pipefail
node -e '
@@ -745,48 +327,24 @@ jobs:
'
- name: Publish
if: ${{ steps.guard.outputs.enabled == 'true' }}
env:
VERSION: ${{ steps.rel.outputs.version }}
NPM_TOKEN: ${{ secrets.NPM_TOKEN }}
VERSION: ${{ needs.release-please.outputs.version }}
run: |
set -euo pipefail
# No token: npm exchanges this job's GitHub OIDC identity with the
# registry (trusted publishing); provenance is attested automatically.
echo "//registry.npmjs.org/:_authToken=${NPM_TOKEN}" > ~/.npmrc
# Platform packages first, then the wrapper (so its optionalDependencies resolve).
# Skip versions already on the registry so a rerun after a partial publish
# is idempotent instead of erroring on the first existing package.
# npm publish moves the latest dist-tag to whatever it publishes, so a
# backfilled older version would hand :latest to stale code. Decide
# the dist-tag per package from that package's own registry latest:
# a partial publish can leave the platform packages ahead of the
# wrapper. E404 means the package has never been published, so this
# release gets latest; any other lookup failure must fail closed
# rather than guess at the dist-tag and risk moving latest backward.
# stderr goes to a file so a warning on a successful lookup cannot
# leak into registry_latest; it is read only when the lookup fails.
npm_err="$(mktemp)"
trap 'rm -f -- "$npm_err"' EXIT
for pkg in gl-darwin-arm64 gl-darwin-x64 gl-linux-arm64 gl-linux-x64 gl; do
name="@gitlawb/$pkg"
if npm view "$name@$VERSION" version >/dev/null 2>&1; then
echo "==> $name@$VERSION already published, skipping"
continue
fi
registry_latest="$(npm view "$name" dist-tags.latest 2>"$npm_err")" || {
if ! grep -q E404 "$npm_err"; then
cat "$npm_err" >&2
echo "::error::npm dist-tags lookup failed for $name; not guessing the dist-tag"
exit 1
fi
registry_latest=""
}
dist_tag="latest"
if [ -n "$registry_latest" ] && \
[ "$VERSION" != "$(printf '%s\n%s\n' "$registry_latest" "$VERSION" | sort -V | tail -1)" ]; then
dist_tag="backfill"
echo "::notice::$VERSION is older than $name@$registry_latest; publishing under dist-tag backfill"
fi
echo "==> npm publish $name@$VERSION (dist-tag $dist_tag)"
npm publish "npm/packages/$pkg" --provenance --access public --tag "$dist_tag"
echo "==> npm publish $name@$VERSION"
npm publish "npm/packages/$pkg" --provenance --access public
done
homebrew-bump:
@@ -796,7 +354,6 @@ jobs:
runs-on: ubuntu-latest
permissions:
contents: read # GITHUB_TOKEN downloads release assets; the tap push uses HOMEBREW_TAP_PAT
environment: release
steps:
- name: Guard on secret
id: guard
@@ -810,27 +367,11 @@ jobs:
echo "enabled=true" >> "$GITHUB_OUTPUT"
fi
- name: Check out workflow scripts
if: ${{ steps.guard.outputs.enabled == 'true' }}
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
persist-credentials: false
- name: Resolve release tag
if: ${{ steps.guard.outputs.enabled == 'true' }}
id: rel
env:
RELEASE_TAG: ${{ needs.release-please.outputs.tag_name }}
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
set -euo pipefail
scripts/resolve-release-tag.sh "$RELEASE_TAG"
- name: Checkout tap repo
if: ${{ steps.guard.outputs.enabled == 'true' }}
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
repository: Twigpine/homebrew-tap
repository: Gitlawb/homebrew-tap
token: ${{ secrets.HOMEBREW_TAP_PAT }}
path: tap
persist-credentials: false
@@ -839,32 +380,15 @@ jobs:
if: ${{ steps.guard.outputs.enabled == 'true' }}
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
VERSION: ${{ steps.rel.outputs.version }}
TAG: ${{ steps.rel.outputs.tag }}
ASSETS: ${{ steps.rel.outputs.assets }}
VERSION: ${{ needs.release-please.outputs.version }}
TAG: ${{ needs.release-please.outputs.tag_name }}
run: |
set -euo pipefail
base="https://github.com/${GITHUB_REPOSITORY}/releases/download/${TAG}"
mkdir -p _sums
# Hash the tarballs themselves, fetched by the asset id the resolver
# captured and uploader-checked. The *.sha256 sidecars are mutable
# release assets too; trusting them would trust the same surface.
sha() {
archive="gitlawb-node-${VERSION}-$1.tar.gz"
want="$(printf '%s' "$archive" | base64 -w0)"
asset_id="$(printf '%s\n' "$ASSETS" | awk -v n="$want" '$1 == n {print $2; exit}')"
if [ -z "$asset_id" ]; then
echo "::error::release $TAG has no asset $archive captured at resolve time" >&2
exit 1
fi
# errexit is off inside the $(...) this runs under, so a failed
# download must exit the function itself instead of hashing a
# truncated file.
gh api "repos/$GITHUB_REPOSITORY/releases/assets/$asset_id" \
-H 'Accept: application/octet-stream' > "_sums/$archive" \
|| exit 1
sha256sum "_sums/$archive" | awk '{print $1}'
}
gh release download "$TAG" --repo "$GITHUB_REPOSITORY" \
--pattern '*.tar.gz.sha256' --dir _sums --clobber
sha() { awk '{print $1}' "_sums/gitlawb-node-${VERSION}-$1.tar.gz.sha256"; }
SHA_MAC_ARM="$(sha aarch64-apple-darwin)"
SHA_MAC_X64="$(sha x86_64-apple-darwin)"
SHA_LNX_ARM="$(sha aarch64-unknown-linux-musl)"
@@ -873,8 +397,8 @@ jobs:
mkdir -p tap/Formula
cat > tap/Formula/gl.rb <<EOF
class Gl < Formula
desc "Twigpine CLI — decentralized git for AI agents and developers"
homepage "https://twigpine.com"
desc "Gitlawb CLI — decentralized git for AI agents and developers"
homepage "https://gitlawb.com"
version "${VERSION}"
license "MIT OR Apache-2.0"
@@ -905,15 +429,6 @@ jobs:
bin.install "git-remote-gitlawb"
end
def caveats
<<~CAVEATS
oh-my-zsh's git plugin aliases gl='git pull', which shadows this
binary in interactive shells. If \`gl\` prints "fatal: not a git
repository", run:
echo 'unalias gl 2>/dev/null' >> ~/.zshrc && source ~/.zshrc
CAVEATS
end
test do
assert_match version.to_s, shell_output("#{bin}/gl --version")
end
@@ -925,7 +440,7 @@ jobs:
working-directory: tap
env:
HOMEBREW_TAP_PAT: ${{ secrets.HOMEBREW_TAP_PAT }}
VERSION: ${{ steps.rel.outputs.version }}
VERSION: ${{ needs.release-please.outputs.version }}
run: |
set -euo pipefail
git config user.name "github-actions[bot]"
@@ -938,7 +453,7 @@ jobs:
fi
git commit -m "gl ${VERSION}"
# Credentials are not persisted in .git/config; supply the token only for the push.
git push "https://x-access-token:${HOMEBREW_TAP_PAT}@github.com/Twigpine/homebrew-tap.git" HEAD:main
git push "https://x-access-token:${HOMEBREW_TAP_PAT}@github.com/Gitlawb/homebrew-tap.git" HEAD:main
web-sync:
name: Sync web (install scripts + version)
@@ -947,7 +462,6 @@ jobs:
runs-on: ubuntu-latest
permissions:
contents: read # GITHUB_TOKEN checks out node; the web PR uses WEB_SYNC_PAT
environment: release
steps:
- name: Guard on secret
id: guard
@@ -961,29 +475,11 @@ jobs:
echo "enabled=true" >> "$GITHUB_OUTPUT"
fi
- name: Check out workflow scripts
if: ${{ steps.guard.outputs.enabled == 'true' }}
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
persist-credentials: false
- name: Resolve release tag
if: ${{ steps.guard.outputs.enabled == 'true' }}
id: rel
env:
RELEASE_TAG: ${{ needs.release-please.outputs.tag_name }}
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
set -euo pipefail
scripts/resolve-release-tag.sh "$RELEASE_TAG"
- name: Checkout node (release tag)
if: ${{ steps.guard.outputs.enabled == 'true' }}
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
# Pin the commit the resolver verified; a tag ref re-resolved here
# could have been moved between resolve and checkout.
ref: ${{ steps.rel.outputs.tag_commit || format('refs/tags/{0}', steps.rel.outputs.tag) }}
ref: ${{ needs.release-please.outputs.tag_name }}
path: node
persist-credentials: false
@@ -991,7 +487,7 @@ jobs:
if: ${{ steps.guard.outputs.enabled == 'true' }}
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
repository: Twigpine/web
repository: Gitlawb/web
token: ${{ secrets.WEB_SYNC_PAT }}
path: web
persist-credentials: false
@@ -1003,8 +499,8 @@ jobs:
if: ${{ steps.guard.outputs.enabled == 'true' }}
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
VERSION: ${{ steps.rel.outputs.version }}
TAG: ${{ steps.rel.outputs.tag }}
VERSION: ${{ needs.release-please.outputs.version }}
TAG: ${{ needs.release-please.outputs.tag_name }}
run: |
set -euo pipefail
gh release view "$TAG" --repo "$GITHUB_REPOSITORY" --json assets \
@@ -1022,8 +518,8 @@ jobs:
if: ${{ steps.guard.outputs.enabled == 'true' }}
env:
GH_TOKEN: ${{ secrets.WEB_SYNC_PAT }}
VERSION: ${{ steps.rel.outputs.version }}
TAG: ${{ steps.rel.outputs.tag }}
VERSION: ${{ needs.release-please.outputs.version }}
TAG: ${{ needs.release-please.outputs.tag_name }}
run: |
set -euo pipefail
# Canonical install scripts (source of truth lives in the node repo).
@@ -1034,7 +530,7 @@ jobs:
# One-time, idempotent drift fixes: drop orphaned binaries, fix stale link.
rm -rf web/public/bin
if [ -f web/public/skill.md ]; then
sed -i 's#github.com/gitlawb/releases#github.com/Twigpine/node/releases#g' web/public/skill.md
sed -i 's#github.com/gitlawb/releases#github.com/Gitlawb/node/releases#g' web/public/skill.md
fi
cd web
@@ -1048,16 +544,16 @@ jobs:
git config user.name "github-actions[bot]"
git config user.email "41898282+github-actions[bot]@users.noreply.github.com"
git checkout -b "$branch"
git commit -m "chore: sync Twigpine ${TAG} (install scripts + version)"
git commit -m "chore: sync gitlawb ${TAG} (install scripts + version)"
# Credentials are not persisted in .git/config; supply the token only for the push.
git push --force "https://x-access-token:${GH_TOKEN}@github.com/Twigpine/web.git" "HEAD:$branch"
gh pr create --repo Twigpine/web --head "$branch" \
--title "Sync Twigpine ${TAG}" \
--body "Automated sync from Twigpine/node ${TAG}: install.sh, install.ps1, public/version.json, and one-time drift fixes (removed orphaned public/bin, fixed skill.md releases link). Review and merge to deploy via Vercel." \
git push --force "https://x-access-token:${GH_TOKEN}@github.com/Gitlawb/web.git" "HEAD:$branch"
gh pr create --repo Gitlawb/web --head "$branch" \
--title "Sync gitlawb ${TAG}" \
--body "Automated sync from Gitlawb/node ${TAG}: install.sh, install.ps1, public/version.json, and one-time drift fixes (removed orphaned public/bin, fixed skill.md releases link). Review and merge to deploy via Vercel." \
|| {
# Only swallow the "PR already exists" case; surface auth/API failures
# (a failed gh pr list must not be misread as "PR exists").
pr_count="$(gh pr list --repo Twigpine/web --head "$branch" --state open --json number --jq 'length')" || exit 1
pr_count="$(gh pr list --repo Gitlawb/web --head "$branch" --state open --json number --jq 'length')" || exit 1
if [ "$pr_count" != "0" ]; then
echo "PR for $branch already exists; branch was updated."
else
+1 -1
View File
@@ -1,3 +1,3 @@
{
".": "0.7.1"
".": "0.5.1"
}
-44
View File
@@ -1,44 +0,0 @@
# Contributor guide for coding agents
This file carries the rules a patch author cannot recover from the code alone, for humans and coding agents alike. For build, test, and PR basics, read `CONTRIBUTING.md` first.
## Authorization invariants (the rules PRs most often break)
Every repo-scoped endpoint must bind the caller to an authorization decision before serving or mutating anything. The `known_ungated` allowlist in `authz_guard::every_repo_scoped_handler_is_gated` is empty on main: `list_webhooks`, `list_replicas`, and `list_protected_branches` are gated (`list_webhooks` with read-visibility then owner; the other two with `authorize_repo_read`). Do not re-open those surfaces as exceptions.
- Owner-only mutations (visibility, webhooks, protected branches, merges) are gated against the repo owner. Three gate forms are in use and all are recognized: webhooks and merges call `require_repo_owner` (`crates/gitlawb-node/src/api/mod.rs`), visibility uses its module-local `require_owner`, and protected branches compare inline with `did_matches`. Use the form the surrounding module already uses.
- Not every write is owner-only, and that is by design: `star_repo`/`unstar_repo` bind to the signer (they still require repo read access), `register_replica`/`unregister_replica` bind to the replica's own DID, the bounty actions (claim, submit, approve, cancel, dispute) have their own multi-party rules, and closing a PR or issue is owner-or-author. Do not add owner gates to these.
- Content-serving reads (blobs, trees, raw files) gate through `authorize_repo_read` with the specific path being served, so a withheld subtree is denied even on an otherwise-public repo; repo-level reads (listings) pass `"/"` by the helper's own contract. The source scans below cannot check this argument's value, so review will.
- A route that reaches repo data through a global id, with no repo in its path, or through an extractor shape the scans do not recognize still needs the same gate; the scans cannot see those.
- Read-surface denials must not reveal existence: a caller who may not read a repo (or a withheld subtree) gets the same 404 as a missing repo, never a 403. (Owner-gated mutations currently return 403 after the repo lookup; do not widen that shape to read paths.)
- Two tests in module `authz_guard` (`crates/gitlawb-node/src/api/mod.rs`) scan handler sources and fail on a repo-scoped handler without a recognized gate: `every_in_scope_mutation_has_its_gate` and `every_repo_scoped_handler_is_gated`. If they fail on your handler, add the gate. The second test's `known_ungated` allowlist is empty; never add an entry to it without an open issue and review.
## Adding or changing routes
- A new gated handler needs tests proving its route-specific gate fires: exercise an unauthorized authenticated caller and an anonymous caller where applicable, assert the exact denial status each way, and assert the body leaks nothing. The non-owner tests in `crates/gitlawb-node/src/test_support.rs` show the shape for owner-gated routes.
- A new mutation handler also gets a row in `every_in_scope_mutation_has_its_gate` naming its gate type; the per-handler row is the point of that test.
- A table-driven deny suite covering every deny-bearing route is in review (#194, #195). If it is present in your checkout, add a probe row there for any new deny-bearing route.
## Verifying signatures and certificates
- Derive the verifying key from something outside the artifact you are checking: the node's own DID, a pinned or configured value, a peers lookup. A key read from a field of the object being verified (`Did::from_str(&cert.node_did)`) only proves the artifact is self-consistent, and identities here are permissionless, so anyone can mint a keypair, name it in the artifact, self-sign, and be told it is valid. A mismatch against that anchor has to fail the result, not log a warning.
- A handler whose correct answer is "valid" needs a positive case driven by a trusted, independently anchored artifact, and a rejection case for a well-formed, fully populated forged artifact. Invalid-only suites miss an implementation that rejects everything; a forged artifact that still returns valid is the regression that matters.
- The fields a signature covers are a versioned format. Adding one invalidates every artifact already signed, so give the payload a version, keep a path that verifies the older form, and add a test that verifies an artifact signed before your change.
## Removing a serving path
When a fix removes a path that used to serve content, the same PR must add a test asserting the removed path now denies. The deletion is what turns that test green; without it the path can quietly come back.
## Database migrations
Schema changes are code-defined in `MIGRATIONS` (`crates/gitlawb-node/src/db/mod.rs`). Always append a new versioned entry; never edit an entry that has merged (operators deploy from main). Test runs build the schema from scratch, so editing an applied version stays green in CI while breaking every existing deployment.
## Client behavior (`gl`, `git-remote-gitlawb`)
When a node denies a request, surface the denial to the user. Never render a denial as an empty list or a silent success; that turns an authorization boundary into invisible data loss.
## Toolchain and CI reality
- MSRV is Rust 1.91 (`rust-version` in the workspace `Cargo.toml`). The MSRV CI job runs `cargo check --workspace --all-targets` on exactly 1.91, so check that an API you rely on is stable there; the full test suite runs on stable.
- The PR workflow runs these jobs on every PR: `cargo fmt --check`, `cargo clippy --workspace --all-targets -- -D warnings`, `cargo test --workspace` (against Postgres), a release build, `cargo audit`, the MSRV check, and a Docker build. The `main` branch ruleset currently requires one approving review but does not require these checks to pass before merge; a failing job should block review approval in practice, but nothing enforces that mechanically today.
- Conventional commit titles (`feat:`, `fix:`, `docs:`, ...) are required by convention and drive releases, but no CI job checks them; a bad title fails review, not the pipeline.
-60
View File
@@ -1,65 +1,5 @@
# Changelog
## [0.7.1](https://github.com/Gitlawb/node/compare/v0.7.0...v0.7.1) (2026-08-03)
### Bug Fixes
* **ci:** make the docker dependency cache work and unbreak Dockerfile.bins ([99b2b9a](https://github.com/Gitlawb/node/commit/99b2b9a16662761a3959a2941798e6823621bda5))
* **ci:** make the docker dependency cache work and unbreak Dockerfile.bins ([a49e04d](https://github.com/Gitlawb/node/commit/a49e04ddcd23f3b666421486ae60c3c4329e2242)), closes [#265](https://github.com/Gitlawb/node/issues/265) [#266](https://github.com/Gitlawb/node/issues/266)
* **db:** pin profile DID matching with lone non-key fixture and boundary matrix ([b4016dc](https://github.com/Gitlawb/node/commit/b4016dce9d1fe450603173f9d92bd880127620fa)), closes [#298](https://github.com/Gitlawb/node/issues/298)
* **node:** add a shared owner/name slug validator ([#272](https://github.com/Gitlawb/node/issues/272)) ([51d82de](https://github.com/Gitlawb/node/commit/51d82ded57a776d9c7e2fe65895efdbf9e227dc8))
* **node:** bound the slug owner by NAME_MAX, not the DID limit ([#272](https://github.com/Gitlawb/node/issues/272)) ([27b0495](https://github.com/Gitlawb/node/commit/27b0495a6f7edeecddbe62b0f075a7e3b1ef5053))
* **node:** fail a sync row terminally when a non-directory holds the owner path ([2e664b7](https://github.com/Gitlawb/node/commit/2e664b7885036ee72ce175af95bb567b18222583))
* **node:** make a stalled sync queue visible, and fix two test claims ([9aa4c17](https://github.com/Gitlawb/node/commit/9aa4c17aefd1885d134b1334b54551366cdd8ef0))
* **node:** move the attempted_at migration to v17, clear of two open branches ([fbc5868](https://github.com/Gitlawb/node/commit/fbc5868bd16946ea184dad26abcdb0e89d68e99a))
* **node:** prove the mirror path is inside repos_dir before running git ([#272](https://github.com/Gitlawb/node/issues/272)) ([3ca9473](https://github.com/Gitlawb/node/commit/3ca94739ed7b90e4c265a0947025dc3b50e7583b))
* **node:** reject a malformed slug at the notify boundary ([#272](https://github.com/Gitlawb/node/issues/272)) ([c587a40](https://github.com/Gitlawb/node/commit/c587a40a508acea9f74c3468d42ff037397c75d0))
* **node:** reset a peer's reachability gate when an announce changes its http_url ([#270](https://github.com/Gitlawb/node/issues/270)) ([1964f88](https://github.com/Gitlawb/node/commit/1964f88378fe6942ffca5836de3a092b5b8f67b4))
* **node:** reset a peer's reachability gate when an announce changes its http_url ([#270](https://github.com/Gitlawb/node/issues/270)) ([e2560ee](https://github.com/Gitlawb/node/commit/e2560ee46c28e4e966d87867a86dc4bad4ca7a3a))
* **node:** stamp every dequeued sync row so a stuck one yields the batch ([6e64d0d](https://github.com/Gitlawb/node/commit/6e64d0d0a7fdd38ef3a56820c1808dd7c9bbd658))
* **node:** stop a peer-supplied repo slug from escaping repos_dir ([#272](https://github.com/Gitlawb/node/issues/272)) ([c70b1df](https://github.com/Gitlawb/node/commit/c70b1df42d5c3f70ec4c272886c85fcc16ed92d1))
* **node:** validate the queued slug before the worker derives a path ([#272](https://github.com/Gitlawb/node/issues/272)) ([9f00e24](https://github.com/Gitlawb/node/commit/9f00e24aec217e4b8256dd437cff4741b1478b96))
* **review:** correct two comment claims and guard two more wrong shapes ([#270](https://github.com/Gitlawb/node/issues/270)) ([811ad50](https://github.com/Gitlawb/node/commit/811ad504bc268c8e9cddac93822adcd00bfe9ff8))
* **review:** stop the dequeue overclaiming, and pin the starvation premise ([457be11](https://github.com/Gitlawb/node/commit/457be11d728a2a34775cf96be2bc5af9a58cb250))
* strict DID matching in get_profile ([acb4714](https://github.com/Gitlawb/node/commit/acb47145beec8ce7b20b7a2479f7a55ed68e2a89))
* strict DID matching in get_profile ([13b565b](https://github.com/Gitlawb/node/commit/13b565bcac09f075a7fd555a6e9acdd27de6c7bb))
* target stored did on profile update aliases ([9836c3a](https://github.com/Gitlawb/node/commit/9836c3a21bc96a2ce95cb1fd12bd84b22142787d))
## [0.7.0](https://github.com/Gitlawb/node/compare/v0.6.0...v0.7.0) (2026-07-22)
### Features
* **gl:** actually verify certificate signatures in gl cert show ([b0a2ed8](https://github.com/Gitlawb/node/commit/b0a2ed830821b5a77b0dafdcf5ce4b1ad904ec25))
* **gl:** doctor warns when a shell alias shadows the gl binary ([0765c42](https://github.com/Gitlawb/node/commit/0765c42feff47b29054b9a277881a7cd5bfd9135))
### Bug Fixes
* **gl:** address review findings — non-fatal DID check, exact-host loopback, detached HEAD, multi-URL remotes, command-aware unalias ([3fa38dc](https://github.com/Gitlawb/node/commit/3fa38dce4e435ef017f607d355266c2892161f8d))
* **gl:** anchor cert --verify to a trusted issuer; pin canonical payload form ([c681af7](https://github.com/Gitlawb/node/commit/c681af70dba3a76b11795d324f2ca0fecf7e42fd))
* **gl:** detect gitlawb remotes beyond origin in gl status ([2d88937](https://github.com/Gitlawb/node/commit/2d88937fbabf21013f28debb4d4f93270a4bb8cd))
* **gl:** doctor treats a reachable local node as configuration, not failure ([94c61f7](https://github.com/Gitlawb/node/commit/94c61f79421fa26ecc1bdf7c255e0c48e66a2048))
* **gl:** make gl init's push guidance match the repo's actual state ([cb42a48](https://github.com/Gitlawb/node/commit/cb42a4830d0d070165e2212f316d009ec716dbee))
* warn about the oh-my-zsh gl alias at install time ([e3df51d](https://github.com/Gitlawb/node/commit/e3df51d76c0275fde33954ff0af0aab95fd0cb7d))
## [0.6.0](https://github.com/Gitlawb/node/compare/v0.5.1...v0.6.0) (2026-07-22)
### Features
* **icaptcha-client:** solve the iCaptcha proof-of-work on answer ([#181](https://github.com/Gitlawb/node/issues/181)) ([c98b503](https://github.com/Gitlawb/node/commit/c98b503f90cade54ce9c588bd2373a4e3486f2fc))
### Bug Fixes
* **core:** honor wildcard capabilities in Ucan::can, matching is_attenuated_by ([#200](https://github.com/Gitlawb/node/issues/200)) ([b6877a6](https://github.com/Gitlawb/node/commit/b6877a65981dbf41c4a7838971eb5c826c7fcaf2))
* **gl:** point doctor's version check at Gitlawb/node, not the frozen Gitlawb/releases repo ([#198](https://github.com/Gitlawb/node/issues/198)) ([98d728c](https://github.com/Gitlawb/node/commit/98d728c325f68f0ee4a62e473dc484b7dd627504))
* **node:** carry full owner DID on ref-update wire event ([#144](https://github.com/Gitlawb/node/issues/144)) ([#145](https://github.com/Gitlawb/node/issues/145)) ([69a62e5](https://github.com/Gitlawb/node/commit/69a62e51c2d2f090f9cae88f4d8ce3b9017b7149))
* **node:** gate /hooks and sibling read surfaces leaking private repo metadata ([#94](https://github.com/Gitlawb/node/issues/94)) ([#113](https://github.com/Gitlawb/node/issues/113)) ([ee8d7c0](https://github.com/Gitlawb/node/commit/ee8d7c03b26a0b678cc90b8801aa340fe81da700))
* strip Cf bidi/format controls in terminal-bound error sanitizers ([#183](https://github.com/Gitlawb/node/issues/183)) ([#188](https://github.com/Gitlawb/node/issues/188)) ([b07d41e](https://github.com/Gitlawb/node/commit/b07d41ea97569d79d3f545ae5d43c373cd6f0f33))
## [0.5.1](https://github.com/Gitlawb/node/compare/v0.5.0...v0.5.1) (2026-07-10)
+6 -9
View File
@@ -1,12 +1,12 @@
# Contributing to Twigpine
# Contributing to gitlawb
Thanks for your interest in contributing. Twigpine is an open protocol — agents and humans welcome.
Thanks for your interest in contributing. gitlawb is an open protocol — agents and humans welcome.
## Quick start for contributors
```sh
# Clone and build
git clone https://github.com/Twigpine/node
git clone https://github.com/Gitlawb/node
cd node
# Build everything
@@ -31,14 +31,12 @@ crates/
├── gitlawb-core/ crypto primitives (DID, CID, HTTP sigs, UCAN, ref certs)
├── gitlawb-node/ axum HTTP server, git smart HTTP, P2P, GraphQL
├── gl/ CLI — identity, repos, MCP server, Base L2 names
├── git-remote-gitlawb/ git remote helper for gitlawb:// URLs
├── gitlawb-attest/ provenance attestations for ref-update certificates
└── icaptcha-client/ client for the iCaptcha proof-of-intelligence service
└── git-remote-gitlawb/ git remote helper for gitlawb:// URLs
docs/ Operator guides
scripts/ Build helpers
```
Smart contracts live in a separate repo: [github.com/Twigpine/contracts](https://github.com/Twigpine/contracts).
Smart contracts live in a separate repo: [github.com/Gitlawb/contracts](https://github.com/Gitlawb/contracts).
## Ways to contribute
@@ -56,7 +54,6 @@ Smart contracts live in a separate repo: [github.com/Twigpine/contracts](https:/
3. **No breaking changes without discussion.** Open an issue first for protocol-level changes.
4. **Conventional commits.** Use `feat:`, `fix:`, `docs:`, `refactor:`, `test:`, `chore:`. Releases are automated by [release-please](https://github.com/googleapis/release-please) — your commit prefixes drive the next version bump.
5. **Format and lint.** Run `cargo fmt --all` and `cargo clippy --workspace --all-targets -- -D warnings` before submitting. CI will reject anything that fails these.
6. **Security-surface rules.** [`AGENTS.md`](AGENTS.md) carries the authorization invariants PRs most often break; it is written for coding agents and humans alike, so read it before touching endpoints, visibility, or migrations.
## What gets merged, what gets closed
@@ -108,7 +105,7 @@ cargo test -p gitlawb-node
- **TypeScript SDK** (`@gitlawb/sdk`) — client library for the HTTP API
- **Python SDK** (`gitlawb`) — for ML/agent pipeline integration
- **UCAN authorization** — add trusted issuer anchoring, capability checks, and authorization-aware revocation
- **UCAN chain validation** — complete the auth middleware
- **Filecoin storage tier** — wire up cold storage deals
- **Documentation** — guides, tutorials, API examples
- **Node operators** — run a public node and report issues
Generated
+552 -1014
View File
File diff suppressed because it is too large. Load diff
+2 -5
View File
@@ -20,8 +20,8 @@ members = [
edition = "2021"
rust-version = "1.91"
license = "MIT OR Apache-2.0"
authors = ["Twigpine contributors"]
repository = "https://github.com/Twigpine/node"
authors = ["gitlawb contributors"]
repository = "https://github.com/gitlawb/node"
[workspace.dependencies]
# async
@@ -53,9 +53,6 @@ chrono = { version = "0.4", features = ["serde"] }
uuid = { version = "1", features = ["v4"] }
# http client
reqwest = { version = "0.12", features = ["blocking", "json", "multipart", "rustls-tls"], default-features = false }
# URL parsing (what reqwest::Url re-exports, so the shared redirect predicate can
# take a parsed URL without pulling reqwest into gitlawb-core)
url = "2"
# HMAC
hmac = "0.12"
+5 -19
View File
@@ -8,33 +8,20 @@ FROM rust:1.91-bookworm AS builder
WORKDIR /build
# Cache dependencies first for faster rebuilds.
# EVERY workspace member's manifest must be copied here, not just the ones this
# image ships: cargo loads the whole workspace before it resolves anything, so a
# missing member's Cargo.toml (or a path dependency pointing at one) aborts the
# layer before a single crate is fetched.
# Cache dependencies first for faster rebuilds
COPY Cargo.toml Cargo.lock ./
COPY crates/gitlawb-core/Cargo.toml crates/gitlawb-core/
COPY crates/gitlawb-node/Cargo.toml crates/gitlawb-node/
COPY crates/gl/Cargo.toml crates/gl/
COPY crates/git-remote-gitlawb/Cargo.toml crates/git-remote-gitlawb/
COPY crates/gitlawb-attest/Cargo.toml crates/gitlawb-attest/
COPY crates/icaptcha-client/Cargo.toml crates/icaptcha-client/
# Fetch deps (this layer caches until Cargo.{toml,lock} change).
# No `|| true`: this layer is expected to succeed, and swallowing its exit code
# is what let it sit dead from the moment gl gained a path dependency on
# icaptcha-client. A failure here should stop the build, not silently turn every
# image build into a cold one.
RUN mkdir -p crates/gitlawb-core/src crates/gitlawb-node/src crates/gl/src \
crates/git-remote-gitlawb/src crates/gitlawb-attest/src crates/icaptcha-client/src && \
# Fetch deps (this layer caches until Cargo.{toml,lock} change)
RUN mkdir -p crates/gitlawb-core/src crates/gitlawb-node/src crates/gl/src crates/git-remote-gitlawb/src && \
echo 'fn main() {}' > crates/gitlawb-node/src/main.rs && \
echo 'fn main() {}' > crates/gl/src/main.rs && \
echo 'fn main() {}' > crates/git-remote-gitlawb/src/main.rs && \
echo '' > crates/gitlawb-core/src/lib.rs && \
echo '' > crates/gitlawb-attest/src/lib.rs && \
echo '' > crates/icaptcha-client/src/lib.rs && \
cargo build --release --locked -p gitlawb-node -p gl -p git-remote-gitlawb
cargo build --release -p gitlawb-node -p gl -p git-remote-gitlawb || true
# Now copy real sources and build for real.
# Force-bump mtimes so cargo's fingerprint check rebuilds — without this,
@@ -47,7 +34,7 @@ RUN find crates -name "*.rs" -exec touch {} + && \
rm -rf target/release/.fingerprint/gitlawb-node-* \
target/release/.fingerprint/gl-* \
target/release/.fingerprint/git-remote-gitlawb-* && \
cargo build --release --locked -p gitlawb-node -p gl -p git-remote-gitlawb && \
cargo build --release -p gitlawb-node -p gl -p git-remote-gitlawb && \
strip target/release/gitlawb-node target/release/gl target/release/git-remote-gitlawb
# ── Runtime stage ───────────────────────────────────────────────────────────
@@ -75,7 +62,6 @@ WORKDIR /data
ENV GITLAWB_REPOS_DIR=/data/repos \
GITLAWB_KEY=/data/keys/identity.pem \
GITLAWB_P2P_KEY=/data/keys/p2p.key \
GITLAWB_HOST=0.0.0.0 \
GITLAWB_PORT=7545 \
GITLAWB_P2P_PORT=7546
+2 -5
View File
@@ -3,10 +3,7 @@
# Used by scripts/build-bins.sh — not used for node deployment.
ARG TARGET=x86_64-unknown-linux-musl
# Must stay at or above the workspace `rust-version` in Cargo.toml. An unmet
# MSRV is a hard cargo error, and nothing in CI builds this file, so a stale
# base image here fails only when someone runs scripts/build-bins.sh.
FROM rust:1.91-bookworm AS builder
FROM rust:1.85-bookworm AS builder
ARG TARGET
WORKDIR /build
@@ -26,7 +23,7 @@ COPY crates/ crates/
# Configure cross-linker for aarch64
ENV CARGO_TARGET_AARCH64_UNKNOWN_LINUX_MUSL_LINKER=aarch64-linux-gnu-gcc
RUN cargo build --release --locked --target "$TARGET" -p gl -p git-remote-gitlawb
RUN cargo build --release --target "$TARGET" -p gl -p git-remote-gitlawb
FROM debian:bookworm-slim
ARG TARGET
+1 -1
View File
@@ -169,7 +169,7 @@
END OF TERMS AND CONDITIONS
Copyright 2026 Twigpine contributors
Copyright 2026 gitlawb contributors
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
+1 -1
View File
@@ -1,6 +1,6 @@
MIT License
Copyright (c) 2026 Twigpine contributors
Copyright (c) 2026 gitlawb contributors
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
+41 -133
View File
@@ -1,10 +1,10 @@
# Twigpine Node
# Gitlawb Node
**Decentralized git infrastructure for developers, AI agents, and app delivery.**
Twigpine Node is the open-source node software behind the Twigpine network. It lets anyone run a self-hosted node, publish repositories under a DID, sign writes with Ed25519 HTTP signatures, replicate git activity across peers, and move toward a resilient app-delivery network where code and build assets can be served closer to users.
Gitlawb Node is the open-source node software behind the Gitlawb network. It lets anyone run a self-hosted node, publish repositories under a DID, sign writes with Ed25519 HTTP signatures, replicate git activity across peers, and move toward a resilient app-delivery network where code and build assets can be served closer to users.
Twigpine is not trying to be only "another git host." The long-term direction is:
Gitlawb is not trying to be only “another git host.” The long-term direction is:
```txt
Decentralized GitHub
@@ -19,35 +19,33 @@ The mission is simple: once code is pushed to the network, it should not disappe
## What is in this repository?
This is a Rust workspace with six crates:
This is a Rust workspace with four crates:
| Crate | Purpose |
|---|---|
| `gitlawb-node` | The node daemon: Axum HTTP server, git smart-HTTP, Postgres metadata, libp2p gossip, optional S3/Tigris/IPFS/Arweave/Base PoS hooks. |
| `gl` | The Twigpine CLI for identity, repos, issues, PRs, bounties, tasks, peers, node status, MCP, and setup flows. |
| `git-remote-gitlawb` | Git remote helper for `gitlawb://` URLs, so normal `git clone`, `git fetch`, and `git push` can talk to Twigpine nodes. |
| `gl` | The Gitlawb CLI for identity, repos, issues, PRs, bounties, tasks, peers, node status, MCP, and setup flows. |
| `git-remote-gitlawb` | Git remote helper for `gitlawb://` URLs, so normal `git clone`, `git fetch`, and `git push` can talk to Gitlawb nodes. |
| `gitlawb-core` | Shared primitives: Ed25519 identities, `did:key`, CIDs, RFC 9421 HTTP signatures, certificates, and UCAN tokens. |
| `gitlawb-attest` | Pluggable external provenance attestations for ref-update certificates. |
| `icaptcha-client` | Client for the iCaptcha proof-of-intelligence flow used to protect spam-prone writes. |
---
## Why Twigpine Nodes?
## Why Gitlawb Nodes?
Most git hosting today depends on a small number of centralized platforms. Twigpine Nodes are designed for a different model:
Most git hosting today depends on a small number of centralized platforms. Gitlawb Nodes are designed for a different model:
- **Own your identity**: every user, agent, and node is an Ed25519 keypair represented as `did:key:z6Mk...`.
- **Signed writes by default**: write requests use RFC 9421 HTTP Signatures instead of passwords.
- **Git-native transport**: repositories are still real git repositories served over smart HTTP.
- **Agent-native workflows**: the `gl` CLI and MCP server expose repo, issue, task, PR, and UCAN flows to AI agents.
- **Peer-aware delivery**: nodes can announce, discover, gossip, and sync with each other.
- **App CDN direction**: the network can evolve from decentralized code storage into code + asset + app delivery.
- **Own your identity** — every user, agent, and node is an Ed25519 keypair represented as `did:key:z6Mk...`.
- **Signed writes by default** — write requests use RFC 9421 HTTP Signatures instead of passwords.
- **Git-native transport** — repositories are still real git repositories served over smart HTTP.
- **Agent-native workflows** — the `gl` CLI and MCP server expose repo, issue, task, PR, and UCAN flows to AI agents.
- **Peer-aware delivery** — nodes can announce, discover, gossip, and sync with each other.
- **App CDN direction** — the network can evolve from decentralized code storage into code + asset + app delivery.
---
## Current status
Twigpine Node is live early infrastructure. It is useful today, but some security and reliability features are intentionally staged for compatibility with existing nodes.
Gitlawb Node is live early infrastructure. It is useful today, but some security and reliability features are intentionally staged for compatibility with existing nodes.
Good today:
@@ -56,7 +54,6 @@ Good today:
- Bare git repository storage.
- Git smart-HTTP clone/fetch/push.
- RFC 9421-signed writes.
- Repository and path-scoped visibility enforcement for repository and Git content reads, with 404-shaped repository denials.
- DID identities.
- `gl` CLI workflows.
- libp2p peer discovery/gossip foundation.
@@ -66,13 +63,11 @@ Good today:
Known limitations:
- Repository write authorization is secure by default: `GITLAWB_ENFORCE_OWNER_PUSH` defaults to `true`, so a valid HTTP Signature identifies a pusher and the node additionally requires that pusher to be the repository owner. Set `GITLAWB_ENFORCE_OWNER_PUSH=false` only for the duration of a rolling upgrade whose pushers are not yet the repo owner. Authentication is not authorization on its own (anyone can mint a `did:key` and sign), so owner enforcement is the only thing that prevents every signed caller from pushing to every repository, private ones included.
- UCAN proof chains are validated when supplied, but UCAN capabilities are not consulted by write authorization and the root issuer is not independently trust-anchored. UCANs therefore do not yet grant scoped collaborator access.
- Agent lifecycle revocation is not enforced by HTTP Signature authorization; do not rely on removing or revoking an agent record to block a compromised signer.
- Read visibility is not a blanket data-classification boundary: task, IPFS-pin, and Arweave-anchor listings are not repository-gated; withheld path names can be visible to a root reader; and later visibility changes cannot retract content already announced or externally anchored.
- Private repository read enforcement is not wired yet. Treat public nodes as public infrastructure unless you restrict access at your proxy/firewall.
- UCAN chain validation and revocation are not complete.
- Repository write authorization is not capability-complete yet; HTTP signatures prove identity, not full authorization policy.
- Peer writes are signed by upgraded nodes, but strict signed-peer enforcement is opt-in during rolling upgrades.
- GraphQL mutations require a signed identity (a verified signer bound to the acting DID), are limited to the agent-task queue, and are covered by a CI completeness fence that requires signer acquisition and actor binding on every registered mutation; repo writes are not exposed over GraphQL and remain REST-only.
- Pull-request review comments do not yet have threaded line-level anchors, and merges do not enforce approval requirements.
- GraphQL mutations need mutation-aware auth before becoming a public write surface.
See:
@@ -87,7 +82,7 @@ See:
The fastest path is Docker Compose. It starts a node and Postgres.
```bash
git clone https://github.com/Twigpine/node.git
git clone https://github.com/Gitlawb/node.git
cd node
cp .env.example .env
docker compose up -d
@@ -132,10 +127,10 @@ npm install -g @gitlawb/gl
brew install gitlawb/tap/gl
# curl (macOS / Linux)
curl -fsSL https://twigpine.com/install.sh | sh
curl -fsSL https://gitlawb.com/install.sh | sh
# PowerShell (Windows)
irm https://twigpine.com/install.ps1 | iex
irm https://gitlawb.com/install.ps1 | iex
```
Or build from source:
@@ -178,7 +173,7 @@ gl register --node http://localhost:7545
Create a repo:
```bash
gl repo create my-repo --description "My first Twigpine repo" --node http://localhost:7545
gl repo create my-repo --description "My first Gitlawb repo" --node http://localhost:7545
```
Use the git remote helper:
@@ -192,21 +187,21 @@ For public-network use, make sure `GITLAWB_NODE` points to the node you want. Th
### Full lifecycle against an iCaptcha-enforcing node
Public nodes (e.g. `node.twigpine.com`) require two things on writes:
Public nodes (e.g. `node.gitlawb.com`) require two things on writes:
1. **RFC 9421 HTTP Signatures**: every write is signed by your identity key. `gl`
1. **RFC 9421 HTTP Signatures** — every write is signed by your identity key. `gl`
and the `git-remote-gitlawb` helper do this automatically. An old/unsigned CLI
fails with `401 not_an_agent`; `gl` will tell you to upgrade and register.
2. **An iCaptcha proof** on the spam-gated writes (**repo create, fork, register**).
`gl` solves this for you: on the node's `403 icaptcha_proof_required` it reads the
`x-icaptcha-url` / `x-icaptcha-level` hints, requests a challenge, solves it
locally (arithmetic / algebra / sequence), and **retries the same signed request**
with the `x-icaptcha-proof` header. No manual steps, no env vars.
with the `x-icaptcha-proof` header — no manual steps, no env vars.
```bash
gl identity new # create did:key identity
gl register --node https://node.twigpine.com # signed + auto-solves iCaptcha
gl repo create memlawb --node https://node.twigpine.com # signed + auto-solves iCaptcha
gl register --node https://node.gitlawb.com # signed + auto-solves iCaptcha
gl repo create memlawb --node https://node.gitlawb.com # signed + auto-solves iCaptcha
git push origin2 main # origin2 = gitlawb://<your-did>/memlawb (signed)
git clone gitlawb://<your-did>/memlawb # public read, no proof needed
gl doctor # preflight: identity, node, version, iCaptcha
@@ -220,61 +215,14 @@ Notes:
- **Proofs are short-lived (~5 min TTL) and single-use.** If one expires between
solving and use, the client transparently solves a fresh one and retries.
- **What needs what:** create / fork / register are signed **and** iCaptcha-gated;
`git push` is **signed-only** (owner signature is the gate, no per-push challenge);
`git push` is **signed-only** (owner signature is the gate — no per-push challenge);
reads (clone / fetch / `repo info`) need no proof. A non-existent repo returns a
clear `404`, never a placeholder.
- **API-key iCaptcha deployments:** set `GITLAWB_ICAPTCHA_URL` to your iCaptcha
origin and `GITLAWB_ICAPTCHA_API_KEY` to its key. The client only talks to an
`https` origin whose host is allowlisted (that URL or the public default), and
sends the bearer token **only** to your configured origin, never to a URL a
node advertises, so a hostile node can't capture the key or redirect the solve.
### Fetching an object by CID
```bash
gl ipfs list # CIDs this node has pinned
gl ipfs get bafkrei... > object.bin # object bytes on stdout
```
Objects that were pinned before the node started recording which repo they came
from are found by scanning its repo inventory, and that scan stops at the
per-request ceilings in the [Configuration](#configuration) table. A stopped scan
answers 503 with a resume token instead of a false "not found", and `gl ipfs get`
follows the token automatically: up to 8 resumes after the first request, so at
most 9 calls to the node, waiting between attempts for as long as the node's
`Retry-After` asks and never longer than 5 seconds.
The whole ladder runs under a 60 second wall-clock deadline. The deadline bounds
the search, not the download: each attempt gets the time left on it to produce
response headers, and once an object is found its bytes stream outside that
deadline. They are not unbounded, though. The client's own 30 second HTTP timeout
is a total request timeout, running from the start of a request until its body
has finished, so a transfer still going 30 seconds after its request began is cut
off. Waits between attempts never run past the deadline either, so a single run
spends at most around 90 seconds on the network: the deadline plus the 30 second
timeout covering the last attempt. Writing the object out sits outside both
bounds, so piping into a reader that stops reading can hold the command open
longer than that.
Two node-side brakes end a ladder early and are reported rather than retried
around. A 429 is terminal, because the node's rate-limit window is an hour and
that wait cannot be honored inside one invocation; a transient overload (a 503
carrying no incomplete-scan code) is retried on the token already held, under the
same cap, clamp and deadline. The per-IP fanout brake can also stop a ladder well
short of the 9 calls, so automatic resumption is not a guarantee of reaching the
object.
When a bound stops the ladder with a usable token in hand, the command prints the
token and the invocation that continues from it before exiting nonzero:
```txt
resume from where this stopped: gl ipfs get bafkrei... --scan <token>
```
Run that to carry on from where the scan stopped. Re-running without `--scan`
restarts at the first row, reproduces the same truncation and spends the node's
per-IP budget again, so the token is the only thing that makes progress. Tokens
are valid for an hour.
sends the bearer token **only** to your configured origin — never to a URL a
node advertises — so a hostile node can't capture the key or redirect the solve.
---
@@ -361,7 +309,7 @@ POST /api/v1/peers/announce
POST /api/v1/sync/notify
```
When `GITLAWB_REQUIRE_SIGNED_PEER_WRITES=false`, unsigned legacy peers are accepted on those two routes, but signed requests are verified when signature headers are present. Staged rollout relaxes who may announce, not who owns a peer row. An unsigned announce may register a previously unseen peer, and it may refresh an existing row whose `http_url` is unchanged, but changing an existing peer's `http_url` requires an RFC 9421 signature from that peer's own DID and is refused with 403 otherwise. An unsigned announce can only register a `did:key`, since that is the only method whose verifying key can be resolved and therefore the only kind of row anyone could ever correct through the signed path; any other DID is refused with 400. Once all live peers upgrade, operators can set:
When `GITLAWB_REQUIRE_SIGNED_PEER_WRITES=false`, unsigned legacy peers are accepted on those two routes, but signed requests are verified when signature headers are present. Once all live peers upgrade, operators can set:
```bash
GITLAWB_REQUIRE_SIGNED_PEER_WRITES=true
@@ -369,12 +317,6 @@ GITLAWB_REQUIRE_SIGNED_PEER_WRITES=true
`POST /api/v1/sync/trigger` is not part of the staged rollout: it always requires a signature in both config modes and returns 401 without one, because each call drives an O(peers) outbound fan-out.
The flag is not HTTP-only. It also gates inbound gossip ref-update events on the libp2p mesh, which carry an Ed25519 payload signature rather than RFC 9421 headers because there are no HTTP headers to sign. Set to `true`, an unsigned gossip event is dropped. Set to `false`, an unsigned event is accepted with a warning during the rolling-upgrade window, and the write budget it consumes is charged to the forwarding peer rather than to the DID it claims, since an unsigned event's claimed DID is asserted and not proven. A present-but-invalid signature is refused in both modes; that is forgery, not an un-upgraded peer.
Because the flag now spans both transports, the rollout order matters in one direction: **upgrade every gossip-publishing peer to a build that signs events before you set this to `true`.** Enabling it while an old publisher is still live drops that publisher's ref-updates on arrival, and the publisher sees no error, because gossip has no response to carry one. Upgrading the HTTP peers alone is not sufficient.
There is a second precondition on gossip ingest that the flag does not control. A publisher's `node_did` must already exist in the receiving node's peers table, or the event is dropped as an unknown peer DID. This holds in both modes, signed and unsigned alike: a valid signature proves key possession, not membership. Rows reach that table over HTTP, either from `GITLAWB_BOOTSTRAP_PEERS` when this node contacts a bootstrap peer and records the DID it reports, or from a prior `POST /api/v1/peers/announce`. So a peer that only ever joined the libp2p mesh, with no HTTP announce and no bootstrap contact in either direction, will have its ref-updates dropped even with a good signature and the flag off. If gossip is silently not landing from a peer you can see on the mesh, check that its DID is in `GET /api/v1/peers` on the receiving side first.
---
## Configuration
@@ -397,27 +339,15 @@ Important node settings:
| `GITLAWB_P2P_PORT` | libp2p QUIC/UDP port. Use `0` to disable. |
| `GITLAWB_BOOTSTRAP_PEERS` | Comma-separated HTTP peer URLs. |
| `GITLAWB_P2P_BOOTSTRAP` | Comma-separated libp2p multiaddrs. |
| `GITLAWB_P2P_KEY` | Path to the node's persistent libp2p identity key file, which fixes the PeerId across restarts. Must name a key file inside a directory, such as `/data/keys/p2p.key` or `./keys/p2p.key`. Two kinds of problem are handled differently. A value that cannot name a securable key file at all (a bare filename, `..` traversal, a trailing `/`, the filesystem root, or a `~/` path that escapes home) is refused before the node binds, so the process exits. Anything wrong with the storage itself (a symlinked or non-directory parent, an unsafe ancestor or working directory, an unreadable or malformed key) leaves p2p off while HTTP keeps serving; the node logs it at error level as `p2p_identity_key_load_failed` and sets the `gitlawb_p2p_identity_key_load_failed` metric to 1. Note that `/health` still reports healthy in that state, so alert on the log event or the metric rather than on the health check. Correcting the storage and restarting restores the same PeerId. On Unix a new key is created `0600` inside a `0700` directory. A key directory that grants access beyond its owner is tightened to `0700` on start; one that is closed too far is refused with a `chmod 700` remedy rather than widened, since a directory locked down on purpose is an operator decision. Rotate the key only when the node logs that the directory was group- or world-writable (or the key file itself was group- or world-readable). An ordinary `0755` directory is tightened too, but others could not read a `0600` key or replace its directory entry. An existing key file is never modified: one with group or other bits set is rejected (`chmod 600`), and so is one this node cannot read. On other platforms no permissions are enforced. Default `~/.gitlawb/p2p.key`; point it at a persistent volume when running in a container. |
| `GITLAWB_BOOTSTRAP_DISABLE_SEEDS` | Disable embedded seed peers for isolated dev/test networks. |
| `GITLAWB_REQUIRE_SIGNED_PEER_WRITES` | Require peer writes to prove their DID on every transport: RFC 9421 signatures on the announce/sync routes, and an Ed25519 payload signature on gossip ref-update events. Defaults to `false` during the staged rollout below. Upgrade every gossip publisher before enabling, or their updates are dropped with no error on their side. |
| `GITLAWB_ENFORCE_OWNER_PUSH` | Require the authenticated pusher to be the repo owner on `git-receive-pack`. **Defaults to `true`.** A `did:key` signature is authentication, not authorization (anyone can mint a key and sign), so with this off every signed caller may push to every repository, private ones included. Delegated and CI keys count as non-owners: a UCAN `git/push` capability is verified but not yet honored for authorization, so they cannot push while this is on. Set `false` only for a rolling upgrade; see [`docs/RUN-A-NODE.md`](docs/RUN-A-NODE.md). |
| `GITLAWB_REQUIRE_SIGNED_PEER_WRITES` | Require signed peer announce/sync writes. |
| `GITLAWB_AUTO_SYNC` | Enable automatic sync from known peers. |
| `GITLAWB_MAX_PACK_BYTES` | Max git pack body size for smart-HTTP routes. |
| `GITLAWB_GIT_SERVICE_TIMEOUT_SECS` | Max seconds a served git upload-pack, receive-pack, or `info/refs` advertisement may run before it is aborted (504). Default 600. Also bounds the withheld-blob classification walk (on both the upload-pack serve and receive-pack replication paths) and the push-side pin-candidate discovery (`rev-list` / `cat-file`), each reaped via process-group teardown at the deadline. On the path-scoped upload-pack path the classification walk and the pack serve share ONE deadline, so this value bounds their combined duration rather than granting each stage a full budget: a walk that consumes it leaves the serve nothing and the clone gets a 504. Serving large path-scoped repos may therefore need a higher value than they did when each stage was budgeted separately. Accepted range is 1 to 3153600000 (100 years), since the node derives deadlines from this value and a larger one cannot be represented. |
| `GITLAWB_GIT_SERVICE_TIMEOUT_SECS` | Max seconds a served git upload-pack, receive-pack, or `info/refs` advertisement may run before it is aborted (504). Default 600. Also bounds the withheld-blob classification walk (on both the upload-pack serve and receive-pack replication paths) and the push-side pin-candidate discovery (`rev-list` / `cat-file`), each reaped via process-group teardown at the deadline. |
| `GITLAWB_GIT_ACQUIRE_TIMEOUT_SECS` | Max seconds the storage-acquisition phase (Tigris HEAD/GET, push advisory-lock) of a served git op may run before the request is shed with a 503, separate from the git-run timeout. The concurrency permit is released on expiry so a stalled backend cannot pin the pool. Default 30. |
| `GITLAWB_MAX_CONCURRENT_GIT_OPS` | Max concurrent served git READ ops (upload-pack and its `info/refs` advertisement) across all callers; over-cap sheds a 503 + Retry-After. Anonymous reads draw from this pool, so pair it with `GITLAWB_MAX_CONCURRENT_READS_PER_CALLER`. Pushes and the receive-pack advertisement have their own pools, so a read flood cannot shed an authenticated push. Default 128. |
| `GITLAWB_MAX_CONCURRENT_GIT_PUSHES` | Max concurrent `git-receive-pack` POST operations, in a pool separate from the read pool. The anon receive-pack `info/refs` advertisement runs in a third pool of the same size, disjoint from both, so an advertisement flood cannot shed a push either. Two per-source push caps are derived from this value (`/8`, floor 1) and have no env var of their own. Over-cap sheds a 503 + Retry-After. Default 32. |
| `GITLAWB_MAX_CONCURRENT_READS_PER_CALLER` | Max concurrent read ops a single caller may hold, so one caller cannot monopolize the read pool. Keyed on the resolved source IP, never the DID, and only as granular as `GITLAWB_TRUSTED_PROXY`: left unset, a node behind an edge or NAT keys every caller on the edge IP and this collapses to one global cap. Default 16. |
| `GITLAWB_MAX_CONCURRENT_PIN_TASKS` | Max post-push pin loops (IPFS + Pinata) running concurrently across all repos. This caps how many loops RUN at once, not how much object-id list memory the node retains: on the local IPFS path a loop parked waiting for a permit still holds its full list. Do not size memory from this knob alone. A loop over cap waits, never drops a pin. Default 8. |
| `GITLAWB_REPO_LEASE_MAX_WAITERS` | Max pushes parked at once waiting for the same repo's write lease. Each waiter pins its buffered pack body, so this bounds that memory for a hot repo; past the cap the newest push sheds a 503 + Retry-After instead of queueing. Pushes to other repos are unaffected, and the lease holder is not counted. Default 8. |
| `GITLAWB_MAX_CONCURRENT_IPFS_WALKS` | Max concurrent `GET /ipfs/{cid}` visibility walks across all callers (own pool, disjoint from the served-git pools); over-cap sheds 503. Default 32. |
| `GITLAWB_IPFS_WALK_PER_SOURCE` | Max concurrent `/ipfs` walks a single source IP may hold. Default 4. |
| `GITLAWB_IPFS_MAX_LEGACY_PROBES` | Max legacy (NULL-provenance) repos probed per `/ipfs/{cid}` request, bounding the scan-fallback fan-out. A truncated scan returns a retryable 503, not a false 404. Default 256. |
| `GITLAWB_IPFS_MAX_LEGACY_SCAN_ROWS` | Max repo rows one `/ipfs/{cid}` request's legacy scan may read from the database. The probe ceiling above only starts counting once a probe runs, and quarantined or private repos are denied before that, so this is what bounds a scan over an inventory that denies the caller everywhere. A truncated scan sheds a retryable 503 carrying an opaque `continuation` token; echoing it as `?scan=` resumes the scan where it stopped. Every per-request ceiling on this path (rows, probes, visits, retained rule bytes) mints one, so each request advances the ladder by at least the rows it read and a holder buried past a ceiling is reached in a bounded number of requests, `ceil(repos / ceiling) + 1` when this row ceiling is the one that binds. No ceiling ever produces a 404. Every page is charged to the caller's `/ipfs` work allowance, so raising this raises that allowance too. Lowering it sharpens an oracle: because a truncation emits a token and a completed wrap does not, laddering to the end reveals the node's total repo count (private and quarantined included) to within one ceiling. Default 2048. |
| `GITLAWB_IPFS_MAX_REPOS_WALKED` | Max expensive path-scope visibility walks a `/ipfs/{cid}` request may run per phase; over-cap repos are skipped and the scan continues, shedding a retryable 503 (not a false 404) if the object is then found nowhere. The effective cap is the tighter of this knob and the node's internal history-walk ceiling of 17 (`MAX_PIN_SOURCES + 1`), so a value above 17 has no effect while a value below it does tighten the cap. It is charged per phase: the provenance lookup and the legacy-scan fallback get separate equal budgets, so one request can run up to twice the cap in total. Default 64. |
| `GITLAWB_IPFS_MAX_REPO_VISITS` | Ceiling on repos one `/ipfs/{cid}` request may visit (acquire + probe) past the visibility gate. Also the worst-case per-request Tigris fetch count. On exhaustion the scan stops with a retryable 503. Default 1024. |
| `GITLAWB_IPFS_REQUEST_BUDGET_SECS` | Absolute wall-clock budget for one admitted `/ipfs/{cid}` request's acquire+walk lifetime. Per-stage clamps bound the acquire and walk stages to the remaining budget, and no stage starts once it is exhausted; the scan then stops with a retryable 503. The object-type probe and content-read `cat-file` subprocesses are budget-checked before starting and each also run under their own deadline (the lesser of `GITLAWB_GIT_SERVICE_TIMEOUT_SECS` and the remaining budget), reaped via process-group teardown, so a hung `cat-file` cannot hold the request's walk slot past it. One hang path is still unbounded: the probe's object-store readability check is a plain filesystem sweep with nothing to reap, so a wedged filesystem can hold the slot past the deadline. Default 600. Accepted range is 1 to 3153600000 (100 years), since the node derives a deadline from this value and a larger one cannot be represented. |
| `GITLAWB_IPFS_RESOLVE_BUDGET_SECS` | Shorter budget for the pre-walk CID resolve inside an admitted `/ipfs/{cid}` request: the lookup that maps the requested CID to its git oid(s), which runs while the scarce walk admission is already held. A well-formed CID with no pin row does no probe and no walk work, so without this it could hold a walk slot for the whole request budget while nothing walked, and enough such requests shed every real retrieval at admission. The effective deadline is the lesser of this and the remaining request budget, so a value above `GITLAWB_IPFS_REQUEST_BUDGET_SECS` degrades to the request budget. Only the resolve is on this clock; walk and probe work stay on the request budget, so a slow but progressing scan is never shed by it. Default 10. Accepted range is 1 to 3153600000 (100 years). |
| `GITLAWB_IPFS_MAX_REPOS_WALKED` | Max legacy (NULL-provenance) repos probed per `/ipfs/{cid}` request, bounding the scan-fallback fan-out. A truncated scan returns a retryable 503, not a false 404. Default 256. |
| `GITLAWB_IPFS_RATE_LIMIT` | Max `/ipfs/{cid}` requests per client IP per hour (route flood brake). 0 disables. Default 600. |
| `GITLAWB_TIGRIS_BUCKET` | Optional S3/Tigris shared repo storage bucket. |
| `GITLAWB_PINATA_JWT` | Optional Pinata/IPFS warm-storage pinning. |
@@ -427,33 +357,11 @@ Production note: change the default Postgres password before exposing a node pub
Legacy-pin window: releases before the CID-resolver work stored the provider CID (Kubo dag-pb / Pinata) as a pinned object's resolver key. The `/ipfs/{cid}` resolver now recomputes the raw-content CID from the object bytes and refuses to serve a key that does not match, so `GET /api/v1/ipfs/pins` can still advertise an unrepaired legacy CID that 404s. Such a row is repaired opportunistically the next time a push carries the object again (its key is rewritten to the raw CID, the old value kept in `legacy_provider_cid`), but git negotiation omits objects the node already has, so most legacy rows never re-enter a push delta. A deferred one-shot startup sweep, not this opportunistic path, is what fully retires the advertise-then-404 window. Rows whose object bytes are gone stay withheld.
### Upgrading: the PeerId rotates once
This node's libp2p identity is now a keypair generated on first start and kept
at `GITLAWB_P2P_KEY`, rather than one derived from the node DID. Every node
therefore gets a new PeerId once, on the first start after upgrading, and keeps
it from then on as long as that key file survives (put it on a persistent volume
in a container).
Two things to check before upgrading:
- Any `GITLAWB_P2P_BOOTSTRAP` multiaddr that pins a peer's old PeerId with a
`/p2p/<PeerId>` suffix stops matching once that peer upgrades. Update the
suffix, or drop it and let identify supply the current one. Addresses without
the suffix keep working untouched.
- `GITLAWB_P2P_KEY` must name a key file inside a directory, such as
`/data/keys/p2p.key` or `./keys/p2p.key`. A bare filename is refused at
startup, since the node will not keep its identity key in the working
directory.
Peers found over `GITLAWB_BOOTSTRAP_PEERS` and the embedded seed list are
unaffected, since those are HTTP URLs and carry no PeerId.
---
## Optional node staking
Twigpine Node includes optional Base L2 node-operator hooks. Operators can register a node DID, stake `$GITLAWB`, and post heartbeats.
Gitlawb Node includes optional Base L2 node-operator hooks. Operators can register a node DID, stake `$GITLAWB`, and post heartbeats.
PoS is disabled unless these are configured:
@@ -523,8 +431,8 @@ Build:
Output:
```txt
dist/Twigpine Node.app
dist/GitlawbNode-macOS.dmg
dist/Gitlawb Node.app
dist/Gitlawb Node.dmg
```
Features:
@@ -538,7 +446,7 @@ Features:
Unsigned local build:
```bash
xattr -cr "dist/Twigpine Node.app"
xattr -cr "dist/Gitlawb Node.app"
```
---
@@ -553,8 +461,8 @@ Short-term priorities:
2. Add Docker and installer smoke tests.
3. Improve operator docs and `gl doctor` checks.
4. Harden peer writes and publish the signed-peer rollout plan.
5. Wire trusted UCAN delegation into repository write authorization.
6. Add threaded, line-level pull-request discussions and enforce approval requirements on merges.
5. Implement repo write authorization: owner checks, protected branches, and UCAN capability checks.
6. Implement private-read enforcement or remove private repo affordances until it exists.
7. Add metrics for pushes, fetches, pack sizes, peer sync, failed auth, and webhooks.
Product direction:
+26 -36
View File
@@ -2,9 +2,9 @@
## Reporting a Vulnerability
If you discover a security vulnerability in Twigpine, please **do not open a public issue**.
If you discover a security vulnerability in gitlawb, please **do not open a public issue**.
Report it privately by emailing **security@twigpine.com** with:
Report it privately by emailing **security@gitlawb.com** with:
- A description of the vulnerability
- Steps to reproduce
- Potential impact assessment
@@ -28,11 +28,10 @@ We will acknowledge receipt within 48 hours and aim to release a fix within 14 d
- Every git object is content-addressed via CIDv1 (SHA-256)
- Tamper-evident by construction — a modified object changes its CID
**UCAN token validation**
- Bootstrap UCAN tokens are issued at registration.
- A supplied token's signature, audience, expiry, and proof-chain attenuation are validated.
- Tokens use a signed JSON wire format with expiry.
- Capability grants are not yet consulted by repository write authorization; see the limitations below.
**UCAN capability tokens**
- Bootstrap UCAN tokens issued at registration
- Capability-scoped: `git:push`, `git:fetch`, `issue:create`, `pr:open`
- JWT-format tokens with expiry
**Smart contracts (Base Sepolia testnet)**
- `GitlawbDIDRegistry` — on-chain DID → document registry
@@ -49,20 +48,21 @@ We will acknowledge receipt within 48 hours and aim to release a fix within 14 d
---
## Known Limitations
## Known Limitations (Planned for v0.2)
These are documented limitations of the current live release. They should be prioritized without breaking existing nodes during rolling upgrades.
These are **documented, accepted limitations** for the current live release and should be prioritized without breaking existing nodes during rolling upgrades.
### Repository write authorization defaults
- `git-receive-pack` verifies HTTP Signatures and, with the default configuration, also requires the pusher DID to be the repository owner: `GITLAWB_ENFORCE_OWNER_PUSH` defaults to `true`.
- **Impact:** With the default setting, a valid signature authenticates the pusher *and* the pusher must be the repository owner. Authentication is not authorization on its own — anyone can mint a `did:key` and sign — so owner enforcement is the only thing that prevents every signed caller from pushing to every repository, including private ones.
- **Mitigation:** `GITLAWB_ENFORCE_OWNER_PUSH=false` is the explicit rolling-upgrade compatibility override. Disable it only for the duration of an upgrade whose pushers are not yet the repo owner, then turn it back on. Delegated and CI keys count as non-owners: a UCAN `git/push` capability is verified but not yet honored for authorization, so they cannot push while this is on.
### UCAN chain validation
- The auth middleware verifies HTTP Signatures and token structure, but does not yet walk the full UCAN delegation chain.
- **Impact:** A node cannot yet enforce fine-grained capability delegation. Currently, any registered agent with a valid HTTP Signature can push.
- **Mitigation:** Keep write endpoints signed, treat public nodes as public infrastructure, and treat trust scores as soft rate-limiting signals rather than authorization.
- **Fix target:** v0.2
### UCAN delegation and revocation
- The middleware validates a supplied UCAN's complete proof chain, but a root token is accepted without an independently trusted issuer anchor. `Ucan::can` is not yet used by write handlers, so a UCAN does not grant scoped repository access.
- Agent lifecycle revocation is not checked by HTTP Signature authorization. Removing or revoking an agent record does not itself block a compromised DID from authenticating.
- **Impact:** Do not use UCANs for collaborator permissions or rely on agent-record revocation as a key-compromise response.
- **Mitigation:** Keep sensitive deployments behind operational network controls and enable owner-only push enforcement where it fits the deployment until trusted delegation and authorization-aware revocation are implemented.
### UCAN revocation
- Issued UCAN tokens cannot be revoked before expiry.
- **Impact:** If a keypair is compromised, the attacker retains access until the UCAN expires (default: 30 days).
- **Mitigation:** Regenerate your identity (`gl identity new --force`) and re-register to issue a new UCAN. Until revocation/blocklisting is implemented, operators should remove compromised DIDs directly from their local database.
- **Fix target:** v0.2
### git-receive-pack authentication
- The `git-receive-pack` endpoint enforces HTTP Signature auth. Plain Git smart-HTTP clients do not generate those headers, so the `git-remote-gitlawb` helper is required for pushes.
@@ -71,20 +71,10 @@ These are documented limitations of the current live release. They should be pri
- **Fix target:** v0.2
### Private repository reads
- Repository and path-scoped visibility checks are enforced for repository API and Git content reads. A denied whole-repository or root read returns the same 404 shape as a missing repository, so the denial does not reveal private-repository existence.
- Sparse-clone support exposes withheld path globs to callers who may read the repository root. Do not put sensitive information in withheld path names.
- `GET /api/v1/tasks`, `/api/v1/ipfs/pins`, and `/api/v1/arweave/anchors` are not repository-gated. Task records include a UCAN token; pin and anchor listings expose object and ref metadata.
- Changing a repository's visibility controls future serving, but cannot retract ref metadata or configured external pins and anchors already announced while the repository was public. Do not push secrets to an announceable repository.
- **Impact:** Visibility policies protect the repository and Git content routes they gate, not every metadata endpoint or previously published content.
- **Remaining boundary:** This read control does not address the independent write-authorization and UCAN-delegation limitations described above.
### Pull-request review enforcement
- Pull-request review comments are not yet threaded or line-anchored, and merges do not enforce required approvals.
- **Impact:** Teams must use their own review policy or external controls for merge approval requirements.
### GraphQL mutation coverage
- Existing GraphQL mutations require an authenticated signer, but a mutation-specific source-level guardrail has not yet been added for future mutations.
- **Impact:** A new mutation could accidentally omit its signer check without an explicit test fence.
- Repository records have an `is_public` field and the node exposes `GITLAWB_PUBLIC_READ`, but per-repository private-read enforcement is not wired in the current live release.
- **Impact:** Do not store private repositories or secrets on public nodes.
- **Mitigation:** Run isolated nodes for non-public data and restrict network access at the reverse proxy or firewall layer.
- **Fix target:** v0.2
### Peer route hardening rollout
- Peer announce and sync notification routes accept signed requests and verify DID matches when a signature is present.
@@ -111,20 +101,20 @@ These are documented limitations of the current live release. They should be pri
| Key storage | PKCS#8 PEM, 0600 permissions |
| Content hashing | SHA-256 via CIDv1 |
| HTTP Signatures | RFC 9421 (Ed25519 + SHA-256 Content-Digest) |
| UCAN tokens | Signed JSON object (Ed25519 signature) |
| UCAN tokens | JWT (Ed25519 signatures) |
| On-chain | ECDSA secp256k1 (Base L2 / Ethereum) |
---
## Threat Model
Twigpine is designed to be secure against:
- **Unauthenticated writes** — HTTP Signature auth on protected write endpoints; see Known Limitations for owner and capability authorization gaps
gitlawb is designed to be secure against:
- **Unauthorized writes** — HTTP Signature auth on all write endpoints
- **Tampered git objects** — CIDv1 content addressing detects modification
- **Identity spoofing** — DIDs derived from public keys, unforgeable without the private key
- **Centralized takedown** — no single point of control; data on IPFS + Arweave
Twigpine is **not yet** designed to defend against:
gitlawb is **not yet** designed to defend against:
- A compromised node operator (node operators are trusted for their own node)
- Sybil attacks on the DHT (trust score system mitigates, not eliminates)
- Timing attacks on signature verification (not constant-time compared in v0.1)
+5 -5
View File
@@ -1,11 +1,11 @@
{
"$comment": "Canonical seed list for the Twigpine network. Merged with GITLAWB_BOOTSTRAP_PEERS at startup. PRs to add public nodes welcome.",
"$comment": "Canonical seed list for the Gitlawb network. Merged with GITLAWB_BOOTSTRAP_PEERS at startup. PRs to add public nodes welcome.",
"version": 1,
"updated": "2026-06-30",
"peers": [
{
"name": "gitlawb",
"operator": "Twigpine",
"operator": "Gitlawb",
"did": "did:key:z6MkqDnb7Siv3Cwj7pGJq4T5EsUisECqR8KpnDLwcaZq5TPr",
"http_url": "https://node.gitlawb.com",
"p2p_multiaddr": null,
@@ -13,7 +13,7 @@
},
{
"name": "gitlawb-node2",
"operator": "Twigpine",
"operator": "Gitlawb",
"did": null,
"http_url": "https://node2.gitlawb.com",
"p2p_multiaddr": null,
@@ -21,7 +21,7 @@
},
{
"name": "gitlawb-node3",
"operator": "Twigpine",
"operator": "Gitlawb",
"did": null,
"http_url": "https://node3.gitlawb.com",
"p2p_multiaddr": null,
@@ -29,7 +29,7 @@
},
{
"name": "gitlawb-manila",
"operator": "Twigpine",
"operator": "Gitlawb",
"did": null,
"http_url": "https://manila.gitlawb.com",
"p2p_multiaddr": null,
-112
View File
@@ -1,112 +0,0 @@
# Dependency allowlist for the `gitlawb-core` crate.
#
# gitlawb-core is the shared-primitives crate every consumer embeds (the `gl`
# CLI, git-remote-gitlawb, the node daemon). It must stay lean so those
# consumers do not inherit daemon-weight dependencies. This file is the source
# of truth for the crates gitlawb-core's NORMAL (non-dev, non-build) transitive
# dependency closure is allowed to contain. CI (`scripts/check-gitlawb-core-deps.sh`,
# wired into the `core-deps-purity` job in .github/workflows/pr-checks.yml)
# fails if gitlawb-core's tree contains any crate not listed here.
#
# The allowlist is exhaustive-by-construction: a NEW dependency reds CI whether
# or not anyone thought to ban it, which a denylist cannot do.
#
# Measured in-workspace, so feature unification across the workspace can, in
# rare cases, surface a crate here that gitlawb-core would not pull standalone.
# An unexpected new entry is a prompt to check what pulled it in, not noise to
# rubber-stamp.
#
# Regenerate (from repo root, after an INTENTIONAL dependency change):
# cargo tree -p gitlawb-core --edges normal --prefix none \
# | sed -E 's/ v[0-9].*$//' \
# | grep -v '^gitlawb-core$' \
# | sort -u
# then paste the result below this header block. (The checker snapshots and
# restores Cargo.lock; a manual regen may refresh it — `git checkout Cargo.lock`
# afterward if you did not intend that.)
#
# Lines starting with `#` and blank lines are ignored by the checker.
aead
anyhow
base-x
base256emoji
base64
base64ct
block-buffer
cfg-if
chacha20
chacha20poly1305
chrono
cid
cipher
const-oid
const-str
core2
cpufeatures
crypto-common
crypto_box
crypto_secretbox
curve25519-dalek
curve25519-dalek-derive
data-encoding
data-encoding-macro
data-encoding-macro-internal
der
digest
ed25519
ed25519-dalek
equivalent
generic-array
getrandom
hashbrown
hex
iana-time-zone
indexmap
inout
itoa
libc
match-lookup
memchr
multibase
multihash
multihash-codetable
multihash-derive
multihash-derive-impl
num-traits
opaque-debug
pem-rfc7468
pkcs8
poly1305
ppv-lite86
proc-macro-crate
proc-macro2
quote
rand
rand_chacha
rand_core
salsa20
serde
serde_core
serde_derive
serde_json
sha2
signature
spki
subtle
syn
synstructure
thiserror
thiserror-impl
toml_datetime
toml_edit
toml_parser
typenum
unicode-ident
universal-hash
unsigned-varint
uuid
winnow
zerocopy
zeroize
zeroize_derive
zmij
+2 -19
View File
@@ -1,7 +1,7 @@
[package]
name = "git-remote-gitlawb"
description = "Git remote helper — enables 'git clone gitlawb://did:gitlawb:...'"
version = "0.7.1" # x-release-please-version
version = "0.5.1" # x-release-please-version
edition.workspace = true
rust-version.workspace = true
license.workspace = true
@@ -11,7 +11,7 @@ name = "git-remote-gitlawb"
path = "src/main.rs"
[dependencies]
gitlawb-core = { path = "../gitlawb-core", features = ["redirect"] }
gitlawb-core = { path = "../gitlawb-core" }
anyhow = { workspace = true }
reqwest = { workspace = true }
tracing = { workspace = true }
@@ -19,20 +19,3 @@ tracing-subscriber = { workspace = true }
[dev-dependencies]
mockito = "1"
tempfile = "3"
[target.'cfg(unix)'.dev-dependencies]
libc = "0.2"
[target.'cfg(windows)'.dev-dependencies]
# Job Object bindings for the real_git_fetch harness: the Windows analog of the
# unix process-group teardown, so the whole `git fetch` tree can be killed
# together on both the timeout and clean-exit paths (INV-22). Win32_Security is
# required by CreateJobObjectW, and Win32_System_Threading by
# JOBOBJECT_EXTENDED_LIMIT_INFORMATION (it embeds IO_COUNTERS).
windows-sys = { version = "0.61", features = [
"Win32_Foundation",
"Win32_Security",
"Win32_System_JobObjects",
"Win32_System_Threading",
] }
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
+2 -2
View File
@@ -1,7 +1,7 @@
[package]
name = "gitlawb-attest"
description = "External Attestation v1: pluggable provenance attachments for Twigpine ref-update certs"
version = "0.7.1" # x-release-please-version
description = "External Attestation v1: pluggable provenance attachments for gitlawb ref-update certs"
version = "0.5.1" # x-release-please-version
edition.workspace = true
rust-version.workspace = true
license.workspace = true
+6 -211
View File
@@ -16,7 +16,7 @@
//! by exact match.
use base64::{engine::general_purpose::URL_SAFE_NO_PAD as B64U, Engine};
use ed25519_dalek::{Signature, Signer, SigningKey, VerifyingKey};
use ed25519_dalek::{Signature, Signer, SigningKey, Verifier, VerifyingKey};
use serde::{de::DeserializeOwned, Deserialize, Serialize};
use crate::error::{Error, Result};
@@ -105,30 +105,16 @@ impl Attestation {
let bytes = canonical_signing_bytes(&self.type_, &self.payload, &self.cert_hash)?;
let vk = verifying_key_from_did_key(&self.signer)?;
self.verify_sig_with_key(&vk, &bytes)?;
Ok(vk)
}
/// The signature check itself, against a key the caller supplies.
///
/// Split out so the STRICTNESS of the check stays provable on its own.
/// `verifying_key_from_did_key` now refuses a small-order key, so a weak
/// signer can no longer reach this line through `verify_signature`. Without
/// this seam there would be nothing left that fails when `verify_strict` is
/// downgraded to the non-strict `verify`, and that regression would land
/// silently. Check order in `verify_signature` is unchanged.
fn verify_sig_with_key(&self, vk: &VerifyingKey, bytes: &[u8]) -> Result<()> {
let sig_bytes: [u8; 64] = B64U
.decode(&self.sig)
.map_err(|e| Error::Signature(format!("base64url: {e}")))?
.try_into()
.map_err(|_| Error::Signature("signature must be 64 bytes".to_string()))?;
let sig = Signature::from_bytes(&sig_bytes);
vk.verify_strict(bytes, &sig)
vk.verify(&bytes, &sig)
.map_err(|e| Error::Signature(format!("ed25519: {e}")))?;
Ok(())
Ok(vk)
}
/// Reparse `payload` as `P`. Errors if the type discriminator does not
@@ -195,7 +181,7 @@ fn canonical_signing_bytes(
Ok(out)
}
pub(crate) fn did_key_from_verifying_key(key: &VerifyingKey) -> String {
fn did_key_from_verifying_key(key: &VerifyingKey) -> String {
let mut buf = Vec::with_capacity(ED25519_MULTICODEC.len() + 32);
buf.extend_from_slice(&ED25519_MULTICODEC);
buf.extend_from_slice(&key.to_bytes());
@@ -216,19 +202,6 @@ fn verifying_key_from_did_key(did: &str) -> Result<VerifyingKey> {
"did:key must use base58btc (z-prefix): {did}"
)));
}
// Refuse an oversized id before decoding. base58 decoding is quadratic in
// its input and `signer` comes off an attacker-supplied attestation, so a
// short request could otherwise buy a large decode. An ed25519 did:key
// method-id is a fixed 48 characters, so this bound is slack rather than a
// behavior change, and it only helps ahead of the decode. Mirrors the same
// cap in gitlawb-core's Did::to_verifying_key, which this crate cannot call
// (gitlawb-core is a dev-dependency here).
const MAX_METHOD_ID_LEN: usize = 64;
if method_id.len() > MAX_METHOD_ID_LEN {
return Err(Error::Did(
"did:key method-specific id too long".to_string(),
));
}
let (base, bytes) =
multibase::decode(method_id).map_err(|e| Error::Did(format!("multibase: {e}")))?;
if base != multibase::Base::Base58Btc {
@@ -244,20 +217,8 @@ fn verifying_key_from_did_key(did: &str) -> Result<VerifyingKey> {
let key_bytes: [u8; 32] = bytes[ED25519_MULTICODEC.len()..]
.try_into()
.expect("length checked above");
let key = VerifyingKey::from_bytes(&key_bytes)
.map_err(|e| Error::Did(format!("invalid ed25519 key: {e}")))?;
// `from_bytes` only decompresses, so it accepts a small-order point. This
// crate parses did:key itself rather than going through gitlawb-core's
// `Did`, and it cannot go through it (gitlawb-core is a dev-dependency
// here), so the rejection is mirrored rather than inherited. Signature
// verification is already strict, so this is defense in depth against a
// future consumer of this parser that does not verify strictly.
if key.is_weak() {
return Err(Error::Did("small-order ed25519 key".to_string()));
}
Ok(key)
VerifyingKey::from_bytes(&key_bytes)
.map_err(|e| Error::Did(format!("invalid ed25519 key: {e}")))
}
#[cfg(test)]
@@ -504,23 +465,6 @@ mod tests {
assert!(matches!(err, Error::Did(_)));
}
#[test]
fn verify_rejects_oversized_method_id_before_decoding() {
let sk = fresh();
let mut att = dummy_attestation(&sk, sample_cert_hash());
// A well-formed prefix on an absurdly long id must fail the length
// bound, not the decoder: base58 decode cost is quadratic in input
// length, so reaching it at all is the defect.
att.signer = format!("did:key:z{}", "1".repeat(256));
let err = att.verify_signature(sample_cert_hash()).unwrap_err();
// Match the bound's own message: any other Did error (a decode or
// length mismatch downstream) means the oversized input was decoded.
match err {
Error::Did(m) => assert!(m.contains("too long"), "unexpected: {m}"),
e => panic!("expected Error::Did, got {e:?}"),
}
}
#[test]
fn verify_rejects_bad_signature_base64() {
let sk = fresh();
@@ -610,153 +554,4 @@ mod tests {
let err = tampered.verify_signature(cert_hash).unwrap_err();
assert!(matches!(err, Error::Signature(_)));
}
/// Regression guard for strict verification: a signature forged under a
/// weak (small-order) public key satisfies the verification equation
/// but must be rejected. The identity point is such a key: with R the
/// identity and S = 0, [S]B - [k]A is the identity for any message.
/// gitlawb-attest parses did:key itself and never routes through
/// gitlawb-core's `Did`, which it cannot: gitlawb-core is a dev-dependency
/// here. So the choke-point rejection has to be mirrored in this parser or
/// this crate keeps handing out keys the rest of the system rejects.
#[test]
fn verifying_key_from_did_key_rejects_a_small_order_key() {
let mut weak = [0u8; 32];
weak[0] = 1; // compressed identity point
let mut buf = Vec::with_capacity(ED25519_MULTICODEC.len() + 32);
buf.extend_from_slice(&ED25519_MULTICODEC);
buf.extend_from_slice(&weak);
let did = format!(
"did:key:{}",
multibase::encode(multibase::Base::Base58Btc, &buf)
);
let err =
verifying_key_from_did_key(&did).expect_err("a small-order did:key must not resolve");
match err {
Error::Did(msg) => assert!(
msg.contains("small-order"),
"rejection must name the small-order key, got: {msg}"
),
other => panic!("expected Error::Did, got {other:?}"),
}
}
/// base58 decoding is quadratic in its input and `signer` is attacker
/// controlled, so an oversized method-id must be refused BEFORE the decode
/// or a hostile attestation buys a large decode for a short request. The
/// assertion is on the length error specifically: a multibase error here
/// would mean the cap ran too late to matter.
#[test]
fn verifying_key_from_did_key_refuses_an_oversized_method_id_before_decoding() {
let oversized = format!("did:key:z{}", "1".repeat(65_536));
let err = verifying_key_from_did_key(&oversized)
.expect_err("an oversized method-id must be refused");
match err {
Error::Did(msg) => assert!(
msg.contains("too long"),
"must fail on the length cap, not after decoding: {msg}"
),
other => panic!("expected Error::Did, got {other:?}"),
}
}
/// Strictness regression guard. The parser now refuses a small-order key,
/// so `verify_signature` can no longer carry a weak signer down to the
/// signature check; this drives that check directly to prove it is still
/// STRICT. Downgrading `verify_strict` to `verify` makes this go red,
/// which nothing else in the suite would catch.
#[test]
fn verify_sig_with_key_is_strict_about_a_weak_key() {
let mut weak_key_bytes = [0u8; 32];
weak_key_bytes[0] = 1; // compressed identity point
let weak_vk = VerifyingKey::from_bytes(&weak_key_bytes).unwrap();
assert!(weak_vk.is_weak());
let cert_hash = sample_cert_hash();
let mut att = dummy_attestation(&SigningKey::generate(&mut OsRng), cert_hash);
// R = identity, S = 0: satisfies the non-strict verification equation
// for any message under a small-order key.
let mut forged = [0u8; 64];
forged[0] = 1;
att.sig = B64U.encode(forged);
let bytes = canonical_signing_bytes(&att.type_, &att.payload, &att.cert_hash).expect("jcs");
let err = att
.verify_sig_with_key(&weak_vk, &bytes)
.expect_err("strict verification must reject a small-order key");
assert!(
matches!(err, Error::Signature(_)),
"expected a signature error, got {err:?}"
);
}
/// Accept control for the seam: a genuine signature still verifies through
/// the same helper, so the guard above is not simply rejecting everything.
#[test]
fn verify_sig_with_key_accepts_a_genuine_signature() {
let cert_hash = sample_cert_hash();
let sk = SigningKey::generate(&mut OsRng);
let att = dummy_attestation(&sk, cert_hash);
let bytes = canonical_signing_bytes(&att.type_, &att.payload, &att.cert_hash).expect("jcs");
att.verify_sig_with_key(&sk.verifying_key(), &bytes)
.expect("a real signature must verify");
}
/// Control for the guard above: a real did:key must still resolve.
#[test]
fn verifying_key_from_did_key_still_accepts_a_real_key() {
let sk = SigningKey::from_bytes(&[3u8; 32]);
let vk = sk.verifying_key();
let mut buf = Vec::with_capacity(ED25519_MULTICODEC.len() + 32);
buf.extend_from_slice(&ED25519_MULTICODEC);
buf.extend_from_slice(&vk.to_bytes());
let did = format!(
"did:key:{}",
multibase::encode(multibase::Base::Base58Btc, &buf)
);
let got = verifying_key_from_did_key(&did).expect("a real did:key must resolve");
assert_eq!(got.to_bytes(), vk.to_bytes());
}
#[test]
fn verify_rejects_weak_key_signature() {
let mut weak_key_bytes = [0u8; 32];
weak_key_bytes[0] = 1; // compressed identity point (y = 1, x = 0)
let weak_vk = VerifyingKey::from_bytes(&weak_key_bytes).unwrap();
assert!(weak_vk.is_weak(), "identity point must be a weak key");
let cert_hash = sample_cert_hash();
let mut forged = [0u8; 64];
forged[0] = 1; // R = identity point, S = 0
let forged_sig = B64U.encode(forged);
// Build an attestation with the weak key as signer and the forged
// signature. Rejection now happens at DID RESOLUTION rather than at
// verify time: `verifying_key_from_did_key` refuses a small-order key
// before `verify_strict` is reached, so the observed error is
// `Error::Did`, not `Error::Signature`. `verify_strict` remains the
// second layer for malleability cases that do not involve a weak
// public key (a small-order R under an honest key), which this fixture
// cannot construct and therefore no longer covers.
let mut att = dummy_attestation(&SigningKey::generate(&mut OsRng), cert_hash);
let mut buf = Vec::with_capacity(ED25519_MULTICODEC.len() + 32);
buf.extend_from_slice(&ED25519_MULTICODEC);
buf.extend_from_slice(&weak_key_bytes);
att.signer = format!(
"did:key:{}",
multibase::encode(multibase::Base::Base58Btc, &buf)
);
att.sig = forged_sig;
let err = att.verify_signature(cert_hash).unwrap_err();
match err {
Error::Did(msg) => assert!(
msg.contains("small-order"),
"weak signer must be refused as a small-order key, got: {msg}"
),
other => panic!("expected Error::Did for a small-order signer, got {other:?}"),
}
}
}
+1 -1
View File
@@ -1,4 +1,4 @@
//! Pluggable provenance attachments for Twigpine ref-update certs.
//! Pluggable provenance attachments for gitlawb ref-update certs.
//!
//! A `gitlawb/ref-update/v1` cert proves who pushed and where a ref moved to.
//! It does not say how the commit was produced. This crate adds an optional
+14 -183
View File
@@ -121,12 +121,7 @@ impl Registry {
attestation: &Attestation,
expected_cert_hash: [u8; 32],
) -> Result<VerifiedAttestation> {
// The key that actually verified is the trust anchor for the signer a
// consumer sees. `verify_signature` recovers it from the artifact's
// `signer` field and rejects a field that does not match the signing
// key, so this is the canonical DID of the key that signed — never a
// raw artifact string that could disagree with it.
let verified_key = attestation.verify_signature(expected_cert_hash)?;
attestation.verify_signature(expected_cert_hash)?;
let fully = match self.by_type.get(attestation.type_.as_str()) {
Some(v) => {
@@ -143,55 +138,30 @@ impl Registry {
Ok(VerifiedAttestation {
type_: attestation.type_.clone(),
signer: crate::attestation::did_key_from_verifying_key(&verified_key),
signer: attestation.signer.clone(),
cert_hash: attestation.cert_hash.clone(),
fully_verified: fully,
})
}
/// Verify a batch, then enforce `RequireAll`.
///
/// Under [`Policy::RequireAll`] this never short-circuits: an attestation
/// that fails to verify — bad signature, cert-hash mismatch, or malformed
/// payload, the actual `Err` paths of [`Registry::verify`] — is dropped
/// from the result rather than aborting the batch. A valid attestation of
/// a type with NO registered verifier is not one of those: under
/// `RequireAll` it verifies to `Ok` with `fully_verified: false` and stays
/// in the result, preserving the distinction between *untrusted* and
/// *invalid* (see `require_all_is_lenient_on_unknown_types_in_the_batch`).
/// Any third party can attach an attestation, so a hard error here would
/// let an unrelated `attacker/spam/v1` deny-of-service the whole cert (the
/// DoS vector the module docstring warns about). The `required_types`
/// check below is the real gate, and it only counts `fully_verified`
/// entries — so a required type represented solely by a malformed
/// attestation still fails with [`Error::RequiredMissing`].
///
/// Under `AcceptKnown` and `RejectUnknown` the batch stays strict: the
/// first verification error is surfaced to the caller.
pub fn verify_all(
&self,
attestations: &[Attestation],
expected_cert_hash: [u8; 32],
) -> Result<Vec<VerifiedAttestation>> {
if self.policy != Policy::RequireAll {
let mut verified = Vec::with_capacity(attestations.len());
for a in attestations {
verified.push(self.verify(a, expected_cert_hash)?);
}
return Ok(verified);
let mut verified = Vec::with_capacity(attestations.len());
for a in attestations {
verified.push(self.verify(a, expected_cert_hash)?);
}
// RequireAll: drop entries that fail to verify instead of aborting.
let verified: Vec<VerifiedAttestation> = attestations
.iter()
.filter_map(|a| self.verify(a, expected_cert_hash).ok())
.collect();
for required in &self.required_types {
let present = verified
.iter()
.any(|v| &v.type_ == required && v.fully_verified);
if !present {
return Err(Error::RequiredMissing(required.clone()));
if self.policy == Policy::RequireAll {
for required in &self.required_types {
let present = verified
.iter()
.any(|v| &v.type_ == required && v.fully_verified);
if !present {
return Err(Error::RequiredMissing(required.clone()));
}
}
}
Ok(verified)
@@ -201,7 +171,7 @@ impl Registry {
#[cfg(test)]
mod tests {
use super::*;
use crate::attestation::{did_key_from_verifying_key, Attestation, AttestationPayload};
use crate::attestation::{Attestation, AttestationPayload};
use ed25519_dalek::SigningKey;
use rand::rngs::OsRng;
use serde::{Deserialize, Serialize};
@@ -349,38 +319,6 @@ mod tests {
.any(|v| v.type_ == "other/v1" && !v.fully_verified));
}
/// The strict policies must surface a batch member's verification error
/// through `verify_all`, not launder it into a partial result. This binds
/// the POLICY BOUNDARY itself: only the `RequireAll` arm may drop failing
/// entries, and an apparently harmless cleanup that applies its
/// `filter_map(Result::ok)` shape to the strict arms would silently turn
/// the fail-closed contracts into fail-open — with every per-attestation
/// `verify` test still green, because none of them go through the batch
/// path. The failing entry is a wrong-cert-hash attestation (signed over a
/// different hash than the batch expects), a genuine `Err` under every
/// policy.
#[test]
fn strict_policies_propagate_a_batch_members_error() {
let sk = fresh();
let cert_hash = sample_hash();
let mut wrong_hash = sample_hash();
wrong_hash[0] ^= 0xff;
for policy in [Policy::AcceptKnown, Policy::RejectUnknown] {
let mut reg = Registry::new().with_policy(policy);
reg.register(DemoVerifier);
let good = signed_demo(&sk, cert_hash, "ok");
let bad = signed_demo(&sk, wrong_hash, "ok");
let err = reg
.verify_all(&[good, bad], cert_hash)
.expect_err("a strict policy must fail the whole batch");
assert!(
matches!(err, Error::Signature(_) | Error::CertHashMismatch),
"expected the underlying verification error, got {err:?}"
);
}
}
/// A required type that is present but unverified (no verifier registered
/// even though the type appears in `required_types`) must fail
/// `verify_all` with `RequiredMissing`. Catches a misconfiguration where
@@ -398,65 +336,6 @@ mod tests {
assert!(matches!(err, Error::RequiredMissing(t) if t == "demo/v1"));
}
/// A cert-hash that does not match `sample_hash()`, so an attestation
/// signed against it fails `verify_signature` when the batch is checked
/// against `sample_hash()`.
fn other_hash() -> [u8; 32] {
let mut h = sample_hash();
h[0] ^= 0xff;
h
}
/// Under `RequireAll`, a malformed attestation (here: signed against the
/// wrong cert hash, so it fails signature verification) attached alongside a
/// valid required one must not abort the batch. The junk is dropped and the
/// cert still verifies. Before the fix, `verify_all` propagated the error
/// via `?` and the whole batch failed — the DoS vector the docstring warns
/// about.
#[test]
fn require_all_drops_a_malformed_extra_and_still_accepts() {
let sk = fresh();
let cert_hash = sample_hash();
let mut reg = Registry::new()
.with_policy(Policy::RequireAll)
.require_types(["demo/v1"]);
reg.register(DemoVerifier);
// Signed against `other_hash()`, so it will not verify under `cert_hash`.
let malformed = signed_demo(&sk, other_hash(), "boom");
let real = signed_demo(&sk, cert_hash, "ok");
let verified = reg
.verify_all(&[malformed, real], cert_hash)
.expect("a malformed extra must not abort the batch under RequireAll");
assert_eq!(verified.len(), 1, "the malformed entry should be dropped");
assert!(verified
.iter()
.any(|v| v.type_ == "demo/v1" && v.fully_verified));
}
/// Leniency must not open a hole: if the *only* attestation of a required
/// type is malformed, dropping it leaves the required type absent and
/// `verify_all` must still fail with `RequiredMissing`. Guards against a fix
/// that skips errors so eagerly it lets an unverified required type pass.
#[test]
fn require_all_rejects_when_only_a_malformed_required_type_is_present() {
let sk = fresh();
let cert_hash = sample_hash();
let mut reg = Registry::new()
.with_policy(Policy::RequireAll)
.require_types(["demo/v1"]);
reg.register(DemoVerifier);
// The sole demo/v1 is bound to the wrong cert hash → fails verification.
let malformed_required = signed_demo(&sk, other_hash(), "ok");
let err = reg
.verify_all(&[malformed_required], cert_hash)
.unwrap_err();
assert!(matches!(err, Error::RequiredMissing(t) if t == "demo/v1"));
}
#[test]
fn payload_check_failure_rejects() {
let sk = fresh();
@@ -495,52 +374,4 @@ mod tests {
let err = reg.verify(&att, cert_hash).unwrap_err();
assert!(matches!(err, Error::Payload(_)));
}
/// The signer a consumer sees must be the canonical DID of the key that
/// actually verified, never a raw artifact string that could disagree.
/// `verify_signature` derives the key from the artifact's `signer` field
/// and the parser is canonical (base58btc, exact length, ed25519
/// multicodec), so a forged field fails verification outright: the
/// must-not here is that no `VerifiedAttestation` ever carries a signer
/// that did not verify.
#[test]
fn verified_signer_is_anchored_to_the_key_that_verified() {
let signer = fresh();
let other = fresh();
assert_ne!(
signer.verifying_key().to_bytes(),
other.verifying_key().to_bytes(),
"fixture precondition: distinct signing keys"
);
let cert_hash = sample_hash();
let reg = Registry::new();
// Control: an honest artifact reports the signer's canonical DID, and
// it agrees with the artifact field.
let att = signed_demo(&signer, cert_hash, "ok");
let v = reg.verify(&att, cert_hash).unwrap();
let expected = did_key_from_verifying_key(&signer.verifying_key());
assert_eq!(v.signer, expected, "signer must be the verified key's DID");
assert_eq!(v.signer, att.signer, "honest artifact: field and key agree");
// Must-not, both directions: a signer field naming a DIFFERENT key
// than the one that signed must fail verification, so the artifact
// string can never redirect the reported signer.
let mut forged = signed_demo(&signer, cert_hash, "ok");
forged.signer = did_key_from_verifying_key(&other.verifying_key());
let err = reg.verify(&forged, cert_hash).unwrap_err();
assert!(
matches!(err, Error::Signature(_)),
"a signer field that does not match the signing key must fail, got {err:?}"
);
let mut forged_rev = signed_demo(&other, cert_hash, "ok");
forged_rev.signer = did_key_from_verifying_key(&signer.verifying_key());
let err_rev = reg.verify(&forged_rev, cert_hash).unwrap_err();
assert!(
matches!(err_rev, Error::Signature(_)),
"mismatched signer field must fail verification, got {err_rev:?}"
);
}
}
+2 -9
View File
@@ -1,7 +1,7 @@
[package]
name = "gitlawb-core"
description = "Core cryptographic primitives for the Twigpine network: DIDs, CIDs, UCAN, HTTP Signatures, ref-update certificates"
version = "0.7.1" # x-release-please-version
description = "Core cryptographic primitives for the gitlawb network: DIDs, CIDs, UCAN, HTTP Signatures, ref-update certificates"
version = "0.5.1" # x-release-please-version
edition.workspace = true
rust-version.workspace = true
license.workspace = true
@@ -22,7 +22,6 @@ multihash-codetable = { workspace = true }
cid = { workspace = true }
chrono = { workspace = true }
uuid = { workspace = true }
url = { workspace = true, optional = true }
zeroize = { version = "1", features = ["derive"] }
pkcs8 = { version = "0.10", features = ["pem", "std"] }
curve25519-dalek = "4"
@@ -31,9 +30,3 @@ chacha20poly1305 = "0.10"
[dev-dependencies]
tokio = { workspace = true }
# Not optional here: the redirect matrix must run under a bare
# `cargo test -p gitlawb-core`, with no feature selected.
url = { workspace = true }
[features]
redirect = ["dep:url"]
+2 -35
View File
@@ -6,8 +6,6 @@
//! The schema is frozen at v1. All fields are mandatory for forward compatibility.
//! Nodes that receive a certificate with an unknown version MUST reject it.
use std::collections::HashSet;
use chrono::{DateTime, Utc};
use serde::{Deserialize, Serialize};
use uuid::Uuid;
@@ -137,14 +135,10 @@ impl RefUpdateCert {
/// Check if this certificate satisfies a threshold of valid signatures
/// from the provided set of authorized maintainer DIDs.
///
/// Counts distinct signer DIDs, not signature entries: a repeated
/// signature from the same maintainer counts once.
pub fn satisfies_threshold(&self, maintainers: &[Did], threshold: usize) -> Result<bool> {
let valid = self.verify_all()?;
let distinct_signers: HashSet<&Did> =
valid.iter().filter(|d| maintainers.contains(d)).collect();
Ok(distinct_signers.len() >= threshold)
let count = valid.iter().filter(|d| maintainers.contains(d)).count();
Ok(count >= threshold)
}
/// Validate the certificate structure (not signatures).
@@ -350,33 +344,6 @@ mod tests {
assert!(!cert.satisfies_threshold(&maintainers, 1).unwrap());
}
#[test]
fn satisfies_threshold_rejects_duplicated_signature() {
let kp1 = Keypair::generate();
let kp2 = Keypair::generate();
let kp3 = Keypair::generate();
let repo_did = kp1.did();
let mut cert = RefUpdateCert::new(
repo_did,
"refs/heads/main".to_string(),
dummy_hash('0'),
dummy_hash('a'),
1,
&kp1,
)
.unwrap();
// Copy-paste the only real signature onto the certificate a second
// time. Same signer, same valid signature, still one real signer.
let dup = cert.signatures[0].clone();
cert.signatures.push(dup);
let maintainers = vec![kp1.did(), kp2.did(), kp3.did()];
// Two signature entries but a single distinct signer must not
// satisfy a 2-of-3 threshold.
assert!(!cert.satisfies_threshold(&maintainers, 2).unwrap());
}
#[test]
fn threshold_zero_is_always_satisfied() {
let kp = Keypair::generate();
+5 -117
View File
@@ -1,11 +1,11 @@
//! Decentralized Identifier (DID) types for the Twigpine network.
//! Decentralized Identifier (DID) types for the gitlawb network.
//!
//! Supported methods:
//! - `did:key` — ephemeral, derived directly from an Ed25519 public key
//! - `did:web` — anchored to a domain
//! - `did:gitlawb` — native, anchored to the libp2p DHT
//!
//! The canonical DID for a Twigpine actor is `did:key` during bootstrap
//! The canonical DID for a gitlawb actor is `did:key` during bootstrap
//! and migrates to `did:gitlawb` once DHT anchoring is live.
use ed25519_dalek::VerifyingKey;
@@ -71,8 +71,7 @@ impl Did {
/// Resolve the Ed25519 verifying key from a `did:key`.
///
/// Returns `Err` if the DID is not a `did:key`, the key bytes are malformed,
/// or the key is a small-order (weak) point.
/// Returns `Err` if the DID is not a `did:key` or the key bytes are invalid.
pub fn to_verifying_key(&self) -> Result<VerifyingKey> {
if !self.is_did_key() {
return Err(Error::InvalidDid(format!(
@@ -81,19 +80,6 @@ impl Did {
)));
}
// Refuse an oversized id before decoding. base58 decoding is quadratic
// in its input, and both callers of this function take the string from
// an untrusted request: the unauthenticated peer-announce gate, and the
// HTTP-signature keyid. An ed25519 did:key method-id is a fixed 48
// characters, so this bound is slack rather than a behavior change, and
// it has to sit ahead of the decode to be worth anything.
const MAX_METHOD_ID_LEN: usize = 64;
if self.method_id().len() > MAX_METHOD_ID_LEN {
return Err(Error::InvalidDid(
"did:key method-specific id too long".to_string(),
));
}
let (_, bytes) =
multibase::decode(self.method_id()).map_err(|e| Error::InvalidDid(e.to_string()))?;
@@ -107,25 +93,7 @@ impl Did {
.try_into()
.map_err(|_| Error::InvalidDid("ed25519 key must be 32 bytes".to_string()))?;
let key =
VerifyingKey::from_bytes(&key_bytes).map_err(|e| Error::InvalidDid(e.to_string()))?;
// `from_bytes` only decompresses, so it accepts a small-order point.
// Such a key satisfies the verification equation for any message, and
// every X25519 shared secret derived from it is the all-zero key: the
// order-1 and order-2 points convert to the all-zero Montgomery u
// directly, and the order-4 and order-8 points are annihilated by the
// scalar clamping instead. `is_weak` covers the whole torsion set.
// Mixed-order points are correctly NOT rejected, since clamping clears
// the cofactor component. Rejecting
// at resolution is what makes this the choke point: every consumer that
// resolves a DID through here inherits the rejection, and a recipient
// that cannot resolve already fails closed downstream.
if key.is_weak() {
return Err(Error::InvalidDid("small-order ed25519 key".to_string()));
}
Ok(key)
VerifyingKey::from_bytes(&key_bytes).map_err(|e| Error::InvalidDid(e.to_string()))
}
/// Return the full DID string as a `&str`.
@@ -171,7 +139,7 @@ impl From<Did> for String {
}
}
/// A DID Document for a Twigpine actor.
/// A DID Document for a gitlawb actor.
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct DidDocument {
#[serde(rename = "@context")]
@@ -340,84 +308,4 @@ mod tests {
assert_eq!(Did::web("example.com").method_id(), "example.com");
assert_eq!(Did::gitlawb("z6MkSomeKey").method_id(), "z6MkSomeKey");
}
/// An oversized method-specific id is refused on its length, before the
/// base58 decode runs.
///
/// base58 decoding is quadratic in the input, and `to_verifying_key` is
/// reached from two places that take an attacker-supplied string: the
/// unauthenticated peer-announce gate, and the HTTP-signature keyid. A
/// 64k-character DID measured 8s through the router without this guard
/// against 4.6ms with it, on a tokio worker with no spawn_blocking, so a
/// single caller could buy CPU-seconds by the thousand.
///
/// Asserting the MESSAGE is what makes this load-bearing: an oversized id
/// errors either way, so a test that only checked `is_err()` would pass
/// with the guard removed, having merely paid the cost it exists to avoid.
#[test]
fn to_verifying_key_rejects_an_oversized_method_id_before_decoding() {
let oversized = format!("did:key:z{}", "z".repeat(64_000));
let did: Did = oversized.parse().expect("method is still `key`");
let start = std::time::Instant::now();
let err = did.to_verifying_key().expect_err("must be refused");
let elapsed = start.elapsed();
assert!(
err.to_string().contains("too long"),
"must be refused on length, not after the decode: {err}"
);
assert!(
elapsed < std::time::Duration::from_secs(1),
"refusal must not pay the quadratic decode: took {elapsed:?}"
);
}
/// The bound must not reject a real key. Every `did:key` this project
/// mints is a fixed 48 characters, so the limit is slack, not a behavior
/// change.
#[test]
fn to_verifying_key_still_accepts_a_real_did_key() {
use ed25519_dalek::SigningKey;
let signing = SigningKey::from_bytes(&[7u8; 32]);
let did = Did::from_verifying_key(&signing.verifying_key());
assert_eq!(
did.method_id().len(),
48,
"did:key method-id is fixed-width"
);
did.to_verifying_key().expect("a real did:key must resolve");
}
/// A `did:key` can encode a small-order (weak) Ed25519 point. Such a key
/// satisfies the verification equation for any message and, converted to
/// Montgomery form for X25519, yields the all-zero shared secret, so an
/// envelope sealed to it is decryptable by anyone. Resolution is the choke
/// point every `Did`-based consumer shares, so it fails closed here.
#[test]
fn to_verifying_key_rejects_a_small_order_key() {
// The compressed identity point, the canonical small-order key.
let mut weak = [0u8; 32];
weak[0] = 1;
let mut prefixed = Vec::with_capacity(ED25519_MULTICODEC.len() + 32);
prefixed.extend_from_slice(ED25519_MULTICODEC);
prefixed.extend_from_slice(&weak);
let did: Did = format!(
"did:key:{}",
multibase::encode(multibase::Base::Base58Btc, &prefixed)
)
.parse()
.expect("a small-order did:key is still well-formed");
let err = did
.to_verifying_key()
.expect_err("a small-order did:key must not resolve");
match err {
Error::InvalidDid(msg) => assert!(
msg.contains("small-order"),
"rejection must name the small-order key, got: {msg}"
),
other => panic!("expected Error::InvalidDid, got {other:?}"),
}
}
}
+2 -501
View File
@@ -11,22 +11,6 @@ use zeroize::Zeroizing;
/// X25519 public key (Montgomery u) for an Ed25519 verifying key.
fn x25519_public(vk: &VerifyingKey) -> Result<[u8; 32]> {
use curve25519_dalek::edwards::CompressedEdwardsY;
// Every X25519 shared secret derived from a small-order point is the
// all-zero key: order-1 and order-2 convert to the all-zero Montgomery u
// directly, order-4 and order-8 are annihilated by the scalar clamping in
// x25519_secret_from_seed. Either way the per-recipient wrap could be
// rebuilt by anyone. Resolution already refuses such a key (see
// Did::to_verifying_key); this guard covers the SEAL side on its own terms
// for a caller that obtained the key some other way. The open side
// (open_blob's attacker-supplied `eph`) skips any entry whose exchange with
// this reader yields the all-zero shared secret
// (`yields_all_zero_shared_secret`), regardless of whether the bytes decode
// to an Edwards point.
if vk.is_weak() {
return Err(anyhow::anyhow!("verifying key is a small-order point"));
}
let edwards = CompressedEdwardsY::from_slice(vk.as_bytes())
.ok()
.and_then(|c| c.decompress())
@@ -34,26 +18,6 @@ fn x25519_public(vk: &VerifyingKey) -> Result<[u8; 32]> {
Ok(edwards.to_montgomery().to_bytes())
}
/// True when the X25519 exchange between an attacker-supplied `u` and this
/// reader's scalar yields the all-zero shared secret.
///
/// Asked of the RESULT, not of the encoding, and that distinction is the whole
/// guard. The previous version decompressed `u` to an Edwards point and asked
/// `is_small_order`, which reports "safe" for any low-order encoding that does
/// not decompress at all. Measured across the seven standard low-order
/// encodings, six were caught and `u = p - 1` was not: it fails to decompress,
/// so the check returned false, and the exchange is still the all-zero secret.
/// An entry built on it unwraps for every reader.
///
/// Enumerating encodings cannot be exhaustive, because the attack depends on the
/// shared secret rather than on the bytes that produced it, and twist and
/// non-canonical inputs keep arriving. Asking the result needs no list.
fn yields_all_zero_shared_secret(u: &[u8; 32], scalar: &[u8; 32]) -> bool {
use curve25519_dalek::montgomery::MontgomeryPoint;
MontgomeryPoint(*u).mul_clamped(*scalar).to_bytes() == [0u8; 32]
}
/// X25519 secret scalar for an Ed25519 seed (SHA-512 of seed, lower 32, clamped).
/// Returns the scalar wrapped in `Zeroizing`, and scrubs the intermediate
/// SHA-512 digest, so no copy of this secret material lingers in freed memory.
@@ -85,13 +49,6 @@ use serde::{Deserialize, Serialize};
const MAGIC: &[u8] = b"GLENC";
const VERSION: u8 = 2;
/// Upper bound on the blinded recipient list. The envelope author controls
/// this count and `open_blob` pays one X25519 exchange per entry before it can
/// answer "not a recipient", so the count has to be capped rather than
/// optimized (#419). Legitimate envelopes carry a handful of authorized
/// readers, so 256 is far above real use.
const MAX_RECIPIENTS: usize = 256;
#[derive(Serialize, Deserialize)]
struct Recipient {
eph: String, // base64 ephemeral x25519 pubkey (32B)
@@ -111,14 +68,6 @@ pub fn seal_blob(plaintext: &[u8], recipients: &[VerifyingKey]) -> Result<Vec<u8
if recipients.is_empty() {
return Err(anyhow::anyhow!("seal_blob: no recipients"));
}
// Same bound open_blob enforces: an envelope over the cap could never be
// opened, so refuse to seal it (#419).
if recipients.len() > MAX_RECIPIENTS {
return Err(anyhow::anyhow!(
"seal_blob: {} recipients, over the {MAX_RECIPIENTS} cap",
recipients.len()
));
}
let mut content_key = [0u8; 32];
OsRng.fill_bytes(&mut content_key);
let body_cipher = XChaCha20Poly1305::new_from_slice(&content_key)
@@ -177,20 +126,9 @@ pub fn open_blob(envelope: &[u8], keypair: &Keypair) -> Result<Vec<u8>> {
let header: Header =
serde_json::from_slice(envelope.get(p..p + hlen).context("truncated header")?)
.context("decode header")?;
// Reject before the recipient scan: every entry below is an X25519
// exchange, and a non-recipient would run the whole list (#419).
if header.recipients.len() > MAX_RECIPIENTS {
return Err(anyhow::anyhow!(
"envelope carries {} recipients, over the {MAX_RECIPIENTS} cap",
header.recipients.len()
));
}
let body = &envelope[p + hlen..];
// The raw scalar is kept alongside the box secret so the exchange can be
// tested BEFORE a box is built from an attacker-supplied ephemeral.
let my_x_scalar = x25519_secret_from_seed(&keypair.to_seed());
let my_x = XSecret::from(*my_x_scalar);
let my_x = XSecret::from(*x25519_secret_from_seed(&keypair.to_seed()));
// Identities are blinded: no entry says which recipient it belongs to, so
// try each one. The ChaChaBox AEAD tag authenticates, so exactly the
@@ -202,12 +140,7 @@ pub fn open_blob(envelope: &[u8], keypair: &Keypair) -> Result<Vec<u8>> {
.ok()
.and_then(|b| <[u8; 32]>::try_from(b.as_slice()).ok())
{
// An ephemeral whose exchange with this reader is the all-zero
// shared secret would unwrap for anyone, so the entry is skipped.
// Tested on the exchange itself: an encoding-shaped check misses
// every low-order input that does not decompress to Edwards.
Some(b) if !yields_all_zero_shared_secret(&b, &my_x_scalar) => XPublic::from(b),
Some(_) => continue,
Some(b) => XPublic::from(b),
None => continue,
};
// from_slice panics on a wrong length, and the envelope is attacker
@@ -360,436 +293,4 @@ mod tests {
header["nonce"] = bad_nonce;
assert!(open_blob(&reframe(&header), &reader).is_err());
}
#[test]
fn over_cap_recipient_list_is_rejected_before_the_scan() {
// #419: the envelope author controls the recipient count and each entry
// costs an X25519 exchange, so the list is capped. Reframe a valid
// envelope with MAX_RECIPIENTS + 1 junk entries.
let reader = Keypair::generate();
let env = seal_blob(b"private blob contents", &[reader.verifying_key()]).unwrap();
let mut p = MAGIC.len() + 1;
let hlen = u32::from_le_bytes(env[p..p + 4].try_into().unwrap()) as usize;
p += 4;
let header_bytes = &env[p..p + hlen];
let body = &env[p + hlen..];
let mut header: serde_json::Value = serde_json::from_slice(header_bytes).unwrap();
let junk = serde_json::json!({
"eph": B64.encode([0u8; 32]),
"nonce": B64.encode([0u8; 24]),
"wrap": B64.encode([0u8; 48]),
});
header["recipients"] = serde_json::Value::Array(vec![junk.clone(); MAX_RECIPIENTS + 1]);
let hj = serde_json::to_vec(&header).unwrap();
let mut fat = Vec::new();
fat.extend_from_slice(MAGIC);
fat.push(VERSION);
fat.extend_from_slice(&(hj.len() as u32).to_le_bytes());
fat.extend_from_slice(&hj);
fat.extend_from_slice(body);
let err = open_blob(&fat, &reader).unwrap_err();
assert!(
err.to_string().contains("recipients"),
"expected the recipient-cap error, got: {err}"
);
// Exactly at the cap the envelope is still processed end to end.
header["recipients"] = serde_json::Value::Array(vec![junk; MAX_RECIPIENTS]);
let hj = serde_json::to_vec(&header).unwrap();
let mut at_cap = Vec::new();
at_cap.extend_from_slice(MAGIC);
at_cap.push(VERSION);
at_cap.extend_from_slice(&(hj.len() as u32).to_le_bytes());
at_cap.extend_from_slice(&hj);
at_cap.extend_from_slice(body);
let err = open_blob(&at_cap, &reader).unwrap_err();
assert!(
!err.to_string().contains("over the"),
"at-cap envelope must reach the scan, got: {err}"
);
}
#[test]
fn seal_refuses_a_recipient_list_over_the_cap() {
// An envelope over the cap can never be opened, so seal must reject it
// instead of minting one.
let keys: Vec<VerifyingKey> = (0..MAX_RECIPIENTS + 1)
.map(|_| Keypair::generate().verifying_key())
.collect();
let err = seal_blob(b"blob", &keys).unwrap_err();
assert!(
err.to_string().contains("recipients"),
"expected the recipient-cap error, got: {err}"
);
}
/// Every standard low-order X25519 encoding must be detected by
/// `yields_all_zero_shared_secret`, including encodings that do not
/// decompress to an Edwards point (notably `u = p - 1`). The open side
/// rejects on the exchange result, not on decode shape, because an
/// encoding-shaped check misses non-decompressing low-order inputs while
/// still yielding the all-zero shared secret.
#[test]
fn all_zero_shared_secret_is_detected_for_every_standard_low_order_encoding() {
// The seven standard X25519 low-order encodings, driven as a set rather
// than as the two that happen to decompress. The encoding-shaped check
// this replaced reported `p - 1` as safe, and its exchange is all-zero.
let mut one = [0u8; 32];
one[0] = 1;
let hex = |h: &str| -> [u8; 32] {
let mut out = [0u8; 32];
for i in 0..32 {
out[i] = u8::from_str_radix(&h[i * 2..i * 2 + 2], 16).unwrap();
}
out
};
let vectors = [
("u=0", [0u8; 32]),
("u=1", one),
(
"order8-a",
hex("e0eb7a7c3b41b8ae1656e3faf19fc46ada098deb9c32b1fd866205165f49b800"),
),
(
"order8-b",
hex("5f9c95bca3508c24b1d0b1559c83ef5b04445cc4581c8e86d8224eddd09f1157"),
),
(
"p-1",
hex("ecffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff7f"),
),
(
"p",
hex("edffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff7f"),
),
(
"p+1",
hex("eeffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff7f"),
),
];
let kp = Keypair::generate();
let scalar = x25519_secret_from_seed(&kp.to_seed());
for (name, u) in vectors {
assert!(
yields_all_zero_shared_secret(&u, &scalar),
"{name} is a low-order encoding and must be detected"
);
}
// And the other direction, or a guard that returned true always would
// pass everything above while making the envelope unopenable.
let real_u = x25519_public(&kp.verifying_key()).unwrap();
assert!(
!yields_all_zero_shared_secret(&real_u, &scalar),
"a real key must not be treated as low-order"
);
}
/// The open side must skip an entry whose header-supplied `eph` is a
/// low-order point: such an entry forces the all-zero shared secret and
/// would unwrap for any reader. A poisoned entry among honest ones must
/// not DoS the envelope — the honest entry still opens.
#[test]
fn open_blob_skips_a_low_order_ephemeral_and_still_opens_honest_entry() {
use curve25519_dalek::montgomery::MontgomeryPoint;
let reader = Keypair::generate();
let env = seal_blob(b"private blob contents", &[reader.verifying_key()]).unwrap();
// Split the envelope framing into header JSON and body.
let mut p = MAGIC.len() + 1;
let hlen = u32::from_le_bytes(env[p..p + 4].try_into().unwrap()) as usize;
p += 4;
let header_bytes = &env[p..p + hlen];
let body = &env[p + hlen..];
// A low-order Montgomery u (u = 0) encoded as the entry's eph.
let low_order_u = MontgomeryPoint([0u8; 32]).to_bytes();
let poisoned = serde_json::json!({
"eph": B64.encode(low_order_u),
"nonce": B64.encode([0u8; 24]),
"wrap": B64.encode([0u8; 32]),
});
let reframe = |header: &serde_json::Value| -> Vec<u8> {
let hj = serde_json::to_vec(header).unwrap();
let mut out = Vec::new();
out.extend_from_slice(MAGIC);
out.push(VERSION);
out.extend_from_slice(&(hj.len() as u32).to_le_bytes());
out.extend_from_slice(&hj);
out.extend_from_slice(body);
out
};
let mut header: serde_json::Value = serde_json::from_slice(header_bytes).unwrap();
// Prepend the poisoned entry. The reader's honest entry must still
// unwrap, so the envelope opens despite the low-order entry.
let recipients = header["recipients"].as_array_mut().unwrap();
recipients.insert(0, poisoned);
assert_eq!(
open_blob(&reframe(&header), &reader).unwrap(),
b"private blob contents",
"the honest entry must still open despite a low-order eph entry"
);
}
/// Must-not for the open-side guard: an envelope whose ONLY entry is built
/// on a low-order ephemeral must not open for any reader. Without the
/// guard the all-zero shared secret unwraps for everyone, so the attacker
/// controls the plaintext and the envelope's confidentiality is void.
#[test]
fn open_blob_rejects_a_poisoned_only_envelope() {
use chacha20poly1305::{aead::Aead, KeyInit, XChaCha20Poly1305, XNonce};
use crypto_box::aead::AeadCore;
use curve25519_dalek::montgomery::MontgomeryPoint;
// Build a poisoned envelope from scratch: one recipient entry whose
// eph is the low-order u = 0. The wrap is the encryption of a content
// key under the all-zero shared secret, so without the guard any
// reader's ChaChaBox against u = 0 decrypts it.
let mut content_key: [u8; 32] = Default::default();
OsRng.fill_bytes(&mut content_key);
let attacker_x = XSecret::generate(&mut OsRng);
let zero_box = ChaChaBox::new(&XPublic::from(<[u8; 32]>::default()), &attacker_x);
let nonce = ChaChaBox::generate_nonce(&mut OsRng);
let wrap = zero_box.encrypt(&nonce, &content_key[..]).unwrap();
let mut body_nonce: [u8; 24] = Default::default();
OsRng.fill_bytes(&mut body_nonce);
let body_cipher = XChaCha20Poly1305::new_from_slice(&content_key).unwrap();
let body = body_cipher
.encrypt(
XNonce::from_slice(&body_nonce),
b"attacker-chosen plaintext".as_slice(),
)
.unwrap();
let header = serde_json::json!({
"alg": "xchacha20poly1305",
"nonce": B64.encode(body_nonce),
"recipients": [{
"eph": B64.encode(MontgomeryPoint([0u8; 32]).to_bytes()),
"nonce": B64.encode(nonce),
"wrap": B64.encode(wrap),
}],
});
let header_json = serde_json::to_vec(&header).unwrap();
let mut env = Vec::new();
env.extend_from_slice(MAGIC);
env.push(VERSION);
env.extend_from_slice(&(header_json.len() as u32).to_le_bytes());
env.extend_from_slice(&header_json);
env.extend_from_slice(&body);
let reader = Keypair::generate();
let err = open_blob(&env, &reader).unwrap_err();
assert!(
err.to_string().contains("not a recipient"),
"a poisoned-only envelope must not open for anyone, got: {err}"
);
}
/// The same must-not, driven with the low-order encoding that does NOT map
/// to an Edwards point.
///
/// This is the case an encoding-shaped guard cannot see. `u = p - 1` is a
/// legal X25519 input whose exchange is the all-zero secret for any scalar,
/// and `MontgomeryPoint::to_edwards` returns None for it, so a check written
/// as "decompress, then ask is_small_order" reports it as safe. Measured
/// across the seven standard low-order encodings: six are caught that way
/// and this one is not.
///
/// Found by a cross-family review of the head that first added the guard,
/// and the comment beside that guard asserted this case was already handled
/// by a decode path that does not exist. That is why the fix rejects on the
/// RESULT of the exchange rather than on a list of encodings: the result is
/// what the attack actually depends on, and it needs no enumeration to be
/// exhaustive.
#[test]
fn open_blob_rejects_a_non_edwards_low_order_ephemeral() {
// No blanket `use ...::Aead` here. Two aead versions are reachable, and a
// blanket import silently resolved the wrap through the one open_blob does
// NOT decrypt with, producing a ciphertext that could never open. The test
// then passed with the guard, without it, and with the original pre-fix
// guard alike, proving nothing. Every call below names its trait.
use chacha20poly1305::{KeyInit, XChaCha20Poly1305, XNonce};
use crypto_box::aead::AeadCore;
// u = p - 1 = 2^255 - 20, little endian.
let mut eph = [0xffu8; 32];
eph[0] = 0xec;
eph[31] = 0x7f;
let mut content_key: [u8; 32] = Default::default();
OsRng.fill_bytes(&mut content_key);
let attacker_x = XSecret::generate(&mut OsRng);
let zero_box = ChaChaBox::new(&XPublic::from(<[u8; 32]>::default()), &attacker_x);
let nonce = ChaChaBox::generate_nonce(&mut OsRng);
let wrap = crypto_box::aead::Aead::encrypt(&zero_box, &nonce, &content_key[..]).unwrap();
let mut body_nonce: [u8; 24] = Default::default();
OsRng.fill_bytes(&mut body_nonce);
let body_cipher = XChaCha20Poly1305::new_from_slice(&content_key).unwrap();
let body = chacha20poly1305::aead::Aead::encrypt(
&body_cipher,
XNonce::from_slice(&body_nonce),
b"attacker-chosen plaintext".as_slice(),
)
.unwrap();
let header = serde_json::json!({
"alg": "xchacha20poly1305",
"nonce": B64.encode(body_nonce),
"recipients": [{
"eph": B64.encode(eph),
"nonce": B64.encode(nonce),
"wrap": B64.encode(wrap),
}],
});
let header_json = serde_json::to_vec(&header).unwrap();
let mut env = Vec::new();
env.extend_from_slice(MAGIC);
env.push(VERSION);
env.extend_from_slice(&(header_json.len() as u32).to_le_bytes());
env.extend_from_slice(&header_json);
env.extend_from_slice(&body);
let reader = Keypair::generate();
let err = open_blob(&env, &reader).unwrap_err();
assert!(
err.to_string().contains("not a recipient"),
"an envelope whose only entry uses a non-Edwards low-order eph must not \
open for anyone, got: {err}"
);
}
/// The compressed identity point: a well-formed Ed25519 encoding that is
/// small-order, so every shared secret derived from it is all-zero.
fn weak_verifying_key() -> VerifyingKey {
let mut weak = [0u8; 32];
weak[0] = 1;
let vk = VerifyingKey::from_bytes(&weak).expect("identity point decompresses");
assert!(vk.is_weak(), "fixture precondition: key is small-order");
vk
}
/// A small-order recipient key converts to Montgomery u = 0, and X25519
/// against u = 0 is the all-zero shared secret for ANY scalar, so the
/// wrapping box is reconstructable with no secret at all. Reject at the
/// primitive so the seal side is safe regardless of how the caller
/// obtained the key.
#[test]
fn x25519_public_rejects_a_small_order_key() {
let weak_vk = weak_verifying_key();
assert!(
x25519_public(&weak_vk).is_err(),
"a small-order key must not yield an x25519 public key"
);
}
/// Control: a legitimate key still converts, and to a non-zero u.
#[test]
fn x25519_public_still_accepts_a_real_key() {
let kp = Keypair::generate();
let u = x25519_public(&kp.verifying_key()).expect("a real key must convert");
assert_ne!(u, [0u8; 32], "a real key must not map to the zero point");
}
/// The guard has to propagate: no envelope may be produced for a weak
/// recipient even when the DID choke point is bypassed entirely.
#[test]
fn seal_blob_refuses_a_small_order_recipient() {
let weak_vk = weak_verifying_key();
assert!(
seal_blob(b"withheld", &[weak_vk]).is_err(),
"sealing to a weak recipient must fail"
);
// And a mixed set must fail too: one weak recipient exposes the shared
// content key, so a partial envelope is not an acceptable outcome.
let honest = Keypair::generate();
assert!(
seal_blob(b"withheld", &[honest.verifying_key(), weak_vk]).is_err(),
"a mixed honest+weak recipient set must fail closed, not seal partially"
);
}
/// Control for the two above: the ordinary seal/open round trip is intact.
#[test]
fn legit_seal_open_round_trip_still_works() {
let reader = Keypair::generate();
let env = seal_blob(b"withheld blob", &[reader.verifying_key()]).expect("seal");
assert_eq!(open_blob(&env, &reader).expect("open"), b"withheld blob");
}
/// The defect this fix exists to close, kept as an executable regression
/// rather than an assertion about absence. It runs the real attack: craft a
/// small-order did:key, get it into a recipient set alongside an honest
/// reader, then rebuild the wrapping box from the all-zero shared secret
/// that a small-order recipient forces. If any guard regresses, this does
/// not merely fail, it fails printing the plaintext it recovered.
#[test]
fn attacker_cannot_recover_plaintext_via_a_weak_recipient() {
use crate::did::Did;
use std::str::FromStr;
let weak_vk = weak_verifying_key();
// Layer 1: the attacker's did:key string must not resolve at all.
let weak_did = Did::from_verifying_key(&weak_vk).to_string();
assert!(
Did::from_str(&weak_did)
.expect("still a well-formed did:key")
.to_verifying_key()
.is_err(),
"a small-order did:key must not resolve"
);
// Layer 2: even handed the key directly, sealing must refuse. One weak
// recipient would expose the single shared content key, so the honest
// reader's blob would be readable by anyone.
let honest = Keypair::generate();
let secret_plaintext = b"WITHHELD BLOB CONTENTS";
let envelope = match seal_blob(secret_plaintext, &[honest.verifying_key(), weak_vk]) {
Err(_) => return, // no envelope exists, nothing to attack
Ok(env) => env,
};
// Only reachable if a guard regressed. Run the attack and report what
// it got, so the failure names the actual exposure.
let mut p = MAGIC.len() + 1;
let hlen = u32::from_le_bytes(envelope[p..p + 4].try_into().unwrap()) as usize;
p += 4;
let header: serde_json::Value = serde_json::from_slice(&envelope[p..p + hlen]).unwrap();
let body = &envelope[p + hlen..];
let body_nonce = B64.decode(header["nonce"].as_str().unwrap()).unwrap();
// X25519 against u = 0 is the all-zero shared secret for any scalar, so
// the sealer's box is reconstructable with a key the attacker picks.
let zero_box = ChaChaBox::new(
&XPublic::from(<[u8; 32]>::default()),
&XSecret::generate(&mut OsRng),
);
for r in header["recipients"].as_array().unwrap() {
let n = B64.decode(r["nonce"].as_str().unwrap()).unwrap();
let w = B64.decode(r["wrap"].as_str().unwrap()).unwrap();
if let Ok(content_key) = zero_box.decrypt(n.as_slice().into(), w.as_slice()) {
let cipher = XChaCha20Poly1305::new_from_slice(&content_key).unwrap();
let recovered = cipher
.decrypt(XNonce::from_slice(&body_nonce), body)
.expect("body decrypts once the content key is out");
panic!(
"REGRESSION: attacker recovered plaintext with no private key: {:?}",
String::from_utf8_lossy(&recovered)
);
}
}
panic!("an envelope was sealed to a small-order recipient; the seal guard regressed");
}
}
+12 -235
View File
@@ -1,6 +1,6 @@
//! HTTP Signatures (RFC 9421) for gitlawb.
//!
//! Every write request to a Twigpine node must be signed by the actor's
//! Every write request to a gitlawb node must be signed by the actor's
//! Ed25519 private key. The signature covers:
//! - `@method` — HTTP method (uppercase)
//! - `@path` — request path and query
@@ -14,13 +14,13 @@
use base64::{engine::general_purpose::STANDARD, Engine};
use chrono::Utc;
use sha2::{Digest, Sha256};
use std::collections::{HashMap, HashSet};
use std::collections::HashMap;
use crate::did::Did;
use crate::identity::Keypair;
use crate::{Error, Result};
/// The component identifiers covered by every Twigpine signature.
/// The component identifiers covered by every gitlawb signature.
pub const COVERED_COMPONENTS: &[&str] = &["@method", "@path", "content-digest"];
/// The three headers produced by RFC 9421 signing.
@@ -67,20 +67,11 @@ impl HttpSignature {
let components_str = &rest[open + 1..close];
let params_str = &rest[close + 1..]; // starts with ';'
// RFC 9421 §2.1: a component identifier appears at most once. Gitlawb
// only supports bare names; parameters are rejected so reordered
// equivalent forms cannot bypass duplicate detection as distinct strings.
let mut components = Vec::new();
let mut seen = HashSet::new();
for token in components_str.split_whitespace() {
let name = parse_covered_component(token)?;
if !seen.insert(name.clone()) {
return Err(Error::HttpSignature(format!(
"duplicate covered component '{name}' in Signature-Input"
)));
}
components.push(name);
}
// "\"@method\" \"@path\" \"content-digest\"" → ["@method", "@path", "content-digest"]
let components: Vec<String> = components_str
.split_whitespace()
.map(|s| s.trim_matches('"').to_string())
.collect();
let params = parse_params(params_str)?;
@@ -122,48 +113,15 @@ impl HttpSignature {
})
}
/// Reject a `created` timestamp outside the acceptance window, which is
/// ASYMMETRIC: up to 5 minutes into the past, but only 60 seconds into the
/// future.
///
/// The two directions are two comparisons rather than one distance, because
/// they are not the same event. A signature that arrives late is ordinary:
/// clocks run slow, requests queue, and networks are not instant, so the
/// past side keeps the full 300-second allowance. A signature stamped ahead
/// of this node's clock is not ordinary, and folding both sides into an
/// `abs()` let a signer extend its own validity: signed at T with
/// `created = T + 300`, it stayed acceptable until roughly T + 600, so the
/// effective window was double the one this function advertises. The
/// forward tolerance exists only to absorb honest drift between two hosts,
/// which is what 60 seconds buys; it is the same allowance the gossip
/// path's freshness check uses for the same reason.
/// Reject if the `created` timestamp is more than 5 minutes from now.
pub fn check_created(&self) -> Result<()> {
const MAX_AGE_SECS: i64 = 300;
const MAX_FUTURE_SECS: i64 = 60;
let now = Utc::now().timestamp();
// Saturating, not plain subtraction. `created` is parsed as an
// unrestricted i64 straight from the Signature-Input header, so a
// sender picks it: at i64::MIN the past-side subtraction overflows,
// which panics a debug build and wraps a release one into an absurd
// value that can read as acceptable. Saturating gives the answer the
// check wants at both extremes, since a timestamp that far away is
// refused by whichever side it saturates toward.
let age = now.saturating_sub(self.created);
if age > MAX_AGE_SECS {
let skew = (now - self.created).abs();
if skew > 300 {
return Err(Error::HttpSignature(format!(
"clock skew too large: created {age}s in the past (max {MAX_AGE_SECS}s)"
"clock skew too large: {skew}s (max 300s)"
)));
}
let ahead = self.created.saturating_sub(now);
if ahead > MAX_FUTURE_SECS {
return Err(Error::HttpSignature(format!(
"clock skew too large: created {ahead}s in the future (max {MAX_FUTURE_SECS}s)"
)));
}
Ok(())
}
@@ -245,33 +203,6 @@ pub fn compute_content_digest(body: &[u8]) -> String {
format!("sha-256=:{}:", STANDARD.encode(hasher.finalize()))
}
/// Parse one covered-component token from Signature-Input's parenthesized list.
///
/// Gitlawb only supports bare component names (`"@method"`, `"@path"`,
/// `"content-digest"`). RFC 9421 permits parameters on identifiers; they are
/// rejected rather than accepted in a form `missing_components()` and
/// `build_signing_string()` cannot handle consistently.
fn parse_covered_component(token: &str) -> Result<String> {
let token = token.trim();
let open = token
.strip_prefix('"')
.ok_or_else(|| Error::HttpSignature("component must be a quoted string".into()))?;
let close = open
.find('"')
.ok_or_else(|| Error::HttpSignature("unterminated component string".into()))?;
let name = &open[..close];
if name.is_empty() {
return Err(Error::HttpSignature("empty component name".into()));
}
let trailing = open[close + 1..].trim();
if !trailing.is_empty() {
return Err(Error::HttpSignature(format!(
"component parameters are not supported on '{name}'"
)));
}
Ok(name.to_string())
}
/// Parse `;key="value";key2=value` parameter string into a map.
fn parse_params(s: &str) -> Result<HashMap<String, String>> {
let mut map = HashMap::new();
@@ -341,91 +272,6 @@ mod tests {
assert!(missing.contains(&"content-digest"));
}
/// RFC 9421 §2.1: a component identifier must not appear twice in the
/// covered list. Repeating one is not a way to say anything, but it does
/// repeat a line in the signing string, so the size of what a verifier
/// builds (and what a node persists alongside a claim) is set by how many
/// times the caller chose to write the same name.
#[test]
fn parse_rejects_component_parameters() {
let kp = Keypair::generate();
let did = kp.did();
let sig_input = format!(
r#"sig1=("@method" "@path" "content-digest";sf);keyid="{did}";alg="ed25519";created=1000"#
);
let err = HttpSignature::parse(
&sig_input,
"sig1=:AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA:",
)
.expect_err("parameterized components must be refused");
assert!(
err.to_string().contains("component parameters"),
"got: {err}"
);
}
#[test]
fn parse_rejects_reordered_equivalent_component_parameters() {
let kp = Keypair::generate();
let did = kp.did();
let sig_input = format!(
r#"sig1=("content-digest";sf;tr "content-digest";tr;sf "@method" "@path");keyid="{did}";alg="ed25519";created=1000"#
);
let err = HttpSignature::parse(
&sig_input,
"sig1=:AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA:",
)
.expect_err("reordered equivalent parameters must not bypass as distinct components");
assert!(
err.to_string().contains("component parameters"),
"got: {err}"
);
}
#[test]
fn parse_many_distinct_components_completes_quickly() {
use std::time::{Duration, Instant};
let kp = Keypair::generate();
let did = kp.did();
let names: Vec<String> = (0..20_000).map(|i| format!("\"c{i}\"")).collect();
let inner = names.join(" ");
let sig_input = format!(r#"sig1=({inner});keyid="{did}";alg="ed25519";created=1000"#);
let start = Instant::now();
let result = HttpSignature::parse(
&sig_input,
"sig1=:AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA:",
);
let elapsed = start.elapsed();
assert!(
elapsed < Duration::from_secs(2),
"duplicate tracking must stay linear, took {elapsed:?}"
);
assert!(result.is_ok(), "distinct names should parse: {:?}", result);
}
#[test]
fn parse_rejects_duplicate_components() {
let kp = Keypair::generate();
let did = kp.did();
let sig_input = format!(
r#"sig1=("@method" "@path" "@path" "content-digest");keyid="{did}";alg="ed25519";created=1000"#
);
let err = HttpSignature::parse(&sig_input, "sig1=:AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA:")
.expect_err("a repeated component must be refused");
assert!(
err.to_string().contains("duplicate"),
"the error must name the duplication, got: {err}"
);
// The control: the same list without the repeat still parses.
let ok = format!(
r#"sig1=("@method" "@path" "content-digest");keyid="{did}";alg="ed25519";created=1000"#
);
HttpSignature::parse(&ok, "sig1=:AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA:")
.expect("a distinct component list must still parse");
}
#[test]
fn verify_signature_end_to_end() {
use crate::identity::verify;
@@ -489,75 +335,6 @@ mod tests {
assert!(sig.check_created().is_err());
}
/// Build a parseable signature whose `created` is `offset` seconds from now
/// (negative for the past), so a test can pin one side of the window.
fn sig_created_at_offset(offset: i64) -> HttpSignature {
let kp = Keypair::generate();
let did = kp.did();
let created = Utc::now().timestamp() + offset;
let sig_input = format!(
r#"sig1=("@method" "@path" "content-digest");keyid="{did}";alg="ed25519";created={created}"#
);
HttpSignature::parse(&sig_input, "sig1=:AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA:").unwrap()
}
/// Build a signature at an ABSOLUTE `created`, so a test can drive the
/// extremes an offset from now cannot reach.
fn sig_created_absolute(created: i64) -> HttpSignature {
let kp = Keypair::generate();
let did = kp.did();
let sig_input = format!(
r#"sig1=("@method" "@path" "content-digest");keyid="{did}";alg="ed25519";created={created}"#
);
HttpSignature::parse(&sig_input, "sig1=:AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA:").unwrap()
}
/// `created` is parsed as an unrestricted i64 from a header the sender
/// writes, so the window check has to survive both ends of the type. Before
/// the saturating form this panicked with "attempt to subtract with
/// overflow" on the past side, which is a debug-build crash any client
/// could trigger by sending one header value.
#[test]
fn extreme_created_values_are_refused_rather_than_overflowing() {
for created in [i64::MIN, i64::MIN + 1, i64::MAX - 1, i64::MAX] {
let err = sig_created_absolute(created)
.check_created()
.expect_err("an extreme created must be refused, not accepted");
let msg = err.to_string();
assert!(
msg.contains("past") || msg.contains("future"),
"created={created} must be refused by a named direction, got: {msg}"
);
}
}
#[test]
fn created_301s_in_past_rejected() {
let err = sig_created_at_offset(-301).check_created().unwrap_err();
assert!(err.to_string().contains("past"), "{err}");
}
#[test]
fn created_299s_in_past_accepted() {
assert!(sig_created_at_offset(-299).check_created().is_ok());
}
#[test]
fn created_61s_in_future_rejected() {
let err = sig_created_at_offset(61).check_created().unwrap_err();
assert!(err.to_string().contains("future"), "{err}");
}
#[test]
fn created_30s_in_future_accepted() {
assert!(sig_created_at_offset(30).check_created().is_ok());
}
#[test]
fn created_exactly_now_accepted() {
assert!(sig_created_at_offset(0).check_created().is_ok());
}
#[test]
fn fresh_signature_passes_clock_skew() {
let kp = Keypair::generate();
+4 -25
View File
@@ -1,6 +1,6 @@
//! Keypair generation, storage, and signing.
//!
//! A Twigpine identity is an Ed25519 keypair. The public key encodes into a
//! A gitlawb identity is an Ed25519 keypair. The public key encodes into a
//! `did:key` DID. The private key is stored as PKCS#8 PEM on disk.
use ed25519_dalek::{Signature, Signer, SigningKey, VerifyingKey};
@@ -11,7 +11,7 @@ use zeroize::Zeroizing;
use crate::did::Did;
use crate::{Error, Result};
/// An Ed25519 keypair that is the root identity for a Twigpine actor.
/// An Ed25519 keypair that is the root identity for a gitlawb actor.
#[derive(Clone)]
pub struct Keypair {
signing_key: SigningKey,
@@ -78,9 +78,10 @@ impl Keypair {
/// Verify an Ed25519 signature.
pub fn verify(verifying_key: &VerifyingKey, msg: &[u8], sig_bytes: &[u8; 64]) -> Result<()> {
use ed25519_dalek::Verifier;
let sig = Signature::from_bytes(sig_bytes);
verifying_key
.verify_strict(msg, &sig)
.verify(msg, &sig)
.map_err(|_| Error::SignatureInvalid)
}
@@ -208,28 +209,6 @@ mod tests {
);
}
#[test]
fn verify_rejects_weak_key_signature() {
// Regression guard for strict verification: a signature forged under a
// weak (small-order) public key satisfies the verification equation
// but must be rejected. The identity point is such a key: with R the
// identity and S = 0, [S]B - [k]A is the identity for any message.
let mut weak_key_bytes = [0u8; 32];
weak_key_bytes[0] = 1; // compressed identity point (y = 1, x = 0)
let weak_vk = VerifyingKey::from_bytes(&weak_key_bytes).unwrap();
assert!(weak_vk.is_weak(), "identity point must be a weak key");
let msg = b"forged message";
let mut forged = [0u8; 64];
forged[0] = 1; // R = identity point, S = 0
let result = verify(&weak_vk, msg, &forged);
assert!(
result.is_err(),
"signature under a weak (small-order) public key must be rejected"
);
}
#[test]
fn signed_payload_round_trip() {
let kp = Keypair::generate();
-10
View File
@@ -5,16 +5,6 @@ pub mod encrypt;
pub mod error;
pub mod http_sig;
pub mod identity;
// `url` is the one dependency here that drags a tail (idna, then the icu
// crates), and gitlawb-core is allowlisted to stay embeddable. Every client that
// needs this predicate already parses URLs, so they opt in and nothing else
// pays. `test` is in the cfg so `cargo test -p gitlawb-core` still compiles and
// runs the matrix below with no feature selected; without it the tests would
// silently not run, which is the failure this module exists to prevent.
#[cfg(any(feature = "redirect", test))]
pub mod redirect;
pub mod sanitize;
pub mod scan_token;
pub mod ucan;
pub use error::Error;
-241
View File
@@ -1,241 +0,0 @@
//! Redirect policy shared by every Twigpine HTTP client that signs its requests.
//!
//! Both `gl` (async) and `git-remote-gitlawb` (blocking) attach RFC 9421
//! `Signature` and `Signature-Input` headers, and reqwest strips only
//! `Authorization`, `Cookie`, `Proxy-Authorization` and `WWW-Authenticate` when a
//! redirect crosses hosts. A signature would survive that hop, and it binds
//! `@method`, `@path` and `content-digest` with no authority component, so a node
//! answering 302 could hand a working credential to a host of its choosing and read
//! as the caller anywhere until the clock-skew window closes. On a 307/308 the
//! request body goes along with it, which for the remote helper is the pack.
//!
//! That same missing authority component is why the predicate also pins the
//! request-target: `@path` is signed as the client sent it and verified as the node
//! received it, so a same-origin hop that rewrites the path or the query (a
//! trailing-slash or query normalization) makes the signature cover a target the
//! node never saw and the read 401s. Only a hop that re-issues the identical target,
//! an http-to-https upgrade being the one that matters in practice, is followed.
//!
//! The decision lives here rather than in either client because the two used to
//! disagree: `gl` was scoped to the origin while the remote helper, the binary that
//! actually runs `git clone gitlawb://`, still ran reqwest's default and followed
//! anywhere. One predicate is what keeps a future third client from repeating that.
//!
//! The type is `url::Url`, which is what `reqwest::Url` re-exports, so both clients
//! pass their attempt URLs straight in.
/// Longest redirect chain followed. `reqwest::redirect::Policy::custom` replaces
/// reqwest's built-in limit, so the bound has to be restated by every client that
/// installs a custom policy; the value is reqwest's own default.
///
/// This counts FOLLOWS, matching `Policy::limited`: reqwest pushes the redirecting
/// URL onto `previous` before consulting the policy, and `Limit(max)` refuses once
/// `previous.len() > max`, so a caller comparing against this constant must use `>`
/// too or it permits one hop fewer than it says.
pub const MAX_REDIRECTS: usize = 10;
/// Follow a redirect only when it stays on the origin that issued it AND re-issues
/// the identical request-target.
///
/// `Policy::none()` would have been the simpler answer, but one same-origin redirect
/// shape is legitimate here: a node fronted by a proxy that upgrades http to https,
/// or otherwise re-issues the same path and query, so the policy is scoped rather
/// than switched off.
///
/// Path and query must match exactly, and that is the request-target clause rather
/// than an origin one. `@path` is signed as the client sent it and verified as the
/// node received it, so a hop that rewrites either half leaves a signature covering
/// a target nobody asked for and the node answers 401. Refusing the hop turns a
/// confusing 401 into the 3xx that names what actually happened. One policy covers
/// signed and unsigned callers alike, for the same reason the predicate is shared:
/// two rules would drift.
///
/// The clause pins `@path`, and only `@path`. A Twigpine signature also covers
/// `@method` and `content-digest`, and both of those are still open on a followed
/// hop: on a 301, 302 or 303, reqwest 0.12.28 delegates to tower-http's
/// `FollowRedirect`, which rewrites a POST to a GET and empties the body
/// (tower-http-0.6.8 `src/follow_redirect/mod.rs:273-285`), while its
/// `drop_payload_headers` removes only `Content-Type`, `Content-Length`,
/// `Content-Encoding` and `Transfer-Encoding`. So `Signature`, `Signature-Input` and
/// `Content-Digest` ride along on a request that no longer has the method or the body
/// they were computed over. Only 307 and 308 preserve both. This predicate returning
/// true therefore makes a GET-shaped hop safe to replay against the node's verifier
/// and says nothing about a bodied one: a signed write must not rely on it alone.
///
/// `Url::query` is `None` for `/a` and `Some("")` for `/a?`, and those are two
/// different request-targets on the node side too, so the comparison is strict and
/// needs no special case.
///
/// Host and port must match exactly. Port is compared as `Url::port`, which is
/// `None` for a scheme's default port, so http -> https on the same host compares
/// equal while http -> http on a different port does not. A downgrade from https to
/// http is refused as well: the target is the same host, but the signature would go
/// out in cleartext, which is the same credential leak by a slower route.
///
/// Host comparison rides on `url`'s parse-time normalization (lowercasing and IDN
/// -> punycode), so the spellings an attacker reaches for do not open a gap. That
/// is a property of the parsed `Url`, not of this function, which is why the test
/// matrix pins it: a move to raw string comparison would silently lose it.
pub fn may_follow(previous: &url::Url, next: &url::Url) -> bool {
let same_origin = next.host_str() == previous.host_str() && next.port() == previous.port();
let same_target = next.path() == previous.path() && next.query() == previous.query();
let downgraded = previous.scheme() == "https" && next.scheme() != "https";
same_origin && same_target && !downgraded
}
#[cfg(test)]
mod tests {
use super::*;
/// Every branch of the decision, both ways, plus the spellings that would slip
/// past a comparison less careful than `Url`'s own normalization.
#[test]
fn may_follow_covers_each_origin_branch() {
let url = |s: &str| url::Url::parse(s).unwrap();
let cases: &[(&str, &str, bool, &str)] = &[
(
"http://node.example/a",
"http://node.example/b",
false,
"same origin but the path changed: @path is signed as sent and verified as received",
),
(
"http://node.example/a",
"https://node.example/a",
true,
"http to https on one host with an identical request-target: both ports are \
the scheme default, so the proxy upgrade is still followed",
),
(
"https://node.example/a",
"https://node.example/a/",
false,
"trailing-slash normalization changes the request-target, so it is refused",
),
(
"https://node.example:8443/a",
"https://node.example:8443/a",
true,
"same explicit port",
),
(
"http://node.example/a",
"http://attacker.example/a",
false,
"different host",
),
(
"http://node.example/a",
"http://node.example:8080/a",
false,
"same host, different port",
),
(
"https://node.example/a",
"http://node.example/a",
false,
"https downgraded to cleartext on the same host",
),
(
"https://node.example/a",
"http://node.example:443/a",
false,
"a downgrade dressed up as the https port",
),
// The rows below pass today because `Url::parse` normalizes the host, not
// because anything here compares case-insensitively or decodes IDN. They
// are the variants an attacker reaches for, so they are pinned: swapping
// this predicate for a raw string comparison must break the suite.
// Each of these pairs an IDENTICAL path on both sides, deliberately. The
// request-target clause below would make every one of them false on the
// path alone, and a row that is false for two reasons has stopped pinning
// either. With the paths equal, the host or port comparison is the only
// thing left that can decide them.
(
"https://node.example/a",
"https://NODE.EXAMPLE/a",
true,
"same host in a different case: parse lowercases it",
),
(
"https://node.example/a",
"https://node.example./a",
false,
"a trailing dot is a different host to url, so the redirect is refused",
),
(
"https://exämple.test/a",
"https://xn--exmple-cua.test/a",
true,
"unicode host and its punycode spelling are one host after parse",
),
(
"https://node.example/a",
"https://user:[email protected]/a",
true,
"userinfo is not part of the origin: same host, still followed",
),
(
"https://node.example/a",
"https://[email protected]/a",
false,
"the node's name smuggled into userinfo: the host is the attacker's",
),
(
"https://node.example/a",
"https://attacker.example/a#node.example",
false,
"the node's name pushed into the fragment: the host is the attacker's",
),
// The request-target clause, both directions. `@path` is the only thing a
// Twigpine signature binds the request to, so a hop that rewrites it hands
// the node a signature over a target it never received.
(
"https://node.example/a?x=1",
"https://node.example/a?x=1",
true,
"identical request-target: the same-origin hop that is still followed",
),
(
"https://node.example/a?x=1",
"https://node.example/a?x=2",
false,
"same path but a different query: the request-target covers the query too",
),
(
"https://node.example/a",
"https://node.example/a?x=1",
false,
"a query added where there was none",
),
(
"http://node.example/a",
"http://node.example/a?",
false,
"an empty query added where there was none: a missing query and an empty \
one are different request-targets",
),
(
"https://node.example/a?x=1",
"https://node.example/a",
false,
"the query dropped where there was one: the comparison is symmetric, and \
nothing else in the matrix pins that direction",
),
(
"https://node.example/a#x",
"https://node.example/a#y",
true,
"a fragment-only difference is still followed: a fragment never reaches \
the wire, so it is no part of the request-target the node verifies",
),
];
for (previous, next, expected, why) in cases {
assert_eq!(
may_follow(&url(previous), &url(next)),
*expected,
"{previous} -> {next} ({why})"
);
}
}
}
-69
View File
@@ -1,69 +0,0 @@
//! Text sanitization predicates shared across the workspace.
//!
//! Untrusted peer/remote bytes that reach an operator's terminal (node error
//! bodies, node-advertised strings) must be stripped of BOTH `Cc` control bytes
//! (`char::is_control()`) AND `Cf` bidi/format controls before display: a
//! right-to-left override or directional isolate can visually reorder the line
//! and spoof the text (Trojan Source, CWE-451). This is INV-6.
//!
//! `is_bidi_format` is the single workspace definition of the second half of
//! that obligation. It was split out of `gl`'s local copy so the two terminal
//! sanitizers that shipped with only the `Cc` half (#183) — and any future
//! sanitizer — reuse one audited predicate instead of re-deriving the set.
/// True for the Unicode bidirectional/directional-isolate format characters
/// (the reordering subset of category `Cf`) that can visually reorder a terminal
/// line. These are NOT matched by [`char::is_control()`], which covers `Cc` only.
///
/// This is deliberately the reordering subset, not the whole `Cf` category:
/// legitimate `Cf` characters such as ZWJ (`U+200D`, emoji sequences) and the
/// soft hyphen (`U+00AD`) are NOT matched, so honest international text passes
/// through unchanged. Widening this set is a behavior change for every consumer.
///
/// Pair it with `!c.is_control()` in a terminal-bound sanitizer (INV-6); see the
/// wrappers in `gl`, `git-remote-gitlawb`, and `icaptcha-client`.
pub fn is_bidi_format(c: char) -> bool {
matches!(c,
'\u{200E}' | '\u{200F}' | '\u{061C}' // LRM, RLM, ALM
| '\u{202A}'..='\u{202E}' // LRE, RLE, PDF, LRO, RLO
| '\u{2066}'..='\u{2069}' // LRI, RLI, FSI, PDI
)
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn strips_every_reordering_code_point() {
// Asserted individually so a partial predicate (e.g. one that covers the
// marks but not the isolates) fails loudly rather than passing.
for c in [
'\u{200E}', '\u{200F}', '\u{061C}', // LRM, RLM, ALM
'\u{202A}', '\u{202B}', '\u{202C}', '\u{202D}',
'\u{202E}', // LRE, RLE, PDF, LRO, RLO
'\u{2066}', '\u{2067}', '\u{2068}', '\u{2069}', // LRI, RLI, FSI, PDI
] {
assert!(
is_bidi_format(c),
"U+{:04X} must be a bidi-format char",
c as u32
);
}
}
#[test]
fn passes_legitimate_and_control_chars() {
// Must NOT over-strip: plain text, a genuine RTL SCRIPT letter (Lo, not a
// format char), ZWJ (legitimate Cf), and soft hyphen (legitimate Cf) are
// not bidi-format. ESC is a Cc control char — owned by is_control(), not
// this predicate, which is bidi-only.
for c in ['a', '\u{0627}', '\u{200D}', '\u{00AD}', '\u{1b}'] {
assert!(
!is_bidi_format(c),
"U+{:04X} must NOT be a bidi-format char",
c as u32
);
}
}
}
-499
View File
@@ -1,499 +0,0 @@
//! Sealed continuation token for the node's bounded legacy CID scan (INV-13).
//!
//! The `/ipfs/{cid}` resolver's legacy scan stops at a row ceiling and sheds a
//! retryable 503. To let a holder buried past that ceiling still be reached, the
//! shed carries the scan position so the caller can echo it back and resume. The
//! whole point of the design is that the node keeps NO server-side scan state: the
//! position rides in the caller's token.
//!
//! That makes the token an EMITTED continuation derived from a FETCHED row, and on
//! a scan that served nothing every fetched row is by construction a private or
//! quarantined repo the caller may not read. The row's `created_at` leaks its
//! creation time and its `id` carries the owner's DID, so both halves are withheld
//! fields and the token must be CONFIDENTIAL, not merely tamper-evident:
//!
//! * AEAD-sealed (XChaCha20-Poly1305), never base64-of-plaintext and never
//! signed plaintext. Integrity is not confidentiality.
//! * A fresh `OsRng` nonce on EVERY seal. Under a stream cipher a repeated nonce
//! means repeated keystream, and an attacker who can force the node to seal a
//! position whose plaintext they know XORs two tokens and recovers a withheld
//! row's fields in full, strictly worse than emitting plaintext.
//! * FIXED-WIDTH plaintext. AEAD ciphertext is plaintext-length plus the tag, and
//! every field of a scan position varies in length, so a variable encoding would
//! make token LENGTH a side channel for the sealed row (a short name under a
//! short owner vs a long one) and for the candidate oid's width (40 hex on a
//! sha1 repo, 64 on a sha256 one). Every token this module mints is byte-identical
//! in length. Each field is padded to its own separate width, which is a per-field
//! constant and so still leaks nothing about a given row or candidate.
//! * The canonical CID as associated data, so a token minted while scanning for
//! one CID does not authenticate when replayed against another.
//!
//! Every failure to open (wrong key, tampered bytes, wrong CID, expired, malformed)
//! returns the same `None`. The caller treats that as "no token" and starts at the
//! front, so no failure class is distinguishable and the token is no oracle.
use base64::{engine::general_purpose::URL_SAFE_NO_PAD as B64URL, Engine};
use chacha20poly1305::{
aead::{Aead, KeyInit, OsRng, Payload},
XChaCha20Poly1305, XNonce,
};
use rand::RngCore;
/// Keyset position of the last row a truncated scan fetched.
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct ScanPosition {
/// The row's raw stored `created_at` text, the first half of the keyset cursor.
pub created_at_key: String,
/// The row's `id`, the tiebreaking half of the keyset cursor.
pub id: String,
/// The oid hex of the CANDIDATE this position resumes. One CID can map to several
/// git oids, so a bare row cursor names a row without naming whose walk it belongs
/// to. The candidate is named by IDENTITY rather than by position in the candidate
/// list: that list is ordered by hex and mutates between rungs, so an index would
/// name a different candidate the moment anything is pinned or unpinned.
pub sha256_hex: String,
}
/// Plaintext version byte, so a future layout change is a clean open-failure
/// (treated as absent) rather than a misparse.
///
/// Bumped to 2 when the two halves stopped sharing one width (see [`ID_WIDTH`]), and
/// to 3 when the position gained the candidate oid it resumes (see [`OID_WIDTH`]).
/// A token minted under an earlier layout is a different length and a different
/// framing, so it opens to `None` and the caller restarts at the front, which is the
/// safe direction: a misparse would resume at a fabricated row and skip coverage.
const VERSION: u8 = 3;
/// Byte width the `created_at` half is padded to. Every value stored here is a
/// serialized timestamp, about 30 bytes, so 64 is roomy for the field's whole domain.
/// It is deliberately NOT widened to match [`ID_WIDTH`]: padding both halves to the id
/// width would nearly double every token for a field that can never use the space.
const CREATED_WIDTH: usize = 64;
/// Byte width the `id` half is padded to.
///
/// The bound is set by the WRITERS, not by what a typical id happens to look like.
/// `upsert_mirror_repo` builds `repos.id` as `{owner}/{name}`, and the slug validators
/// in the node's `repo_store` admit an owner of up to 255 bytes and a name of up to
/// 100, so 356 bytes is reachable through the ordinary write path and repo names are
/// peer-controllable. 384 clears that with margin.
///
/// Under-sizing this is not a cosmetic bug. A row at a truncation boundary whose id
/// exceeds the width fails the seal, the handler sheds a 503 with no continuation, and
/// a tokenless shed is byte-identical to the wrapped-scan response whose contract is
/// "the absence of a token means the ladder is over". The boundary row is deterministic
/// for a stable inventory, so every retry reproduces it and every row past it becomes
/// permanently unreachable. Anything past the width still fails loudly rather than
/// silently truncating a cursor into one that resumes at the wrong row.
const ID_WIDTH: usize = 384;
/// Byte width the candidate oid half is padded to.
///
/// Git mints exactly two oid widths and this field carries BOTH. A production repo is
/// created by `store::init_bare` with `git init --bare --object-format=sha1`, so its
/// oids are 40 hex; only the sha256 test fixtures mint 64. The field is therefore
/// length-prefixed like the two row halves rather than a bare fixed 64: a 64-only
/// field would fail every seal on a real deployment, and a failed seal sheds a
/// tokenless 503 that is byte-identical to "your ladder is over".
///
/// The padding to 64 is what keeps the WIDTH off the wire. Without it a 40-hex token
/// is 24 bytes shorter than a 64-hex one, and token length would say which object
/// format the holder's repo uses.
const OID_WIDTH: usize = 64;
/// `version | created_len:u16 | created[CREATED_WIDTH] | id_len:u16 | id[ID_WIDTH] | oid_len:u16 | oid[OID_WIDTH] | expires:i64`
const PLAINTEXT_LEN: usize = 1 + 2 + CREATED_WIDTH + 2 + ID_WIDTH + 2 + OID_WIDTH + 8;
/// Nonce width for XChaCha20-Poly1305.
const NONCE_LEN: usize = 24;
/// A fresh random 32-byte sealing key from the OS CSPRNG.
pub fn new_key() -> [u8; 32] {
let mut key = [0u8; 32];
OsRng.fill_bytes(&mut key);
key
}
// The two halves below are deliberately separate and adjacent. The framing pair owns
// "every token is the same length"; the AEAD pair owns "the contents are confidential
// and CID-bound". Keeping them apart is what lets each property be exercised (and
// broken) without disturbing the other.
/// Encode a position into the FIXED-WIDTH plaintext:
/// `version | created_len:u16 | created[CREATED_WIDTH] | id_len:u16 | id[ID_WIDTH] | oid_len:u16 | oid[OID_WIDTH] | expires:i64`
///
/// The padding is the point. AEAD ciphertext is plaintext-length plus the tag, and every
/// field of a scan position varies in length, so a length-prefixed encoding with no
/// padding would make token LENGTH a side channel for the sealed row and for the
/// candidate's object format. Each field is padded to its OWN fixed width, which keeps
/// every minted token the same length while letting the id half carry the range the
/// write path actually admits.
fn encode_position(pos: &ScanPosition, expires_at_unix: i64) -> anyhow::Result<Vec<u8>> {
let mut out = vec![0u8; PLAINTEXT_LEN];
out[0] = VERSION;
let mut at = 1;
for (field, width) in [
(pos.created_at_key.as_bytes(), CREATED_WIDTH),
(pos.id.as_bytes(), ID_WIDTH),
(pos.sha256_hex.as_bytes(), OID_WIDTH),
] {
if field.len() > width {
// Loud rather than truncating: a clipped cursor resumes at the wrong row and
// silently skips coverage, which is the availability half of the bug this
// token exists to fix.
anyhow::bail!(
"scan token field is {} bytes, over the {width}-byte fixed width",
field.len()
);
}
out[at..at + 2].copy_from_slice(&(field.len() as u16).to_le_bytes());
at += 2;
out[at..at + field.len()].copy_from_slice(field);
at += width;
}
out[at..at + 8].copy_from_slice(&expires_at_unix.to_le_bytes());
Ok(out)
}
/// Decode what [`encode_position`] wrote. `None` on any structural mismatch.
fn decode_position(bytes: &[u8]) -> Option<(ScanPosition, i64)> {
if bytes.len() != PLAINTEXT_LEN || bytes[0] != VERSION {
return None;
}
let mut at = 1;
let mut fields = [const { String::new() }; 3];
for (slot, width) in fields.iter_mut().zip([CREATED_WIDTH, ID_WIDTH, OID_WIDTH]) {
let len = u16::from_le_bytes([bytes[at], bytes[at + 1]]) as usize;
at += 2;
if len > width {
return None;
}
*slot = String::from_utf8(bytes[at..at + len].to_vec()).ok()?;
at += width;
}
let expires_at = i64::from_le_bytes(bytes[at..at + 8].try_into().ok()?);
let [created_at_key, id, sha256_hex] = fields;
// A zero-length candidate is a third state the encoder never mints. The front-of-table
// sentinel is empty ROW halves with a real oid, so an empty oid would hand the resume
// path a candidate that names nothing; refuse it like every other malformed frame.
if sha256_hex.is_empty() {
return None;
}
Some((
ScanPosition {
created_at_key,
id,
sha256_hex,
},
expires_at,
))
}
/// AEAD-seal `plaintext` under `key`, bound to `cid`, framed as `nonce || ciphertext`.
fn seal_bytes(key: &[u8; 32], cid: &str, plaintext: &[u8]) -> anyhow::Result<Vec<u8>> {
let cipher = XChaCha20Poly1305::new_from_slice(key)
.map_err(|e| anyhow::anyhow!("scan token key: {e}"))?;
// A FRESH nonce per seal, from the OS CSPRNG. Under a stream cipher a repeated nonce
// repeats the keystream, and two tokens sealed under one nonce XOR to the difference
// of their plaintexts, which recovers a withheld row in full when the attacker can
// force one of the two positions. This draw is the property the whole confidentiality
// claim rests on.
let mut nonce = [0u8; NONCE_LEN];
OsRng.fill_bytes(&mut nonce);
let sealed = cipher
.encrypt(
XNonce::from_slice(&nonce),
Payload {
msg: plaintext,
// The canonical CID as associated data: a token minted while scanning for
// one CID does not authenticate against another, so it cannot be replayed
// to seed a different scan.
aad: cid.as_bytes(),
},
)
.map_err(|e| anyhow::anyhow!("scan token seal: {e}"))?;
let mut out = Vec::with_capacity(NONCE_LEN + sealed.len());
out.extend_from_slice(&nonce);
out.extend_from_slice(&sealed);
Ok(out)
}
/// Open what [`seal_bytes`] framed. `None` on any failure, including a wrong `cid`.
fn open_bytes(key: &[u8; 32], cid: &str, raw: &[u8]) -> Option<Vec<u8>> {
if raw.len() <= NONCE_LEN + 16 {
return None;
}
let (nonce, sealed) = raw.split_at(NONCE_LEN);
let cipher = XChaCha20Poly1305::new_from_slice(key).ok()?;
cipher
.decrypt(
XNonce::from_slice(nonce),
Payload {
msg: sealed,
aad: cid.as_bytes(),
},
)
.ok()
}
/// Seal `pos` under `key`, bound to `cid`, expiring at `expires_at_unix`.
///
/// Returns the base64url (no pad) token. Errors only when a field exceeds
/// its own fixed width ([`CREATED_WIDTH`], [`ID_WIDTH`], [`OID_WIDTH`]) or the AEAD
/// itself fails, never silently truncates.
pub fn seal_scan_token(
key: &[u8; 32],
cid: &str,
pos: &ScanPosition,
expires_at_unix: i64,
) -> anyhow::Result<String> {
let plaintext = encode_position(pos, expires_at_unix)?;
Ok(B64URL.encode(seal_bytes(key, cid, &plaintext)?))
}
/// Open a token minted by [`seal_scan_token`] under the same key and CID.
///
/// `None` for every failure class alike (wrong key, tampered, foreign CID, expired,
/// malformed, wrong version), so the caller can treat all of them as "absent" without
/// leaking which one occurred.
pub fn open_scan_token(
key: &[u8; 32],
cid: &str,
token: &str,
now_unix: i64,
) -> Option<ScanPosition> {
let raw = B64URL.decode(token).ok()?;
let plaintext = open_bytes(key, cid, &raw)?;
let (pos, expires_at) = decode_position(&plaintext)?;
if now_unix >= expires_at {
return None;
}
Some(pos)
}
#[cfg(test)]
mod tests {
use super::*;
/// A production-shaped candidate: `git init --bare --object-format=sha1`, so 40 hex.
const OID_40: &str = "a1b2c3d4e5f60718293a4b5c6d7e8f9012345678";
/// A test-fixture-shaped candidate: the sha256 repos the suite creates mint 64 hex.
const OID_64: &str = "0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef";
fn pos(created: &str, id: &str) -> ScanPosition {
pos_for(created, id, OID_40)
}
fn pos_for(created: &str, id: &str, sha256_hex: &str) -> ScanPosition {
ScanPosition {
created_at_key: created.to_string(),
id: id.to_string(),
sha256_hex: sha256_hex.to_string(),
}
}
/// Seal a hand-built plaintext through the AEAD half, so a test can frame bytes the
/// encoder would never mint (an old version byte, a zero-length oid) and still exercise
/// the real open path.
fn seal_raw(key: &[u8; 32], cid: &str, plaintext: &[u8]) -> String {
B64URL.encode(seal_bytes(key, cid, plaintext).unwrap())
}
/// Byte offset of the oid length prefix inside the plaintext, derived from the widths
/// rather than hardcoded so a width change moves it with the layout.
const OID_LEN_AT: usize = 1 + 2 + CREATED_WIDTH + 2 + ID_WIDTH;
/// Scenario 1: both of git's oid widths round trip. The 40-hex case is the PRODUCTION
/// shape (`git init --bare --object-format=sha1`), and an all-64 fixture suite would
/// never exercise it, which is exactly how a fixed-64 field would ship broken.
#[test]
fn round_trips_at_both_oid_widths() {
let key = new_key();
for hex in [OID_40, OID_64] {
let p = pos_for("2020-01-01T00:00:03+00:00", "z6MkOwner/private-repo", hex);
let t =
seal_scan_token(&key, "bafkcid", &p, 1 << 40).expect("both oid widths must seal");
assert_eq!(
open_scan_token(&key, "bafkcid", &t, 0),
Some(p),
"a {}-hex candidate must open to the identical position",
hex.len()
);
}
}
/// Scenario 2: a token framed under the OLD version opens to `None`, never a misparse.
/// Both legs matter: the version-2 plaintext was a different LENGTH, and a future
/// same-length layout would only be caught by the version byte itself.
#[test]
fn a_prior_layout_version_opens_to_none() {
let key = new_key();
// The version-2 layout verbatim: no oid field, so 461 bytes.
let mut old = vec![0u8; 1 + 2 + CREATED_WIDTH + 2 + ID_WIDTH + 8];
old[0] = 2;
assert_eq!(old.len(), 461, "the version-2 plaintext was 461 bytes");
assert_eq!(
open_scan_token(&key, "bafkcid", &seal_raw(&key, "bafkcid", &old), 0),
None,
"a version-2 token must open to None so the caller restarts at the front"
);
// Same length, stale version byte: only the version check can refuse this one.
let mut stamped = encode_position(
&pos("2020-01-01T00:00:03+00:00", "z6MkOwner/private-repo"),
1 << 40,
)
.unwrap();
stamped[0] = VERSION - 1;
assert_eq!(
open_scan_token(&key, "bafkcid", &seal_raw(&key, "bafkcid", &stamped), 0),
None,
"a stale version byte must open to None even at the current width"
);
}
/// Scenario 3: token length is invariant across candidate VALUE and candidate WIDTH.
/// The padding is what keeps the oid width off the wire: without it a 40-hex token is
/// 24 bytes shorter than a 64-hex one and the length says which repo format the
/// holder uses.
#[test]
fn token_length_is_invariant_across_oid_widths() {
let key = new_key();
let created = "2020-01-01T00:00:00+00:00";
let id = "z6MkOwner/private-repo";
let short =
seal_scan_token(&key, "bafkcid", &pos_for(created, id, OID_40), 1 << 40).unwrap();
let long =
seal_scan_token(&key, "bafkcid", &pos_for(created, id, OID_64), 1 << 40).unwrap();
assert_eq!(
short.len(),
long.len(),
"a 40-hex and a 64-hex candidate must mint tokens of identical length, or the \
oid width is a side channel"
);
// The absolute width, pinned by execution rather than by arithmetic on paper. The
// gl client's mock fixtures hardcode this number (`TOKEN_LEN` in
// crates/gl/src/ipfs_cmd.rs), and nothing in that crate seals a real token, so this
// assertion is the only executable check that the two agree.
assert_eq!(
short.len(),
756,
"24 nonce + {PLAINTEXT_LEN} plaintext + 16 tag, base64url no pad"
);
}
/// Scenario 4: the front-of-table sentinel. Empty row halves with a REAL candidate
/// round trip, which is what lets a seal say "this candidate, no row cursor yet".
#[test]
fn empty_row_fields_round_trip_with_a_real_candidate() {
let key = new_key();
let p = pos_for("", "", OID_40);
let t =
seal_scan_token(&key, "bafkcid", &p, 1 << 40).expect("the front sentinel must seal");
assert_eq!(open_scan_token(&key, "bafkcid", &t, 0), Some(p));
}
/// Scenario 6, third leg: a zero-length oid is a distinguishable third state that the
/// encoder never mints, and accepting it would hand the sentinel machinery a candidate
/// naming nothing. The decode path refuses it.
#[test]
fn a_zero_length_candidate_opens_to_none() {
let key = new_key();
let mut plaintext = encode_position(
&pos("2020-01-01T00:00:03+00:00", "z6MkOwner/private-repo"),
1 << 40,
)
.unwrap();
plaintext[OID_LEN_AT..OID_LEN_AT + 2].copy_from_slice(&0u16.to_le_bytes());
assert_eq!(
open_scan_token(&key, "bafkcid", &seal_raw(&key, "bafkcid", &plaintext), 0),
None,
"a zero-length candidate must open to None, not to a position naming nothing"
);
}
/// Scenario 6, second leg: an oid past the 64-byte width fails the seal loudly rather
/// than being clipped into a hex that names a different candidate.
#[test]
fn an_oid_over_the_fixed_width_fails_loudly() {
let key = new_key();
let p = pos_for(
"2020-01-01T00:00:03+00:00",
"z6MkOwner/private-repo",
&"a".repeat(OID_WIDTH + 1),
);
assert!(
seal_scan_token(&key, "bafkcid", &p, 1 << 40).is_err(),
"an over-wide candidate must fail the seal, never be truncated into a hex that \
resumes the wrong candidate"
);
}
#[test]
fn round_trips_under_the_same_key_and_cid() {
let key = new_key();
let p = pos("2020-01-01T00:00:03+00:00", "z6MkOwner/private-repo");
let t = seal_scan_token(&key, "bafkcid", &p, 1 << 40).unwrap();
assert_eq!(open_scan_token(&key, "bafkcid", &t, 0), Some(p));
}
#[test]
fn every_failure_class_opens_to_none() {
let key = new_key();
let other = new_key();
let p = pos("2020-01-01T00:00:03+00:00", "z6MkOwner/private-repo");
let t = seal_scan_token(&key, "bafkcid", &p, 1 << 40).unwrap();
assert_eq!(open_scan_token(&other, "bafkcid", &t, 0), None, "wrong key");
assert_eq!(
open_scan_token(&key, "bafkOTHER", &t, 0),
None,
"foreign CID"
);
assert_eq!(
open_scan_token(&key, "bafkcid", &t, 1 << 41),
None,
"expired"
);
assert_eq!(open_scan_token(&key, "bafkcid", "!!not b64", 0), None);
assert_eq!(open_scan_token(&key, "bafkcid", "", 0), None);
let mut flipped: Vec<u8> = t.bytes().collect();
let last = flipped.len() - 1;
flipped[last] = if flipped[last] == b'A' { b'B' } else { b'A' };
assert_eq!(
open_scan_token(&key, "bafkcid", &String::from_utf8(flipped).unwrap(), 0),
None,
"tampered"
);
}
/// The id half must clear the LARGEST repo id the node's own write path admits,
/// not merely a typical one. `upsert_mirror_repo` builds `repos.id` as
/// `{owner}/{name}`, and the slug validators in `repo_store` admit 255 bytes of
/// owner and 100 of name, so 356 is reachable. A width under that turns the
/// boundary row into a seal failure, which sheds a tokenless 503 that is
/// byte-identical to "your ladder is over" and strands every row past it forever.
#[test]
fn round_trips_a_repo_id_at_the_write_paths_maximum() {
let key = new_key();
let id = format!("{}/{}", "o".repeat(255), "n".repeat(100));
assert_eq!(id.len(), 356, "255 owner + '/' + 100 name");
let p = pos("2020-01-01T00:00:03+00:00", &id);
let t = seal_scan_token(&key, "bafkcid", &p, 1 << 40)
.expect("a repo id the write path admits must seal, never fail the width");
assert_eq!(open_scan_token(&key, "bafkcid", &t, 0), Some(p));
}
#[test]
fn a_field_over_the_fixed_width_fails_loudly() {
let key = new_key();
let p = pos("2020-01-01T00:00:03+00:00", &"x".repeat(ID_WIDTH + 1));
assert!(
seal_scan_token(&key, "bafkcid", &p, 1 << 40).is_err(),
"an over-wide field must fail the seal, never be truncated into a cursor \
that resumes at the wrong row"
);
}
}
+6 -69
View File
@@ -1,7 +1,7 @@
//! UCAN (User Controlled Authorization Networks) — capability token types.
//!
//! UCANs let a DID delegate specific capabilities to another DID,
//! with optional expiry and revocation. Twigpine uses UCANs for:
//! with optional expiry and revocation. gitlawb uses UCANs for:
//! - Delegating push access to a branch to a CI agent
//! - Granting a reviewer the ability to approve PRs
//! - Bootstrap tokens issued at registration
@@ -57,7 +57,7 @@ impl Capability {
}
}
/// Well-known Twigpine capability strings.
/// Well-known gitlawb capability strings.
pub mod caps {
pub const GIT_PUSH: &str = "git/push";
pub const GIT_FETCH: &str = "git/fetch";
@@ -187,17 +187,11 @@ impl Ucan {
}
/// Check if this UCAN grants a specific capability on a resource.
///
/// Mirrors `Capability::is_attenuated_by`'s wildcard semantics: a stored
/// capability of `with: "*"` or `can: "*"` / `"repo/admin"` covers any
/// requested resource/action, since a valid delegation chain can produce
/// exactly that capability (see `is_attenuated_by`).
pub fn can(&self, resource: &str, action: &str) -> bool {
self.payload.att.iter().any(|cap| {
let resource_ok = cap.with == resource || cap.with == "*";
let action_ok = cap.can == action || cap.can == "*" || cap.can == caps::REPO_ADMIN;
resource_ok && action_ok
})
self.payload
.att
.iter()
.any(|cap| cap.with == resource && cap.can == action)
}
/// Encode to a compact JSON string (the wire format).
@@ -318,63 +312,6 @@ mod tests {
assert!(!ucan.can("gitlawb://repos/test/repo", caps::PR_MERGE));
}
#[test]
fn can_honors_resource_wildcard() {
let issuer = Keypair::generate();
let audience = Keypair::generate().did();
let ucan = Ucan::issue(
&issuer,
audience,
vec![Capability::new("*", caps::GIT_PUSH)],
None,
)
.unwrap();
assert!(ucan.can("gitlawb://repos/test/repo", caps::GIT_PUSH));
assert!(ucan.can("gitlawb://repos/other/repo", caps::GIT_PUSH));
assert!(!ucan.can("gitlawb://repos/test/repo", caps::PR_MERGE));
}
#[test]
fn can_honors_repo_admin_action_wildcard() {
let issuer = Keypair::generate();
let audience = Keypair::generate().did();
let ucan = Ucan::issue(
&issuer,
audience,
vec![Capability::new(
"gitlawb://repos/test/repo",
caps::REPO_ADMIN,
)],
None,
)
.unwrap();
assert!(ucan.can("gitlawb://repos/test/repo", caps::GIT_PUSH));
assert!(ucan.can("gitlawb://repos/test/repo", caps::PR_MERGE));
assert!(!ucan.can("gitlawb://repos/other/repo", caps::GIT_PUSH));
}
#[test]
fn can_honors_action_wildcard() {
let issuer = Keypair::generate();
let audience = Keypair::generate().did();
let ucan = Ucan::issue(
&issuer,
audience,
vec![Capability::new("gitlawb://repos/test/repo", "*")],
None,
)
.unwrap();
assert!(ucan.can("gitlawb://repos/test/repo", caps::GIT_PUSH));
assert!(ucan.can("gitlawb://repos/test/repo", caps::PR_MERGE));
assert!(!ucan.can("gitlawb://repos/other/repo", caps::GIT_PUSH));
}
#[test]
fn bootstrap_ucan() {
let issuer = Keypair::generate();
+11 -15
View File
@@ -1,7 +1,7 @@
[package]
name = "gitlawb-node"
description = "The Twigpine node daemon — git hosting over HTTP with DID auth"
version = "0.7.1" # x-release-please-version
description = "The gitlawb node daemon — git hosting over HTTP with DID auth"
version = "0.5.1" # x-release-please-version
edition.workspace = true
rust-version.workspace = true
license.workspace = true
@@ -33,7 +33,6 @@ sqlx = { version = "0.8", features = ["postgres", "runtime-tokio-rustls", "chron
clap = { version = "4", features = ["derive", "env"] }
bytes = "1"
libc = "0.2"
zeroize = "1"
cid = { workspace = true }
hex = { workspace = true }
sha2 = { workspace = true }
@@ -45,14 +44,14 @@ dirs-next = "2"
# withholds a canonically-equivalent path regardless of NFC/NFD form (#101).
unicode-normalization = "0.1"
reqwest = { workspace = true }
libp2p-core = "0.44"
libp2p-gossipsub = "0.50.1"
libp2p-identify = "0.48"
libp2p-identity = { version = "0.3", features = ["ed25519", "peerid", "rand"] }
libp2p-kad = "0.49"
libp2p-quic = { version = "0.14", features = ["tokio"] }
libp2p-swarm = { version = "0.48", features = ["macros", "tokio"] }
multiaddr = "0.19"
libp2p-core = "0.43"
libp2p-gossipsub = "0.49.4"
libp2p-identify = "0.47"
libp2p-identity = { version = "0.2", features = ["ed25519", "peerid", "rand"] }
libp2p-kad = "0.48"
libp2p-quic = { version = "0.13", features = ["tokio"] }
libp2p-swarm = { version = "0.47", features = ["macros", "tokio"] }
multiaddr = "0.18"
futures = "0.3"
aws-sdk-s3 = { version = "1", default-features = false, features = ["sigv4a", "default-https-client", "rt-tokio"] }
aws-config = { version = "1", features = ["behavior-version-latest"] }
@@ -73,11 +72,8 @@ alloy = { version = "1", default-features = false, features = [
"network",
"rpc-types-eth",
] }
libp2p-dns = { version = "0.45.0", features = ["tokio"] }
libp2p-dns = { version = "0.44.0", features = ["tokio"] }
[dev-dependencies]
mockito = "1"
tempfile = "3"
# `test-util` for `#[tokio::test(start_paused = true)]`: lets the deadline tests assert
# on which timer fired instead of on wall clock.
tokio = { workspace = true, features = ["test-util"] }
File diff suppressed because it is too large. Load diff
+8 -102
View File
@@ -1,13 +1,12 @@
//! GET /api/v1/arweave/anchors — list Arweave ref-update anchors.
use axum::{
extract::{Extension, Query, State},
extract::{Query, State},
Json,
};
use serde::Deserialize;
use crate::db::normalize_owner_key;
use crate::error::{AppError, Result};
use crate::error::Result;
use crate::state::AppState;
#[derive(Debug, Deserialize)]
@@ -22,112 +21,19 @@ fn default_limit() -> i64 {
}
/// GET /api/v1/arweave/anchors
///
/// `?repo=<owner>/<name>` returns anchors for one repository and binds the
/// request to the canonical read-authorization path used by every other
/// repo-scoped read: the same `authorize_repo_read` helper, with the caller
/// identity attached. Missing repositories, quarantined mirrors, and signed
/// non-readers all collapse to the standard `404` (no existence oracle). The
/// unscoped listing is auth-only — the #121 contract — and does not admit
/// visibility filtering, which is the #136 stale-index class and explicitly
/// out of scope for this auth slice.
pub async fn list_anchors(
State(state): State<AppState>,
Query(q): Query<ListAnchorsQuery>,
auth: Option<Extension<crate::auth::AuthenticatedDid>>,
) -> Result<Json<serde_json::Value>> {
// The route's `optional_signature` layer is permissive (it admits unsigned
// legacy callers) so this in-handler check is the actual gate. A signed
// non-reader still passes this check and proceeds to the authz step below.
let caller = auth.as_ref().map(|e| e.0 .0.as_str());
if caller.is_none() {
return Err(AppError::Unauthorized(
"authentication required for anchor listing".into(),
));
}
// Clamp before the SQL query. A negative or non-numeric limit must not
// reach Postgres as `LIMIT -1` (which it rejects as a db error / 500) — the
// contract is the same default ceiling as the listing page itself.
let limit = q.limit.clamp(0, 200);
let anchors = if let Some(repo_slug) = q.repo.as_deref() {
// Parse the user-supplied "owner/name" through the same slug validator
// the sync path uses, so a malformed query is a 400, not a 500.
let (owner, name) = match crate::git::repo_store::validate_repo_slug(repo_slug) {
Ok(parts) => parts,
Err(e) => return Err(AppError::BadRequest(format!("invalid ?repo: {e}"))),
};
// Read gate. Returns `RepoNotFound` (→ 404) indistinguishably for
// missing repos, quarantined mirrors, and signed non-readers.
let (record, _rules) =
crate::api::authorize_repo_read(&state, owner, name, caller, "/").await?;
// Build the SQL filter from the canonical stored slug, NOT from the
// user-supplied `?repo=` string. Anchor rows are written as
// `{normalize_owner_key(owner_did)}/{name}` (the short form), so a
// request that passes authz with the full `did:key:…/name` form would
// match zero rows and return a false empty page. The gate already
// verified the repo is readable; `record.owner_did` is the canonical
// identity and `record.name` is the stored name.
let stored_slug = format!("{}/{}", normalize_owner_key(&record.owner_did), record.name);
state
.db
.list_arweave_anchors(Some(&stored_slug), limit)
.await?
} else {
// No scope → no repo-read decision. Auth-only.
state.db.list_arweave_anchors(None, limit).await?
};
let limit = q.limit.min(200);
let anchors = state
.db
.list_arweave_anchors(q.repo.as_deref(), limit)
.await
.map_err(crate::error::AppError::Internal)?;
Ok(Json(serde_json::json!({
"anchors": anchors,
"count": anchors.len(),
})))
}
#[cfg(test)]
mod closed_pool_tests {
use super::*;
use axum::http::{Request, StatusCode};
use axum::Router;
use serde_json::Value;
use sqlx::PgPool;
use tower::ServiceExt;
/// #251: a closed pool on /api/v1/arweave/anchors must be 503 db_unavailable.
#[sqlx::test]
async fn list_anchors_closed_pool_returns_503_db_unavailable(pool: PgPool) {
let state = crate::test_support::test_state(pool.clone()).await;
pool.close().await;
let resp = Router::new()
.route("/api/v1/arweave/anchors", axum::routing::get(list_anchors))
.with_state(state)
.oneshot(
Request::builder()
.uri("/api/v1/arweave/anchors")
.extension(crate::auth::AuthenticatedDid("did:key:test".into()))
.body(axum::body::Body::empty())
.unwrap(),
)
.await
.unwrap();
assert_eq!(
resp.status(),
StatusCode::SERVICE_UNAVAILABLE,
"closed-pool outage must be retryable 503, not 500"
);
let bytes = axum::body::to_bytes(resp.into_body(), usize::MAX)
.await
.expect("read body");
let v: Value = serde_json::from_slice(&bytes).expect("json body");
assert_eq!(
v,
serde_json::json!({
"error": crate::error::DB_UNAVAILABLE_CODE,
"message": crate::error::DB_UNAVAILABLE_MESSAGE,
})
);
}
}
+29 -123
View File
@@ -285,22 +285,11 @@ pub async fn submit_bounty(
Path(id): Path<String>,
Json(req): Json<SubmitBountyRequest>,
) -> Result<Json<BountyRecord>> {
let not_found = || AppError::NotFound(format!("bounty {id} not found"));
let bounty = state.db.get_bounty(&id).await?.ok_or_else(not_found)?;
if crate::api::authorize_repo_read(
&state,
&bounty.repo_owner,
&bounty.repo_name,
Some(auth.0.as_str()),
"/",
)
.await
.is_err()
{
return Err(not_found());
}
let bounty = state
.db
.get_bounty(&id)
.await?
.ok_or_else(|| AppError::NotFound(format!("bounty {id} not found")))?;
if bounty.status != "claimed" {
return Err(AppError::BadRequest(format!(
@@ -336,22 +325,11 @@ pub async fn approve_bounty(
Path(id): Path<String>,
Json(req): Json<ApproveBountyRequest>,
) -> Result<Json<BountyRecord>> {
let not_found = || AppError::NotFound(format!("bounty {id} not found"));
let bounty = state.db.get_bounty(&id).await?.ok_or_else(not_found)?;
if crate::api::authorize_repo_read(
&state,
&bounty.repo_owner,
&bounty.repo_name,
Some(auth.0.as_str()),
"/",
)
.await
.is_err()
{
return Err(not_found());
}
let bounty = state
.db
.get_bounty(&id)
.await?
.ok_or_else(|| AppError::NotFound(format!("bounty {id} not found")))?;
if bounty.status != "submitted" {
return Err(AppError::BadRequest(format!(
@@ -394,22 +372,11 @@ pub async fn cancel_bounty(
Extension(auth): Extension<AuthenticatedDid>,
Path(id): Path<String>,
) -> Result<Json<BountyRecord>> {
let not_found = || AppError::NotFound(format!("bounty {id} not found"));
let bounty = state.db.get_bounty(&id).await?.ok_or_else(not_found)?;
if crate::api::authorize_repo_read(
&state,
&bounty.repo_owner,
&bounty.repo_name,
Some(auth.0.as_str()),
"/",
)
.await
.is_err()
{
return Err(not_found());
}
let bounty = state
.db
.get_bounty(&id)
.await?
.ok_or_else(|| AppError::NotFound(format!("bounty {id} not found")))?;
if bounty.status != "open" {
return Err(AppError::BadRequest(format!(
@@ -440,22 +407,11 @@ pub async fn dispute_bounty(
Extension(auth): Extension<AuthenticatedDid>,
Path(id): Path<String>,
) -> Result<Json<BountyRecord>> {
let not_found = || AppError::NotFound(format!("bounty {id} not found"));
let bounty = state.db.get_bounty(&id).await?.ok_or_else(not_found)?;
if crate::api::authorize_repo_read(
&state,
&bounty.repo_owner,
&bounty.repo_name,
Some(auth.0.as_str()),
"/",
)
.await
.is_err()
{
return Err(not_found());
}
let bounty = state
.db
.get_bounty(&id)
.await?
.ok_or_else(|| AppError::NotFound(format!("bounty {id} not found")))?;
if bounty.status != "claimed" && bounty.status != "submitted" {
return Err(AppError::BadRequest(format!(
@@ -500,68 +456,21 @@ pub async fn dispute_bounty(
Ok(Json(updated))
}
/// Resolve the set of `(owner_did, name)` pairs for repos an anonymous caller
/// can list. Mirrors the `stats()` pattern in `server.rs` (#104): batch-load
/// all deduped repos, batch-load their visibility rules, keep only those that
/// pass `listable_at_root`. Pure I/O after the two DB round-trips — no
/// per-repo authorization queries.
///
/// Both bounty stats handlers mount behind `optional_signature` (see
/// `bounty_read_routes` in server.rs), so the caller is always `None` for
/// anonymous access. Every caller, signed or not, currently receives the
/// anonymous-scoped aggregates (matching the #104 pattern); per-caller
/// visibility can be threaded through here when that work lands.
///
/// A database error is returned to the caller, never collapsed to an empty
/// set: an empty set would render as HTTP 200 with zero counts, which a client
/// reads as confirmed absence of activity.
async fn visible_repo_pairs(state: &AppState) -> Result<Vec<(String, String)>> {
let rows = state.db.list_all_repos_deduped().await?;
let ids: Vec<String> = rows.iter().map(|r| r.id.clone()).collect();
let rules_by_repo = state.db.list_visibility_rules_for_repos(&ids).await?;
let pairs = rows
.iter()
.filter(|r| {
let rules = rules_by_repo.get(&r.id).map(Vec::as_slice).unwrap_or(&[]);
// mirror-rows-handled: a mirror with a canonical twin is deduped to the
// canonical row and its rules; a mirror-only repo counts as public, the
// same answer authorize_repo_read and /stats give. Pinned by
// bounty_stats_mirror_rows_follow_canonical_twin_or_count_when_alone.
crate::visibility::listable_at_root(rules, r.is_public, &r.owner_did, None)
})
.map(|r| {
(
crate::db::normalize_owner_key(&r.owner_did).to_string(),
r.name.clone(),
)
})
.collect();
Ok(pairs)
}
/// GET /api/v1/bounties/stats
///
/// Aggregates are restricted to anonymously-listable repos so private-repo
/// bounty activity is not exposed (#477). The visible-repo set is resolved
/// once per request via `visible_repo_pairs` (two SQL round-trips), then
/// passed into the filtered aggregate queries.
pub async fn bounty_stats(State(state): State<AppState>) -> Result<Json<BountyStatsResponse>> {
let visible = visible_repo_pairs(&state).await?;
let open = state
.db
.count_bounties_by_status_visible("open", &visible)
.await?;
let open = state.db.count_bounties_by_status("open").await.unwrap_or(0);
let claimed = state
.db
.count_bounties_by_status_visible("claimed", &visible)
.await?;
.count_bounties_by_status("claimed")
.await
.unwrap_or(0);
let completed = state
.db
.count_bounties_by_status_visible("completed", &visible)
.await?;
.count_bounties_by_status("completed")
.await
.unwrap_or(0);
let leaders = state.db.bounty_leaderboard_visible(10, &visible).await?;
let leaders = state.db.bounty_leaderboard(10).await.unwrap_or_default();
let leaderboard = leaders
.into_iter()
.map(|(did, cnt, total)| AgentBountyEntry {
@@ -580,14 +489,11 @@ pub async fn bounty_stats(State(state): State<AppState>) -> Result<Json<BountySt
}
/// GET /api/v1/agents/{did}/bounties
///
/// Per-agent earnings restricted to anonymously-listable repos (#477).
pub async fn agent_bounty_stats(
State(state): State<AppState>,
Path(did): Path<String>,
) -> Result<Json<serde_json::Value>> {
let visible = visible_repo_pairs(&state).await?;
let (count, total) = state.db.agent_bounty_stats_visible(&did, &visible).await?;
let (count, total) = state.db.agent_bounty_stats(&did).await.unwrap_or((0, 0));
Ok(Json(serde_json::json!({
"did": did,
"completed_bounties": count,
+24 -89
View File
@@ -138,33 +138,9 @@ pub async fn list_ref_updates(
let caller = auth.as_ref().map(|e| e.0 .0.as_str());
let updates = collect_visible_ref_updates(&state.db, None, limit, caller).await?;
// Resolve the trusted display owner_did per row. The stored wire value is
// untrusted (arrives over gossip / unsigned peer notify), so it is echoed
// only when it matches the canonical owner of the local repo the slug
// names (#P1); legacy None rows are attributed via an exact unique local
// match (#P3). Both surfaces share this resolver so they cannot drift.
// The batch resolver issues at most one query per distinct local repo
// rather than one per event row (#P2).
let raw_dids: Vec<Option<String>> = state
.db
.resolve_ref_update_owner_dids(
&updates
.iter()
.map(|u| (u.repo.as_str(), u.owner_did.as_deref()))
.collect::<Vec<_>>(),
)
.await?;
let owner_dids: Vec<serde_json::Value> = raw_dids
.into_iter()
.map(|o| o.map_or(serde_json::Value::Null, |s| serde_json::json!(s)))
.collect();
let events: Vec<serde_json::Value> = updates
.iter()
.enumerate()
.map(|(i, u)| {
let owner_did = owner_dids[i].clone();
.map(|u| {
serde_json::json!({
"id": u.id,
"node_did": u.node_did,
@@ -177,7 +153,6 @@ pub async fn list_ref_updates(
"cert_id": u.cert_id,
"received_at": u.received_at,
"from_peer": u.from_peer,
"owner_did": owner_did,
})
})
.collect();
@@ -250,7 +225,6 @@ pub async fn list_repo_events(
"pusher_did": c.pusher_did,
"node_did": c.node_did,
"timestamp": c.issued_at,
"owner_did": record.owner_did,
"source": "local",
})
})
@@ -266,46 +240,28 @@ pub async fn list_repo_events(
// collect_visible_ref_updates drops any row whose slug matches a local repo the
// caller cannot read (fail-closed), and propagates DB errors instead of swallowing
// them.
let gossip_updates =
collect_visible_ref_updates(&state.db, Some(&repo_id_str), limit, caller).await?;
let gossip_raw: Vec<Option<String>> = state
.db
.resolve_ref_update_owner_dids(
&gossip_updates
.iter()
.map(|u| (u.repo.as_str(), u.owner_did.as_deref()))
.collect::<Vec<_>>(),
)
.await?;
let gossip_owner_dids: Vec<serde_json::Value> = gossip_raw
.into_iter()
.map(|o| o.map_or(serde_json::Value::Null, |s| serde_json::json!(s)))
.collect();
let gossip_events: Vec<serde_json::Value> = gossip_updates
.iter()
.enumerate()
.map(|(i, u)| {
let owner_did = gossip_owner_dids[i].clone();
serde_json::json!({
"type": "gossipsub",
"id": u.id,
"repo": u.repo,
"ref_name": u.ref_name,
"old_sha": u.old_sha,
"new_sha": u.new_sha,
"pusher_did": u.pusher_did,
"node_did": u.node_did,
"timestamp": u.timestamp,
"cert_id": u.cert_id,
"received_at": u.received_at,
"from_peer": u.from_peer,
"owner_did": owner_did,
"source": "gossipsub",
let gossip_events: Vec<serde_json::Value> =
collect_visible_ref_updates(&state.db, Some(&repo_id_str), limit, caller)
.await?
.iter()
.map(|u| {
serde_json::json!({
"type": "gossipsub",
"id": u.id,
"repo": u.repo,
"ref_name": u.ref_name,
"old_sha": u.old_sha,
"new_sha": u.new_sha,
"pusher_did": u.pusher_did,
"node_did": u.node_did,
"timestamp": u.timestamp,
"cert_id": u.cert_id,
"received_at": u.received_at,
"from_peer": u.from_peer,
"source": "gossipsub",
})
})
})
.collect();
.collect();
// Merge both lists
let mut all_events: Vec<serde_json::Value> = cert_events;
@@ -370,7 +326,6 @@ mod ref_updates_feed_tests {
cert_id: None,
received_at: Utc::now().to_rfc3339(),
from_peer: "peer1".into(),
owner_did: None,
}
}
@@ -1150,14 +1105,7 @@ mod ref_updates_feed_tests {
.await
.unwrap();
assert_eq!(resp.status(), StatusCode::OK);
let body = body_json(resp).await;
assert_eq!(count(&body), 2);
for event in body["events"].as_array().unwrap() {
assert_eq!(
event["owner_did"], OWNER,
"each event must carry the local owner_did"
);
}
assert_eq!(count(&body_json(resp).await), 2);
}
// Anon reads a quarantined mirror → 404 (withheld without disclosing existence
@@ -1440,8 +1388,7 @@ mod ref_updates_feed_tests {
// A DB error in the gate fails closed as 500, not swallowed into an empty 200 (the
// regression the old get_repo().ok().flatten() allowed). Inject by dropping a
// column get_repo selects so its query errors. Also pin the anonymous body to
// the exact opaque object (#226).
// column get_repo selects so its query errors.
#[sqlx::test]
async fn repo_events_db_error_fails_closed_500(pool: PgPool) {
let state = test_state(pool.clone()).await;
@@ -1464,18 +1411,6 @@ mod ref_updates_feed_tests {
StatusCode::INTERNAL_SERVER_ERROR,
"a DB error must fail closed (500), never serve an empty 200"
);
let body = axum::body::to_bytes(resp.into_body(), usize::MAX)
.await
.expect("read body");
let body = String::from_utf8_lossy(&body);
let v: serde_json::Value = serde_json::from_str(&body).expect("json body");
assert_eq!(
v,
serde_json::json!({
"error": "db_error",
"message": crate::error::DB_ERROR_MESSAGE,
})
);
}
// Symmetric to the gate DB-error test: a DB error in the CERT fetch (after the gate
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
+9 -28
View File
@@ -14,13 +14,14 @@ pub struct LabelRequest {
pub label: String,
}
/// Canonicalize and validate a label string. `add_label` and `remove_label`
/// must agree on the form stored in the DB: `add_label` already trims and
/// lowercases before storing, so `remove_label` has to do the same to find
/// the row. Validation is shared so neither handler can drift on charset or
/// length (#344).
fn canonicalize_label(raw: &str) -> std::result::Result<String, AppError> {
let label = raw.trim().to_lowercase();
/// POST /api/v1/repos/:owner/:repo/labels
pub async fn add_label(
State(state): State<AppState>,
Extension(auth): Extension<AuthenticatedDid>,
Path((owner, name)): Path<(String, String)>,
Json(req): Json<LabelRequest>,
) -> Result<(StatusCode, Json<serde_json::Value>)> {
let label = req.label.trim().to_lowercase();
if label.is_empty() || label.len() > 50 {
return Err(AppError::BadRequest("label must be 1–50 characters".into()));
}
@@ -32,17 +33,6 @@ fn canonicalize_label(raw: &str) -> std::result::Result<String, AppError> {
"label must contain only alphanumeric characters, hyphens, and colons".into(),
));
}
Ok(label)
}
/// POST /api/v1/repos/:owner/:repo/labels
pub async fn add_label(
State(state): State<AppState>,
Extension(auth): Extension<AuthenticatedDid>,
Path((owner, name)): Path<(String, String)>,
Json(req): Json<LabelRequest>,
) -> Result<(StatusCode, Json<serde_json::Value>)> {
let label = canonicalize_label(&req.label)?;
let record = state
.db
@@ -69,8 +59,6 @@ pub async fn remove_label(
Extension(auth): Extension<AuthenticatedDid>,
Path((owner, name, label)): Path<(String, String, String)>,
) -> Result<Json<serde_json::Value>> {
let label = canonicalize_label(&label)?;
let record = state
.db
.get_repo(&owner, &name)
@@ -78,18 +66,11 @@ pub async fn remove_label(
.ok_or_else(|| AppError::RepoNotFound(format!("{owner}/{name}")))?;
crate::api::require_repo_owner(&record, &auth.0)?;
let removed = state.db.remove_label(&record.id, &label).await?;
if !removed {
return Err(AppError::NotFound(format!("label '{label}'")));
}
state.db.remove_label(&record.id, &label).await?;
Ok(Json(serde_json::json!({ "label": label, "removed": true })))
}
/// GET /api/v1/repos/:owner/:repo/labels
///
/// Read-visibility-gated (INV-2 root listing): a public repo's labels stay
/// anonymously listable; a private repo's label names are hidden (404) from
/// anyone who cannot read it at the root.
pub async fn list_labels(
State(state): State<AppState>,
Path((owner, name)): Path<(String, String)>,
+8 -606
View File
@@ -92,25 +92,6 @@ pub(crate) fn require_repo_owner(record: &RepoRecord, caller: &str) -> Result<()
}
}
/// Validate a branch name using the configured git binary (storage boundaries).
pub(crate) fn validate_git_ref_with_git(
git_bin: &str,
name: &str,
) -> std::result::Result<(), crate::git::store::GitRefValidationError> {
crate::git::store::validate_git_ref_with_git(git_bin, name)
}
/// Map ref-validation failures to the correct HTTP surface: malformed caller
/// input stays 400; an inability to run git stays on the git/server-error path.
pub(crate) fn map_git_ref_validation_error(
err: crate::git::store::GitRefValidationError,
) -> AppError {
match err {
crate::git::store::GitRefValidationError::Invalid(msg) => AppError::BadRequest(msg),
crate::git::store::GitRefValidationError::GitUnavailable(msg) => AppError::Git(msg),
}
}
#[cfg(test)]
mod did_tests {
use super::did_matches;
@@ -191,7 +172,6 @@ mod authz_guard {
let issues = include_str!("issues.rs");
let bounties = include_str!("bounties.rs");
let replicas = include_str!("replicas.rs");
let events = include_str!("events.rs");
let tasks = include_str!("tasks.rs");
let stars = include_str!("stars.rs");
let protect = include_str!("protect.rs");
@@ -207,13 +187,6 @@ mod authz_guard {
(pulls, "merge_pr", "require_repo_owner("),
(webhooks, "create_webhook", "require_repo_owner("),
(webhooks, "delete_webhook", "require_repo_owner("),
// Bucket A/B hybrid — list_webhooks is read-visibility THEN owner:
// authorize_repo_read 404s a non-reader of a private repo, then
// require_repo_owner 403s a non-owner of a public one. Both halves
// are pinned: the authorize_repo_read marker guards the read half
// (existence hiding) and require_repo_owner guards the owner half.
(webhooks, "list_webhooks", "authorize_repo_read("),
(webhooks, "list_webhooks", "require_repo_owner("),
(labels, "add_label", "require_repo_owner("),
(labels, "remove_label", "require_repo_owner("),
// Bucket A' — owner OR author (did_matches against the author)
@@ -228,18 +201,10 @@ mod authz_guard {
(bounties, "create_bounty", "authorize_repo_read("),
(repos, "fork_repo", "authorize_repo_read("),
// get_by_cid resolves each candidate (provenance path + legacy scan) through
// the shared `gate_and_serve`; the gate markers themselves are asserted
// below. This row proves the delegation is real, so the gate is actually
// reached rather than dead code. The delegated gate still calls
// `visibility_check` directly and never `authorize_repo_read`, so it keeps
// the property the pre-merge marker enforced: no fuzzy re-resolve.
// the shared `gate_and_serve` (#173 round 2); the gate markers themselves are
// asserted below. This row proves the delegation is real — the gate is
// actually reached, not dead code.
(ipfs, "get_by_cid", "gate_and_serve("),
// Sibling read surfaces: gate private-repo metadata on read visibility
// (public repos stay anonymous; private repos 404).
(replicas, "list_replicas", "authorize_repo_read("),
(protect, "list_protected_branches", "authorize_repo_read("),
(labels, "list_labels", "authorize_repo_read("),
(events, "list_repo_events", "authorize_repo_read("),
// Bucket C — signer-self: the acting DID is matched/bound to auth.0
(tasks, "create_task", "did_matches("),
(tasks, "claim_task", "did_matches("),
@@ -298,573 +263,6 @@ mod authz_guard {
}
}
/// Completeness fence over the GraphQL mutation surface: the mirror of
/// `every_in_scope_mutation_has_its_gate` for `graphql/mutation.rs`. Kept a
/// SEPARATE guard with its own table (never folded into the REST rows above):
/// a guard over a heterogeneous set must classify per member, and these
/// same-named task rows belong to the GraphQL surface, not `api/tasks.rs`.
///
/// COMPLETENESS comes from the engine's own SDL (`schema.sdl()`), parsed as a
/// structured AST (#219), NOT introspection and NOT a text parse of source.
/// This matters: async-graphql OMITS `#[graphql(visible = false)]` fields from
/// introspection while leaving them executable, so an introspection-sourced fence
/// could be defeated by a hidden mutation. SDL includes those fields, so no
/// visibility flag (and, via the AST parse, no brace-in-description quirk) can hide
/// a live mutation. The SDL field set must equal the registered set: a new field with
/// no row fails (a repo-write cannot slip in unlisted); a registered name absent from
/// the schema fails (renamed or removed). The set is also cross-checked against
/// introspection, so any field present in SDL but hidden from introspection fails
/// named as a forbidden hidden mutation.
///
/// The MARKER check is the only source-scraped part, and it is not the
/// completeness guarantee. Three honest limits, so a green check is not misread
/// as full authorization coverage:
/// 1. A present marker is not load-bearingness (the gate may not redden on a
/// hostile probe: the #203 present-but-vacuous case). The adversarial tests
/// in `graphql/mutation.rs` are the load-bearing layer.
/// 2. A present marker is not gate-correctness (it may be the wrong bucket for
/// the operation's resource: a repo-write registered as signer-self would
/// pass, so INV-1 review remains the defense).
/// 3. The marker check scrapes the comment-masked, brace-bounded resolver body;
/// a `{`/`}` or a `did_matches(` inside a STRING literal is not masked, so it
/// could mis-slice that body or satisfy the marker vacuously (the same
/// vacuous-marker class as limit 1). Completeness is unaffected: it comes
/// from the SDL AST, not from this scrape.
#[test]
fn every_graphql_mutation_has_its_gate() {
// (Rust snake name, schema field in camelCase, expected gate markers). The
// four are Bucket C signer-self: `/graphql` uses optional_signature, so each
// resolver must call `require_signer(ctx)?` and bind the acting DID with
// `did_matches(`. A repo-write would be Bucket A (`require_repo_owner(`).
// Same-named REST rows live in the guard above against `api/tasks.rs`.
let registered: &[(&str, &str, &[&str])] = &[
(
"create_task",
"createTask",
&["require_signer(ctx)?", "did_matches("],
),
(
"claim_task",
"claimTask",
&["require_signer(ctx)?", "did_matches("],
),
(
"complete_task",
"completeTask",
&["require_signer(ctx)?", "did_matches("],
),
(
"fail_task",
"failTask",
&["require_signer(ctx)?", "did_matches("],
),
];
// Completeness + hidden-mutation cross-check against the REAL schema (#219),
// delegated to the generic `assert_mutation_fence` so the SDL sourcing and the
// cross-check are ALSO exercised by a synthetic hidden schema in
// `fence_rejects_a_hidden_mutation` (INV-21). Because the real MutationRoot has no
// hidden field today, that synthetic test — not a hypothetical production canary —
// is what turns reverting the SDL source or deleting the cross-check RED.
{
use crate::graphql::mutation::MutationRoot;
use crate::graphql::query::QueryRoot;
use crate::graphql::subscription::SubscriptionRoot;
let schema =
async_graphql::Schema::build(QueryRoot, MutationRoot, SubscriptionRoot).finish();
let registered_camel: Vec<&str> = registered.iter().map(|(_, c, _)| *c).collect();
assert_mutation_fence(&schema, &registered_camel);
}
// Marker present (source scrape; the vacuity limit, backstopped by the
// adversarial tests): each resolver body carries every required gate marker.
let masked = mask_comments(include_str!("../graphql/mutation.rs"));
for (snake, _, markers) in registered {
let body = resolver_body(&masked, snake);
for marker in *markers {
assert!(
body.contains(marker),
"GraphQL mutation `{snake}` is missing its gate marker `{marker}`: gate removed"
);
}
}
}
/// Version-safety self-check (#219, INV-21): the fence's completeness guarantee rests
/// on `schema.sdl()` INCLUDING `visible = false` fields while introspection omits
/// them. That is observed async-graphql behavior, not a documented contract, so pin
/// it here: a synthetic schema with a hidden mutation must have that field in its
/// SDL-derived set and NOT in its introspection-derived set. If a future
/// async-graphql upgrade ever filters SDL by visibility, the first assertion flips
/// RED — the assumption is load-bearing, not trusted.
#[test]
fn sdl_source_sees_hidden_mutations_that_introspection_omits() {
use async_graphql::{EmptySubscription, Object, Schema};
struct Q;
#[Object]
impl Q {
async fn ping(&self) -> i32 {
0
}
}
struct M;
#[Object]
impl M {
async fn visible_mut(&self) -> i32 {
1
}
#[graphql(visible = false)]
async fn hidden_mut(&self) -> i32 {
2
}
}
let schema = Schema::build(Q, M, EmptySubscription).finish();
// SDL source (the fix): includes the hidden field.
let sdl_fields = mutation_field_names_from_sdl(&schema.sdl());
assert!(
sdl_fields.iter().any(|f| f == "hiddenMut"),
"SDL-sourced set MUST include the visible=false mutation; if this is RED, \
async-graphql started filtering SDL by visibility and the fence's \
completeness guarantee is broken: {sdl_fields:?}"
);
// Introspection (the fence's own helper): OMITS the hidden field. This is the
// exact gap the fix closes — an introspection-sourced fence never sees `hiddenMut`.
let introspected = introspection_mutation_fields(&schema);
assert!(
!introspected.iter().any(|f| f == "hiddenMut"),
"introspection is expected to OMIT the visible=false mutation (that is the \
gap the SDL source closes); got {introspected:?}"
);
// The exact diff the fence computes: hidden ∈ (sdl \ introspection).
let hidden: Vec<&String> = sdl_fields
.iter()
.filter(|f| !introspected.iter().any(|i| i == *f))
.collect();
assert_eq!(
hidden,
vec!["hiddenMut"],
"the SDL-vs-introspection diff must surface exactly the hidden mutation"
);
}
/// INV-21 wiring guard (#219): drives the generic `assert_mutation_fence` — the SAME
/// helper the real fence calls — against a synthetic schema whose mutation is
/// `visible = false` and IS registered, so completeness passes and ONLY the
/// hidden-from-introspection cross-check can fail. This is what makes the fence's SDL
/// sourcing load-bearing without a production canary: reverting the source to
/// introspection (the hidden field then absent from the SDL set) or deleting the
/// cross-check removes this panic (or changes its message), turning the
/// `should_panic` test RED. The real MutationRoot has no hidden field, so nothing on
/// the real schema could catch that regression.
#[test]
#[should_panic(expected = "hidden from introspection")]
fn fence_rejects_a_hidden_mutation() {
use async_graphql::{EmptySubscription, Object, Schema};
struct Q;
#[Object]
impl Q {
async fn ping(&self) -> i32 {
0
}
}
struct M;
#[Object]
impl M {
async fn visible_mut(&self) -> i32 {
1
}
#[graphql(visible = false)]
async fn hidden_mut(&self) -> i32 {
2
}
}
let schema = Schema::build(Q, M, EmptySubscription).finish();
// Both fields registered, so completeness passes and only the cross-check fires.
assert_mutation_fence(&schema, &["visibleMut", "hiddenMut"]);
}
/// Deprecated-mutation guard (#219): a `#[graphql(deprecation = "...")]` mutation is
/// executable and present in SDL, but GraphQL introspection omits it by default. The
/// fence's introspection helper uses `includeDeprecated: true`, so a deprecated (but
/// visible) mutation is NOT mistaken for a hidden one. With it registered, the fence
/// must NOT panic. Remove `includeDeprecated: true` and this goes RED — the deprecated
/// field would look hidden and false-red as a forbidden visible=false mutation.
#[test]
fn fence_admits_a_deprecated_mutation() {
use async_graphql::{EmptySubscription, Object, Schema};
struct Q;
#[Object]
impl Q {
async fn ping(&self) -> i32 {
0
}
}
struct M;
#[Object]
impl M {
async fn visible_mut(&self) -> i32 {
1
}
#[graphql(deprecation = "use visibleMut")]
async fn deprecated_mut(&self) -> i32 {
2
}
}
let schema = Schema::build(Q, M, EmptySubscription).finish();
// Must not panic: the deprecated field is visible (present in introspection with
// includeDeprecated) and registered, so it is neither hidden nor unclassified.
assert_mutation_fence(&schema, &["visibleMut", "deprecatedMut"]);
}
/// Completeness + hidden-mutation cross-check for a BUILT schema (#219). Generic over
/// the schema type so the real fence AND the synthetic `fence_rejects_a_hidden_mutation`
/// / `fence_admits_a_deprecated_mutation` tests drive the SAME sourcing and cross-check.
/// Completeness is sourced from SDL (which includes `visible = false` fields); the
/// cross-check fails any SDL field hidden from introspection. This genericity is the
/// INV-21 fix: because the real MutationRoot has no hidden field, only a synthetic
/// hidden schema through THIS helper makes reverting the SDL source or removing the
/// cross-check RED, rather than leaving the class latent behind a production canary.
fn assert_mutation_fence<Q, M, S>(
schema: &async_graphql::Schema<Q, M, S>,
registered_camel: &[&str],
) where
Q: async_graphql::ObjectType + 'static,
M: async_graphql::ObjectType + 'static,
S: async_graphql::SubscriptionType + 'static,
{
let sdl_fields = mutation_field_names_from_sdl(&schema.sdl());
let introspected = introspection_mutation_fields(schema);
// No SDL field may be hidden from introspection (a visible=false mutation).
for f in &sdl_fields {
assert!(
introspected.iter().any(|i| i == f),
"GraphQL mutation `{f}` is in the schema but hidden from introspection \
(a `#[graphql(visible = false)]` field): hidden mutations are forbidden \
by this fence — make it visible so its classification is enforceable, or \
remove it"
);
}
// Completeness both directions against the registered (camelCase) set.
for f in &sdl_fields {
assert!(
registered_camel.contains(&f.as_str()),
"unclassified GraphQL mutation `{f}`: add it to \
every_graphql_mutation_has_its_gate with a deliberate gate \
(`require_repo_owner(` for a repo-write, `did_matches(` for a task op)"
);
}
for c in registered_camel {
assert!(
sdl_fields.iter().any(|f| f == c),
"registered GraphQL mutation `{c}` is not in the schema \
(renamed or removed); update the table"
);
}
}
/// The mutation field names via the engine's introspection, run against a passed
/// schema. STRUCTURED (names arrive as data). Uses `fields(includeDeprecated: true)`:
/// GraphQL's default omits `@deprecated` fields, but a `#[graphql(deprecation = "...")]`
/// mutation is still executable and present in SDL, so without this the cross-check
/// above would false-red a deprecated mutation as a hidden (visible=false) one. Only a
/// real visibility filter should trip that arm; deprecation is a lifecycle marker.
fn introspection_mutation_fields<Q, M, S>(
schema: &async_graphql::Schema<Q, M, S>,
) -> Vec<String>
where
Q: async_graphql::ObjectType + 'static,
M: async_graphql::ObjectType + 'static,
S: async_graphql::SubscriptionType + 'static,
{
use async_graphql::Value;
let resp = tokio::runtime::Runtime::new()
.expect("tokio runtime")
.block_on(schema.execute(
"{ __schema { mutationType { fields(includeDeprecated: true) { name } } } }",
));
assert!(
resp.errors.is_empty(),
"schema introspection failed: {:?}",
resp.errors
);
let mut names = Vec::new();
if let Value::Object(root) = &resp.data {
if let Some(Value::Object(sc)) = root.get("__schema") {
if let Some(Value::Object(mt)) = sc.get("mutationType") {
if let Some(Value::List(field_list)) = mt.get("fields") {
for field in field_list {
if let Value::Object(fo) = field {
if let Some(Value::String(n)) = fo.get("name") {
names.push(n.clone());
}
}
}
}
}
}
}
names
}
/// The mutation field names sourced from the built schema's SDL — the completeness
/// source (#219). Builds the same DB-free schema as the introspection helper and runs
/// its `.sdl()` (sync, no runtime needed) through [`mutation_field_names_from_sdl`].
/// SDL includes `#[graphql(visible = false)]` fields, so this source cannot be
/// defeated by hiding a mutation from introspection.
fn graphql_mutation_fields_from_sdl() -> Vec<String> {
use crate::graphql::mutation::MutationRoot;
use crate::graphql::query::QueryRoot;
use crate::graphql::subscription::SubscriptionRoot;
let schema =
async_graphql::Schema::build(QueryRoot, MutationRoot, SubscriptionRoot).finish();
mutation_field_names_from_sdl(&schema.sdl())
}
/// The mutation type's field names extracted from an async-graphql SDL string via
/// the engine's OWN parser AST (`async_graphql::parser::parse_schema`), NOT
/// introspection. This is the completeness source (#219): unlike introspection,
/// which async-graphql filters by `visible` (a `#[graphql(visible = false)]`
/// mutation is executable but ABSENT from introspection), `schema.sdl()` includes
/// every registered field, so a hidden mutation still appears here and must carry a
/// classified row. Parsing the AST (not a brace/line text scan) means a `}` inside a
/// field description cannot mis-slice the type block. camelCase, as async-graphql
/// emits. An empty result on a navigation miss is safe: the set-equality in
/// `every_graphql_mutation_has_its_gate` turns "no fields" into a loud failure for
/// every registered name, never a vacuous pass.
fn mutation_field_names_from_sdl(sdl: &str) -> Vec<String> {
use async_graphql::parser::parse_schema;
use async_graphql::parser::types::{TypeKind, TypeSystemDefinition};
let doc = parse_schema(sdl).expect("async-graphql SDL must parse");
// The mutation root type name: from the `schema { mutation: X }` block if
// present, else the GraphQL default "Mutation".
let mutation_type = doc
.definitions
.iter()
.find_map(|def| match def {
TypeSystemDefinition::Schema(s) => s
.node
.mutation
.as_ref()
.map(|m| m.node.as_str().to_string()),
_ => None,
})
.unwrap_or_else(|| "Mutation".to_string());
// The fields of that object type.
doc.definitions
.iter()
.find_map(|def| match def {
TypeSystemDefinition::Type(t) if t.node.name.node.as_str() == mutation_type => {
match &t.node.kind {
TypeKind::Object(obj) => Some(
obj.fields
.iter()
.map(|f| f.node.name.node.as_str().to_string())
.collect::<Vec<_>>(),
),
_ => None,
}
}
_ => None,
})
.unwrap_or_default()
}
#[test]
fn sdl_extractor_includes_a_hidden_visible_false_field() {
// A field that would be hidden from introspection must still be listed from SDL.
let sdl = "schema { query: QueryRoot mutation: MutationRoot }\n\
type QueryRoot { ping: Int! }\n\
type MutationRoot { createTask: Int! hiddenMut: Int! }\n";
let fields = mutation_field_names_from_sdl(sdl);
assert!(
fields.iter().any(|f| f == "createTask"),
"visible field must be listed: {fields:?}"
);
assert!(
fields.iter().any(|f| f == "hiddenMut"),
"a visible=false field (present in SDL, hidden from introspection) MUST be \
listed by the completeness source: {fields:?}"
);
}
#[test]
fn sdl_extractor_is_not_fooled_by_a_brace_in_a_field_description() {
// A `}` inside a field description must not truncate the type block (the AST
// parse is immune to the text-parse quirk the old design feared).
let sdl = "schema { query: QueryRoot mutation: MutationRoot }\n\
type QueryRoot { ping: Int! }\n\
type MutationRoot {\n\
\"\"\"a description with a } brace and { another\"\"\"\n\
createTask: Int!\n\
claimTask: Int!\n\
}\n";
let fields = mutation_field_names_from_sdl(sdl);
assert!(
fields.iter().any(|f| f == "createTask") && fields.iter().any(|f| f == "claimTask"),
"both fields must be extracted despite a brace in a description: {fields:?}"
);
}
#[test]
fn sdl_extractor_returns_empty_for_a_query_only_schema() {
let sdl = "schema { query: QueryRoot }\ntype QueryRoot { ping: Int! }\n";
assert!(
mutation_field_names_from_sdl(sdl).is_empty(),
"a schema with no mutation root yields no mutation fields (no panic)"
);
}
/// The comment-masked body of `async fn <snake>` in `graphql/mutation.rs`, bounded
/// to the resolver's OWN braces (its `{ .. }`), for the marker check. Brace-matching
/// on the comment-masked source means a `}` in a trailing helper cannot extend the
/// body and a comment marker does not count. A `{`/`}` inside a STRING literal is
/// not masked, a documented marker-check-only residual (limit 3); completeness does
/// not use this.
fn resolver_body(masked_src: &str, snake: &str) -> String {
let needle = format!("async fn {snake}(");
let start = masked_src.find(&needle).unwrap_or_else(|| {
panic!("resolver `async fn {snake}(` not found in graphql/mutation.rs")
});
let open = start
+ masked_src[start..]
.find('{')
.expect("resolver body opening brace");
let mut depth = 0i32;
let mut end = masked_src.len();
for (i, c) in masked_src[open..].char_indices() {
match c {
'{' => depth += 1,
'}' => {
depth -= 1;
if depth == 0 {
end = open + i + 1;
break;
}
}
_ => {}
}
}
masked_src[open..end].to_string()
}
/// `src` with `//` line comments and `/* */` block comments (nested) blanked to
/// spaces, WITHOUT opening a comment inside a `"..."` string (so a `"https://"` does
/// not erase the rest of its line). String CONTENTS are left intact, so a brace or
/// a gate marker inside a string is not masked; for the marker check that is a
/// documented vacuity residual (limit 3), and completeness does not use this at
/// all. Char literals and raw/byte strings are not tracked (absent in resolver
/// bodies); a `'"'` char or a raw string is the remaining narrow residual.
fn mask_comments(src: &str) -> String {
let chars: Vec<char> = src.chars().collect();
let mut out = String::with_capacity(src.len());
let mut i = 0;
let mut block_depth = 0usize;
let mut in_string = false;
while i < chars.len() {
let c = chars[i];
let next = chars.get(i + 1).copied();
if block_depth > 0 {
if c == '/' && next == Some('*') {
block_depth += 1;
out.push_str(" ");
i += 2;
} else if c == '*' && next == Some('/') {
block_depth -= 1;
out.push_str(" ");
i += 2;
} else {
out.push(if c == '\n' { '\n' } else { ' ' });
i += 1;
}
} else if in_string {
out.push(c);
if c == '\\' {
if let Some(n) = next {
out.push(n);
i += 2;
} else {
i += 1;
}
} else {
if c == '"' {
in_string = false;
}
i += 1;
}
} else if c == '"' {
in_string = true;
out.push('"');
i += 1;
} else if c == '/' && next == Some('/') {
while i < chars.len() && chars[i] != '\n' {
out.push(' ');
i += 1;
}
} else if c == '/' && next == Some('*') {
block_depth = 1;
out.push_str(" ");
i += 2;
} else {
out.push(c);
i += 1;
}
}
out
}
/// The SDL field set (the completeness source of truth, #219) is exactly the
/// registered mutations. If async-graphql adds or renames a field, this reddens
/// and forces the `every_graphql_mutation_has_its_gate` table to be updated. Uses
/// SDL, not introspection: introspection omits `visible = false` fields, so an
/// introspection-based pin here would go green while a hidden mutation existed.
#[test]
fn schema_lists_exactly_the_registered_mutations() {
let mut fields = graphql_mutation_fields_from_sdl();
fields.sort();
assert_eq!(
fields,
vec!["claimTask", "completeTask", "createTask", "failTask"],
"the SDL mutation field set is the authoritative discovered set"
);
}
/// The marker check is bounded to a resolver's OWN body and is comment- and
/// string-aware: a `//` inside a `"https://"` string does not erase a real gate on
/// that line (no false-red), and a trailing module helper's `did_matches(` does not
/// rescue a resolver whose own gate was removed (no false-green).
#[test]
fn resolver_body_is_bounded_and_comment_and_string_aware() {
let src = "\
impl MutationRoot {
async fn ok(&self) -> Result<u32> {
let _ = (\"see https://x/y\", did_matches(a, b)); // URL string then gate, one line
Ok(0)
}
async fn last(&self) -> Result<u32> {
// did_matches( only in a comment
Ok(0)
}
}
fn helper() { let _ = did_matches(z, z); }
#[cfg(test)]
mod tests { async fn t() { let _ = did_matches(x, y); } }
";
let masked = mask_comments(src);
assert!(
resolver_body(&masked, "ok").contains("did_matches("),
"a `//` inside a string must not erase the real gate on that line"
);
assert!(
!resolver_body(&masked, "last").contains("did_matches("),
"the marker check must be bounded to the resolver's own body (a trailing \
helper's marker must not rescue a resolver whose gate was removed)"
);
}
/// Proves the comment-stripping that GUARD-1 added: a marker that appears only
/// in a full-line comment (the real `replicas.rs` false-pass shape) must NOT
/// satisfy a row.
@@ -1129,7 +527,11 @@ mod tests { async fn t() { let _ = did_matches(x, y); } }
// Repo-scoped reads known to be ungated today, each tracked by an issue.
// Remove an entry the moment its gate lands (the staleness assert enforces it).
let known_ungated: &[(&str, &str)] = &[];
let known_ungated: &[(&str, &str)] = &[
("list_webhooks", "#94 (PR #113)"),
("list_replicas", "PR #113"),
("list_protected_branches", "PR #113"),
];
let is_known = |n: &str| known_ungated.iter().any(|(k, _)| *k == n);
// Any one of these = the handler binds the caller to an authz decision: the
// first two are read gates, the rest are the write/owner/self-binding forms.
File diff suppressed because it is too large. Load diff
+5 -7
View File
@@ -79,14 +79,12 @@ pub async fn unprotect_branch(
pub async fn list_protected_branches(
State(state): State<AppState>,
Path((owner, repo)): Path<(String, String)>,
auth: Option<Extension<AuthenticatedDid>>,
) -> Result<Json<serde_json::Value>> {
// Read-visibility-gated (INV-2 root listing): a public repo's protected
// branches stay anonymously listable; a private repo's branch names are
// hidden (404) from anyone who cannot read it at the root.
let caller = auth.as_ref().map(|e| e.0 .0.as_str());
let (record, _rules) =
crate::api::authorize_repo_read(&state, &owner, &repo, caller, "/").await?;
let record = state
.db
.get_repo(&owner, &repo)
.await?
.ok_or_else(|| AppError::RepoNotFound(format!("{owner}/{repo}")))?;
let branches = state.db.list_protected_branches(&record.id).await?;
+19 -19
View File
@@ -50,17 +50,6 @@ pub async fn create_pr(
let target_branch = req
.target_branch
.unwrap_or_else(|| record.default_branch.clone());
// Validate both refs before they are stored, since both are later
// interpolated into git argv (git diff / worktree add / merge). Validate the
// RESOLVED target, not just a caller-supplied one: create_repo also gates
// default_branch, but validating here as well means a PR can never feed the
// git sink an unchecked ref even if a default was poisoned by an older row
// or a future path that skips create_repo's gate.
crate::api::validate_git_ref_with_git(&state.git_bin, &req.source_branch)
.map_err(crate::api::map_git_ref_validation_error)?;
crate::api::validate_git_ref_with_git(&state.git_bin, &target_branch)
.map_err(crate::api::map_git_ref_validation_error)?;
let number = state.db.next_pr_number(&record.id).await?;
let now = Utc::now().to_rfc3339();
@@ -220,6 +209,9 @@ pub async fn merge_pr(
return Err(AppError::BadRequest(format!("PR is already {}", pr.status)));
}
// Shed 503 + Retry-After on an exhausted write-lock POOL instead of a generic
// git 500 (#173 F1). Merging holds no admission permit, so it reaches the pool
// unthrottled; reuse the push handler's mapping so the two cannot drift.
let guard = state
.repo_store
.acquire_write(&record.owner_did, &record.name)
@@ -236,10 +228,7 @@ pub async fn merge_pr(
);
// Always release the advisory lock — even on error; upload to Tigris only on success.
// Short-circuit on a refused publish before the PR is marked merged and
// before the webhook fires. Both are irreversible announcements of a merge
// commit that only exists on this node's disk.
guard.release(merge_result.is_ok()).await.into_result()?;
guard.release(merge_result.is_ok()).await;
let merge_sha = merge_result.map_err(|e| AppError::Git(e.to_string()))?;
@@ -439,8 +428,12 @@ pub async fn list_comments(
Ok(Json(serde_json::json!({ "comments": comments })))
}
/// #173 F1 follow-up: merge holds no admission permit, so an exhausted write-lock
/// pool must shed 503 + Retry-After.
/// #173 F1 follow-up: `merge_pr` reaches `acquire_write` holding NO admission permit
/// (unlike the push handler, which is capped by the git-push semaphore), so it is one
/// of the callers most likely to meet an exhausted write-lock POOL under load. An
/// exhausted pool is a capacity signal, so the merge must shed 503 + Retry-After
/// (`AppError::Overloaded`) the way the push handler does, not report the generic
/// 500 git error that says nothing about retrying.
#[cfg(test)]
mod lock_pool_shed_tests {
use super::*;
@@ -464,15 +457,20 @@ mod lock_pool_shed_tests {
}
}
/// RED-before/GREEN-after for `merge_pr`. Both directions: the shed while the only
/// lock-pool connection is held by a guard on a DIFFERENT repo (so this is pool
/// capacity, not advisory-lock contention on this repo), and the must-not case
/// once that connection is back.
#[sqlx::test]
async fn merge_pr_lock_pool_exhaustion_sheds_503_not_500(pool: PgPool) {
let owner = "did:key:zMERGELOCKPOOLOWNERAAAAAAAAAAAAAAAAAAAAA";
let mut state = crate::test_support::test_state(pool.clone()).await;
// One lock-pool connection with a short checkout timeout, so a single held
// guard exhausts it promptly rather than at the pool default.
state.repo_store = crate::git::repo_store::RepoStore::new(
std::path::PathBuf::from("/tmp/gitlawb-pulls-lockpool"),
None,
crate::git::repo_store::build_lock_pool(&pool, 1, std::time::Duration::from_secs(1)),
std::time::Duration::from_secs(300),
);
let repo = seed_repo(owner, "lp-merge");
@@ -526,7 +524,9 @@ mod lock_pool_shed_tests {
"merge_pr: a capacity shed must tell the client when to retry"
);
let _ = held.release(false).await;
// MUST-NOT: with the pool free again the merge is not shed as capacity (it
// fails later on the nonexistent on-disk repo, which is a git 500).
held.release(false).await;
let admitted = merge_pr(
State(state.clone()),
Extension(AuthenticatedDid(owner.to_string())),
+16 -16
View File
@@ -39,11 +39,11 @@ pub async fn register_replica(
) -> Result<(StatusCode, Json<serde_json::Value>)> {
validate_replica_url(&req.url)?;
// The mutation response carries repo metadata (name, replica count), so
// registration applies the same read-visibility decision as the listing:
// a caller who may not read the repo gets the same 404 as a missing one.
let (record, _rules) =
crate::api::authorize_repo_read(&state, &owner, &repo, Some(auth.0.as_str()), "/").await?;
let record = state
.db
.get_repo(&owner, &repo)
.await?
.ok_or_else(|| AppError::RepoNotFound(format!("{owner}/{repo}")))?;
let replica_did = &auth.0;
@@ -93,10 +93,11 @@ pub async fn unregister_replica(
Extension(auth): Extension<AuthenticatedDid>,
Path((owner, repo)): Path<(String, String)>,
) -> Result<Json<serde_json::Value>> {
// Same gate as register_replica: removal mutates repo-scoped metadata and
// its response discloses the replica count.
let (record, _rules) =
crate::api::authorize_repo_read(&state, &owner, &repo, Some(auth.0.as_str()), "/").await?;
let record = state
.db
.get_repo(&owner, &repo)
.await?
.ok_or_else(|| AppError::RepoNotFound(format!("{owner}/{repo}")))?;
let replica_did = &auth.0;
state.db.unregister_replica(&record.id, replica_did).await?;
@@ -112,17 +113,16 @@ pub async fn unregister_replica(
}
/// GET /api/v1/repos/:owner/:repo/replicas
/// Read-visibility-gated: a PUBLIC repo's replica list stays anonymously
/// listable (mirror-discovery), but a PRIVATE repo's replica URLs are hidden
/// (404) from anyone who cannot read the repo at its root.
/// Public — returns the list of replicas (DID + URL + registration timestamp).
pub async fn list_replicas(
State(state): State<AppState>,
Path((owner, repo)): Path<(String, String)>,
auth: Option<Extension<AuthenticatedDid>>,
) -> Result<Json<serde_json::Value>> {
let caller = auth.as_ref().map(|e| e.0 .0.as_str());
let (record, _rules) =
crate::api::authorize_repo_read(&state, &owner, &repo, caller, "/").await?;
let record = state
.db
.get_repo(&owner, &repo)
.await?
.ok_or_else(|| AppError::RepoNotFound(format!("{owner}/{repo}")))?;
let replicas = state.db.list_replicas(&record.id).await?;
File diff suppressed because it is too large. Load diff
+2 -2
View File
@@ -1,4 +1,4 @@
//! DID resolution — look up a node by its Twigpine DID.
//! DID resolution — look up a node by its gitlawb DID.
//!
//! First checks the local peers table (HTTP-layer peer discovery),
//! then queries the Kademlia DHT via the libp2p swarm.
@@ -14,7 +14,7 @@ use crate::state::AppState;
/// GET /api/v1/resolve/{did}
///
/// Resolves a Twigpine DID to an HTTP URL and p2p multiaddr.
/// Resolves a gitlawb DID to an HTTP URL and p2p multiaddr.
/// Checks local peer cache first, then falls back to Kademlia DHT lookup.
pub async fn resolve_did(
State(state): State<AppState>,
+12 -89
View File
@@ -1,8 +1,8 @@
//! Webhook CRUD API.
//!
//! POST /api/v1/repos/:owner/:repo/hooks — create (owner only)
//! GET /api/v1/repos/:owner/:repo/hooks — list (owner only; auth required)
//! DELETE /api/v1/repos/:owner/:repo/hooks/:id — delete (owner only)
//! POST /api/v1/repos/:owner/:repo/hooks — create (auth required)
//! GET /api/v1/repos/:owner/:repo/hooks — list
//! DELETE /api/v1/repos/:owner/:repo/hooks/:id — delete (auth required)
use axum::extract::{Extension, Path, State};
use axum::http::StatusCode;
@@ -46,10 +46,9 @@ pub async fn create_webhook(
// endpoints (SSRF). Delivery runs on the shared no-redirect client
// (main.rs), which closes the 3xx-to-internal bounce.
if !crate::api::peers::is_public_http_url(&req.url) {
return Err(AppError::BadRequest(format!(
"webhook URL {}",
crate::api::peers::PUBLIC_HTTP_URL_REQUIREMENT
)));
return Err(AppError::BadRequest(
"webhook URL must be a public http(s) URL (no loopback, private, or .internal/.local hosts)".into(),
));
}
let events = req.events.unwrap_or_else(|| vec!["*".into()]);
@@ -74,26 +73,12 @@ pub async fn create_webhook(
pub async fn list_webhooks(
State(state): State<AppState>,
Path((owner, name)): Path<(String, String)>,
auth: Option<Extension<AuthenticatedDid>>,
) -> Result<Json<serde_json::Value>> {
// This route sits on `optional_signature`, so the DID is optional. Webhook
// callback URLs are owner-secret config and there is no anonymous form, so a
// headerless caller is rejected before any lookup (401, which fires for an
// existing-private and an absent repo alike, so it leaks no existence).
let Some(Extension(AuthenticatedDid(caller))) = auth else {
return Err(AppError::Unauthorized(
"listing webhooks requires authentication".into(),
));
};
// Read-visibility first, then owner. authorize_repo_read returns 404 on a
// visibility deny, so a non-reader of a private repo cannot tell it exists
// (uniform with the sibling read surfaces); require_repo_owner then yields
// 403 only for a non-owner of a public/readable repo, where existence is not
// secret.
let (record, _rules) =
crate::api::authorize_repo_read(&state, &owner, &name, Some(&caller), "/").await?;
crate::api::require_repo_owner(&record, &caller)?;
let record = state
.db
.get_repo(&owner, &name)
.await?
.ok_or_else(|| AppError::RepoNotFound(format!("{owner}/{name}")))?;
let hooks = state.db.list_webhooks(&record.id).await?;
// Redact secrets in list response
@@ -142,19 +127,7 @@ pub async fn delete_webhook(
#[cfg(test)]
mod tests {
use axum::extract::{Extension, Path, State};
use axum::Json;
use chrono::Utc;
use gitlawb_core::identity::Keypair;
use sqlx::PgPool;
use uuid::Uuid;
use super::{create_webhook, CreateWebhookRequest};
use crate::api::peers::{is_public_http_url, PUBLIC_HTTP_URL_REQUIREMENT};
use crate::auth::AuthenticatedDid;
use crate::db::RepoRecord;
use crate::error::AppError;
use crate::test_support::test_state;
use crate::api::peers::is_public_http_url;
// create_webhook gates req.url through is_public_http_url. Pin the exact
// SSRF targets from issue #81 so the webhook path can never regress to the
@@ -165,7 +138,6 @@ mod tests {
"http://127.0.0.1:5432/",
"http://169.254.169.254/latest/meta-data/",
"http://localhost/",
"http://node.localhost/",
"http://10.0.0.5/",
"http://[::1]/",
// IPv6 transition encodings smuggling loopback v4 (6to4 / NAT64).
@@ -183,53 +155,4 @@ mod tests {
assert!(is_public_http_url("https://hooks.example.com/gitlawb"));
assert!(is_public_http_url("http://203.0.113.10:7545/"));
}
#[sqlx::test]
async fn webhook_localhost_rejection_uses_shared_public_url_contract(pool: PgPool) {
let state = test_state(pool).await;
let owner = Keypair::generate().did().to_string();
let repo_id = Uuid::new_v4().to_string();
state
.db
.create_repo(&RepoRecord {
id: repo_id.clone(),
name: "webhook-contract".to_string(),
owner_did: owner.clone(),
description: None,
is_public: true,
default_branch: "main".to_string(),
created_at: Utc::now(),
updated_at: Utc::now(),
disk_path: "/tmp/webhook-contract.git".to_string(),
forked_from: None,
machine_id: None,
})
.await
.unwrap();
let result = create_webhook(
State(state.clone()),
Extension(AuthenticatedDid(owner.clone())),
Path((owner, "webhook-contract".to_string())),
Json(CreateWebhookRequest {
url: "https://node.localhost/hook".to_string(),
secret: None,
events: None,
}),
)
.await;
match result {
Err(AppError::BadRequest(message)) => assert_eq!(
message,
format!("webhook URL {PUBLIC_HTTP_URL_REQUIREMENT}")
),
Err(error) => panic!("unexpected webhook rejection: {error}"),
Ok(_) => panic!("a .localhost webhook must be rejected"),
}
assert!(
state.db.list_webhooks(&repo_id).await.unwrap().is_empty(),
"a rejected .localhost webhook must leave no row behind"
);
}
}
-62
View File
@@ -1,62 +0,0 @@
//! #26 Split PR 2 — three-outcome recovery policy types.
//!
//! This module owns the recovery classification policy the reviewer
//! demanded: only a trustworthy, protocol-defined absence authorizes
//! a paid re-upload. Everything else stays non-terminal.
//!
//! Deliberately narrow: the ANS-104 codec lives in `ans104`, and the
//! gateway probe, the `verify_anchor` path, and the public verify
//! endpoint are deferred until the uploader, retrieval proof, and
//! recovery consumer can be proven as one vertical slice (reviewer 2,
//! round 5). A probe against a real provider contract — same-network
//! upload/read pair, redirect-safe client, envelope-supplying read
//! API, provider-documented absence evidence — belongs in that slice,
//! not here. What ships here is the policy type plus the rule that
//! only `DefinitivelyAbsent` permits spending another upload.
/// Outcome of a gateway probe for a persisted `item_id`.
// Policy surface for the recovery slice: no construction on this
// head (the probe arrives with the provider contract); the unit
// test pins the rule meanwhile.
#[allow(dead_code)]
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum ProbeOutcome {
/// The persisted item was served back with a verifiable envelope
/// bound to the requested id. No re-upload allowed.
Present,
/// Trustworthy, protocol-defined proof the item was never
/// accepted. Authorizes re-upload. The exact evidence that
/// establishes this is defined by the provider contract in the
/// vertical slice that introduces the probe — never by a
/// mock-only body shape.
DefinitivelyAbsent,
/// Anything else: transport failure, oversized body, bad
/// signature, id mismatch, ambiguous gateway response. The
/// outbox stays non-terminal.
Indeterminate,
}
impl ProbeOutcome {
/// True iff the recovery code is allowed to spend another paid
/// upload request. Only `DefinitivelyAbsent` qualifies.
#[allow(dead_code)] // the recovery consumer lives in a later slice
pub fn permits_reupload(self) -> bool {
matches!(self, ProbeOutcome::DefinitivelyAbsent)
}
}
#[cfg(test)]
mod tests {
use super::*;
/// Only `DefinitivelyAbsent` authorizes a paid re-upload. This is
/// the policy the reviewer named: collapsing any ambiguous outcome
/// to absent spends a second immutable upload for the same
/// transition.
#[test]
fn only_definitively_absent_authorizes_reupload() {
assert!(!ProbeOutcome::Present.permits_reupload());
assert!(ProbeOutcome::DefinitivelyAbsent.permits_reupload());
assert!(!ProbeOutcome::Indeterminate.permits_reupload());
}
}
+36 -256
View File
@@ -34,38 +34,6 @@ use gitlawb_core::http_sig::{
};
use gitlawb_core::identity::verify;
/// Per-route-group ceiling on how much request body [`require_signature`] will
/// buffer BEFORE it has authenticated anything.
///
/// This is middleware state rather than a constant because a single number cannot
/// serve both a 2 GB git push and a 2 MB profile update: the git write group
/// passes `GITLAWB_MAX_PACK_BYTES`, every other signed group passes
/// [`crate::server::SIGNED_BODY_LIMIT`]. Making it a required piece of state means
/// a new signed route group cannot be wired up without choosing a limit.
#[derive(Clone, Copy, Debug)]
pub struct SignatureBodyLimit(pub usize);
/// The hint for an unresolvable DID, chosen by method. The hint is only true
/// for a method we have no resolver for: a did:key that parsed and then failed
/// on its key material (a small-order key, wrong multicodec, wrong length)
/// would be sent looking for the wrong problem by the alpha-support wording.
fn unresolvable_did_hint(key_id: &Did) -> &'static str {
if key_id.is_did_key() {
"the DID is a did:key whose key material did not resolve"
} else {
"only did:key is supported in alpha"
}
}
/// Bound the echoed keyid in the unresolvable-DID message. `key_id` is
/// unauthenticated request input, so reflecting it back at whatever length the
/// caller chose would amplify attacker input into the response body. The bound
/// is slack (a did:key method-id is a fixed 48 characters) and only defends
/// the echo, not the resolution.
fn bounded_did_echo(key_id: &str) -> String {
key_id.chars().take(96).collect()
}
/// Axum middleware that enforces HTTP Signature authentication (RFC 9421).
///
/// Every write request must carry:
@@ -80,52 +48,20 @@ fn bounded_did_echo(key_id: &str) -> String {
/// 4. Resolves the did:key to an Ed25519 VerifyingKey
/// 5. Rebuilds the signing string and verifies the Ed25519 signature
/// 6. Verifies Content-Digest matches the request body
pub async fn require_signature(
State(SignatureBodyLimit(limit)): State<SignatureBodyLimit>,
request: Request,
next: Next,
) -> Response {
verify_signature(limit, request, next).await
}
/// The signature check itself, shared by [`require_signature`] and the optional
/// variant. `limit` bounds the pre-auth buffer; see [`SignatureBodyLimit`].
async fn verify_signature(limit: usize, request: Request, next: Next) -> Response {
// Buffer the body so we can verify content-digest and pass it downstream.
//
// The buffer is bounded, because this runs BEFORE any identity check: an
// unauthenticated caller with no DID, no signature and no repo reaches this
// line. Nothing upstream supplies the bound. On the git write routes
// `RequestBodyLimitLayer` sits on the inner router, and a later `.layer()` is
// the OUTER service, so this middleware runs first and sees the raw body. On
// every other signed group axum's 2 MB `DefaultBodyLimit` is enforced by the
// `Bytes`/`Json` extractors through a request extension, which collecting the
// body here bypasses. `Limited` stops reading at the limit rather than reading
// everything and then measuring it, which is the property that matters.
pub async fn require_signature(request: Request, next: Next) -> Response {
// Buffer the body so we can verify content-digest and pass it downstream
let (parts, body) = request.into_parts();
let body_bytes = match http_body_util::Limited::new(body, limit).collect().await {
Ok(collected) => collected.to_bytes(),
// Over the limit: a distinct status from the 401 an unsigned request gets
// and the 400 below, so a client can tell "too big" from "not authenticated"
// and from "your body did not arrive".
Err(e) if e.is::<http_body_util::LengthLimitError>() => return (
StatusCode::PAYLOAD_TOO_LARGE,
Json(json!({
"error": "body_too_large",
"message": format!("request body exceeds the {limit}-byte limit for this route"),
})),
)
.into_response(),
Err(_) => {
return (
let body_bytes =
match body.collect().await {
Ok(collected) => collected.to_bytes(),
Err(_) => return (
StatusCode::BAD_REQUEST,
Json(
json!({ "error": "unreadable_body", "message": "could not read request body" }),
),
)
.into_response()
}
};
.into_response(),
};
let sig_input = parts
.headers
@@ -184,7 +120,7 @@ async fn verify_signature(limit: usize, request: Request, next: Next) -> Respons
COVERED_COMPONENTS.join(", "),
missing.join(", ")
),
"hint": "See https://twigpine.com/agents#authentication",
"hint": "See https://gitlawb.com/agents#authentication",
})),
)
.into_response();
@@ -209,14 +145,11 @@ async fn verify_signature(limit: usize, request: Request, next: Next) -> Respons
StatusCode::BAD_REQUEST,
Json(json!({
"error": "unresolvable_did",
"message": format!(
"cannot resolve DID '{}': {e}",
bounded_did_echo(&sig.key_id.to_string())
),
"hint": unresolvable_did_hint(&sig.key_id),
"message": format!("cannot resolve DID '{}': {e}", sig.key_id),
"hint": "only did:key is supported in alpha",
})),
)
.into_response();
.into_response()
}
};
@@ -229,34 +162,17 @@ async fn verify_signature(limit: usize, request: Request, next: Next) -> Respons
.unwrap_or("/")
.to_string();
// Required, never defaulted. The signing string is rebuilt from the request,
// so an absent header would make the covered `content-digest` empty on both
// sides: verification would pass and the digest-versus-body comparison below
// would have nothing to compare, leaving a signed request free to carry a
// body its signature never covered. `sign_request` always emits the header,
// so refusing here costs no conforming client anything.
let content_digest = match parts
let content_digest = parts
.headers
.get("content-digest")
.and_then(|v| v.to_str().ok())
{
Some(v) => v.to_string(),
None => {
return (
StatusCode::BAD_REQUEST,
Json(json!({
"error": "missing_content_digest",
"message": "Content-Digest header is required on a signed request",
})),
)
.into_response()
}
};
.unwrap_or("")
.to_string();
let mut request_values: HashMap<String, String> = HashMap::new();
request_values.insert("@method".to_string(), method);
request_values.insert("@path".to_string(), path_and_query);
request_values.insert("content-digest".to_string(), content_digest.clone());
request_values.insert("content-digest".to_string(), content_digest);
// The @signature-params value is the part of Signature-Input after "sig1="
let sig_params_value = sig_input.strip_prefix("sig1=").unwrap_or(&sig_input);
@@ -301,18 +217,23 @@ async fn verify_signature(limit: usize, request: Request, next: Next) -> Respons
.into_response();
}
// Verify Content-Digest matches the actual request body. Unconditional: the
// header's presence was established above, so there is no branch here that
// skips the comparison.
if content_digest != compute_content_digest(&body_bytes) {
return (
StatusCode::BAD_REQUEST,
Json(json!({
"error": "content_digest_mismatch",
"message": "Content-Digest does not match request body",
})),
)
.into_response();
// Verify Content-Digest matches the actual request body
if let Some(claimed) = parts
.headers
.get("content-digest")
.and_then(|v| v.to_str().ok())
{
let actual = compute_content_digest(&body_bytes);
if claimed != actual {
return (
StatusCode::BAD_REQUEST,
Json(json!({
"error": "content_digest_mismatch",
"message": "Content-Digest does not match request body",
})),
)
.into_response();
}
}
tracing::info!(did = %sig.key_id, "✓ authenticated request");
@@ -327,17 +248,11 @@ async fn verify_signature(limit: usize, request: Request, next: Next) -> Respons
/// Optional variant for rolling upgrades: verify and inject `AuthenticatedDid` when
/// RFC 9421 signature headers are present, but allow legacy unsigned requests to
/// continue when no signature attempt was made.
pub async fn optional_signature(
State(SignatureBodyLimit(limit)): State<SignatureBodyLimit>,
request: Request,
next: Next,
) -> Response {
pub async fn optional_signature(request: Request, next: Next) -> Response {
let has_signature_headers = request.headers().contains_key("signature-input")
|| request.headers().contains_key("signature");
if has_signature_headers {
// Same pre-auth buffer, same bound: presenting signature headers is not
// authentication, so this path is as reachable as the required one.
return verify_signature(limit, request, next).await;
return require_signature(request, next).await;
}
next.run(request).await
}
@@ -465,7 +380,7 @@ fn human_detected(message: &str) -> impl IntoResponse {
"error": "not_an_agent",
"message": message,
"hint": "gl identity new && gl register",
"docs": "https://twigpine.com/agents",
"docs": "https://gitlawb.com/agents",
})),
)
}
@@ -508,58 +423,6 @@ mod tests {
assert!(validate_ucan_chain(&token, &node_did, &agent_did).is_ok());
}
/// The two-branch hint selection is the point of the unresolvable-DID
/// change: a did:key that failed on its key material must be told the key
/// material did not resolve, and any other method must be told only did:key
/// is supported. A regression that drops the is_did_key() check would send
/// a did:key caller after the wrong problem, so both branches are pinned.
#[test]
fn unresolvable_did_hint_distinguishes_key_material_from_method() {
// A real did:key whose key material fails to resolve (small-order):
// key-material hint.
let mut weak = [0u8; 32];
weak[0] = 1; // compressed identity point
let weak_vk = ed25519_dalek::VerifyingKey::from_bytes(&weak).unwrap();
let weak_did = Did::from_verifying_key(&weak_vk);
assert!(
weak_did.to_verifying_key().is_err(),
"fixture precondition: small-order did:key must not resolve"
);
assert_eq!(
unresolvable_did_hint(&weak_did),
"the DID is a did:key whose key material did not resolve"
);
// A method with no resolver: alpha-support hint.
let web = Did::web("example.com");
assert!(
web.to_verifying_key().is_err(),
"fixture precondition: did:web must not resolve locally"
);
assert_eq!(
unresolvable_did_hint(&web),
"only did:key is supported in alpha"
);
}
/// The unresolvable-DID message echoes the keyid, which is unauthenticated
/// request input. The echoed value must be bounded so a hostile caller
/// cannot amplify arbitrary-length input into the response body.
#[test]
fn unresolvable_did_message_bounds_the_echoed_keyid() {
let long_keyid = format!("did:key:z{}", "A".repeat(2000));
let shown = bounded_did_echo(&long_keyid);
assert!(
shown.len() <= 96,
"echoed keyid must be bounded, got {} chars",
shown.len()
);
assert!(
!shown.contains(&"A".repeat(1000)),
"the full keyid must not survive into the message"
);
}
#[test]
fn validate_ucan_chain_wrong_issuer() {
let node = Keypair::generate();
@@ -627,7 +490,6 @@ mod tests {
use clap::Parser;
let keypair = Keypair::generate();
let scan_token_key = crate::state::AppState::derive_scan_token_key(&keypair);
let (ref_tx, _) = tokio::sync::broadcast::channel(1);
let (task_tx, _) = tokio::sync::broadcast::channel(1);
let pool = sqlx::postgres::PgPoolOptions::new()
@@ -654,16 +516,10 @@ mod tests {
rate_limiter: RateLimiter::new(100, Duration::from_secs(60)),
create_ip_rate_limiter: RateLimiter::new(1000, Duration::from_secs(3600)),
push_rate_limiter: RateLimiter::new(600, Duration::from_secs(3600)),
close_issue_rate_limiter: RateLimiter::new(120, Duration::from_secs(3600)),
ipfs_rate_limiter: RateLimiter::new(600, Duration::from_secs(3600)),
ipfs_work_rate_limiter: RateLimiter::new(600, Duration::from_secs(3600)),
ipfs_max_history_walks: crate::api::ipfs::MAX_HISTORY_WALKS_PER_REQUEST,
ipfs_max_legacy_probes: crate::api::ipfs::MAX_LEGACY_PROBES_PER_REQUEST,
ipfs_legacy_scan_page_rows: crate::api::ipfs::LEGACY_SCAN_PAGE_ROWS,
ipfs_max_legacy_scan_rows: crate::api::ipfs::MAX_LEGACY_SCAN_ROWS_PER_REQUEST,
ipfs_max_legacy_scan_rule_bytes:
crate::api::ipfs::MAX_LEGACY_SCAN_RULE_BYTES_PER_REQUEST,
ipfs_scan_token_key: Arc::new(scan_token_key),
ipfs_max_served_object_bytes: crate::api::ipfs::MAX_SERVED_OBJECT_BYTES,
push_limiter_trust: crate::rate_limit::TrustedProxy::None,
sync_trigger_rate_limiter: RateLimiter::new(60, Duration::from_secs(3600)),
@@ -673,9 +529,7 @@ mod tests {
git_write_semaphore: Arc::new(tokio::sync::Semaphore::new(64)),
git_push_advert_semaphore: Arc::new(tokio::sync::Semaphore::new(64)),
git_encrypt_semaphore: Arc::new(tokio::sync::Semaphore::new(64)),
pin_semaphore: Arc::new(tokio::sync::Semaphore::new(64)),
encrypt_inflight: crate::state::EncryptInflight::new(),
repo_write_leases: crate::state::RepoWriteLeases::new(8),
git_read_per_caller: crate::rate_limit::PerCallerConcurrency::with_default_max_keys(16),
git_push_advert_per_caller:
crate::rate_limit::PerCallerConcurrency::with_default_max_keys(8),
@@ -774,78 +628,4 @@ mod tests {
let body_json: serde_json::Value = serde_json::from_slice(&body_bytes).unwrap();
assert_eq!(body_json["error"], "invalid_ucan");
}
/// A request that signs `content-digest` as the EMPTY string and then sends
/// no Content-Digest header at all must be refused before any handler runs.
///
/// This is the whole point of requiring the header rather than defaulting it:
/// the middleware rebuilds the signing string from the request, so an absent
/// header makes the covered digest empty on both sides. The Ed25519 check
/// passes, the body-versus-digest comparison is skipped for want of a header,
/// and a signed request carries a body the signature never covered.
#[tokio::test]
async fn signed_request_without_content_digest_is_rejected() {
use base64::{engine::general_purpose::STANDARD, Engine};
use std::sync::atomic::{AtomicBool, Ordering};
static REACHED: AtomicBool = AtomicBool::new(false);
let kp = Keypair::generate();
let did = kp.did();
let created = chrono::Utc::now().timestamp();
let signature_input = format!(
r#"sig1=("@method" "@path" "content-digest");keyid="{did}";alg="ed25519";created={created}"#
);
let sig_params_value = &signature_input["sig1=".len()..];
let mut values: HashMap<String, String> = HashMap::new();
values.insert("@method".to_string(), "POST".to_string());
values.insert("@path".to_string(), "/x".to_string());
// Exactly what the middleware derives when the header is absent.
values.insert("content-digest".to_string(), String::new());
let signing_string =
build_signing_string(COVERED_COMPONENTS, sig_params_value, &values).unwrap();
let signature = format!(
"sig1=:{}:",
STANDARD.encode(kp.sign(signing_string.as_bytes()).to_bytes())
);
let app = Router::new()
.route(
"/x",
axum::routing::post(|| async {
REACHED.store(true, Ordering::SeqCst);
StatusCode::OK
}),
)
.layer(middleware::from_fn_with_state(
SignatureBodyLimit(crate::server::SIGNED_BODY_LIMIT),
require_signature,
));
let req = Request::builder()
.method("POST")
.uri("/x")
.header("signature-input", signature_input)
.header("signature", signature)
.body(Body::from("a body no signature covers"))
.unwrap();
let resp = app.oneshot(req).await.unwrap();
assert_eq!(
resp.status(),
StatusCode::BAD_REQUEST,
"a signed request with no Content-Digest header must be refused"
);
let bytes = axum::body::to_bytes(resp.into_body(), 2048).await.unwrap();
let body: serde_json::Value = serde_json::from_slice(&bytes).unwrap();
assert_eq!(
body["error"], "missing_content_digest",
"the refusal must name the absent Content-Digest header"
);
assert!(
!REACHED.load(Ordering::SeqCst),
"the handler must never see a request whose body no signature covers"
);
}
}
+1 -1
View File
@@ -1,4 +1,4 @@
//! Embedded seed list of public Twigpine network nodes.
//! Embedded seed list of public Gitlawb network nodes.
//!
//! This module parses `bootstrap-peers.json` (embedded at compile time) and
//! merges its contents into the runtime config so a fresh `docker compose up`
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
+7 -123
View File
@@ -69,12 +69,6 @@ fn resolve_all_recipients(dids: &BTreeSet<String>) -> Result<Vec<VerifyingKey>,
enum SealPlan {
/// An existing envelope already covers exactly this recipient set; nothing to do.
SkipUnchanged,
/// The stored tag matches, but the stored reader set contains a DID that
/// must not resolve (e.g. a small-order did:key sealed before the choke
/// point landed). The old envelope is already public and cannot be
/// withdrawn, but it must be surfaced for operator action rather than
/// silently skipped as unchanged.
SkipUnresolvableStored(Vec<String>),
/// No recipient DID resolved to a key, so there is nothing to seal to.
SkipNoRecipients,
/// At least one recipient DID is unresolvable. Fail closed: never seal to a
@@ -96,30 +90,15 @@ enum SealPlan {
/// recover the blob; reader removal is not retroactive (the old envelope is
/// already public). The comparison is on the opaque node-keyed tag, never the
/// DID list.
///
/// A matching stored tag is normally a skip, but the set is still resolved
/// before skipping: a reader that used to resolve and now must not (a
/// small-order did:key sealed before the choke point landed) is surfaced as
/// [`SealPlan::SkipUnresolvableStored`] instead of being silently treated as
/// unchanged. The old envelope is already public and cannot be withdrawn, but
/// the operator must be able to see that a persisted envelope was sealed to a
/// weak recipient.
fn plan_seal(node_seed: &[u8; 32], dids: &BTreeSet<String>, stored_tag: Option<&str>) -> SealPlan {
let tag = recipients_tag(node_seed, dids);
if stored_tag == Some(tag.as_str()) {
// The stored envelope already covers exactly this set, so there is
// nothing to re-seal. Resolve anyway so a now-unresolvable reader in an
// unchanged set is surfaced rather than hidden by the tag match.
match resolve_all_recipients(dids) {
Ok(_) => SealPlan::SkipUnchanged,
Err(unresolved) => SealPlan::SkipUnresolvableStored(unresolved),
}
} else {
match resolve_all_recipients(dids) {
Ok(keys) if keys.is_empty() => SealPlan::SkipNoRecipients,
Ok(keys) => SealPlan::Seal { keys, tag },
Err(unresolved) => SealPlan::SkipUnresolvable(unresolved),
}
return SealPlan::SkipUnchanged;
}
match resolve_all_recipients(dids) {
Ok(keys) if keys.is_empty() => SealPlan::SkipNoRecipients,
Ok(keys) => SealPlan::Seal { keys, tag },
Err(unresolved) => SealPlan::SkipUnresolvable(unresolved),
}
}
@@ -152,21 +131,6 @@ pub async fn encrypt_and_pin(
// covered (which would permanently lock out the dropped readers, #47).
let (keys, tag) = match plan_seal(node_seed, dids, stored_tag.as_deref()) {
SealPlan::SkipUnchanged => continue,
SealPlan::SkipUnresolvableStored(unresolved) => {
// A stored envelope already covers this exact reader set, and
// the set now contains a DID that must not resolve (a
// small-order did:key sealed before the choke point landed).
// The old envelope is already public and cannot be withdrawn
// or re-sealed; surface it so the operator can act.
let sample: Vec<&String> = unresolved.iter().take(3).collect();
tracing::warn!(
oid = %oid,
unresolved_count = unresolved.len(),
unresolved_sample = ?sample,
"stored envelope was sealed to a recipient that must not resolve; operator action may be required"
);
continue;
}
SealPlan::SkipNoRecipients => {
tracing::warn!(oid = %oid, "no resolvable recipient keys; skipping encrypted pin");
continue;
@@ -206,7 +170,7 @@ pub async fn encrypt_and_pin(
continue;
}
};
let cid = match crate::ipfs_pin::pin_git_object(ipfs_api, oid, &envelope, None).await {
let cid = match crate::ipfs_pin::pin_git_object(ipfs_api, oid, &envelope).await {
Ok(c) if !c.is_empty() => c,
Ok(_) => {
tracing::warn!(oid = %oid, "pin_git_object returned no cid; skipping");
@@ -242,49 +206,6 @@ mod tests {
use super::*;
use ed25519_dalek::SigningKey;
/// A did:key encoding a small-order point. Well-formed, indistinguishable
/// from a real one by eye, and refused at resolution since the choke-point
/// guard landed.
fn weak_did_key() -> String {
let mut weak = [0u8; 32];
weak[0] = 1; // compressed identity point
let vk = ed25519_dalek::VerifyingKey::from_bytes(&weak).expect("decompresses");
Did::from_verifying_key(&vk).to_string()
}
/// The #47 fail-closed arm already refuses to seal to a partial recipient
/// set. Because a small-order did:key no longer RESOLVES, it lands in the
/// unresolved set and that existing arm catches it: nothing is sealed, and
/// no new code path was added to make that happen.
#[test]
fn weak_recipient_falls_into_the_existing_fail_closed_arm() {
let weak = weak_did_key();
let mut dids = BTreeSet::new();
dids.insert(did_key(9));
dids.insert(weak.clone());
match plan_seal(&[0u8; 32], &dids, None) {
SealPlan::SkipUnresolvable(unresolved) => assert!(
unresolved.contains(&weak),
"the weak DID must be named in the unresolved set, got {unresolved:?}"
),
other => panic!("expected SkipUnresolvable, got {other:?}"),
}
}
/// Must-not-over-reject control: an all-legitimate recipient set still seals.
#[test]
fn legitimate_recipient_set_still_seals() {
let mut dids = BTreeSet::new();
dids.insert(did_key(9));
dids.insert(did_key(11));
match plan_seal(&[0u8; 32], &dids, None) {
SealPlan::Seal { keys, .. } => assert_eq!(keys.len(), 2, "both recipients resolve"),
other => panic!("expected Seal, got {other:?}"),
}
}
fn did_key(seed: u8) -> String {
let vk = SigningKey::from_bytes(&[seed; 32]).verifying_key();
Did::from_verifying_key(&vk).to_string()
@@ -427,43 +348,6 @@ mod tests {
));
}
/// P1 remediation (review): an unchanged reader set is normally a skip, but
/// the skip must not hide a recipient that now fails to resolve. A blob
/// sealed to a small-order did:key before the choke-point guard landed keeps
/// its old CID and stays decryptable-without-a-private-key unless the
/// matching-tag path surfaces it. This pins that the stored-tag match still
/// resolves the set instead of short-circuiting.
#[test]
fn plan_seal_surfaces_unresolvable_recipient_in_unchanged_reader_set() {
let weak = weak_did_key();
let mut dids = BTreeSet::new();
dids.insert(did_key(9));
dids.insert(weak.clone());
let stored = recipients_tag(&SEED, &dids);
match plan_seal(&SEED, &dids, Some(&stored)) {
SealPlan::SkipUnresolvableStored(unresolved) => assert!(
unresolved.contains(&weak),
"the weak DID must be named in the unresolved set, got {unresolved:?}"
),
other => panic!("expected SkipUnresolvableStored, got {other:?}"),
}
}
/// Degenerate control: an empty set with a matching stored tag stays a
/// silent skip. The state is unreachable in practice (a seal is never
/// recorded for an empty reader set), so this pins only that the
/// resolve-on-match path must not turn it into a re-seal or a warning.
#[test]
fn plan_seal_unchanged_empty_set_still_skips() {
let empty = BTreeSet::new();
let empty_tag = recipients_tag(&SEED, &empty);
assert!(matches!(
plan_seal(&SEED, &empty, Some(&empty_tag)),
SealPlan::SkipUnchanged
));
}
#[test]
fn plan_seal_reseals_when_recipient_set_changed() {
// A stored tag for a DIFFERENT set is a miss: a newly added reader must
+15 -237
View File
@@ -35,30 +35,14 @@ pub enum AppError {
#[error("invalid request: {0}")]
BadRequest(String),
/// A DID was well-formed enough to carry to a resolver but no verifying key
/// could be derived from it. Its own code rather than plain `bad_request`
/// because the auth middleware already answers `unresolvable_did` for the
/// same failure on a request's keyid, and a client should not have to
/// substring-match a message to tell this apart from the other validation
/// failures on the same route.
#[error("unresolvable did: {0}")]
UnresolvableDid(String),
#[error("too many requests: {0}")]
TooManyRequests(String),
#[error("incomplete: {0}")]
Incomplete(String),
/// A bounded search that could not complete. `continuation`, when present, is the
/// sealed scan position the caller echoes as `?scan=` to resume where the search
/// stopped (#173 round 13, F2). It is AEAD-sealed at the mint site, never plaintext:
/// the row it names is by construction one the caller was denied (INV-13).
#[error("search incomplete: {message}")]
SearchIncomplete {
message: String,
continuation: Option<String>,
},
#[error("search incomplete: {0}")]
SearchIncomplete(String),
#[error("git error: {0}")]
Git(String),
@@ -69,15 +53,6 @@ pub enum AppError {
#[error("server overloaded: {0}")]
Overloaded(String),
#[error("repository is busy")]
RepoBusy,
#[error("repository is temporarily unavailable")]
RepoUnavailable,
#[error("repository write was fenced by a concurrent publish")]
RepoWriteFenced,
#[error("database error: {0}")]
Db(#[from] sqlx::Error),
@@ -91,14 +66,6 @@ pub enum AppError {
pub const DB_UNAVAILABLE_CODE: &str = "db_unavailable";
pub const DB_UNAVAILABLE_MESSAGE: &str = "database is temporarily unavailable";
/// Generic client-facing message for `AppError::Internal`. The real error is
/// logged server-side; never put sqlx/anyhow detail in the HTTP body (#226).
pub const INTERNAL_ERROR_MESSAGE: &str = "an internal error occurred";
/// Generic client-facing message for non-unavailable `AppError::Db`. Query /
/// schema errors stay in logs; the HTTP body must not leak them (#226).
pub const DB_ERROR_MESSAGE: &str = "a database error occurred";
/// Connection-level sqlx failures that mean the database is unreachable right
/// now (retryable, 503), as opposed to server-reported query errors.
fn db_unavailable(e: &sqlx::Error) -> bool {
@@ -119,31 +86,7 @@ impl From<anyhow::Error> for AppError {
fn from(err: anyhow::Error) -> Self {
match err.downcast::<sqlx::Error>() {
Ok(sql) => AppError::Db(sql),
// Lock contention is transient and ordinary, so it must not land as a
// 500. The internal message names the owner slug and repo, so the
// variant carries nothing: the detail stays in the log at the raise
// site and the client gets a fixed retryable body.
Err(err) => match err.downcast::<crate::git::repo_store::LockPoolBusy>() {
Ok(_) => AppError::Overloaded("git write locks at capacity, retry shortly".into()),
Err(err) => match err.downcast::<crate::git::repo_store::RepoBusy>() {
Ok(_) => AppError::RepoBusy,
// Same reasoning one rung down: a refused under-lock refresh is a
// transient storage condition, and its internal message names the
// owner slug and repo, so the variant carries nothing.
Err(err) => match err.downcast::<crate::git::repo_store::RepoUnavailable>() {
Ok(_) => AppError::RepoUnavailable,
// And one more rung: a publish the store refused twice is
// transient in the same way, and the retry is the client's
// to make. The variant carries nothing for the same reason
// as the two above.
Err(err) => match err.downcast::<crate::git::repo_store::RepoWriteFenced>()
{
Ok(_) => AppError::RepoWriteFenced,
Err(err) => AppError::Internal(err),
},
},
},
},
Err(err) => AppError::Internal(err),
}
}
}
@@ -195,9 +138,6 @@ impl IntoResponse for AppError {
// IcaptchaProofRequired is handled above (it carries extra headers/fields).
AppError::IcaptchaProofRequired { .. } => unreachable!("handled before this match"),
AppError::BadRequest(msg) => (StatusCode::BAD_REQUEST, "bad_request", msg.clone()),
AppError::UnresolvableDid(msg) => {
(StatusCode::BAD_REQUEST, "unresolvable_did", msg.clone())
}
AppError::TooManyRequests(msg) => {
(StatusCode::TOO_MANY_REQUESTS, "rate_limited", msg.clone())
}
@@ -208,38 +148,15 @@ impl IntoResponse for AppError {
// legacy-probe or walk ceiling), distinct from the 404 that asserts a
// definitive not-found: absence was NOT proven, so the caller should
// retry rather than treat it as gone (#173, F2). 503, retryable.
AppError::SearchIncomplete { message, .. } => (
AppError::SearchIncomplete(msg) => (
StatusCode::SERVICE_UNAVAILABLE,
"search_incomplete",
message.clone(),
msg.clone(),
),
AppError::Git(msg) => (StatusCode::INTERNAL_SERVER_ERROR, "git_error", msg.clone()),
// 504, distinct from the 500 git_error and from the read-gate's 404 /
// the auth 401, so the client can tell a deadline from a failure.
AppError::Timeout(msg) => (StatusCode::GATEWAY_TIMEOUT, "git_timeout", msg.clone()),
// 503 with a FIXED body: the caller should retry, and must not be told
// which repo is contended or for how long.
AppError::RepoBusy => (
StatusCode::SERVICE_UNAVAILABLE,
"repo_busy",
"repository is busy — retry".into(),
),
// 503 with a FIXED body for the same reason: the caller should retry, and
// must not be told which repo could not be refreshed or why.
AppError::RepoUnavailable => (
StatusCode::SERVICE_UNAVAILABLE,
"repo_unavailable",
"repository is temporarily unavailable, retry".into(),
),
// 503 with a FIXED body again, and its own code: the caller should
// retry, but the condition is not contention, so a client that
// distinguishes them should be able to. The body must not say which
// repo lost its publish or to whom.
AppError::RepoWriteFenced => (
StatusCode::SERVICE_UNAVAILABLE,
"repo_write_fenced",
"repository changed underneath this write, retry".into(),
),
AppError::Db(e) if db_unavailable(e) => (
StatusCode::SERVICE_UNAVAILABLE,
DB_UNAVAILABLE_CODE,
@@ -251,60 +168,27 @@ impl IntoResponse for AppError {
AppError::Overloaded(msg) => {
(StatusCode::SERVICE_UNAVAILABLE, "overloaded", msg.clone())
}
// Opaque body + server log: bare `?` on sqlx paths becomes `AppError::Db`
// via `From`, so this arm (not `Internal`) is the common leak for open
// routes like GET /api/v1/repos and GET /api/v1/peers (#226).
AppError::Db(e) => {
tracing::error!(error = %e, "database error");
(
StatusCode::INTERNAL_SERVER_ERROR,
"db_error",
DB_ERROR_MESSAGE.into(),
)
}
// Opaque body: handlers that map with `.map_err(AppError::Internal)`
// (e.g. GET /ipfs/{cid}) land here; other DB failures usually hit `Db`.
// Log `{e:#}` so context-wrapped anyhow chains keep the leaf cause
// (Display alone is only the outermost layer; see api/repos.rs).
AppError::Internal(e) => {
tracing::error!(error = %format!("{e:#}"), "internal error");
(
StatusCode::INTERNAL_SERVER_ERROR,
"internal_error",
INTERNAL_ERROR_MESSAGE.into(),
)
}
AppError::Db(e) => (StatusCode::INTERNAL_SERVER_ERROR, "db_error", e.to_string()),
AppError::Internal(e) => (
StatusCode::INTERNAL_SERVER_ERROR,
"internal_error",
e.to_string(),
),
};
let mut body = json!({
let body = Json(json!({
"error": code,
"message": message,
});
// A truncated CID search may carry the sealed position the caller echoes as
// `?scan=` to resume. Rendered as a third body field, present only when the
// shed actually left something to resume: a wrapped scan and a throttled
// request both omit it, and its ABSENCE is what tells a caller the ladder is
// over. It is opaque ciphertext; see `gitlawb_core::scan_token`.
if let AppError::SearchIncomplete {
continuation: Some(token),
..
} = &self
{
body["continuation"] = json!(token);
}
}));
let mut resp = (status, Json(body)).into_response();
let mut resp = (status, body).into_response();
// Both retryable 503s advertise when to retry: Overloaded (capacity shed) and
// SearchIncomplete (a bounded CID search cut short by a cap — retry may complete
// it). They ride the shared tail above for body/status, so the header is attached
// here rather than in bespoke early returns, keeping each variant handled once.
if matches!(
self,
AppError::Overloaded(_)
| AppError::SearchIncomplete { .. }
| AppError::RepoBusy
| AppError::RepoUnavailable
| AppError::RepoWriteFenced
AppError::Overloaded(_) | AppError::SearchIncomplete(_)
) {
resp.headers_mut().insert(
axum::http::header::RETRY_AFTER,
@@ -343,110 +227,4 @@ mod tests {
"1"
);
}
/// #226: raw sqlx/DB detail must never appear in the Internal 500 body.
#[tokio::test]
async fn internal_error_body_is_opaque() {
use serde_json::{json, Value};
let leak = "error returned from database: relation \"repos\" does not exist";
let resp = AppError::Internal(anyhow::anyhow!("{leak}")).into_response();
assert_eq!(resp.status(), StatusCode::INTERNAL_SERVER_ERROR);
let bytes = axum::body::to_bytes(resp.into_body(), usize::MAX)
.await
.expect("read body");
let v: Value = serde_json::from_slice(&bytes).expect("json body");
// Exact object: a new `detail` field with different sensitive text must
// also fail, not only a repeat of the original error string.
assert_eq!(
v,
json!({
"error": "internal_error",
"message": INTERNAL_ERROR_MESSAGE,
})
);
}
/// #226: `AppError::Db` query errors (the common `?` path) must also be opaque.
#[tokio::test]
async fn db_error_body_is_opaque() {
use serde_json::{json, Value};
let resp = AppError::Db(sqlx::Error::Protocol(
"error returned from database: column \"is_public\" does not exist".into(),
))
.into_response();
assert_eq!(resp.status(), StatusCode::INTERNAL_SERVER_ERROR);
let bytes = axum::body::to_bytes(resp.into_body(), usize::MAX)
.await
.expect("read body");
let v: Value = serde_json::from_slice(&bytes).expect("json body");
assert_eq!(
v,
json!({
"error": "db_error",
"message": DB_ERROR_MESSAGE,
})
);
}
/// Connection-level failures must stay 503 `db_unavailable`, not collapse
/// into the opaque 500 `db_error` arm if `db_unavailable` loses a variant.
#[tokio::test]
async fn db_pool_timeout_stays_503_unavailable() {
use serde_json::{json, Value};
let resp = AppError::Db(sqlx::Error::PoolTimedOut).into_response();
assert_eq!(resp.status(), StatusCode::SERVICE_UNAVAILABLE);
let bytes = axum::body::to_bytes(resp.into_body(), usize::MAX)
.await
.expect("read body");
let v: Value = serde_json::from_slice(&bytes).expect("json body");
assert_eq!(
v,
json!({
"error": DB_UNAVAILABLE_CODE,
"message": DB_UNAVAILABLE_MESSAGE,
})
);
}
/// #251: bare `?` on anyhow-wrapped sqlx relies on this downcast so a
/// closed pool becomes 503 `db_unavailable`, not 500 `internal_error`.
#[test]
fn pool_closed_via_anyhow_from_is_503_db_unavailable() {
let err: AppError = anyhow::Error::from(sqlx::Error::PoolClosed).into();
let resp = err.into_response();
assert_eq!(resp.status(), StatusCode::SERVICE_UNAVAILABLE);
}
#[test]
fn lock_pool_busy_via_anyhow_from_is_503_overloaded() {
let err: AppError = anyhow::Error::new(crate::git::repo_store::LockPoolBusy).into();
let resp = err.into_response();
assert_eq!(resp.status(), StatusCode::SERVICE_UNAVAILABLE);
assert_eq!(
resp.headers().get("retry-after").unwrap().to_str().unwrap(),
"1"
);
}
#[test]
fn repo_busy_unavailable_and_fenced_advertise_retry_after() {
for err in [
AppError::RepoBusy,
AppError::RepoUnavailable,
AppError::RepoWriteFenced,
] {
let resp = err.into_response();
assert_eq!(resp.status(), StatusCode::SERVICE_UNAVAILABLE);
assert_eq!(
resp.headers().get("retry-after").unwrap().to_str().unwrap(),
"1"
);
}
}
}
+28 -158
View File
@@ -61,27 +61,6 @@ pub fn create_issue(repo_path: &Path, issue_id: &str, json: &str) -> Result<()>
Ok(())
}
/// Remove a single issue ref while the write guard still holds the advisory lock,
/// after a definite publish refusal. Best-effort: a failed delete leaves a
/// local-only orphan, which is still better than telling the client to retry into a
/// duplicate id. Not used on `UploadUnknowable`, where the PUT may still land.
pub fn delete_issue_ref(
git_bin: &str,
repo_path: &Path,
issue_id: &str,
deadline: std::time::Instant,
) -> Result<()> {
let ref_name = format!("refs/gitlawb/issues/{issue_id}");
crate::git::visibility_pack::run_bounded_git(
git_bin,
&["update-ref", "-d", &ref_name],
repo_path,
b"",
deadline,
)?;
Ok(())
}
/// List all issue refs and return their JSON content.
pub fn list_issues(repo_path: &Path) -> Result<Vec<String>> {
// List all refs under refs/gitlawb/issues/
@@ -129,22 +108,19 @@ pub fn list_issues(repo_path: &Path) -> Result<Vec<String>> {
/// Returns Ok(Some(full_id)) on unique match, Ok(None) if not found,
/// Err if the prefix is ambiguous (matches more than one issue).
pub fn resolve_issue_id(repo_path: &Path, id_or_prefix: &str) -> Result<Option<String>> {
// The id comes from the request path and is spliced into a git revision and a
// for-each-ref pattern below. Issue ids are UUIDs, so accept only ASCII
// alphanumerics and '-': anything else (`^{blob}`, `@{0}`, `*`, `/`, `..`) is
// revision or glob syntax, never an issue id, and is reported as not found.
if id_or_prefix.is_empty()
|| !id_or_prefix
.bytes()
.all(|b| b.is_ascii_alphanumeric() || b == b'-')
{
return Ok(None);
// Try exact match first — fast path for callers passing the full UUID.
let exact_ref = format!("refs/gitlawb/issues/{id_or_prefix}");
let check = Command::new("git")
.args(["cat-file", "-e", &exact_ref])
.current_dir(repo_path)
.output()
.context("failed to run git cat-file -e")?;
if check.status.success() {
return Ok(Some(id_or_prefix.to_string()));
}
// List all refs that start with the given string; the glob also matches the exact
// ref. The id handed back is always git's own refname, never the request input.
let exact_ref = format!("refs/gitlawb/issues/{id_or_prefix}");
let prefix_glob = format!("{exact_ref}*");
// Prefix search: list all refs that start with the given string.
let prefix_glob = format!("refs/gitlawb/issues/{id_or_prefix}*");
let list = Command::new("git")
.args(["for-each-ref", "--format=%(refname)", &prefix_glob])
.current_dir(repo_path)
@@ -156,44 +132,25 @@ pub fn resolve_issue_id(repo_path: &Path, id_or_prefix: &str) -> Result<Option<S
}
let output = String::from_utf8_lossy(&list.stdout);
let matches: Vec<&str> = output
.lines()
.map(str::trim)
.filter(|l| !l.is_empty())
.collect();
let matches: Vec<&str> = output.lines().filter(|l| !l.trim().is_empty()).collect();
// An exact id wins over longer ids that share it as a prefix, but only if its
// object exists: for-each-ref also lists a ref whose blob is missing, so a
// dangling exact ref falls through to the prefix rules below.
let exact = match matches.iter().find(|m| **m == exact_ref) {
Some(m) => {
let check = Command::new("git")
.args(["cat-file", "-e", m])
.current_dir(repo_path)
.output()
.context("failed to run git cat-file -e")?;
check.status.success().then_some(*m)
match matches.len() {
0 => Ok(None),
1 => {
// Strip the "refs/gitlawb/issues/" prefix to get the bare ID.
let full_id = matches[0]
.trim()
.strip_prefix("refs/gitlawb/issues/")
.unwrap_or(matches[0].trim())
.to_string();
Ok(Some(full_id))
}
None => None,
};
let found = match exact {
Some(m) => m,
None => match matches.as_slice() {
[] => return Ok(None),
[m] => *m,
_ => anyhow::bail!(
"ambiguous issue prefix '{}': matches {} issues",
id_or_prefix,
matches.len()
),
},
};
// Strip the "refs/gitlawb/issues/" prefix to get the bare ID.
let full_id = found
.strip_prefix("refs/gitlawb/issues/")
.unwrap_or(found)
.to_string();
Ok(Some(full_id))
_ => anyhow::bail!(
"ambiguous issue prefix '{}': matches {} issues",
id_or_prefix,
matches.len()
),
}
}
/// Close an issue by updating its status to "closed" in the git ref.
@@ -277,43 +234,6 @@ mod tests {
assert_eq!(resolved, Some(full_id.to_string()));
}
#[test]
fn test_resolve_exact_id_wins_over_longer_id() {
let dir = TempDir::new().unwrap();
init_repo(&dir);
create_issue(dir.path(), "abc12345", r#"{"status":"open"}"#).unwrap();
create_issue(dir.path(), "abc12345-0000", r#"{"status":"open"}"#).unwrap();
let resolved = resolve_issue_id(dir.path(), "abc12345").unwrap();
assert_eq!(resolved, Some("abc12345".to_string()));
}
#[test]
fn test_resolve_dangling_exact_id_does_not_win_over_longer_id() {
let dir = TempDir::new().unwrap();
init_repo(&dir);
create_issue(dir.path(), "abc12345", r#"{"status":"open","n":1}"#).unwrap();
create_issue(dir.path(), "abc12345-0000", r#"{"status":"open","n":2}"#).unwrap();
// Dangle the exact ref: the ref stays listed by for-each-ref, but its blob is gone.
let out = Command::new("git")
.args(["rev-parse", "refs/gitlawb/issues/abc12345"])
.current_dir(dir.path())
.output()
.unwrap();
let hash = String::from_utf8(out.stdout).unwrap();
let hash = hash.trim();
std::fs::remove_file(
dir.path()
.join(".git/objects")
.join(&hash[..2])
.join(&hash[2..]),
)
.unwrap();
let result = resolve_issue_id(dir.path(), "abc12345");
assert!(result.unwrap_err().to_string().contains("ambiguous"));
assert!(close_issue(dir.path(), "abc12345").is_err());
}
#[test]
fn test_resolve_prefix_matches_unique() {
let dir = TempDir::new().unwrap();
@@ -329,35 +249,6 @@ mod tests {
assert_eq!(resolved, Some(full_id.to_string()));
}
#[test]
fn test_resolve_rejects_revision_syntax() {
let dir = TempDir::new().unwrap();
init_repo(&dir);
let full_id = "abc12345-0000-0000-0000-000000000000";
create_issue(
dir.path(),
full_id,
r#"{"id":"abc12345-0000-0000-0000-000000000000","status":"open"}"#,
)
.unwrap();
// Each of these resolves to (or globs over) the issue blob in git, but none
// is an issue id, so the lookup must not accept it.
for crafted in [
"abc12345-0000-0000-0000-000000000000^{blob}",
"abc12345-0000-0000-0000-000000000000^{}",
"abc12345-0000-0000-0000-000000000000~0",
"*",
"",
] {
assert_eq!(
resolve_issue_id(dir.path(), crafted).unwrap(),
None,
"{crafted:?}"
);
assert_eq!(get_issue(dir.path(), crafted).unwrap(), None, "{crafted:?}");
}
}
#[test]
fn test_resolve_prefix_not_found() {
let dir = TempDir::new().unwrap();
@@ -387,27 +278,6 @@ mod tests {
assert!(result.unwrap_err().to_string().contains("ambiguous"));
}
#[test]
fn delete_issue_ref_removes_a_created_ref() {
let dir = TempDir::new().unwrap();
init_repo(&dir);
let full_id = "eee88888-0000-0000-0000-000000000000";
create_issue(
dir.path(),
full_id,
r#"{"id":"eee88888-0000-0000-0000-000000000000","status":"open"}"#,
)
.unwrap();
delete_issue_ref(
"git",
dir.path(),
full_id,
std::time::Instant::now() + std::time::Duration::from_secs(30),
)
.unwrap();
assert_eq!(resolve_issue_id(dir.path(), full_id).unwrap(), None);
}
#[test]
fn test_close_issue_via_prefix() {
let dir = TempDir::new().unwrap();
-1
View File
@@ -1,5 +1,4 @@
pub mod issues;
pub mod publish;
pub mod push_delta;
pub mod repo_store;
pub mod smart_http;
-358
View File
@@ -1,358 +0,0 @@
//! The publication boundary: WHO wrote, and HOW FAR that write got.
//!
//! Four lifecycle defects on this branch shared one root cause. A resource was
//! tracked by a coarse state — "release started", "the path exists", "a row with
//! this name exists" — when the safety question was about the IDENTITY and the
//! PUBLICATION STAGE of one specific write attempt. A tail replicated refs from
//! an attempt that had not dispatched a PUT; a read served a tree whose
//! generation nobody had confirmed; a fork claimed a concurrent attempt's row and
//! deleted a successor's object; a response-loss failure was compensated as if it
//! proved non-publication.
//!
//! This module is the vocabulary that makes those questions answerable, and it is
//! deliberately free of any object-storage type:
//!
//! - [`PublishAttemptId`] — minted before the request is built, carried with the
//! bytes as user metadata, read back off the store to decide whether what is
//! published is THIS attempt's work.
//! - [`PublishStage`] / [`PublishStageCell`] — how far one attempt got, observable
//! from outside the future that is performing it, so a CANCELLED attempt can
//! still be classified.
//! - [`UploadError`] — what the client KNOWS about dispatch and commit, not just
//! which HTTP status came back. Destructive compensation is licensed only by
//! [`UploadError::proves_not_published`].
//!
//! # Porting note (#79)
//!
//! PR #79 (`feat/storage-abstraction`) deletes `git/tigris.rs` and replaces it
//! with a `BlobStore`/`RepoArchive` layer. Everything in this file is
//! backend-agnostic on purpose and is meant to survive that swap unchanged: a new
//! backend supplies its own error classifier (the one in `tigris.rs` is a single
//! private function) and keeps these types as the contract its callers read.
use std::sync::Mutex;
/// A durable identity for ONE publish attempt.
///
/// The point is reconciliation. A conditional PUT whose response was lost leaves
/// the client unable to say whether its bytes committed; an attempt id stored
/// alongside those bytes turns that into a question the store can answer, because
/// "is the published object mine?" is decidable where "did my request succeed?"
/// is not.
///
/// Fork creation passes the DB row id it is about to insert, so the object, the
/// on-disk clone and the database row are all stamped with the same attempt
/// identity and every cleanup can be made conditional on still owning all three.
#[derive(Clone, Debug, PartialEq, Eq)]
pub struct PublishAttemptId(String);
impl PublishAttemptId {
/// A fresh identity for an attempt that has nothing else to be named after.
pub fn new() -> Self {
Self(uuid::Uuid::new_v4().to_string())
}
/// Name the attempt after a caller-owned identity — fork creation uses the
/// `record.id` it is about to insert, which is what ties the object back to
/// the exact row rather than to the logical owner/name.
pub fn from_owned(id: impl Into<String>) -> Self {
Self(id.into())
}
pub fn as_str(&self) -> &str {
&self.0
}
}
impl Default for PublishAttemptId {
fn default() -> Self {
Self::new()
}
}
impl std::fmt::Display for PublishAttemptId {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.write_str(&self.0)
}
}
/// The object user-metadata key an attempt id travels in.
///
/// S3 lowercases user-metadata keys, so this must already be lowercase or the
/// read back would never match what was written.
pub const ATTEMPT_METADATA_KEY: &str = "gitlawb-attempt";
/// The precondition an upload is fenced on.
///
/// Object storage is the only place a fence can hold. Dropping the future of an
/// in-flight PUT does not cancel the request the server is already processing,
/// so no amount of local locking stops an abandoned writer's bytes from landing
/// after a successor has published. A conditional PUT the store itself refuses
/// is what actually stops it.
#[derive(Clone, Debug)]
pub enum UploadPrecondition {
/// Publish only if the stored object is still the generation we observed.
IfMatch(String),
/// Publish only if nothing is stored under the key yet.
IfAbsent,
/// No fence. Last writer wins.
Unconditional,
}
/// What the store holds under a key right now.
#[derive(Clone, Debug, PartialEq, Eq)]
pub struct StoredGeneration {
/// The generation a later conditional upload can fence itself on.
pub etag: Option<String>,
/// The attempt that published it, when the object carries our metadata.
/// `None` for anything written by something that does not stamp attempts
/// (an operator upload, a pre-attempt-id archive).
pub attempt: Option<String>,
}
impl StoredGeneration {
/// Is what the store holds the work of `attempt`?
///
/// This is the whole reconciliation primitive. It is deliberately a
/// three-way question collapsed to a boolean only at the point of use: an
/// object with no attempt metadata answers `false`, because "somebody else's
/// bytes" and "bytes nobody stamped" license exactly the same caution.
pub fn belongs_to(&self, attempt: &PublishAttemptId) -> bool {
self.attempt.as_deref() == Some(attempt.as_str())
}
}
/// The receipt of a publish the store ACKNOWLEDGED.
///
/// Only produced on a response the client actually read, so holding one is proof
/// of publication in a way that "the upload future returned" is not.
#[derive(Clone, Debug)]
pub struct UploadReceipt {
pub attempt: PublishAttemptId,
pub etag: Option<String>,
}
/// How far ONE publish attempt got.
///
/// Observable through [`PublishStageCell`] from outside the future doing the
/// work, which is the property the whole design turns on: a handler future that
/// is DROPPED never returns an outcome, so the only way to classify a cancelled
/// attempt is to read the stage it had reached when it was abandoned.
#[derive(Clone, Debug, PartialEq, Eq)]
pub enum PublishStage {
/// Nothing has been attempted.
Idle,
/// No object-storage backend is configured, so there is no publication to
/// confirm and the local write IS the durable copy. Distinct from `Idle`,
/// which means an attempt was possible and had not started: a cancellation
/// under `NoBackend` loses nothing, while a cancellation under `Idle` means
/// a publish that could have happened never did.
NoBackend,
/// The archive is being built. The request has not been constructed, let
/// alone sent. A cancellation here is a DEFINITE "no publication was
/// attempted" — not an ambiguous in-flight PUT.
PreparingArchive,
/// The request is on the wire. Its fate is unknown until the store answers,
/// and a cancellation here does NOT stop the server processing it.
PutDispatched { attempt: PublishAttemptId },
/// The store acknowledged the write. This is the ONLY stage that licenses
/// replication, a 2xx, or treating the local tree as durable.
Published {
attempt: PublishAttemptId,
etag: Option<String>,
},
/// The store definitively did not publish this attempt (a refused
/// precondition, or a failure that proves the bytes never committed).
Refused,
/// The attempt may or may not have committed and has not been reconciled.
/// Destructive compensation is NEVER licensed from here.
Ambiguous { attempt: PublishAttemptId },
}
impl PublishStage {
/// Did this attempt reach a state where the store may hold its bytes?
///
/// True from dispatch onward. `false` is what makes deleting the attempt's
/// object or local tree safe.
pub fn may_have_published(&self) -> bool {
matches!(
self,
PublishStage::PutDispatched { .. }
| PublishStage::Published { .. }
| PublishStage::Ambiguous { .. }
)
}
/// The attempt whose fate is unresolved, when there is one. This is what a
/// reconciliation HEAD is compared against.
pub fn unresolved_attempt(&self) -> Option<&PublishAttemptId> {
match self {
PublishStage::PutDispatched { attempt } | PublishStage::Ambiguous { attempt } => {
Some(attempt)
}
_ => None,
}
}
}
/// A [`PublishStage`] an in-flight upload writes and an outside observer reads.
///
/// `std::sync::Mutex` rather than an async lock on purpose: every write is a
/// field assignment with no await inside, and the reader that matters most runs
/// inside a `Drop` impl, where an async lock cannot be awaited at all.
#[derive(Debug)]
pub struct PublishStageCell(Mutex<PublishStage>);
impl PublishStageCell {
pub fn new() -> Self {
Self::seeded(PublishStage::Idle)
}
pub fn seeded(stage: PublishStage) -> Self {
Self(Mutex::new(stage))
}
pub fn set(&self, stage: PublishStage) {
*self.0.lock().expect("publish stage mutex poisoned") = stage;
}
pub fn get(&self) -> PublishStage {
self.0.lock().expect("publish stage mutex poisoned").clone()
}
}
impl Default for PublishStageCell {
fn default() -> Self {
Self::new()
}
}
/// What the client knows about whether a request it could not complete ever
/// reached the wire.
///
/// The classifier that produces this is the ONLY backend-aware code in the
/// publish path; a pluggable store (#79) supplies its own and every caller below
/// keeps working off [`UploadError`].
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub enum DispatchKnowledge {
/// The request could not be built or sent. The store cannot hold these bytes.
NeverSent,
/// The request may have been sent, and may have committed.
MaybeSent,
}
/// Why an upload did not publish, split by what it leaves the caller ENTITLED TO
/// DO — not by which HTTP status came back.
///
/// The split exists because the previous shape ("precondition lost" vs "some
/// other error") made every caller treat a lost response as proof of failure.
/// Smithy timeout, dispatch and response errors all allow that the request was
/// sent and committed: a server can accept a complete conditional PUT and lose
/// or corrupt the response before the client observes success. Compensating that
/// as a definite failure deletes state that IS published.
#[derive(Debug, thiserror::Error)]
pub enum UploadError {
/// The store explicitly refused the fence (412, or 409 under create-only).
/// DEFINITE: this attempt did not publish, and a successor did.
#[error("upload precondition lost (HTTP {status})")]
PreconditionLost { status: u16 },
/// The attempt provably never committed: it failed before the request could
/// be dispatched, or the store answered a definite client-side refusal.
/// Destructive compensation is safe here and ONLY here.
#[error("upload did not reach the store: {0:#}")]
NotPublished(#[source] anyhow::Error),
/// The request may have been dispatched and may have committed; the client
/// never learned which. The attempt id is carried so the caller can ask the
/// store rather than guess.
#[error("upload outcome is unknowable (attempt {attempt}): {source:#}")]
Ambiguous {
attempt: PublishAttemptId,
#[source]
source: anyhow::Error,
},
}
impl UploadError {
/// May this caller destroy state that would be needed if the write HAD
/// landed — delete the object, drop the only local clone, invalidate the
/// cache?
///
/// Only a proven non-publication says yes. Every new variant must default to
/// `false`, which is why this is a match on the safe arms rather than a
/// negation of the unsafe one.
pub fn proves_not_published(&self) -> bool {
matches!(
self,
UploadError::PreconditionLost { .. } | UploadError::NotPublished(_)
)
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn only_definite_outcomes_license_destructive_compensation() {
assert!(UploadError::PreconditionLost { status: 412 }.proves_not_published());
assert!(
UploadError::NotPublished(anyhow::anyhow!("no route to host")).proves_not_published()
);
assert!(
!UploadError::Ambiguous {
attempt: PublishAttemptId::new(),
source: anyhow::anyhow!("response body truncated"),
}
.proves_not_published(),
"a response-loss failure must never license deleting state the write may own"
);
}
#[test]
fn a_stage_may_have_published_only_from_dispatch_onward() {
let attempt = PublishAttemptId::new();
assert!(!PublishStage::Idle.may_have_published());
assert!(!PublishStage::NoBackend.may_have_published());
assert!(
!PublishStage::PreparingArchive.may_have_published(),
"compression has not constructed a request, let alone sent one"
);
assert!(!PublishStage::Refused.may_have_published());
assert!(PublishStage::PutDispatched {
attempt: attempt.clone()
}
.may_have_published());
assert!(PublishStage::Ambiguous {
attempt: attempt.clone()
}
.may_have_published());
assert!(PublishStage::Published {
attempt,
etag: None
}
.may_have_published());
}
#[test]
fn an_unstamped_object_never_belongs_to_an_attempt() {
let attempt = PublishAttemptId::new();
assert!(StoredGeneration {
etag: Some("\"e\"".into()),
attempt: Some(attempt.as_str().to_string()),
}
.belongs_to(&attempt));
assert!(
!StoredGeneration {
etag: Some("\"e\"".into()),
attempt: None,
}
.belongs_to(&attempt),
"bytes nobody stamped must not be claimed by this attempt"
);
assert!(!StoredGeneration {
etag: Some("\"e\"".into()),
attempt: Some("someone-else".into()),
}
.belongs_to(&attempt));
}
}
+12 -257
View File
@@ -19,7 +19,7 @@
//! objects the reachable withheld set never classified, so subtracting that set
//! is not enough — a dangling private blob would slip through (#99). The caller
//! signals a full scan via [`PinCandidateSet::full_scan`] and must then apply
//! the fail-closed blob-and-tree filter (`visibility_pack::replicable_objects_fail_closed`)
//! the fail-closed blob filter (`visibility_pack::replicable_objects_fail_closed`)
//! instead of the plain reachable-only subtraction.
//!
//! Because the pin candidate set needs only the OID *set* (never the per-path
@@ -211,9 +211,8 @@ pub fn list_all_objects(repo_path: &Path, git_bin: &str, deadline: Instant) -> R
/// Like [`list_all_objects`] but pairs each OID with its object type, via
/// `--batch-check='%(objectname) %(objecttype)'`. The pin path's fail-closed
/// filter needs to tell blobs (content, withholdable) and trees (content,
/// withholdable since #135/#173) from commits/tags (structural, never withheld)
/// without typing the candidate list itself.
/// filter needs to tell blobs (content, withholdable) from commits/trees
/// (structural, never withheld) without typing the candidate list itself.
pub fn list_all_objects_with_type(
repo_path: &Path,
git_bin: &str,
@@ -245,48 +244,24 @@ pub fn list_all_objects_with_type(
/// Every blob OID in the repo, including unreachable/dangling ones. The
/// fail-closed pin filter drops any candidate blob absent from the reachable,
/// visibility-allowed set; a dangling private blob is in this set but not the
/// allowed set, so it never replicates (#99). Production uses
/// [`all_blob_and_tree_oids`] (one typed pass); this wrapper stays for tests.
#[cfg(test)]
/// allowed set, so it never replicates (#99).
pub fn all_blob_oids(
repo_path: &Path,
git_bin: &str,
deadline: Instant,
) -> Result<HashSet<String>> {
Ok(all_blob_and_tree_oids(repo_path, git_bin, deadline)?.0)
}
/// Every blob OID and every tree OID in the repo, including unreachable/dangling
/// ones, from one typed enumeration. The fail-closed pin filter drops any
/// candidate blob or tree absent from the matching reachable visibility-allowed
/// set; a dangling or withheld object is in this universe but not the allowed
/// set, so it never replicates (#99, #172).
pub fn all_blob_and_tree_oids(
repo_path: &Path,
git_bin: &str,
deadline: Instant,
) -> Result<(HashSet<String>, HashSet<String>)> {
let mut blobs = HashSet::new();
let mut trees = HashSet::new();
for (oid, ty) in list_all_objects_with_type(repo_path, git_bin, deadline)? {
match ty.as_str() {
"blob" => {
blobs.insert(oid);
}
"tree" => {
trees.insert(oid);
}
_ => {}
}
}
Ok((blobs, trees))
Ok(list_all_objects_with_type(repo_path, git_bin, deadline)?
.into_iter()
.filter(|(_, ty)| ty == "blob")
.map(|(oid, _)| oid)
.collect())
}
/// The pin candidate OIDs for a push plus whether they came from a full-repo
/// scan. `full_scan` is true when the delta could not be used and the whole
/// object DB (including dangling objects) was enumerated — the caller must then
/// apply the fail-closed blob-and-tree filter, because the reachable-only withheld set
/// cannot classify dangling blobs or trees (#99, #172).
/// apply the fail-closed blob filter, because the reachable-only withheld set
/// cannot classify dangling blobs (#99).
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct PinCandidateSet {
pub candidates: Vec<String>,
@@ -307,61 +282,19 @@ pub struct PinCandidateSet {
/// it can never leak because the withheld/fail-closed filter still runs on
/// whatever set is returned. `full_scan` rides on the returned set so the caller
/// knows when the dangling-inclusive filter is required.
///
/// `scan_sem` is the post-receive scan admission pool (`git_encrypt_semaphore`,
/// #174 F4): both git-spawning stages — the per-tip `cat-file` probe + delta
/// rev-list, and the full-scan fallback — run under ONE permit held for the
/// whole blocking scan. The deletion-only fast path (no new tips) spawns no git
/// and never parks.
///
/// `force_full_scan` forces the full-scan fallback regardless of the tips — the
/// coalesced-drain `PendingWork::FullScan` marker (#174 F5). It composes with
/// the env kill-switch (either forces), and it disqualifies the empty-tips fast
/// path: a forced-scan call with no tips must still enumerate the repo, never
/// silently resolve to an empty delta (which would pin nothing and lose the
/// drained pushes' work — the F5 bug shape).
pub async fn resolve_candidates_for_push(
scan_sem: std::sync::Arc<tokio::sync::Semaphore>,
repo_path: PathBuf,
new_tips: Vec<String>,
old_tips: Vec<String>,
git_bin: String,
timeout: Duration,
force_full_scan: bool,
) -> PinCandidateSet {
let force = force_full_scan || self::force_full_scan();
// No-git fast path: a deletion-only push resolves to an empty delta without
// spawning any git child, so it must not park on the scan pool. A forced
// full scan (caller flag or kill-switch) disqualifies it — the scan spawns git.
if new_tips.is_empty() && !force {
tracing::info!(delta = 0usize, repo = %repo_path.display(), "pin candidate set from push delta");
return PinCandidateSet {
candidates: Vec::new(),
full_scan: false,
};
}
// Scan admission (#174 F4): DEFER, never shed — a dropped scan would
// silently under-pin this push. The permit moves into the blocking closure
// so a started scan always completes holding it. Residuals at
// `acquire_scan_permit`.
let permit =
crate::state::acquire_scan_permit(scan_sem, &repo_path, "pin-candidate scan").await;
tokio::task::spawn_blocking(move || {
let _permit = permit;
// ONE shared deadline for the whole scan, per jatmn ("the same deadline").
let deadline = Instant::now() + timeout;
let new_refs: Vec<&str> = new_tips.iter().map(String::as_str).collect();
let old_refs: Vec<&str> = old_tips.iter().map(String::as_str).collect();
// `force` already folds in the env kill-switch, so the forced arm skips the
// delta machinery outright; the unforced arm goes through the normal
// resolver (whose own env read is false here by construction).
let resolved = if force {
tracing::debug!("full scan forced (coalesced-drain marker or kill-switch)");
PinCandidates::FullScanRequired
} else {
resolve_push_delta(&repo_path, &new_refs, &old_refs, &git_bin, deadline)
};
match resolved {
match resolve_push_delta(&repo_path, &new_refs, &old_refs, &git_bin, deadline) {
PinCandidates::Delta(objs) => {
tracing::info!(delta = objs.len(), repo = %repo_path.display(), "pin candidate set from push delta");
PinCandidateSet { candidates: objs, full_scan: false }
@@ -614,13 +547,11 @@ mod tests {
let c1 = repo.commit_file("a.txt", "one\n");
let c2 = repo.commit_file("b.txt", "two\n");
let set = resolve_candidates_for_push(
std::sync::Arc::new(tokio::sync::Semaphore::new(64)),
repo.path.clone(),
vec![c2.clone()],
vec![c1.clone()],
"git".to_string(),
std::time::Duration::from_secs(600),
false,
)
.await;
assert!(!set.full_scan, "happy-path delta is not a full scan");
@@ -645,13 +576,11 @@ mod tests {
.into_iter()
.collect();
let set = resolve_candidates_for_push(
std::sync::Arc::new(tokio::sync::Semaphore::new(64)),
repo.path.clone(),
vec![blob],
vec![],
"git".to_string(),
std::time::Duration::from_secs(600),
false,
)
.await;
assert!(set.full_scan, "non-commit tip is signalled as a full scan");
@@ -659,180 +588,6 @@ mod tests {
assert_eq!(got, all, "non-commit tip falls back to full repo scan");
}
/// F4 defer proof 2: `resolve_candidates_for_push`'s git stages (the per-tip
/// cat-file type probe + delta rev-list, and the full-scan fallback) run under a
/// scan-admission permit: with a zero-permit pool the call parks and spawns no
/// git; once a permit is available the SAME call runs (defer, not shed). On
/// ungated code the git runs regardless of the pool (RED).
#[cfg(unix)]
#[tokio::test]
async fn resolve_candidates_defers_when_scan_pool_exhausted() {
use std::os::unix::fs::PermissionsExt;
use std::sync::Arc;
use std::time::Duration;
use tokio::sync::Semaphore;
let dir = tempfile::TempDir::new().unwrap();
let marker = dir.path().join("git.ran");
// Fake git records ANY invocation; cat-file reports a commit tip so the
// delta stage proceeds, rev-list yields an empty delta.
let fake = dir.path().join("fakegit");
std::fs::write(
&fake,
format!(
"#!/bin/sh\necho ran >> \"{}\"\ncase \"$1\" in\n cat-file) echo commit ;;\n *) : ;;\nesac\nexit 0\n",
marker.display()
),
)
.unwrap();
let mut perm = std::fs::metadata(&fake).unwrap().permissions();
perm.set_mode(0o755);
std::fs::set_permissions(&fake, perm).unwrap();
let git_bin = fake.to_str().unwrap().to_string();
let tip = "deadbeefdeadbeefdeadbeefdeadbeefdeadbeef".to_string();
let sem: Arc<Semaphore> = Arc::new(Semaphore::new(0));
let blocked = tokio::time::timeout(
Duration::from_millis(500),
resolve_candidates_for_push(
sem.clone(),
dir.path().to_path_buf(),
vec![tip.clone()],
vec![],
git_bin.clone(),
Duration::from_secs(5),
false,
),
)
.await;
assert!(
blocked.is_err(),
"the pin-candidate scan must defer (park on admission) when the pool is exhausted"
);
assert!(
!marker.exists(),
"the scan's git must not spawn while its admission permit is unavailable (F4)"
);
// Release admission: the SAME scan now runs (defer, not shed).
sem.add_permits(1);
let set = resolve_candidates_for_push(
sem,
dir.path().to_path_buf(),
vec![tip],
vec![],
git_bin,
Duration::from_secs(5),
false,
)
.await;
assert!(
marker.exists(),
"once admission is available the deferred scan runs its git"
);
assert!(!set.full_scan, "commit tip + empty rev-list is a delta");
assert!(set.candidates.is_empty());
}
/// F4 fast-path negative arm: a deletion-only push (no new tips, kill-switch
/// off) computes its empty delta without spawning ANY git child, so it must
/// complete without acquiring from the scan pool — even at zero permits. The
/// per-tip cat-file probe means every push with a non-empty new tip DOES spawn
/// git, so this is the only genuinely git-free stage.
#[cfg(unix)]
#[tokio::test]
async fn resolve_candidates_no_git_fast_path_skips_admission() {
use std::os::unix::fs::PermissionsExt;
use std::sync::Arc;
use std::time::Duration;
use tokio::sync::Semaphore;
let dir = tempfile::TempDir::new().unwrap();
let marker = dir.path().join("git.ran");
let fake = dir.path().join("fakegit");
std::fs::write(
&fake,
format!("#!/bin/sh\necho ran >> \"{}\"\nexit 0\n", marker.display()),
)
.unwrap();
let mut perm = std::fs::metadata(&fake).unwrap().permissions();
perm.set_mode(0o755);
std::fs::set_permissions(&fake, perm).unwrap();
let set = tokio::time::timeout(
Duration::from_millis(500),
resolve_candidates_for_push(
Arc::new(Semaphore::new(0)),
dir.path().to_path_buf(),
vec![],
vec!["deadbeefdeadbeefdeadbeefdeadbeefdeadbeef".to_string()],
fake.to_str().unwrap().to_string(),
Duration::from_secs(5),
false,
),
)
.await
.expect("the no-new-tips fast path must not park on the scan pool");
assert_eq!(
set,
PinCandidateSet {
candidates: Vec::new(),
full_scan: false
}
);
assert!(!marker.exists(), "the fast path must spawn no git at all");
}
/// #174 F5: the coalesced-drain FullScan marker is signalled via the explicit
/// `force_full_scan` flag, and a flagged call with NO tips must still enumerate
/// the whole repo (full_scan=true, non-empty candidates). The RED arm of the
/// encoding question: if the marker were encoded as a plain empty-tips call
/// (flag off — the fast path), the drain would resolve to an empty delta and
/// pin nothing, silently losing the coalesced pushes' work.
#[tokio::test]
async fn forced_full_scan_with_no_tips_enumerates_the_repo() {
let repo = Repo::new();
repo.commit_file("a.txt", "one\n");
let all: HashSet<String> = list_all_objects(&repo.path, "git", td())
.unwrap()
.into_iter()
.collect();
assert!(!all.is_empty(), "fixture repo has objects");
let set = resolve_candidates_for_push(
std::sync::Arc::new(tokio::sync::Semaphore::new(64)),
repo.path.clone(),
vec![],
vec![],
"git".to_string(),
std::time::Duration::from_secs(600),
true,
)
.await;
assert!(set.full_scan, "the forced call is signalled as a full scan");
let got: HashSet<String> = set.candidates.into_iter().collect();
assert_eq!(
got, all,
"a forced full scan with no tips enumerates the repo — never an empty delta"
);
// The discriminator: the SAME empty-tips call without the flag is the
// deletion-only fast path (empty delta, pin nothing). The two must differ,
// or the marker encoding has collapsed into the silent-loss shape.
let unforced = resolve_candidates_for_push(
std::sync::Arc::new(tokio::sync::Semaphore::new(64)),
repo.path.clone(),
vec![],
vec![],
"git".to_string(),
std::time::Duration::from_secs(600),
false,
)
.await;
assert!(!unforced.full_scan);
assert!(unforced.candidates.is_empty());
}
#[test]
fn missing_oid_tip_forces_full_scan() {
let repo = Repo::new();
File diff suppressed because it is too large. Load diff
+88 -644
View File
@@ -24,9 +24,7 @@ use tokio::process::Command;
///
/// The `be0cdd6` path-scoped upload-pack walk already applies this discipline by
/// moving its permits into the `spawn_blocking`; this generalizes it to the plain
/// (non-path-scoped) `info_refs` / `run_git_service` spawn paths and, via the
/// hand-back contract on `drive_git_child`, across both stages of the filtered
/// (`upload_pack_excluding`) pack build (F1).
/// (non-path-scoped) `info_refs` / `run_git_service` spawn paths.
pub struct AdmissionGuard {
// Boxed so the concrete permit types stay private to the caller (repos.rs) — the
// guard only needs to hold them and drop them, never inspect them. `Send +
@@ -35,29 +33,16 @@ pub struct AdmissionGuard {
_caller: Option<Box<dyn Send + 'static>>,
// Any further work-scoped hold that must outlive the process group; see `with_hold`.
_hold: Option<Box<dyn Send + 'static>>,
// Per-repo write lease (#174 U2/F3), `Some` ONLY on the receive-pack write path
// (via [`with_lease`](Self::with_lease)); `None` on every read path and every
// non-receive-pack write path. It rides this guard into `KillGroupOnDrop`'s detached
// reaper, so on a client disconnect the lease frees only after the disconnected
// push's git group is reaped — serializing a second same-node push against the
// still-writing group. A `RepoWriteLease` clone is `Send + 'static` and holds no pg
// connection, so it travels with the reaper cleanly. A stray `Some` on a READ path
// would wrongly serialize upload-pack against pushes, hence the None-everywhere-else
// discipline.
_lease: Option<crate::state::RepoWriteLease>,
}
impl AdmissionGuard {
/// Take ownership of the global permit and an optional per-caller permit. Both are
/// erased to `Box<dyn Send>` — the guard's only job is to hold them until it drops.
/// No lease is attached here; the receive-pack write path adds one via
/// [`with_lease`](Self::with_lease), so every other call site is lease-free.
pub fn new(global: impl Send + 'static, caller: Option<impl Send + 'static>) -> Self {
Self {
_global: Some(Box::new(global)),
_caller: caller.map(|c| Box::new(c) as Box<dyn Send + 'static>),
_hold: None,
_lease: None,
}
}
@@ -75,13 +60,6 @@ impl AdmissionGuard {
self._hold = Some(Box::new(hold));
self
}
/// Attach the per-repo write lease (#174 U2/F3). Called ONLY on the receive-pack
/// write path, so the lease rides the disconnect reaper; read paths never call this.
pub fn with_lease(mut self, lease: crate::state::RepoWriteLease) -> Self {
self._lease = Some(lease);
self
}
}
/// Handle `GET /:owner/:repo/info/refs?service=git-upload-pack`
@@ -106,16 +84,14 @@ pub async fn info_refs(
let mut command = Command::new(git_bin);
command
.arg(service_to_command(service)?)
.arg(service_to_command(service))
.arg("--stateless-rpc")
.arg("--advertise-refs")
.arg(repo_path);
// No request body: advertise-refs does not read stdin.
let (stdout, admission) =
// No request body — advertise-refs does not read stdin. Single stage: the returned
// guard is dropped here (advertise-refs holds admission for its own child only).
let (stdout, _admission) =
drive_git_child(command, Bytes::new(), timeout, "advertise-refs", admission).await?;
// Single-stage op: the advertisement's group is reaped by now; release admission
// rather than holding it across the pure response framing below.
drop(admission);
let content_type = format!("application/x-{service}-advertisement");
@@ -139,18 +115,13 @@ pub async fn info_refs(
///
/// Serves pack data for a clone or fetch. This is stateless — the entire
/// negotiation happens in a single request/response.
///
/// Returns `(response, served_pack)` where `served_pack` is whether the
/// git-upload-pack-result actually delivered a packfile (vs a negotiation-only
/// ACK/NAK round). The handler counts a completed fetch from this outcome rather
/// than from the request's `done` flag (#192 F1).
pub async fn upload_pack(
git_bin: &str,
repo_path: &Path,
request_body: Bytes,
timeout: Duration,
admission: Option<AdmissionGuard>,
) -> Result<(Response, bool)> {
) -> Result<Response> {
let output = run_git_service(
git_bin,
"git-upload-pack",
@@ -161,59 +132,11 @@ pub async fn upload_pack(
)
.await?;
let served_pack = response_served_pack(&output);
let resp = Response::builder()
Ok(Response::builder()
.status(StatusCode::OK)
.header("Content-Type", "application/x-git-upload-pack-result")
.header("Cache-Control", "no-cache")
.body(Body::from(output))?;
Ok((resp, served_pack))
}
/// True when a `git-upload-pack-result` stream actually delivered a packfile,
/// as opposed to a negotiation-only response (ACK/NAK with no pack).
///
/// Lets the fetch-completion metric key off the response outcome instead of the
/// request's `done` flag (#192 F1): a `no-done` fetch that the server answers
/// with `ACK <oid> ready` + pack is a completion; a flush-terminated negotiation
/// round (the server replies only ACK/NAK, or nothing) is not.
///
/// Handles both framings the serve paths emit:
/// - side-band-64k: the pack rides in band-1 (`0x01`) pkt-lines; the first such
/// chunk begins with the `PACK` magic (band-2 progress lines can precede it).
/// - non-sideband: the raw packfile follows the ACK/NAK pkt-lines and begins with
/// the `PACK` magic (which is not valid pkt-line hex, so parsing falls through
/// to the raw-stream check).
///
/// Fails closed (returns false) on a truncated or malformed stream.
pub fn response_served_pack(output: &[u8]) -> bool {
let mut i = 0;
while i + 4 <= output.len() {
let Ok(hdr) = std::str::from_utf8(&output[i..i + 4]) else {
// Not a pkt-line header: the raw (non-sideband) pack stream has begun.
return output[i..].starts_with(b"PACK");
};
let Ok(len) = usize::from_str_radix(hdr, 16) else {
// `PACK` (and any other non-hex 4 bytes) lands here: the raw pack
// follows the NAK pkt-line in the non-sideband framing.
return output[i..].starts_with(b"PACK");
};
// 0000/0001/0002 are flush/delim/response-end markers, no payload.
if len < 4 {
i += 4;
continue;
}
if i + len > output.len() {
return false; // truncated
}
let payload = &output[i + 4..i + len];
// side-band-64k: band 1 carries pack data; its first chunk is the PACK magic.
if payload.first() == Some(&0x01) && payload[1..].starts_with(b"PACK") {
return true;
}
i += len;
}
false
.body(Body::from(output))?)
}
/// Handle `POST /:owner/:repo/git-receive-pack`
@@ -270,12 +193,11 @@ struct KillGroupOnDrop {
// own SIGCHLD-driven orphan reaper.
child: Option<tokio::process::Child>,
pgid: Option<i32>,
// The global + per-source admission permits for this op, if any. repos.rs moves
// its permits in here (directly on the plain spawn paths, via the stage threading
// on the filtered rev-list/pack-objects ones) so admission is released only when
// the group is confirmed reaped, on every exit: complete, timeout, or
// client-disconnect (#174 P1-a, F1). `None` when the op carries no admission
// (e.g. the smart_http test harness).
// The global + per-source admission permits for this op, if any. On the plain
// (non-path-scoped) spawn paths repos.rs moves its permits in here so admission is
// released only when the group is confirmed reaped, on every exit — complete,
// timeout, or client-disconnect (#174 P1-a). The rev-list/pack-objects
// visibility-walk callers hold no admission and pass `None`.
admission: Option<AdmissionGuard>,
}
@@ -288,10 +210,9 @@ impl KillGroupOnDrop {
/// Disarm on the success/timeout path: the body has already reaped the child (its
/// `wait()` returned, or `reap_group_on_timeout` ran), so drop the handle and clear
/// the pgid, leaving the guard's Drop a no-op. Returns the admission guard: the
/// group is already reaped on both callers of this, so the success path hands it
/// back to drive_git_child's caller (a multi-stage build carries it to the next
/// stage) and the timeout path drops it there and then.
/// the pgid, leaving the guard's Drop a no-op. Returns the admission guard so the
/// caller drops it at the earliest provably-free point (the group is already reaped
/// on both callers of this) rather than at end-of-bookkeeping.
fn disarm(&mut self) -> Option<AdmissionGuard> {
self.pgid = None;
self.child = None;
@@ -426,32 +347,30 @@ async fn run_git_service(
) -> Result<Vec<u8>> {
let mut command = Command::new(git_bin);
command
.arg(service_to_command(service)?)
.arg(service_to_command(service))
.arg("--stateless-rpc")
.arg(repo_path);
let (out, admission) = drive_git_child(command, input, timeout, service, admission).await?;
// Single-stage op: the child group is already reaped when drive_git_child
// returns, so releasing admission here keeps the permits held exactly for the
// process lifetime, no longer.
drop(admission);
// Single stage: drop the returned guard when this function returns (its permits are
// released once the child's group is reaped, per drive_git_child).
let (out, _admission) = drive_git_child(command, input, timeout, service, admission).await?;
Ok(out)
}
/// Drive a spawned git child under `timeout` with process-group teardown, returning
/// its stdout together with the admission guard. Shared core for
/// [`run_git_service`], [`info_refs`], and the filtered-pack stages: the caller
/// passes a `Command` with its args set; this adds piped stdio and `process_group(0)`.
/// On the deadline the whole group is torn down and reaped before returning
/// [`GitServiceTimeout`]; on a dropped future (client disconnect) the
/// [`KillGroupOnDrop`] guard fires. `input` is written to the child's stdin (empty
/// for the advertise-refs path, which has no request body); `what` labels errors.
/// its stdout AND the disarmed admission guard on success. Shared core for
/// [`run_git_service`] and [`info_refs`]: the caller passes a `Command` with its args
/// set; this adds piped stdio and `process_group(0)`. On the deadline the whole group
/// is torn down and reaped before returning [`GitServiceTimeout`]; on a dropped future
/// (client disconnect) the [`KillGroupOnDrop`] guard fires. `input` is written to the
/// child's stdin (empty for the advertise-refs path, which has no request body); `what`
/// labels errors.
///
/// Admission contract: on success the guard is handed BACK so a multi-stage caller
/// (the filtered-pack build) can carry one admission across consecutive children
/// instead of releasing it between stages. On every Err return (timeout, spawn
/// failure, interaction error, non-zero exit) the guard is dropped internally, and
/// only after the child is reaped: an Err cannot carry the guard, and by then the
/// group is confirmed gone, so releasing admission there is correct.
/// On success the guard is RETURNED rather than dropped internally (#174 KTD3), so a
/// caller running two sequential git stages under one admission — `build_filtered_pack`'s
/// rev-list then pack-objects — can hand the same guard from the first stage to the
/// second and keep the permits held across both, releasing them only when the second
/// stage's process group is reaped. Callers that run a single stage just let the
/// returned guard drop.
async fn drive_git_child(
mut command: Command,
input: Bytes,
@@ -493,10 +412,10 @@ async fn drive_git_child(
pgid,
admission,
};
// On non-unix there is no process-group teardown; the `admission` parameter
// itself stays live for the child's whole interaction (nothing moves it until
// the match below), so the success path hands it back like the unix path and
// the timeout path drops it after the child is waited on.
// On non-unix there is no process-group teardown, so hold the admission guard here
// for the child's whole interaction; it is returned to the caller below.
#[cfg(not(unix))]
let admission_holder = admission;
let mut out = Vec::new();
let mut err = Vec::new();
@@ -530,18 +449,25 @@ async fn drive_git_child(
};
let timed = tokio::time::timeout(timeout, interact).await;
let (write_result, status, admission) = match timed {
// The disarmed admission guard, handed back to the caller on success so it can ride
// a following sequential git stage (#174 KTD3) instead of releasing between stages.
let admission_out: Option<AdmissionGuard>;
let (write_result, status) = match timed {
Ok(result) => {
// The join runs all arms to completion, so the child is reaped: disarm
// before surfacing any interaction error (a read/wait error), else the
// guard's drop would reap an already-reaped child / signal a reused pgid.
// Disarming hands the admission guard back; it rides to the Ok return
// below, and every error return between here and there drops it, always
// AFTER the reap, the earliest provably-free point.
// The guard is returned (not dropped) so a two-stage caller keeps the
// permits held across both stages; a single-stage caller drops it on return.
#[cfg(unix)]
let admission = group_guard.disarm();
let (write_result, status) = result?;
(write_result, status, admission)
{
admission_out = group_guard.disarm();
}
#[cfg(not(unix))]
{
admission_out = admission_holder;
}
result?
}
Err(_elapsed) => {
// Timeout: tear the whole group down and reap it before returning so a
@@ -557,7 +483,6 @@ async fn drive_git_child(
{
let _ = child.start_kill();
let _ = tokio::time::timeout(Duration::from_secs(2), child.wait()).await;
drop(admission);
}
return Err(GitServiceTimeout.into());
}
@@ -566,8 +491,7 @@ async fn drive_git_child(
// Surface git's own failure (its stderr, which the handler may classify as a
// 400) before any stdin-write error: when git rejects a malformed body it
// exits non-zero and closes stdin, so the write's EPIPE would otherwise mask
// the real cause. `admission` drops on these error returns; the child is
// already reaped by the completed join above.
// the real cause.
if !status.success() {
let stderr = String::from_utf8_lossy(&err);
bail!("{what} failed: {stderr}");
@@ -575,16 +499,14 @@ async fn drive_git_child(
write_result.context("failed to write to git stdin")?;
Ok((out, admission))
Ok((out, admission_out))
}
/// Map a validated service name to its git subcommand. Only the two fixed
/// subcommands can come out, so an unrecognized service can never reach git's argv.
fn service_to_command(service: &str) -> Result<&'static str> {
fn service_to_command(service: &str) -> &str {
match service {
"git-upload-pack" => Ok("upload-pack"),
"git-receive-pack" => Ok("receive-pack"),
other => bail!("unknown git service: {other}"),
"git-upload-pack" => "upload-pack",
"git-receive-pack" => "receive-pack",
_ => service,
}
}
@@ -621,9 +543,10 @@ async fn rev_list_keep(
command
.args(["rev-list", "--objects", "--all"])
.current_dir(repo_path);
// The filtered-serve caller's admission rides through drive_git_child so a
// disconnect mid-enumeration keeps the permits held until the rev-list group is
// reaped; on success the guard comes back for the pack-objects stage (F1).
// First of `build_filtered_pack`'s two sequential stages: it holds the caller's
// admission for the rev-list child and hands the disarmed guard back so the same
// permits ride the pack-objects stage (#174 KTD3, R2). The inter-stage window holds
// no live git group, so nothing is reaped-late by carrying the guard between them.
let (stdout, admission) =
drive_git_child(command, Bytes::new(), timeout, "rev-list", admission).await?;
let mut keep = Vec::new();
@@ -647,24 +570,20 @@ async fn rev_list_keep(
/// disconnect. An outer `tokio::time::timeout` around a `spawn_blocking` cannot
/// cancel the blocking thread, so neither stage may live off the async side
/// (#174, KTD5).
///
/// `admission` carries the filtered-serve permits across BOTH stages: stage 1 hands
/// it back on success, stage 2 takes it, and the caller receives it after stage 2 so
/// release happens only once the last child group is reaped. On a disconnect the
/// guard rides the active stage's detached reaper instead of dropping with the
/// future (F1). A stage that fails or times out (including a stage 2 whose
/// remaining budget has saturated to zero) drops the guard inside drive_git_child,
/// after that stage's reap.
pub async fn build_filtered_pack(
git_bin: &str,
repo_path: &Path,
withheld: &HashSet<String>,
timeout: Duration,
admission: Option<AdmissionGuard>,
) -> Result<(Vec<u8>, Option<AdmissionGuard>)> {
) -> Result<Vec<u8>> {
// One deadline spans both git stages so a slow rev-list eats into the pack
// budget rather than granting each stage a fresh `timeout` (2x the permit hold).
let deadline = Instant::now() + timeout;
// The caller's admission rides both stages: rev-list holds it, hands it back, then
// pack-objects holds it until ITS process group is reaped on disconnect. That closes
// the path-scoped cap bypass — permits release on group reap, not the instant the
// request future drops (#174 R2, KTD3). The plain upload-pack path already does this.
let (keep, admission) = rev_list_keep(
git_bin,
repo_path,
@@ -679,7 +598,7 @@ pub async fn build_filtered_pack(
command
.args(["pack-objects", "--stdout"])
.current_dir(repo_path);
let (out, admission) = drive_git_child(
let (out, _admission) = drive_git_child(
command,
Bytes::from(data),
deadline.saturating_duration_since(Instant::now()),
@@ -687,7 +606,7 @@ pub async fn build_filtered_pack(
admission,
)
.await?;
Ok((out, admission))
Ok(out)
}
/// Serve a clone/fetch with the withheld blobs removed from the response pack.
@@ -714,24 +633,19 @@ pub async fn build_filtered_pack(
/// instead of a thin delta. Honoring negotiation for smaller fetch packs is an
/// optimization follow-up, not a correctness requirement.
pub async fn upload_pack_excluding(
git_bin: &str,
repo_path: &Path,
request_body: Bytes,
withheld: &HashSet<String>,
timeout: Duration,
admission: Option<AdmissionGuard>,
) -> Result<(Response, bool)> {
// Both filtered-pack stages run async under drive_git_child (duration-bounded,
// process group reaped on disconnect, #174), and `admission` threads through
// them so the caller's permits release only after the active stage's group is
// reaped, never the instant a disconnect drops this future (F1). `git_bin` is
// injectable for the same fake-git testing reason as `run_git_service`
// (production passes the configured git binary).
let (pack, admission) =
build_filtered_pack(git_bin, repo_path, withheld, timeout, admission).await?;
// Both git stages are done and their groups reaped; release admission before the
// pure in-memory response framing below.
drop(admission);
) -> Result<Response> {
// Both git stages are duration-bounded and their process groups are reaped on
// disconnect via drive_git_child (#174), so a hung build no longer pins its
// concurrency slot and a client disconnect no longer orphans the git child. The
// caller's read + per-caller admission is threaded through both stages so the
// permits are held until the pack-objects group is reaped, matching the plain path
// (#174 R2).
let pack = build_filtered_pack("git", repo_path, withheld, timeout, admission).await?;
// The client lists its capabilities on the first `want` line. Honor
// side-band-64k when offered (every modern smart-HTTP client offers it);
@@ -755,19 +669,11 @@ pub async fn upload_pack_excluding(
body.extend_from_slice(&pack);
}
// The filtered path always builds and serves a self-contained pack, so this
// is always true; computing it (rather than hardcoding) keeps the signal
// honest if the framing ever changes. The handler does NOT count off this
// flag alone — a filtered response can't distinguish an accepted fresh clone
// from a rejected mid-negotiation one, so the count is gated on `done` too
// (#192 F2, see `should_count_fetch`).
let served_pack = response_served_pack(&body);
let resp = Response::builder()
Ok(Response::builder()
.status(StatusCode::OK)
.header("Content-Type", "application/x-git-upload-pack-result")
.header("Cache-Control", "no-cache")
.body(Body::from(body))?;
Ok((resp, served_pack))
.body(Body::from(body))?)
}
/// True if `needle` occurs anywhere in `haystack`. Small substring scan used to
@@ -781,111 +687,12 @@ fn memmem(haystack: &[u8], needle: &[u8]) -> bool {
.any(|window| window == needle)
}
/// Build real `git upload-pack --stateless-rpc` result fixtures for tests:
/// `(pack_bearing, negotiation_only)`. The first is a completing clone request
/// (`want` + flush + `done`) which git answers with `NAK` + a packfile; the
/// second is a negotiation round (`want` + flush + an unknown `have` + flush, no
/// `done`) which git answers with `NAK` and no pack. Used to exercise
/// [`response_served_pack`] and the completion-count rule against real protocol
/// output rather than a hand-rolled approximation.
#[cfg(test)]
pub(crate) fn upload_pack_result_fixtures() -> (Vec<u8>, Vec<u8>) {
use std::io::Write as _;
let dir = tempfile::TempDir::new().unwrap();
let work = dir.path().join("work");
let bare = dir.path().join("bare.git");
std::fs::create_dir_all(work.join("sub")).unwrap();
std::fs::write(work.join("a.txt"), b"hello\n").unwrap();
std::fs::write(work.join("sub/b.txt"), b"world\n").unwrap();
let g = |args: &[&str], d: &Path| {
assert!(std::process::Command::new("git")
.args(args)
.current_dir(d)
.status()
.unwrap()
.success());
};
g(&["init", "-q"], &work);
g(&["config", "user.email", "t@t"], &work);
g(&["config", "user.name", "t"], &work);
g(&["add", "."], &work);
g(&["commit", "-qm", "init"], &work);
g(
&[
"clone",
"-q",
"--bare",
work.to_str().unwrap(),
bare.to_str().unwrap(),
],
dir.path(),
);
let sha = {
let o = std::process::Command::new("git")
.args(["rev-parse", "HEAD"])
.current_dir(&work)
.output()
.unwrap();
String::from_utf8_lossy(&o.stdout).trim().to_string()
};
let caps = "multi_ack_detailed side-band-64k thin-pack ofs-delta agent=git/test";
let run = |req: &[u8]| -> Vec<u8> {
let mut child = std::process::Command::new("git")
.args(["upload-pack", "--stateless-rpc"])
.arg(&bare)
.stdin(Stdio::piped())
.stdout(Stdio::piped())
.stderr(Stdio::piped())
.spawn()
.unwrap();
child.stdin.take().unwrap().write_all(req).unwrap();
let out = child.wait_with_output().unwrap();
assert!(
out.status.success(),
"upload-pack failed: {}",
String::from_utf8_lossy(&out.stderr)
);
out.stdout
};
// Completing: want + flush + done -> NAK + pack.
let mut completing = pkt_line(&format!("want {sha} {caps}\n"));
completing.extend_from_slice(b"0000");
completing.extend_from_slice(&pkt_line("done\n"));
let pack_bearing = run(&completing);
// Negotiation-only: want + flush + an unknown have + flush, no done -> NAK,
// no pack.
let mut nego = pkt_line(&format!("want {sha} {caps}\n"));
nego.extend_from_slice(b"0000");
nego.extend_from_slice(&pkt_line("have 0000000000000000000000000000000000000000\n"));
nego.extend_from_slice(b"0000");
let negotiation_only = run(&nego);
(pack_bearing, negotiation_only)
}
#[cfg(test)]
mod tests {
use super::*;
use std::process::Command;
use tempfile::TempDir;
#[test]
fn service_to_command_maps_allowlist_and_rejects_passthrough() {
assert_eq!(
service_to_command("git-upload-pack").unwrap(),
"upload-pack"
);
assert_eq!(
service_to_command("git-receive-pack").unwrap(),
"receive-pack"
);
for bad in ["git-fetch-pack", "--version", "upload-pack", ""] {
assert!(
service_to_command(bad).is_err(),
"must reject passthrough for {bad:?}"
);
}
}
/// List OIDs in a pack by writing it to a temp dir and running verify-pack.
pub(super) fn pack_object_ids(pack: &[u8]) -> std::collections::HashSet<String> {
let dir = TempDir::new().unwrap();
@@ -957,10 +764,9 @@ mod tests {
let mut withheld = std::collections::HashSet::new();
withheld.insert(secret.clone());
let (pack, _admission) =
build_filtered_pack("git", &bare, &withheld, Duration::from_secs(30), None)
.await
.unwrap();
let pack = build_filtered_pack("git", &bare, &withheld, Duration::from_secs(30), None)
.await
.unwrap();
let ids = pack_object_ids(&pack);
assert!(ids.contains(&public), "public blob must be in the pack");
assert!(
@@ -1022,14 +828,9 @@ mod tests {
b"0098want 0000000000000000000000000000000000000000 \
side-band-64k ofs-delta agent=git/2\n00000009done\n",
);
let (resp, served_pack) =
upload_pack_excluding("git", &bare, req, &withheld, Duration::from_secs(30), None)
.await
.unwrap();
assert!(
served_pack,
"the filtered serve path always delivers a pack"
);
let resp = upload_pack_excluding(&bare, req, &withheld, Duration::from_secs(30), None)
.await
.unwrap();
let body = to_bytes(resp.into_body(), usize::MAX).await.unwrap();
let ids = pack_object_ids(&extract_pack(&body));
assert!(
@@ -1088,17 +889,9 @@ mod tests {
axum::extract::State(st): axum::extract::State<std::sync::Arc<FilterState>>,
body: Bytes,
) -> Response {
upload_pack_excluding(
"git",
&st.repo,
body,
&st.withheld,
Duration::from_secs(30),
None,
)
.await
.unwrap()
.0
upload_pack_excluding(&st.repo, body, &st.withheld, Duration::from_secs(30), None)
.await
.unwrap()
}
/// Spawn the server for `bare`, withholding `withheld`. Returns the clone URL
@@ -1625,12 +1418,7 @@ mod tests {
"build_filtered_pack must return within the watchdog — the git stage \
must be timeout-bounded, not an uncancellable spawn_blocking",
);
// `match` rather than `expect_err`: the Ok arm carries an
// AdmissionGuard, which has no Debug impl for expect_err to print.
let err = match result {
Ok(_) => panic!("a hung git stage must return an error, not hang"),
Err(e) => e,
};
let err = result.expect_err("a hung git stage must return an error, not hang");
if is_transient_exec_race(&err) {
// Fresh-fake-git ETXTBSY: back off (growing) so a bursty fork-pressure
// spike subsides before retrying, per fake_git_run_with_pids.
@@ -2056,296 +1844,6 @@ mod tests {
);
}
// ── F1: filtered-serve admission threaded through both pack stages ──────
//
// The handler-level disconnect regression lives in api/repos.rs
// (upload_pack_filtered_permit_held_through_group_reap_after_disconnect); these
// exercise the guard contract at the smart_http seam: held through a
// mid-rev-list disconnect, handed back on success, and recovered (post-reap)
// on every error return.
/// F1: a client disconnect mid-REV-LIST (stage 1 of the filtered build) must
/// keep the threaded admission held until the rev-list group is reaped, and
/// stage 2 (pack-objects) must never spawn, since the dropped future cannot
/// advance to it.
#[cfg(unix)]
#[tokio::test]
async fn build_filtered_pack_holds_admission_through_rev_list_reap_on_disconnect() {
let tmp = tempfile::TempDir::new().unwrap();
let descfile = tmp.path().join("desc.pid");
let packfile = tmp.path().join("pack.ran");
// rev-list: SIGTERM-trapping descendant records its pid and loops (bounded
// so a broken fix leaks no permanent orphan); the leader waits, keeping the
// interaction pending. pack-objects: records that it ran (it must never run).
let body = format!(
"#!/bin/sh\n\
case \"$1\" in\n\
rev-list)\n\
sh -c 'trap \"\" TERM; echo $$ > \"{desc}\"; i=0; while [ $i -lt 20 ]; do sleep 1; i=$((i+1)); done' &\n\
wait ;;\n\
pack-objects) : > \"{pack}\" ;;\n\
*) : ;;\n\
esac\n\
exit 0\n",
desc = descfile.display(),
pack = packfile.display()
);
let git_bin = write_fake_git(tmp.path(), &body);
let sem = std::sync::Arc::new(tokio::sync::Semaphore::new(1));
let guard = AdmissionGuard::new(sem.clone().try_acquire_owned().unwrap(), None::<()>);
let withheld = HashSet::new();
let mut fut = Box::pin(build_filtered_pack(
git_bin.to_str().unwrap(),
tmp.path(),
&withheld,
Duration::from_secs(60),
Some(guard),
));
// Drive until the rev-list descendant records its pid.
let mut desc: Option<i32> = None;
for _ in 0..500 {
let _ = tokio::time::timeout(Duration::from_millis(10), &mut fut).await;
if let Some(p) = std::fs::read_to_string(&descfile)
.ok()
.and_then(|s| s.trim().parse::<i32>().ok())
{
desc = Some(p);
break;
}
}
let desc = desc.expect("the fake rev-list must have spawned its descendant");
let _cleanup = ReapOnPanic(vec![desc]);
assert_eq!(
sem.available_permits(),
0,
"admission held while rev-list runs"
);
// Client disconnect mid-enumeration.
drop(fut);
// Load-bearing: the permit must stay held while the SIGTERM-ignoring group
// member is still alive (check before the reaper's ~2s SIGKILL escalation).
tokio::time::sleep(Duration::from_millis(100)).await;
assert!(
alive(desc),
"the SIGTERM-ignoring descendant must still be alive during the hold window"
);
assert_eq!(
sem.available_permits(),
0,
"on disconnect mid-rev-list the admission must be held until the group is \
reaped, not released the instant the future drops (F1)"
);
let mut freed = false;
for _ in 0..400 {
if sem.available_permits() == 1 {
freed = true;
break;
}
tokio::time::sleep(Duration::from_millis(50)).await;
}
assert!(
freed,
"the reaper must release admission once the rev-list group is gone"
);
assert!(
!packfile.exists(),
"pack-objects must never spawn after a disconnect mid-rev-list"
);
}
/// F1 success path: a filtered serve that completes hands the guard back
/// through both stages and releases admission when the serve returns. No leak,
/// no double-release.
#[tokio::test]
async fn upload_pack_excluding_releases_admission_after_success() {
let td = TempDir::new().unwrap();
let (_work, bare, secret_oid, _public_oid) = fixture_with_secret(&td);
let sem = std::sync::Arc::new(tokio::sync::Semaphore::new(1));
let guard = AdmissionGuard::new(sem.clone().try_acquire_owned().unwrap(), None::<()>);
let withheld = HashSet::from([secret_oid]);
let (resp, _served_pack) = upload_pack_excluding(
"git",
&bare,
Bytes::from_static(b"0000"),
&withheld,
Duration::from_secs(30),
Some(guard),
)
.await
.unwrap();
assert_eq!(resp.status(), StatusCode::OK);
assert_eq!(
sem.available_permits(),
1,
"admission must be released once the filtered serve returns"
);
}
/// F1 error path: a rev-list that exits non-zero fails the build AND recovers
/// the threaded admission: drive_git_child drops the guard only after the
/// completed join has reaped the child, and the error return must not leak it.
#[cfg(unix)]
#[tokio::test]
async fn build_filtered_pack_releases_admission_after_rev_list_failure() {
let tmp = tempfile::TempDir::new().unwrap();
let body =
"#!/bin/sh\ncase \"$1\" in\n rev-list) echo boom >&2; exit 2 ;;\n *) : ;;\nesac\nexit 0\n";
let git_bin = write_fake_git(tmp.path(), body);
let withheld = HashSet::new();
let sem = std::sync::Arc::new(tokio::sync::Semaphore::new(1));
for i in 0..FAKE_GIT_RETRY_ATTEMPTS {
let guard = AdmissionGuard::new(sem.clone().try_acquire_owned().unwrap(), None::<()>);
let result = build_filtered_pack(
git_bin.to_str().unwrap(),
tmp.path(),
&withheld,
Duration::from_secs(30),
Some(guard),
)
.await;
let err = match result {
Ok(_) => panic!("a non-zero rev-list exit must fail the build"),
Err(e) => e,
};
// Load-bearing on EVERY attempt: even a transient spawn-failure return
// must recover the permit (the guard drops before the Err surfaces).
assert_eq!(
sem.available_permits(),
1,
"the admission permit must be recovered on the error return, not leaked"
);
if is_transient_exec_race(&err) {
tokio::time::sleep(Duration::from_millis(FAKE_GIT_BACKOFF_STEP_MS * (i + 1))).await;
continue;
}
assert!(
err.to_string().contains("rev-list failed"),
"the failure must surface rev-list's own error, got: {err}"
);
return;
}
panic!(
"fake git kept hitting ETXTBSY after {FAKE_GIT_RETRY_ATTEMPTS} attempts \
(persistent exec failure, not a transient parallel-runner miss)"
);
}
/// F1: a pack-objects stage that times out (stage 2 of the filtered build,
/// guard threaded through) recovers the admission after the reap: the timeout
/// Err cannot carry the guard, so drive_git_child must drop it post-reap.
#[cfg(unix)]
#[tokio::test]
async fn build_filtered_pack_recovers_admission_after_pack_objects_timeout() {
let tmp = tempfile::TempDir::new().unwrap();
// rev-list returns one oid fast (guard handed back); pack-objects hangs.
let body = "#!/bin/sh\ncase \"$1\" in\n rev-list) echo deadbeefdeadbeefdeadbeefdeadbeefdeadbeef ;;\n pack-objects) sleep 300 ;;\n *) exit 1 ;;\nesac\n";
let git_bin = write_fake_git(tmp.path(), body);
let withheld = HashSet::new();
let sem = std::sync::Arc::new(tokio::sync::Semaphore::new(1));
for i in 0..FAKE_GIT_RETRY_ATTEMPTS {
let guard = AdmissionGuard::new(sem.clone().try_acquire_owned().unwrap(), None::<()>);
let result = tokio::time::timeout(
Duration::from_secs(10),
build_filtered_pack(
git_bin.to_str().unwrap(),
tmp.path(),
&withheld,
Duration::from_millis(200),
Some(guard),
),
)
.await
.expect(
"build_filtered_pack must return within the watchdog; the git stage \
must be timeout-bounded",
);
let err = match result {
Ok(_) => panic!("a hung pack-objects must fail the build"),
Err(e) => e,
};
assert_eq!(
sem.available_permits(),
1,
"admission must be recovered after the stage-2 timeout reap, not leaked"
);
if is_transient_exec_race(&err) {
tokio::time::sleep(Duration::from_millis(FAKE_GIT_BACKOFF_STEP_MS * (i + 1))).await;
continue;
}
assert!(
err.downcast_ref::<GitServiceTimeout>().is_some(),
"a hung pack-objects must abort with GitServiceTimeout, got: {err}"
);
return;
}
panic!(
"fake git kept hitting ETXTBSY after {FAKE_GIT_RETRY_ATTEMPTS} attempts \
(persistent exec failure, not a transient parallel-runner miss)"
);
}
/// F1 zero-budget contract: build_filtered_pack hands pack-objects
/// `deadline.saturating_duration_since(now)`, which saturates to ZERO once
/// rev-list has consumed the whole budget. A zero-budget stage must still route
/// the guard through the timeout's reap-then-drop rather than leak it: the
/// child spawns, the deadline fires immediately, the group is reaped, and only
/// then does the guard drop.
#[cfg(unix)]
#[tokio::test]
async fn drive_git_child_zero_budget_drops_admission_after_reap() {
let tmp = tempfile::TempDir::new().unwrap();
let git_bin = write_fake_git(tmp.path(), "#!/bin/sh\nsleep 300\n");
let sem = std::sync::Arc::new(tokio::sync::Semaphore::new(1));
for i in 0..FAKE_GIT_RETRY_ATTEMPTS {
let guard = AdmissionGuard::new(sem.clone().try_acquire_owned().unwrap(), None::<()>);
let mut command = tokio::process::Command::new(git_bin.to_str().unwrap());
command
.args(["pack-objects", "--stdout"])
.current_dir(tmp.path());
let result = tokio::time::timeout(
Duration::from_secs(10),
drive_git_child(
command,
Bytes::new(),
Duration::ZERO,
"pack-objects",
Some(guard),
),
)
.await
.expect("a zero-budget stage must abort via its own deadline, not hang");
let err = match result {
Ok(_) => panic!("a zero-budget stage must return an error"),
Err(e) => e,
};
assert_eq!(
sem.available_permits(),
1,
"the guard must drop after the zero-budget timeout reap, not leak"
);
if is_transient_exec_race(&err) {
tokio::time::sleep(Duration::from_millis(FAKE_GIT_BACKOFF_STEP_MS * (i + 1))).await;
continue;
}
assert!(
err.downcast_ref::<GitServiceTimeout>().is_some(),
"a zero-budget stage must abort with GitServiceTimeout, got: {err}"
);
return;
}
panic!(
"fake git kept hitting ETXTBSY after {FAKE_GIT_RETRY_ATTEMPTS} attempts \
(persistent exec failure, not a transient parallel-runner miss)"
);
}
// A request that runs to completion must DISARM the guard after reaping, so
// no stray group SIGTERM fires. The fake exits non-zero (surfacing as Err)
// but leaves a grandchild alive; the grandchild must survive. Goes RED if the
@@ -2640,58 +2138,4 @@ mod tests {
stdin-write EPIPE (a generic 500); got: {msg}"
);
}
// ── #192 F1: response_served_pack detector (real git output) ────────────
// A completing request (want+flush+done) yields NAK + a real packfile;
// response_served_pack must detect the delivered pack. A negotiation-only
// round (want+flush+have+flush, no done) yields NAK with no pack and must
// detect false. Fixtures are captured from real `git upload-pack
// --stateless-rpc`, so the detector is validated against protocol output, not
// a hand-rolled approximation.
#[test]
fn response_served_pack_detects_real_pack_vs_negotiation_only() {
let (pack_bearing, negotiation_only) = upload_pack_result_fixtures();
// Sanity-check the fixtures are the shapes we think (a served pack carries
// the PACK magic; a NAK-only round does not).
assert!(
pack_bearing.windows(4).any(|w| w == b"PACK"),
"the completing fixture must carry a packfile"
);
assert!(
!negotiation_only.windows(4).any(|w| w == b"PACK"),
"the negotiation-only fixture must carry no packfile"
);
assert!(
response_served_pack(&pack_bearing),
"a real NAK + pack response must be detected as a served pack"
);
assert!(
!response_served_pack(&negotiation_only),
"a real ACK/NAK-only negotiation round must be detected as no pack"
);
// An empty response (git produced nothing for a want+flush with no done)
// is a negotiation round, not a completion.
assert!(!response_served_pack(b""));
}
// The non-sideband framing (raw pack after the NAK pkt-line) must also detect
// true: the raw PACK magic is not valid pkt-line hex, so parsing falls through
// to the raw-stream check. Built by hand to pin that branch independent of the
// client's advertised capabilities.
#[test]
fn response_served_pack_detects_non_sideband_raw_pack() {
let mut raw = pkt_line("NAK\n");
raw.extend_from_slice(b"PACK\x00\x00\x00\x02\x00\x00\x00\x00");
assert!(
response_served_pack(&raw),
"a raw (non-sideband) pack after NAK must be detected"
);
// NAK with no following pack is a negotiation round.
let nak_only = pkt_line("NAK\n");
assert!(!response_served_pack(&nak_only));
}
}
File diff suppressed because it is too large. Load diff
+55 -774
View File
@@ -5,121 +5,17 @@
use std::collections::HashMap;
use std::path::{Path, PathBuf};
use std::sync::atomic::AtomicBool;
use std::sync::{Arc, Mutex, OnceLock};
use anyhow::{Context, Result};
use aws_sdk_s3::Client as S3Client;
use tracing::{debug, info, warn};
// The publication vocabulary lives in `git::publish`, which carries no
// object-storage types, and is re-exported here so existing `git::tigris::…`
// imports keep resolving. #79 deletes this file; `git::publish` is what survives
// the swap, and the only backend-aware code below is `classify_dispatch`.
pub use super::publish::{
DispatchKnowledge, PublishAttemptId, PublishStage, PublishStageCell, StoredGeneration,
UploadError, UploadPrecondition, UploadReceipt, ATTEMPT_METADATA_KEY,
};
/// What happened to a conditional, attempt-guarded delete.
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub enum AttemptDelete {
/// The object was this attempt's and is gone.
Deleted,
/// Nothing is stored under the key.
Absent,
/// Something is stored, and it is NOT this attempt's work. Left alone.
NotOurs,
}
/// THE ONLY BACKEND-AWARE CLASSIFIER in the publish path.
///
/// It answers one question: does this failure PROVE the request never committed?
/// Anything short of proof is [`DispatchKnowledge::MaybeSent`], because the
/// caller's next move on a definite failure is destructive (delete the object,
/// drop the only local clone, invalidate the cache) and being wrong there
/// destroys published state.
///
/// - An HTTP status means the server answered. A 4xx is a refusal it took
/// BEFORE storing anything, so it proves non-publication. A 5xx does not: the
/// store can commit and then fail to report it.
/// - No status at all means no response was read. Only a construction failure
/// proves the request never left; Smithy's timeout, dispatch and response
/// errors all allow that the bytes arrived and committed while the answer was
/// lost, corrupted, or never parsed.
/// - `SdkError` is `#[non_exhaustive]`, so the fallback arm must be the CAUTIOUS
/// one. A future variant defaulting to "definitely failed" would silently
/// re-open exactly the class this closes.
///
/// #79 replaces `tigris.rs` with a `BlobStore` layer; this function is the piece
/// each backend reimplements, and nothing above it changes.
fn classify_put_failure<E, R>(
err: &aws_sdk_s3::error::SdkError<E, R>,
status: Option<u16>,
) -> DispatchKnowledge {
if let Some(status) = status {
return if (400..500).contains(&status) {
DispatchKnowledge::NeverSent
} else {
DispatchKnowledge::MaybeSent
};
}
match err {
aws_sdk_s3::error::SdkError::ConstructionFailure(_) => DispatchKnowledge::NeverSent,
_ => DispatchKnowledge::MaybeSent,
}
}
use tracing::{debug, info};
/// Wrapper around the S3 client with the configured bucket.
#[derive(Clone)]
pub struct TigrisClient {
s3: S3Client,
bucket: String,
/// Test-only seam: when set, the blocking compression inside `upload_tracked`
/// parks on this barrier. That is the ONLY window in which a publish attempt
/// can be cancelled with `PublishStage::PreparingArchive` still holding, and
/// it is unreachable from outside without a seam because compression of a
/// test-sized repo finishes in microseconds.
#[cfg(test)]
compress_gate: Option<Arc<BlockingGate>>,
}
/// A gate a BLOCKING thread parks on until a test opens it.
///
/// A condvar rather than a held `MutexGuard`: the test's assertions run while the
/// gate is shut, and holding a `std::sync::MutexGuard` across those awaits is
/// both a lint violation and a real hazard. Here the test owns no guard at all —
/// it flips a flag and notifies.
#[cfg(test)]
#[derive(Default)]
pub struct BlockingGate {
open: Mutex<bool>,
opened: std::sync::Condvar,
}
#[cfg(test)]
impl BlockingGate {
/// A gate that starts SHUT.
pub fn shut() -> Self {
Self::default()
}
fn wait(&self) {
let mut open = self.open.lock().expect("compression gate poisoned");
while !*open {
open = self
.opened
.wait(open)
.expect("compression gate poisoned while waiting");
}
}
/// Let every parked compression through. Call this at teardown so the
/// blocking thread is not stranded for the life of the process.
pub fn open(&self) {
*self.open.lock().expect("compression gate poisoned") = true;
self.opened.notify_all();
}
}
impl TigrisClient {
@@ -132,48 +28,9 @@ impl TigrisClient {
Ok(Self {
s3,
bucket: bucket.to_string(),
#[cfg(test)]
compress_gate: None,
})
}
/// Build a client pointed at an arbitrary endpoint, for tests.
///
/// The production constructor reads the endpoint and credentials from the
/// environment, which a test cannot steer without mutating process-global
/// state. This takes both explicitly so a test can aim the client at a
/// closed port and get a prompt transport error out of `exists()`.
///
/// `RetryConfig::disabled()` is load-bearing, not tidiness: the SDK's default
/// policy retries a connection refusal with backoff, which turns each failing
/// call into seconds of waiting.
#[cfg(test)]
pub fn for_testing_with_endpoint(bucket: &str, endpoint: &str) -> Self {
use aws_sdk_s3::config::{retry::RetryConfig, Credentials, Region};
let config = aws_sdk_s3::config::Config::builder()
.endpoint_url(endpoint)
.credentials_provider(Credentials::new("test", "test", None, None, "test"))
.region(Region::new("auto"))
.retry_config(RetryConfig::disabled())
.behavior_version_latest()
.build();
Self {
s3: S3Client::from_conf(config),
bucket: bucket.to_string(),
compress_gate: None,
}
}
/// Test-only: park this client's blocking compression on `gate` until the
/// holder releases it, so a cancellation can be aimed at
/// [`PublishStage::PreparingArchive`] rather than at the PUT.
#[cfg(test)]
pub fn with_compress_gate(mut self, gate: Arc<BlockingGate>) -> Self {
self.compress_gate = Some(gate);
self
}
/// S3 key for a given repo: `repos/v1/{owner_slug}/{repo_name}.tar.zst`
fn repo_key(owner_slug: &str, repo_name: &str) -> String {
format!("repos/v1/{owner_slug}/{repo_name}.tar.zst")
@@ -201,325 +58,34 @@ impl TigrisClient {
}
}
/// Read the ETag of a repo archive, or `None` when nothing is stored under
/// the key. The ETag identifies the generation a later conditional upload
/// can fence itself on.
///
/// Separate from `exists` rather than folded into it: `exists` has callers
/// that only want the boolean, and widening its return type would churn
/// every one of them for no benefit.
pub async fn head_etag(&self, owner_slug: &str, repo_name: &str) -> Result<Option<String>> {
Ok(self
.head_generation(owner_slug, repo_name)
.await?
.map(|g| g.etag)
.unwrap_or(None))
}
/// The full stored generation: the ETag AND the attempt that published it.
///
/// The attempt half is what makes a lost response recoverable. An ETag alone
/// answers "has the generation moved", which every writer sees the same way;
/// the attempt id answers "are the published bytes MINE", which is the
/// question a client whose PUT response vanished actually needs answered.
pub async fn head_generation(
&self,
owner_slug: &str,
repo_name: &str,
) -> Result<Option<StoredGeneration>> {
/// Upload a local bare repo directory to Tigris as a tar.zst archive.
pub async fn upload(&self, owner_slug: &str, repo_name: &str, local_path: &Path) -> Result<()> {
let key = Self::repo_key(owner_slug, repo_name);
match self
.s3
.head_object()
.bucket(&self.bucket)
.key(&key)
.send()
.await
{
Ok(out) => {
let etag = out
.e_tag()
.context(format!("tigris HEAD {key}: hit carried no ETag"))?
.to_string();
let attempt = out
.metadata()
.and_then(|m| m.get(ATTEMPT_METADATA_KEY))
.cloned();
Ok(Some(StoredGeneration {
etag: Some(etag),
attempt,
}))
}
Err(e) => {
if e.as_service_error().is_some_and(|e| e.is_not_found()) {
Ok(None)
} else {
Err(anyhow::anyhow!("tigris HEAD {key}: {e}"))
}
}
}
}
debug!(key = %key, path = %local_path.display(), "uploading repo to tigris");
/// Did `attempt`'s bytes land? The reconciliation an ambiguous dispatch owes
/// before anything downstream may treat it as confirmation.
///
/// `Ok(false)` is deliberately NOT "the PUT failed": an abandoned request may
/// still be in flight, so a negative answer licenses refusing and retrying,
/// never deleting. Only `Ok(true)` upgrades an unresolved attempt to
/// published.
pub async fn attempt_landed(
&self,
owner_slug: &str,
repo_name: &str,
attempt: &PublishAttemptId,
) -> Result<bool> {
Ok(self
.head_generation(owner_slug, repo_name)
.await?
.is_some_and(|g| g.belongs_to(attempt)))
}
/// Upload a local bare repo directory to Tigris as a tar.zst archive,
/// fenced by `precondition`, under a freshly minted attempt identity.
pub async fn upload(
&self,
owner_slug: &str,
repo_name: &str,
local_path: &Path,
precondition: UploadPrecondition,
) -> std::result::Result<UploadReceipt, UploadError> {
self.upload_tracked(
owner_slug,
repo_name,
local_path,
precondition,
PublishAttemptId::new(),
None,
)
.await
}
/// The full form: a caller-chosen attempt identity, and a stage cell the
/// upload reports its progress into.
///
/// The stage cell is the answer to "a dropped future never returns an
/// outcome". Compression runs inside `spawn_blocking` and the conditional PUT
/// is not even constructed until it finishes, so a handler cancelled during
/// compression definitely never attempted publication — but nothing could
/// observe that, because the only report was the return value of a future
/// that no longer exists. Marking [`PublishStage::PreparingArchive`] before
/// the blocking call and [`PublishStage::PutDispatched`] immediately before
/// `send()` makes that boundary readable from outside.
pub async fn upload_tracked(
&self,
owner_slug: &str,
repo_name: &str,
local_path: &Path,
precondition: UploadPrecondition,
attempt: PublishAttemptId,
stage: Option<&PublishStageCell>,
) -> std::result::Result<UploadReceipt, UploadError> {
let key = Self::repo_key(owner_slug, repo_name);
debug!(key = %key, path = %local_path.display(), attempt = %attempt, "uploading repo to tigris");
if let Some(stage) = stage {
stage.set(PublishStage::PreparingArchive);
}
// Create tar.zst in memory. Nothing has been sent at this point and
// nothing can be: the request below is not constructed until this
// returns. A failure or a cancellation here is a definite
// non-publication.
let compressed = {
// Create tar.zst in memory
let archive_bytes = tokio::task::spawn_blocking({
let local_path = local_path.to_path_buf();
#[cfg(test)]
let gate = self.compress_gate.clone();
tokio::task::spawn_blocking(move || {
// Test-only seam: park INSIDE the blocking compression, which is
// the window a handler can be cancelled in while
// `PublishStage::PreparingArchive` still holds.
#[cfg(test)]
if let Some(gate) = gate {
// Blocks until the test opens the gate.
gate.wait();
}
compress_repo(&local_path)
})
.await
};
let archive_bytes = match compressed {
Ok(Ok(bytes)) => bytes,
Ok(Err(e)) => {
if let Some(stage) = stage {
stage.set(PublishStage::Refused);
}
return Err(UploadError::NotPublished(e.context("compressing repo")));
}
Err(e) => {
if let Some(stage) = stage {
stage.set(PublishStage::Refused);
}
return Err(UploadError::NotPublished(
anyhow::Error::new(e).context("tar task panicked"),
));
}
};
move || compress_repo(&local_path)
})
.await
.context("tar task panicked")?
.context("compressing repo")?;
let body = aws_sdk_s3::primitives::ByteStream::from(archive_bytes);
let mut req = self
.s3
self.s3
.put_object()
.bucket(&self.bucket)
.key(&key)
// The attempt identity travels WITH the bytes. That is what makes a
// lost response recoverable: the client can HEAD the key afterwards
// and ask whether the published object is its own work, which is
// decidable, instead of asking whether its request succeeded, which
// is not.
.metadata(ATTEMPT_METADATA_KEY, attempt.as_str())
.body(body)
.content_type("application/zstd");
match &precondition {
UploadPrecondition::IfMatch(etag) => req = req.if_match(etag),
UploadPrecondition::IfAbsent => req = req.if_none_match("*"),
UploadPrecondition::Unconditional => {}
}
.content_type("application/zstd")
.send()
.await
.context(format!("tigris PUT {key}"))?;
// From HERE the bytes may reach the store. Everything downstream that
// could destroy state has to treat this stage as "maybe published".
if let Some(stage) = stage {
stage.set(PublishStage::PutDispatched {
attempt: attempt.clone(),
});
}
let sent = req.send().await;
let out = match sent {
Ok(out) => out,
Err(e) => {
// `PutObjectError` models no PreconditionFailed variant (its arms are
// EncryptionTypeMismatch, InvalidRequest, InvalidWriteOffset,
// TooManyParts, Unhandled), so a refused precondition arrives as
// `Unhandled` and matching the enum would classify it as a generic
// failure. The raw HTTP status off the response is the only place the
// answer actually lives.
//
// Read it via `raw_response()`, not a `ServiceError`-only match: the
// SDK exposes the raw response for BOTH `ServiceError` and
// `ResponseError`, and a refused conditional PUT whose error body the
// SDK cannot parse (malformed XML, premature close) surfaces as
// `ResponseError`. Matching only `ServiceError` would classify that
// unparsable 409/412 as a generic failure, and `RepoWriteGuard::release`
// would log-and-succeed instead of taking the supersede retry,
// acknowledging a write that was definitively not published.
let status = e.raw_response().map(|raw| raw.status().as_u16());
// 412 is always a lost precondition. 409 is one only when we asked
// for create-only, which is how S3-compatible stores report "the key
// already exists". Everything else, 404 included, is a real failure:
// archive keys are never deleted by the write path, so a 404 here
// means something permanent like a missing bucket or a misrouted
// endpoint, and reporting that as a lost precondition would tell a
// caller to expect a successor that does not exist.
let lost = match status {
Some(412) => true,
Some(409) => matches!(precondition, UploadPrecondition::IfAbsent),
_ => false,
};
if lost {
if let Some(stage) = stage {
stage.set(PublishStage::Refused);
}
return Err(UploadError::PreconditionLost {
status: status.expect("a lost precondition came from a status"),
});
}
let knowledge = classify_put_failure(&e, status);
let ctx = anyhow::Error::new(e).context(format!("tigris PUT {key}"));
return Err(match knowledge {
DispatchKnowledge::NeverSent => {
if let Some(stage) = stage {
stage.set(PublishStage::Refused);
}
UploadError::NotPublished(ctx)
}
DispatchKnowledge::MaybeSent => {
warn!(
key = %key,
attempt = %attempt,
status = ?status,
"tigris PUT failed WITHOUT proving it did not commit — the outcome \
is ambiguous and must be reconciled, not compensated"
);
if let Some(stage) = stage {
stage.set(PublishStage::Ambiguous {
attempt: attempt.clone(),
});
}
UploadError::Ambiguous {
attempt: attempt.clone(),
source: ctx,
}
}
});
}
};
let etag = out.e_tag().map(str::to_string);
if let Some(stage) = stage {
stage.set(PublishStage::Published {
attempt: attempt.clone(),
etag: etag.clone(),
});
}
info!(key = %key, attempt = %attempt, "uploaded repo to tigris");
Ok(UploadReceipt { attempt, etag })
}
/// Delete a repo archive ONLY while it is still `attempt`'s work.
///
/// The unconditional delete this replaces used the logical owner/name as its
/// cleanup authority, so a failed attempt compensating late could erase the
/// object a SUCCESSOR had already published under the same name and already
/// returned 201 for. A second name lookup before the delete only narrows that
/// window; reading the attempt id off the object and fencing the delete on
/// the generation it came from closes it.
pub async fn delete_if_attempt_matches(
&self,
owner_slug: &str,
repo_name: &str,
attempt: &PublishAttemptId,
) -> Result<AttemptDelete> {
let Some(generation) = self.head_generation(owner_slug, repo_name).await? else {
return Ok(AttemptDelete::Absent);
};
if !generation.belongs_to(attempt) {
return Ok(AttemptDelete::NotOurs);
}
let key = Self::repo_key(owner_slug, repo_name);
let mut req = self.s3.delete_object().bucket(&self.bucket).key(&key);
// The If-Match guard is what makes this atomic rather than merely
// narrowed: between the HEAD above and this call a successor can publish,
// and the store refusing on the moved generation is the only thing that
// stops the delete landing on their object.
if let Some(etag) = generation.etag.as_deref() {
req = req.if_match(etag);
}
match req.send().await {
Ok(_) => Ok(AttemptDelete::Deleted),
Err(e) => {
// A refused conditional delete means the generation moved: the
// object is no longer ours, which is a successful outcome for a
// guard whose whole job is not to touch somebody else's bytes.
if e.raw_response()
.map(|raw| raw.status().as_u16())
.is_some_and(|s| s == 412 || s == 409)
{
return Ok(AttemptDelete::NotOurs);
}
Err(anyhow::anyhow!("tigris conditional DELETE {key}: {e}"))
}
}
info!(key = %key, "uploaded repo to tigris");
Ok(())
}
/// Download a repo archive from Tigris and extract to local disk.
@@ -527,32 +93,10 @@ impl TigrisClient {
&self,
owner_slug: &str,
repo_name: &str,
local_path: &super::repo_store::ValidatedRepoDiskPath,
swap_authority: Option<Arc<AtomicBool>>,
local_path: &Path,
) -> Result<()> {
self.download_to(owner_slug, repo_name, local_path, true, swap_authority)
.await
.map(|_| ())
}
/// Download a repo archive from Tigris and extract it, returning the
/// directory that was populated.
///
/// `publish` controls whether the extract is swapped into `target` in place
/// (the live-path mutation used by writes; returns `target`) or unpacked
/// into a fresh temp directory under `target`'s parent (a non-mutating
/// snapshot read; returns the temp dir, which the caller owns and cleans
/// up). The snapshot form never touches the live repo path.
pub async fn download_to(
&self,
owner_slug: &str,
repo_name: &str,
target: &super::repo_store::ValidatedRepoDiskPath,
publish: bool,
swap_authority: Option<Arc<AtomicBool>>,
) -> Result<DownloadExtract> {
let key = Self::repo_key(owner_slug, repo_name);
debug!(key = %key, path = %target.as_path().display(), "downloading repo from tigris");
debug!(key = %key, path = %local_path.display(), "downloading repo from tigris");
let resp = self
.s3
@@ -570,59 +114,31 @@ impl TigrisClient {
.context("reading tigris response body")?
.into_bytes();
// The snapshot temp dir is decided HERE, in the async layer, before the
// extraction runs. Cleanup ownership moves into the blocking task so a
// cancelled async future cannot drop it while extraction is still running.
let snapshot_tmp = if publish {
None
} else {
let parent = target.parent().context("snapshot path has no parent")?;
std::fs::create_dir_all(parent).context("creating parent dir")?;
let file_name = target
.file_name()
.context("snapshot path has no file name")?
.to_string_lossy();
Some(parent.join(format!(
".{file_name}.tmp-snapshot.{}",
uuid::Uuid::new_v4()
)))
};
// Extract tar.zst to a directory.
let extracted = tokio::task::spawn_blocking({
let target = target.clone();
let snapshot_tmp = snapshot_tmp.clone();
move || -> Result<DownloadExtract> {
let result = (|| -> Result<DownloadExtract> {
if publish {
decompress_repo(&data, &target, swap_authority.as_ref())?;
return Ok(DownloadExtract::Published(()));
}
// Non-mutating snapshot: unpack into the temp dir decided above.
// The live repo path is never touched.
let tmp_dir = snapshot_tmp.expect("snapshot path was decided above");
std::fs::create_dir_all(&tmp_dir).context("creating temp extract dir")?;
let unpack = (|| -> Result<()> {
let decoder = zstd::stream::Decoder::new(&data[..])?;
let mut archive = tar::Archive::new(decoder);
archive.unpack(&tmp_dir).context("unpacking tar.zst")?;
Ok(())
})();
if let Err(e) = unpack {
let _ = std::fs::remove_dir_all(&tmp_dir);
return Err(e);
}
Ok(DownloadExtract::Snapshot(TempSnapshotDir { path: tmp_dir }))
})();
result
}
// Extract tar.zst to local path
tokio::task::spawn_blocking({
let local_path = local_path.to_path_buf();
move || decompress_repo(&data, &local_path)
})
.await
.context("extract task panicked")?
.context("extracting repo")?;
info!(key = %key, path = %target.as_path().display(), "downloaded repo from tigris");
Ok(extracted)
info!(key = %key, path = %local_path.display(), "downloaded repo from tigris");
Ok(())
}
/// Delete a repo archive from Tigris.
#[allow(dead_code)]
pub async fn delete(&self, owner_slug: &str, repo_name: &str) -> Result<()> {
let key = Self::repo_key(owner_slug, repo_name);
self.s3
.delete_object()
.bucket(&self.bucket)
.key(&key)
.send()
.await
.context(format!("tigris DELETE {key}"))?;
Ok(())
}
}
@@ -645,7 +161,7 @@ fn compress_repo(repo_path: &Path) -> Result<Vec<u8>> {
/// `decompress_repo`. Concurrent extractions unpack into isolated temp dirs in
/// parallel, but the final `remove_dir_all` + `rename` must not interleave for
/// the same `local_path`, or they race to a nondeterministic overwrite/failure.
pub(crate) fn publish_lock(local_path: &Path) -> Arc<Mutex<()>> {
fn publish_lock(local_path: &Path) -> Arc<Mutex<()>> {
// KNOWN LIMITATION: this map is never evicted — one (PathBuf, Arc<Mutex>)
// entry accrues per distinct repo path for the process lifetime. Bounded by
// the number of repos a node hosts, so it's negligible for normal use, but
@@ -658,35 +174,6 @@ pub(crate) fn publish_lock(local_path: &Path) -> Arc<Mutex<()>> {
.clone()
}
/// Owns a snapshot temp dir from extraction through handoff to [`RepoSnapshot`].
/// Dropped when the `spawn_blocking` join result is abandoned, so cancellation
/// before the outer future resumes still removes the directory.
pub(crate) struct TempSnapshotDir {
path: PathBuf,
}
impl TempSnapshotDir {
pub(crate) fn into_repo_snapshot(self) -> super::repo_store::RepoSnapshot {
let path = self.path.clone();
std::mem::forget(self);
super::repo_store::RepoSnapshot::from_owned_path(path)
}
}
impl Drop for TempSnapshotDir {
fn drop(&mut self) {
let _ = std::fs::remove_dir_all(&self.path);
}
}
/// What [`TigrisClient::download_to`] produced.
pub enum DownloadExtract {
/// Published into the validated live repo path.
Published(()),
/// Unpacked into a throwaway temp dir that cleans up on drop until adopted.
Snapshot(TempSnapshotDir),
}
/// Decompress a tar.zst byte vector into a local directory.
///
/// Extraction is atomic with respect to `local_path`: the archive is unpacked
@@ -694,16 +181,11 @@ pub enum DownloadExtract {
/// fully succeeds. A corrupt or truncated archive therefore can never clobber a
/// good existing copy at `local_path` — on failure we discard the temp dir and
/// leave `local_path` exactly as it was.
fn decompress_repo(
data: &[u8],
local_path: &super::repo_store::ValidatedRepoDiskPath,
swap_authority: Option<&Arc<AtomicBool>>,
) -> Result<()> {
let live = local_path.as_path();
let parent = live.parent().context("repo path has no parent")?;
fn decompress_repo(data: &[u8], local_path: &Path) -> Result<()> {
let parent = local_path.parent().context("repo path has no parent")?;
std::fs::create_dir_all(parent).context("creating parent dir")?;
let file_name = live
let file_name = local_path
.file_name()
.context("repo path has no file name")?
.to_string_lossy();
@@ -728,218 +210,17 @@ fn decompress_repo(
return Err(e);
}
// Swap through the validated-path helper so CodeQL sees the barrier before the
// remove/rename sink (`rust/path-injection`).
super::repo_store::swap_extracted_into_validated_repo(local_path, &tmp_dir, swap_authority)?;
// Swap the freshly-extracted repo into place. rename within the same parent
// is effectively atomic, but most platforms refuse to rename onto a
// non-empty dir, so remove the old copy first. Serialize this per repo path:
// concurrent extractions unpack into isolated temp dirs, but their swaps
// must not interleave or they race to a nondeterministic overwrite/failure.
let lock = publish_lock(local_path);
let _publish = lock.lock().expect("publish lock poisoned");
if local_path.exists() {
std::fs::remove_dir_all(local_path).context("removing stale repo dir")?;
}
std::fs::rename(&tmp_dir, local_path).context("swapping extracted repo into place")?;
Ok(())
}
#[cfg(test)]
mod tests {
use super::*;
use aws_sdk_s3::primitives::ByteStream;
use futures::FutureExt;
/// The envs the probe needs, all of them, or it does not run.
///
/// `AWS_ENDPOINT_URL_S3` is included on purpose: without it the SDK resolves
/// to real AWS S3, and a probe that passed there would say nothing about
/// Tigris.
fn probe_env() -> Option<String> {
if std::env::var("GITLAWB_TIGRIS_PROBE").ok().as_deref() != Some("1") {
return None;
}
for name in [
"AWS_ACCESS_KEY_ID",
"AWS_SECRET_ACCESS_KEY",
"AWS_ENDPOINT_URL_S3",
] {
if std::env::var(name).is_err() {
eprintln!("tigris conditional-write probe: {name} is unset, skipping");
return None;
}
}
match std::env::var("GITLAWB_TIGRIS_BUCKET") {
Ok(b) if !b.is_empty() => Some(b),
_ => {
eprintln!(
"tigris conditional-write probe: GITLAWB_TIGRIS_BUCKET is unset, skipping"
);
None
}
}
}
/// One conditional PUT, reported as the status that REFUSED it, or `None`
/// when the store accepted the write.
///
/// Accepted is the interesting answer here, not an error: it means the
/// endpoint ignored the header we fenced on.
async fn conditional_put(
s3: &S3Client,
bucket: &str,
key: &str,
body: &'static [u8],
if_match: Option<&str>,
if_none_match: Option<&str>,
) -> Result<Option<u16>, String> {
let mut req = s3
.put_object()
.bucket(bucket)
.key(key)
.body(ByteStream::from_static(body));
if let Some(v) = if_match {
req = req.if_match(v);
}
if let Some(v) = if_none_match {
req = req.if_none_match(v);
}
match req.send().await {
Ok(_) => Ok(None),
Err(e) => match e.raw_response() {
// Same raw-response rule as `upload`: a refused conditional PUT
// whose body the SDK cannot parse surfaces as `ResponseError`, and
// the status has to come off the raw response for both variants.
// Without it, an unparsable 409/412 would report "no HTTP
// response" here and skip the supersede retry.
Some(raw) => Ok(Some(raw.status().as_u16())),
None => Err(format!("conditional PUT {key}: no HTTP response: {e}")),
},
}
}
/// The probe body, written to RETURN its failures rather than panic on
/// them, so the caller's cleanup is reached on every arm.
async fn conditional_write_probe(s3: &S3Client, bucket: &str, key: &str) -> Result<(), String> {
// 1. A plain PUT under a throwaway key.
let seeded = s3
.put_object()
.bucket(bucket)
.key(key)
.body(ByteStream::from_static(b"probe-one"))
.send()
.await
.map_err(|e| format!("seeding PUT {key}: {e}"))?;
// 2. Its ETag, which is the generation the next arm fences against.
let etag = seeded
.e_tag()
.ok_or_else(|| format!("seeding PUT {key} returned no ETag"))?
.to_string();
// 3. A deliberately wrong If-Match. A store honoring it answers 412.
let wrong = format!("\"{}\"", "0".repeat(32));
if etag.trim_matches('"') == wrong.trim_matches('"') {
return Err(format!(
"the seeded ETag {etag} collides with the deliberately wrong one, \
so this arm would prove nothing"
));
}
match conditional_put(s3, bucket, key, b"probe-two", Some(&wrong), None).await? {
Some(412) => {}
Some(status) => {
return Err(format!(
"a stale If-Match must be refused with 412, the endpoint answered {status}"
))
}
None => {
return Err(
"a stale If-Match was ACCEPTED: this endpoint does not honor If-Match, so \
the release fence cannot hold here"
.to_string(),
)
}
}
// 4. If-None-Match `*` over the object that now exists. This arm matters
// MORE than the one above. An ignored If-Match eventually surfaces as
// odd behavior, because a stale writer overwrites and someone notices
// the lost tree. An ignored If-None-Match just returns 200, so a publish
// that should have been fenced lands with no error anywhere: the silent
// no-op the bucket-type caveat on this test describes.
match conditional_put(s3, bucket, key, b"probe-three", None, Some("*")).await? {
// Either status is a pass, and the asymmetry with the If-Match arm
// above mirrors `upload`'s classifier exactly: 412 is always a lost
// precondition, and 409 is one too when we asked for create-only.
// AWS documents 409 for a create-only conflict racing a delete, so a
// store answering it is enforcing the precondition and we already
// handle it. Pinning 412 alone here would fail the probe against a
// backend that is behaving correctly, which sends whoever runs it
// chasing a fault that is not there.
Some(412) | Some(409) => {}
Some(status) => {
return Err(format!(
"create-only over an existing object must be refused with 412 or 409, \
the endpoint answered {status}"
))
}
None => {
return Err(
"If-None-Match * was ACCEPTED over an existing object: this endpoint does \
not honor create-only, so a fenced publish lands silently"
.to_string(),
)
}
}
Ok(())
}
/// Probe the REAL Tigris endpoint for the conditional-write semantics the
/// release fence depends on.
///
/// UNTIL THIS IS RUN AGAINST REAL CREDENTIALS, the fence is verified against
/// vendor documentation and an in-process mock, not against the backend it
/// runs on. The mock implements the semantics we believe Tigris has; it
/// cannot tell us whether Tigris actually has them.
///
/// The bucket matters, not just the endpoint. Tigris documents conditional
/// operations as supported on Single-region and Multi-region buckets only.
/// Global and Dual-region buckets are eventually consistent, and a
/// conditional PUT evaluated against a stale replica would make the fence a
/// silent no-op rather than an error. So point `GITLAWB_TIGRIS_BUCKET` at a
/// throwaway bucket of the SAME type production uses.
///
/// Ignored by default and additionally gated on `GITLAWB_TIGRIS_PROBE=1`,
/// because it writes to a real bucket and costs real requests. Run with:
/// `GITLAWB_TIGRIS_PROBE=1 cargo test -p gitlawb-node --bin gitlawb-node
/// tigris_honors_conditional_writes -- --ignored --nocapture`
#[tokio::test]
#[ignore = "writes to a real Tigris bucket; needs GITLAWB_TIGRIS_PROBE=1 plus credentials"]
async fn tigris_honors_conditional_writes() {
let Some(bucket) = probe_env() else {
eprintln!(
"tigris conditional-write probe: skipped. Set GITLAWB_TIGRIS_PROBE=1, \
GITLAWB_TIGRIS_BUCKET, AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY and \
AWS_ENDPOINT_URL_S3 to run it."
);
return;
};
let config = aws_config::load_defaults(aws_config::BehaviorVersion::latest()).await;
let s3 = S3Client::new(&config);
// A fresh key per run, so a probe that somehow orphaned an object on an
// earlier run cannot change what this one observes.
let key = format!("probe/conditional-write-{}.bin", uuid::Uuid::new_v4());
// CLEANUP MUST RUN ON EVERY ARM, and a failing assertion is precisely
// the case this probe exists to catch, so the delete cannot sit after
// the checks. The body returns its failures rather than panicking, and
// `catch_unwind` covers the panic an SDK call could still raise; either
// way the delete below is reached before the verdict is re-raised.
let outcome = std::panic::AssertUnwindSafe(conditional_write_probe(&s3, &bucket, &key))
.catch_unwind()
.await;
if let Err(e) = s3.delete_object().bucket(&bucket).key(&key).send().await {
eprintln!("tigris conditional-write probe: cleanup of {key} failed: {e}");
}
match outcome {
Ok(Ok(())) => {}
Ok(Err(msg)) => panic!("tigris conditional-write probe: {msg}"),
Err(payload) => std::panic::resume_unwind(payload),
}
}
}
+35 -410
View File
@@ -1,9 +1,7 @@
//! Resolve which object OIDs must be withheld from a caller because every path
//! at which the object appears is denied by the repo's visibility rules. Blobs
//! and subtree trees are withholdable (mode B; trees since #135/#173). Commits
//! and tags are never withheld: they are root-level metadata a "/" reader already
//! clears. The replication pin path uses [`withheld_object_oids_bounded`] so a
//! withheld-subtree tree is not exported to public IPFS (#172).
//! Resolve which blob OIDs must be withheld from a caller because every path
//! at which the blob appears is denied by the repo's visibility rules. Trees
//! and commits are never withheld (mode B keeps SHAs intact); only blob
//! content is held back.
use crate::db::VisibilityRule;
use crate::visibility::{visibility_check, Decision};
@@ -69,13 +67,13 @@ fn child_terminated_without_reaping(pid: i32) -> bool {
}
#[cfg(unix)]
pub(crate) fn run_bounded_git_raw(
pub(crate) fn run_bounded_git(
git_bin: &str,
args: &[&str],
repo_path: &Path,
stdin_bytes: &[u8],
deadline: Instant,
) -> Result<(std::process::ExitStatus, Vec<u8>, Vec<u8>)> {
) -> Result<Vec<u8>> {
use std::io::{Read, Write};
use std::os::unix::process::CommandExt;
use std::sync::mpsc::RecvTimeoutError;
@@ -193,23 +191,6 @@ pub(crate) fn run_bounded_git_raw(
if killed && !status.success() {
return Err(crate::git::smart_http::GitServiceTimeout.into());
}
Ok((status, out, err))
}
/// Bounded git returning only stdout, `bail!`ing on any nonzero exit. The thin
/// wrapper the walk callers use. Probes that must distinguish exit classes —
/// `git cat-file` absence vs an object-store access failure — call
/// [`run_bounded_git_raw`] and classify the status/stderr themselves.
#[cfg(unix)]
pub(crate) fn run_bounded_git(
git_bin: &str,
args: &[&str],
repo_path: &Path,
stdin_bytes: &[u8],
deadline: Instant,
) -> Result<Vec<u8>> {
let label = args.first().copied().unwrap_or("git");
let (status, out, err) = run_bounded_git_raw(git_bin, args, repo_path, stdin_bytes, deadline)?;
if !status.success() {
anyhow::bail!("git {label} failed: {}", String::from_utf8_lossy(&err));
}
@@ -229,13 +210,13 @@ pub(crate) fn run_bounded_git(
/// the Unix version's signature and result semantics so every caller compiles on all
/// targets (#174).
#[cfg(not(unix))]
pub(crate) fn run_bounded_git_raw(
pub(crate) fn run_bounded_git(
git_bin: &str,
args: &[&str],
repo_path: &Path,
stdin_bytes: &[u8],
deadline: Instant,
) -> Result<(std::process::ExitStatus, Vec<u8>, Vec<u8>)> {
) -> Result<Vec<u8>> {
use std::io::{Read, Write};
use std::sync::mpsc::RecvTimeoutError;
@@ -302,20 +283,6 @@ pub(crate) fn run_bounded_git_raw(
if killed && !status.success() {
return Err(crate::git::smart_http::GitServiceTimeout.into());
}
Ok((status, out, err))
}
/// Non-Unix thin wrapper matching the Unix [`run_bounded_git`] semantics.
#[cfg(not(unix))]
pub(crate) fn run_bounded_git(
git_bin: &str,
args: &[&str],
repo_path: &Path,
stdin_bytes: &[u8],
deadline: Instant,
) -> Result<Vec<u8>> {
let label = args.first().copied().unwrap_or("git");
let (status, out, err) = run_bounded_git_raw(git_bin, args, repo_path, stdin_bytes, deadline)?;
if !status.success() {
anyhow::bail!("git {label} failed: {}", String::from_utf8_lossy(&err));
}
@@ -542,79 +509,6 @@ pub fn withheld_blob_oids_bounded(
))
}
/// Reachable blob AND withheld-subtree tree OIDs the anonymous (or `caller`)
/// replication audience must not receive. The single source for
/// `replication_withheld_set`: a blob-only set handed to [`replicable_objects`]
/// reintroduces #172 (withheld trees pin to public IPFS).
///
/// Strict where [`tree_paths`] is lenient. This feeds a withheld filter, so
/// absence means replicate; a missed reachable tree would leak. The walk
/// therefore matches [`blob_paths`]: [`assert_all_refs_are_commits`] first, then
/// a failing `rev-list --all [HEAD]` is an error, then [`object_paths`] (already
/// fail-closed on ls-tree / non-UTF-8) plus [`root_tree_pairs`]. [`tree_paths`]
/// stays the lenient allow-list walk for GET /ipfs/{cid} and the full-scan
/// allowed-tree set, where absence withholds.
pub fn withheld_object_oids_bounded(
repo_path: &Path,
git_bin: &str,
timeout: Duration,
rules: &[VisibilityRule],
is_public: bool,
owner_did: &str,
caller: Option<&str>,
) -> Result<HashSet<String>> {
let deadline = Instant::now() + timeout;
assert_all_refs_are_commits(repo_path, git_bin, deadline)?;
let head_resolves = run_bounded_git(
git_bin,
&["rev-parse", "--verify", "HEAD"],
repo_path,
b"",
deadline,
)
.is_ok();
let mut rev_args = vec!["rev-list", "--all"];
if head_resolves {
rev_args.push("HEAD");
}
let commits_out = run_bounded_git(git_bin, &rev_args, repo_path, b"", deadline)?;
let commits: Vec<String> = String::from_utf8_lossy(&commits_out)
.lines()
.map(|l| l.trim().to_string())
.filter(|l| !l.is_empty())
.collect();
let triples = object_paths(repo_path, git_bin, &commits, deadline)?;
let mut pairs: Vec<(String, String)> = triples
.into_iter()
.filter(|(_, _, kind)| kind == "blob" || kind == "tree")
.map(|(oid, path, _)| (oid, path))
.collect();
pairs.extend(root_tree_pairs(repo_path, git_bin, &commits, deadline)?);
Ok(withheld_from_pairs(
&pairs, rules, is_public, owner_did, caller,
))
}
/// [`withheld_object_oids_bounded`] with the test walk budget.
#[cfg(test)]
pub fn withheld_object_oids(
repo_path: &Path,
rules: &[VisibilityRule],
is_public: bool,
owner_did: &str,
caller: Option<&str>,
) -> Result<HashSet<String>> {
withheld_object_oids_bounded(
repo_path,
"git",
WALK_TIMEOUT,
rules,
is_public,
owner_did,
caller,
)
}
/// Withheld set from an already-computed (oid, "/path") listing: a blob is
/// withheld only when visibility denies the caller at *every* path it appears
/// at. Split out so a caller that already walked `blob_paths` (e.g.
@@ -663,9 +557,8 @@ pub fn has_path_scoped_rule(rules: &[VisibilityRule]) -> bool {
/// Objects that may replicate to the public: everything not in `withheld`.
/// Order-preserving. The single seam every replication site (IPFS, Pinata)
/// passes its object list through on the delta path. `withheld` MUST be the
/// combined blob-and-tree set from [`withheld_object_oids_bounded`] (via
/// `replication_withheld_set`); a blob-only set reintroduces #172.
/// passes its object list through; option B would later reroute the withheld
/// ones through encrypt-then-pin instead of dropping them.
pub fn replicable_objects(all: Vec<String>, withheld: &HashSet<String>) -> Vec<String> {
all.into_iter()
.filter(|oid| !withheld.contains(oid))
@@ -703,23 +596,6 @@ pub fn replicable_blob_set_bounded(
)
}
/// Reachable tree OIDs that visibility ALLOWS the anonymous replication
/// audience at some path. The tree analog of [`replicable_blob_set_bounded`],
/// used on the fail-closed full-scan pin path so a withheld or dangling tree
/// is absent and dropped (#172).
pub fn replicable_tree_set_bounded(
repo_path: &Path,
git_bin: &str,
timeout: Duration,
rules: &[VisibilityRule],
is_public: bool,
owner_did: &str,
) -> Result<HashSet<String>> {
allowed_tree_set_for_caller_bounded(
repo_path, git_bin, timeout, rules, is_public, owner_did, None,
)
}
/// Reachable blob OIDs that visibility ALLOWS `caller` at some path. The
/// caller-aware generalization of `replicable_blob_set` (which is the anonymous
/// `caller = None` case). Used by `GET /ipfs/{cid}` to gate fail-closed against
@@ -920,12 +796,11 @@ fn root_tree_pairs(
/// Every `(tree_oid, "/path")` pair reachable in `repo_path`: the `kind == "tree"`
/// slice of [`object_paths`] (subtree trees at their directory paths) PLUS every
/// reachable commit's root tree at "/" (see [`root_tree_pairs`]). Computes the
/// reachable-commit set ONCE (leniently, see [`reachable_commit_oids`]; the tree
/// allowed-set feeds the `/ipfs/{cid}` tree gate AND the full-scan pin allow-list,
/// where absence = fail-closed) and drives both the ls-tree walk and the root-tree
/// pass from it, so the two cannot diverge and neither re-enumerates. The tree analog
/// of [`blob_paths`], bounded by the same shared `deadline`. Not for a withheld
/// filter: that path is [`withheld_object_oids_bounded`].
/// reachable-commit set ONCE (leniently — see [`reachable_commit_oids`]; the tree
/// allowed-set feeds ONLY the `/ipfs/{cid}` tree gate, where absence = fail-closed
/// 404) and drives both the ls-tree walk and the root-tree pass from it, so the two
/// cannot diverge and neither re-enumerates. The tree analog of [`blob_paths`],
/// bounded by the same shared `deadline`.
fn tree_paths(
repo_path: &Path,
git_bin: &str,
@@ -1244,29 +1119,22 @@ pub fn reachable_commit_tag_oids_bounded(
Ok(set)
}
/// Objects safe to replicate, failing closed on blobs AND trees (#99, #172).
/// A candidate replicates iff it is neither a known blob nor a known tree
/// (commits and tags: structural, a "/" reader already clears) OR it is in
/// `allowed_blobs` OR it is in `allowed_trees`. Trees have been content-withholdable
/// since #135/#173; the old premise that they always pass is falsified. A dangling
/// or withheld tree is in `all_tree_oids` and absent from `allowed_trees`, so it
/// drops. Used on the full-scan pin path, where the candidate set can contain
/// dangling objects the reachable-only withheld set cannot cover; the delta path
/// keeps [`replicable_objects`].
/// Objects safe to replicate, failing closed on blobs (#99). A candidate
/// replicates iff it is NOT a blob (`all_blob_oids` — commits and trees are
/// structural, never content-withheld) OR it is in `allowed_blobs` (reachable
/// and visibility-allowed). This drops both withheld reachable blobs and
/// dangling/unreachable blobs the reachable walk never classified, without
/// tagging the candidate list with per-object types. Used on the full-scan pin
/// path, where the candidate set can contain dangling objects the reachable-only
/// withheld set cannot cover; the delta path keeps `replicable_objects`.
pub fn replicable_objects_fail_closed(
candidates: Vec<String>,
allowed_blobs: &HashSet<String>,
all_blob_oids: &HashSet<String>,
allowed_trees: &HashSet<String>,
all_tree_oids: &HashSet<String>,
) -> Vec<String> {
candidates
.into_iter()
.filter(|oid| {
let is_blob = all_blob_oids.contains(oid);
let is_tree = all_tree_oids.contains(oid);
(!is_blob && !is_tree) || allowed_blobs.contains(oid) || allowed_trees.contains(oid)
})
.filter(|oid| !all_blob_oids.contains(oid) || allowed_blobs.contains(oid))
.collect()
}
@@ -1373,10 +1241,7 @@ mod tests {
assert!(
err.downcast_ref::<crate::git::smart_http::GitServiceTimeout>()
.is_some(),
// `{err:#}` prints the whole anyhow chain. Plain `{err}` shows only the top
// context ("failed to spawn git for-each-ref") and drops the underlying io
// error, which left a real beta-lane CI failure undiagnosable.
"a hung walk must abort with GitServiceTimeout (mapped to 504), got: {err:#}"
"a hung walk must abort with GitServiceTimeout (mapped to 504), got: {err}"
);
// The recorded process-group leader must be gone: the watchdog reaps the
@@ -1567,17 +1432,6 @@ esac\n";
// rev-list records its (leader) pid, closes stdout+stderr so the drain EOFs
// immediately, then sleeps without trapping TERM. `sleep 30` bounds the worst
// case so a RED run cannot wedge the suite; the recv budget fires first.
//
// The deadline below is deliberately far larger than the work it bounds: the
// watchdog tears the whole group down at the deadline, so a deadline tight
// enough to race `/bin/sh` reaching its first statement kills the leader
// BEFORE it records its pid, and the `leader.pid` read below then fails with
// NotFound. That is a harness race, not a product failure, and it is exactly
// what a loaded CI runner reproduced (test (beta), 2026-08-04). Sizing it at
// 2s keeps the property under test intact, since the pre-fix build blocks on
// `child.wait()` for the child's full `sleep 30` and still trips the 10s recv
// budget below (RED), while leaving a 20x margin over the ~100ms that a
// healthy start actually needs.
let body = "#!/bin/sh\ncase \"$1\" in\n rev-list) echo $$ > leader.pid; exec 1>&- 2>&-; sleep 30 ;;\n *) : ;;\nesac\nexit 0\n";
let git_bin = write_fake_git(tmp.path(), body);
let (tx, rx) = std::sync::mpsc::channel();
@@ -1588,7 +1442,7 @@ esac\n";
&["rev-list"],
&path,
b"",
Instant::now() + Duration::from_secs(2),
Instant::now() + Duration::from_millis(100),
));
});
let out = rx.recv_timeout(Duration::from_secs(10)).expect(
@@ -2485,49 +2339,28 @@ esac\n";
}
#[test]
fn fail_closed_keeps_commits_allowed_blobs_and_allowed_trees_only() {
// Deliberate invariant flip of fail_closed_keeps_nonblobs_and_allowed_blobs_only
// (#172). That test's premise ("commits and trees are structural, never
// content-withheld") was falsified by #135/#173: a withheld-subtree tree is
// content, same as a withheld blob. Commits and tags still pass; a blob or
// tree passes only if it is in its allowed set. Withheld and dangling blobs
// and trees (in the all-set, absent from allowed) drop.
let allowed_blobs: HashSet<String> = ["b_pub".to_string()].into_iter().collect();
fn fail_closed_keeps_nonblobs_and_allowed_blobs_only() {
// Non-blob objects (commit/tree) always pass; a blob passes only if it
// is in the allowed set. A withheld blob and a dangling blob (both in
// all_blob_oids, neither in allowed) are dropped.
let allowed: HashSet<String> = ["b_pub".to_string()].into_iter().collect();
let all_blobs: HashSet<String> = ["b_pub", "b_secret", "b_dangling"]
.into_iter()
.map(String::from)
.collect();
let allowed_trees: HashSet<String> =
["t_root", "t_pub"].into_iter().map(String::from).collect();
let all_trees: HashSet<String> = ["t_root", "t_pub", "t_secret", "t_dangling"]
.into_iter()
.map(String::from)
.collect();
let candidates = vec![
"commit1".to_string(),
"tag1".to_string(),
"t_root".to_string(),
"t_pub".to_string(),
"t_secret".to_string(),
"t_dangling".to_string(),
"tree1".to_string(),
"b_pub".to_string(),
"b_secret".to_string(),
"b_dangling".to_string(),
];
let got = replicable_objects_fail_closed(
candidates,
&allowed_blobs,
&all_blobs,
&allowed_trees,
&all_trees,
);
let got = replicable_objects_fail_closed(candidates, &allowed, &all_blobs);
assert_eq!(
got,
vec![
"commit1".to_string(),
"tag1".to_string(),
"t_root".to_string(),
"t_pub".to_string(),
"tree1".to_string(),
"b_pub".to_string()
]
);
@@ -2604,12 +2437,8 @@ esac\n";
);
// Full-scan candidate set includes the dangling blob; fail-closed drops it.
// Empty tree sets keep today's keep-all-trees behavior for this blob-only
// case (U1 signature adapt; U3 wires the real tree walks).
let no_trees: HashSet<String> = HashSet::new();
let candidates = vec![dangling_oid.clone(), public_oid.clone()];
let replicable =
replicable_objects_fail_closed(candidates, &allowed, &all_blobs, &no_trees, &no_trees);
let replicable = replicable_objects_fail_closed(candidates, &allowed, &all_blobs);
assert!(
!replicable.contains(&dangling_oid),
"#99: a dangling private blob must not replicate"
@@ -2620,210 +2449,6 @@ esac\n";
);
}
#[test]
fn withheld_object_set_contains_withheld_subtree_tree() {
// #172 repro as a test: the withheld /secret tree must enter the combined
// withheld set (today's blob-only walk omits it because trees are never in
// all_blob_oids). Nested secret/nested is the same leak class.
let td = TempDir::new().unwrap();
let work = td.path().join("work");
let bare = td.path().join("bare.git");
let run = |args: &[&str], dir: &Path| {
assert!(
Command::new("git")
.args(args)
.current_dir(dir)
.status()
.unwrap()
.success(),
"git {args:?} failed"
);
};
std::fs::create_dir_all(work.join("public")).unwrap();
std::fs::create_dir_all(work.join("secret/nested")).unwrap();
std::fs::write(work.join("public/a.txt"), b"public bytes\n").unwrap();
std::fs::write(work.join("secret/b.txt"), b"TOP SECRET\n").unwrap();
std::fs::write(work.join("secret/nested/c.txt"), b"NESTED SECRET\n").unwrap();
run(&["init", "-q"], &work);
run(&["config", "user.email", "t@t"], &work);
run(&["config", "user.name", "t"], &work);
run(&["add", "."], &work);
run(&["commit", "-qm", "init"], &work);
run(
&[
"clone",
"-q",
"--bare",
work.to_str().unwrap(),
bare.to_str().unwrap(),
],
td.path(),
);
let oid = |rev: &str| {
let out = Command::new("git")
.args(["rev-parse", rev])
.current_dir(&bare)
.output()
.unwrap();
assert!(out.status.success(), "rev-parse {rev}");
String::from_utf8_lossy(&out.stdout).trim().to_string()
};
let secret_tree = oid("HEAD:secret");
let nested_tree = oid("HEAD:secret/nested");
let secret_blob = oid("HEAD:secret/b.txt");
let public_tree = oid("HEAD:public");
let public_blob = oid("HEAD:public/a.txt");
let root_tree = oid("HEAD^{tree}");
let head = oid("HEAD");
let rules = [rule("/secret/**", &[])];
let withheld = withheld_object_oids(&bare, &rules, true, OWNER, None).unwrap();
assert!(
withheld.contains(&secret_tree),
"withheld /secret tree must be in the combined withheld set (#172)"
);
assert!(
withheld.contains(&nested_tree),
"nested withheld tree secret/nested must also be withheld"
);
assert!(
withheld.contains(&secret_blob),
"withheld blob remains withheld"
);
assert!(!withheld.contains(&root_tree), "root tree is allowed at /");
assert!(
!withheld.contains(&public_tree),
"public subtree tree is not withheld"
);
assert!(
!withheld.contains(&public_blob),
"public blob is not withheld"
);
assert!(!withheld.contains(&head), "commits are never withheld");
let candidates_out = Command::new("git")
.args(["rev-list", "--objects", "--no-object-names", "HEAD"])
.current_dir(&bare)
.output()
.unwrap();
assert!(candidates_out.status.success());
let candidates: Vec<String> = String::from_utf8_lossy(&candidates_out.stdout)
.lines()
.map(|l| l.trim().to_string())
.filter(|l| !l.is_empty())
.collect();
let replicable = replicable_objects(candidates, &withheld);
assert!(
replicable.contains(&head) && replicable.contains(&root_tree),
"commit and root tree still replicate"
);
assert!(
replicable.contains(&public_tree) && replicable.contains(&public_blob),
"public tree and blob still replicate"
);
assert!(
!replicable.contains(&secret_tree) && !replicable.contains(&secret_blob),
"secret tree and blob must drop from the replicable set (#172)"
);
assert!(
!replicable.contains(&nested_tree),
"nested withheld tree must drop too"
);
}
#[test]
fn withheld_object_set_blob_slice_matches_withheld_blob_oids() {
let (_td, bare, _s, _p) = fixture();
let rules = [rule("/secret/**", &[])];
let combined = withheld_object_oids(&bare, &rules, true, OWNER, None).unwrap();
let blobs = withheld_blob_oids(&bare, &rules, true, OWNER, None).unwrap();
let all_blobs =
crate::git::push_delta::all_blob_oids(&bare, "git", Instant::now() + WALK_TIMEOUT)
.unwrap();
let blob_slice: HashSet<String> = combined.intersection(&all_blobs).cloned().collect();
assert_eq!(
blob_slice, blobs,
"combined withheld set's blob slice must equal withheld_blob_oids (R4)"
);
}
#[test]
fn withheld_object_set_keeps_tree_shared_across_allowed_and_denied_paths() {
let td = TempDir::new().unwrap();
let work = td.path().join("work");
std::fs::create_dir_all(work.join("pub/sub")).unwrap();
std::fs::create_dir_all(work.join("sec/sub")).unwrap();
std::fs::write(work.join("pub/sub/f.txt"), b"same bytes\n").unwrap();
std::fs::write(work.join("sec/sub/f.txt"), b"same bytes\n").unwrap();
let run = |args: &[&str]| {
assert!(
Command::new("git")
.args(args)
.current_dir(&work)
.status()
.unwrap()
.success(),
"git {args:?}"
);
};
run(&["init", "-q"]);
run(&["config", "user.email", "t@t"]);
run(&["config", "user.name", "t"]);
run(&["add", "."]);
run(&["commit", "-qm", "seed"]);
let oid = |rev: &str| {
let out = Command::new("git")
.args(["rev-parse", rev])
.current_dir(&work)
.output()
.unwrap();
String::from_utf8_lossy(&out.stdout).trim().to_string()
};
let pub_sub = oid("HEAD:pub/sub");
let sec_sub = oid("HEAD:sec/sub");
assert_eq!(pub_sub, sec_sub, "identical content dedups to one tree oid");
let rules = [rule("/sec/**", &[])];
let withheld = withheld_object_oids(&work, &rules, true, OWNER, None).unwrap();
assert!(
!withheld.contains(&pub_sub),
"a tree reachable at an allowed path is not withheld even when also at a denied path"
);
}
#[test]
fn withheld_object_set_empty_with_root_only_rules() {
let (_td, bare, _s, _p) = fixture();
// A caller who has already passed the "/" gate: the listed reader of a
// root-only rule. Path-scoped withholding cannot apply, so the set is empty
// (R7, the has_path_scoped_rule short-circuit premise).
let rules = [rule("/", &["did:key:zFriend"])];
let withheld =
withheld_object_oids(&bare, &rules, true, OWNER, Some("did:key:zFriend")).unwrap();
assert!(
withheld.is_empty(),
"root-only rules cannot withhold a blob or tree from a caller who passed the / gate (R7)"
);
}
#[test]
fn withheld_object_walk_fails_closed_on_non_commit_ref() {
let (_td, bare, _s, _p) = fixture();
let secret_tree = {
let out = Command::new("git")
.args(["rev-parse", "HEAD:secret"])
.current_dir(&bare)
.output()
.unwrap();
String::from_utf8_lossy(&out.stdout).trim().to_string()
};
std::fs::write(bare.join("refs/heads/treeref"), format!("{secret_tree}\n")).unwrap();
let rules = [rule("/secret/**", &[])];
let result = withheld_object_oids(&bare, &rules, true, OWNER, None);
assert!(
result.is_err(),
"a ref that peels to a tree must fail the withheld walk closed (Err), not a partial set"
);
}
#[test]
fn allowed_set_excludes_dangling_blob_for_every_caller() {
// #126: a blob written via `git hash-object -w` but never referenced has
-164
View File
@@ -14,68 +14,6 @@ use subscription::SubscriptionRoot;
pub type GitlawbSchema = Schema<QueryRoot, MutationRoot, SubscriptionRoot>;
/// Client-facing message for GraphQL resolver failures that wrap a real
/// `sqlx::Error`. The real error is logged server-side; never put sqlx/Postgres
/// detail in the GraphQL `errors` array (#250).
///
/// Kept as its own constant on this PR's base (main still renders
/// `AppError::Db` with `e.to_string()`). If/when #247's `DB_ERROR_MESSAGE`
/// lands, fold this into that shared constant.
pub const GRAPHQL_DB_ERROR_MESSAGE: &str = "a database error occurred";
fn anyhow_has_sqlx(e: &anyhow::Error) -> bool {
e.chain().any(|c| c.downcast_ref::<sqlx::Error>().is_some())
}
/// Map an `anyhow` failure from the db layer to a GraphQL error.
///
/// - Real DB faults (`sqlx::Error` anywhere in the chain) → opaque client
/// message + `error!` log with the full `{e:#}` cause chain.
/// - Application/business errors (e.g. claim race, not-in-claimed-state) →
/// keep the actionable message; log at `warn!` so they are not mistaken for
/// infrastructure failures (#250 review).
pub(crate) fn graphql_db_err(e: anyhow::Error) -> async_graphql::Error {
if anyhow_has_sqlx(&e) {
tracing::error!(error = %format!("{e:#}"), "graphql database error");
async_graphql::Error::new(GRAPHQL_DB_ERROR_MESSAGE)
} else {
tracing::warn!(error = %format!("{e:#}"), "graphql application error");
async_graphql::Error::new(e.to_string())
}
}
/// Map an `AppError` from a shared collector (e.g. ref-update feed) to a
/// GraphQL error.
///
/// Fail closed: only explicitly curated variants surface their `Display`
/// text. Unnamed variants (including `Git`, which may embed on-disk paths)
/// render opaque so a future addition cannot leak by default (#255 review).
pub(crate) fn graphql_app_err(e: crate::error::AppError) -> async_graphql::Error {
match e {
crate::error::AppError::Db(sql) => graphql_db_err(sql.into()),
crate::error::AppError::Internal(err) => {
tracing::error!(error = %format!("{err:#}"), "graphql internal error");
async_graphql::Error::new(GRAPHQL_DB_ERROR_MESSAGE)
}
// Curated client-safe variants — `Display` is intentional API text.
safe @ (crate::error::AppError::RepoNotFound(_)
| crate::error::AppError::RepoExists(_)
| crate::error::AppError::NotFound(_)
| crate::error::AppError::Unauthorized(_)
| crate::error::AppError::Forbidden(_)
| crate::error::AppError::BadRequest(_)
| crate::error::AppError::TooManyRequests(_)
| crate::error::AppError::Incomplete(_)) => {
tracing::warn!(error = %safe, "graphql application error");
async_graphql::Error::new(safe.to_string())
}
other => {
tracing::error!(error = %other, "graphql unclassified AppError (opaque)");
async_graphql::Error::new(GRAPHQL_DB_ERROR_MESSAGE)
}
}
}
pub fn build_schema(
db: Arc<Db>,
ref_update_tx: tokio::sync::broadcast::Sender<RefUpdateBroadcast>,
@@ -87,105 +25,3 @@ pub fn build_schema(
.data(task_event_tx)
.finish()
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn graphql_db_err_opaques_sqlx_chain() {
let leak = "error returned from database: column \"is_public\" does not exist";
// Context layer must not hide sqlx from the chain walk (db helpers
// wrap with `.context(...)` in several places).
let err = graphql_db_err(
anyhow::Error::from(sqlx::Error::Protocol(leak.into())).context("loading repos"),
);
assert_eq!(err.message, GRAPHQL_DB_ERROR_MESSAGE);
assert!(!err.message.contains("is_public"));
assert!(!err.message.contains(leak));
assert!(!err.message.contains("loading repos"));
}
#[test]
fn graphql_db_err_keeps_business_message() {
let msg = "task not claimable: not found or already claimed";
let err = graphql_db_err(anyhow::anyhow!("{msg}"));
assert_eq!(err.message, msg);
}
#[test]
fn graphql_app_err_opaques_db_and_internal() {
let leak = "column \"is_public\" does not exist";
let db_err = graphql_app_err(crate::error::AppError::Db(sqlx::Error::Protocol(
leak.into(),
)));
assert_eq!(db_err.message, GRAPHQL_DB_ERROR_MESSAGE);
assert!(!db_err.message.contains("is_public"));
let internal = graphql_app_err(crate::error::AppError::Internal(anyhow::anyhow!(
"loading repo: {leak}"
)));
assert_eq!(internal.message, GRAPHQL_DB_ERROR_MESSAGE);
assert!(!internal.message.contains("is_public"));
}
#[test]
fn graphql_app_err_keeps_safe_variant_messages() {
let err = graphql_app_err(crate::error::AppError::NotFound("widget".into()));
assert!(
err.message.contains("widget"),
"safe NotFound message must reach the client: {}",
err.message
);
assert_ne!(err.message, GRAPHQL_DB_ERROR_MESSAGE);
let err = graphql_app_err(crate::error::AppError::BadRequest("bad cid".into()));
assert!(
err.message.contains("bad cid"),
"safe BadRequest message must reach the client: {}",
err.message
);
assert_ne!(err.message, GRAPHQL_DB_ERROR_MESSAGE);
}
#[test]
fn graphql_app_err_opaques_unclassified_variants() {
// `Git` may embed on-disk paths from libgit2; fail closed.
let path = "/var/lib/gitlawb/repos/owner/secret.git";
let err = graphql_app_err(crate::error::AppError::Git(format!(
"failed to open '{path}'"
)));
assert_eq!(err.message, GRAPHQL_DB_ERROR_MESSAGE);
assert!(!err.message.contains(path));
assert!(!err.message.contains("failed to open"));
}
/// Every `.map_err(` in the GraphQL query/mutation resolvers must route
/// through the opaque helpers, or discard the error (`|_|`). Same source-
/// scrape pattern as `api::authz_guard` (#255 review).
#[test]
fn every_graphql_map_err_uses_opaque_helpers() {
for (file, src) in [
("query.rs", include_str!("query.rs")),
("mutation.rs", include_str!("mutation.rs")),
] {
for (lineno, line) in src.lines().enumerate() {
let code = line.split("//").next().unwrap_or(line);
let Some(idx) = code.find(".map_err(") else {
continue;
};
let after = code[idx + ".map_err(".len()..].trim_start();
let ok = after.starts_with("crate::graphql::graphql_db_err")
|| after.starts_with("crate::graphql::graphql_app_err")
|| after.starts_with("|_|")
|| after.starts_with("|_ ");
assert!(
ok,
"{file}:{}: `.map_err(` must use graphql_db_err / graphql_app_err \
or discard (`|_|`): {line}",
lineno + 1
);
}
}
}
}
+38 -101
View File
@@ -54,7 +54,7 @@ impl MutationRoot {
};
db.create_task(&task)
.await
.map_err(crate::graphql::graphql_db_err)?;
.map_err(|e| async_graphql::Error::new(e.to_string()))?;
Ok(AgentTaskType::from(task))
}
@@ -76,7 +76,7 @@ impl MutationRoot {
let task = db
.claim_task(&id, &assignee_did)
.await
.map_err(crate::graphql::graphql_db_err)?;
.map_err(|e| async_graphql::Error::new(e.to_string()))?;
let _ = tx.send(TaskEventBroadcast {
task_id: id,
old_status: "pending".to_string(),
@@ -108,7 +108,7 @@ impl MutationRoot {
let existing = db
.get_task(&id)
.await
.map_err(crate::graphql::graphql_db_err)?
.map_err(|e| async_graphql::Error::new(e.to_string()))?
.ok_or_else(|| async_graphql::Error::new("task not found"))?;
if !crate::api::did_matches(caller, existing.assignee_did.as_deref().unwrap_or_default()) {
return Err(async_graphql::Error::new(
@@ -118,7 +118,7 @@ impl MutationRoot {
let task = db
.finish_task(&id, "completed", input.result.as_deref())
.await
.map_err(crate::graphql::graphql_db_err)?;
.map_err(|e| async_graphql::Error::new(e.to_string()))?;
let _ = tx.send(TaskEventBroadcast {
task_id: id,
old_status: "claimed".to_string(),
@@ -149,7 +149,7 @@ impl MutationRoot {
let existing = db
.get_task(&id)
.await
.map_err(crate::graphql::graphql_db_err)?
.map_err(|e| async_graphql::Error::new(e.to_string()))?
.ok_or_else(|| async_graphql::Error::new("task not found"))?;
if !crate::api::did_matches(caller, existing.assignee_did.as_deref().unwrap_or_default()) {
return Err(async_graphql::Error::new(
@@ -160,7 +160,7 @@ impl MutationRoot {
let task = db
.finish_task(&id, "failed", Some(&reason))
.await
.map_err(crate::graphql::graphql_db_err)?;
.map_err(|e| async_graphql::Error::new(e.to_string()))?;
let _ = tx.send(TaskEventBroadcast {
task_id: id,
old_status: "claimed".to_string(),
@@ -186,110 +186,47 @@ mod tests {
.join("; ")
}
/// N2: every GraphQL mutation requires a verified signer and binds the acting
/// DID to it. Unsigned is rejected; a signer other than the claimed actor is
/// N2: GraphQL mutations require a verified signer and bind the acting DID to
/// it. Unsigned is rejected; a signer other than the claimed actor is
/// rejected; the matching signer passes the auth gate.
#[sqlx::test]
async fn each_mutation_requires_and_binds_signer(pool: PgPool) {
async fn mutation_requires_and_binds_signer(pool: PgPool) {
let state = crate::test_support::test_state(pool).await;
let schema = state.graphql_schema.as_ref();
let actor = "did:key:zGQLACTORAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA";
let other = "did:key:zGQLOTHERBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB";
let cases: Vec<(&str, String)> = vec![
(
"createTask",
format!(
r#"mutation {{ createTask(delegatorDid: "{actor}", input: {{ kind: "build", capability: "repo:write" }}) {{ id }} }}"#
),
),
(
"claimTask",
format!(
r#"mutation {{ claimTask(id: "no-such-task", assigneeDid: "{actor}") {{ id }} }}"#
),
),
(
"completeTask",
format!(
r#"mutation {{ completeTask(id: "no-such-task", byDid: "{actor}", input: {{}}) {{ id }} }}"#
),
),
(
"failTask",
format!(
r#"mutation {{ failTask(id: "no-such-task", byDid: "{actor}", input: {{}}) {{ id }} }}"#
),
),
];
for (name, q) in &cases {
let resp = schema.execute(Request::new(q.as_str())).await;
assert!(
errors(&resp).contains("authentication required"),
"{name}: unsigned mutation must be rejected: {}",
errors(&resp)
);
let resp = schema
.execute(Request::new(q.as_str()).data(AuthenticatedDid(other.into())))
.await;
assert!(
errors(&resp).contains("authenticated signer"),
"{name}: DID mismatch must be rejected: {}",
errors(&resp)
);
let resp = schema
.execute(Request::new(q.as_str()).data(AuthenticatedDid(actor.into())))
.await;
let errs = errors(&resp);
assert!(
!errs.contains("authentication required") && !errs.contains("authenticated signer"),
"{name}: matching signer must pass the auth gate: {errs}"
);
if *name == "claimTask" {
assert!(
errs.contains("task not claimable"),
"claim race / missing task must keep its business message: {errs}"
);
assert!(
!errs.contains(crate::graphql::GRAPHQL_DB_ERROR_MESSAGE),
"business error must not be rewritten as opaque DB error: {errs}"
);
}
}
}
/// #250: mutation DB faults must be opaque; create_task hits agent_tasks.
#[sqlx::test]
async fn create_task_db_error_message_is_opaque(pool: PgPool) {
let state = crate::test_support::test_state(pool.clone()).await;
sqlx::query("ALTER TABLE agent_tasks DROP COLUMN status")
.execute(&pool)
.await
.unwrap();
let delegator = "did:key:zGQLDELEGATORAAAAAAAAAAAAAAAAAAAAAAAAAA";
let assignee = "did:key:zASSIGNEEAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA";
let q = format!(
r#"mutation {{
createTask(
delegatorDid: "{delegator}",
input: {{ kind: "build", capability: "repo:write" }}
) {{ id }}
}}"#
r#"mutation {{ claimTask(id: "no-such-task", assigneeDid: "{assignee}") {{ id }} }}"#
);
let resp = state
.graphql_schema
.execute(Request::new(&q).data(AuthenticatedDid(delegator.into())))
// 1. Unsigned → rejected before any DB work.
let resp = schema.execute(Request::new(&q)).await;
assert!(
errors(&resp).contains("authentication required"),
"unsigned mutation must be rejected: {}",
errors(&resp)
);
// 2. Signed as someone other than the claimed assignee → rejected.
let resp = schema
.execute(Request::new(&q).data(AuthenticatedDid(
"did:key:zOTHERBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB".into(),
)))
.await;
assert!(
errors(&resp).contains("authenticated signer"),
"DID mismatch must be rejected: {}",
errors(&resp)
);
// 3. Signed as the claimed assignee → passes the auth gate (any remaining
// error is the missing task, not an auth error).
let resp = schema
.execute(Request::new(&q).data(AuthenticatedDid(assignee.into())))
.await;
let errs = errors(&resp);
assert!(
errs.contains(crate::graphql::GRAPHQL_DB_ERROR_MESSAGE),
"sqlx fault must be opaque: {errs}"
);
assert!(
!errs.contains("column") && !errs.contains("status"),
"schema text leaked: {errs}"
!errs.contains("authentication required") && !errs.contains("authenticated signer"),
"matching signer must pass the auth gate: {errs}"
);
}
+11 -130
View File
@@ -14,7 +14,7 @@ impl QueryRoot {
let repos = db
.list_all_repos_deduped()
.await
.map_err(crate::graphql::graphql_db_err)?;
.map_err(|e| async_graphql::Error::new(e.to_string()))?;
// Apply the same "/" visibility gate the REST/per-repo endpoints use so
// this surface does not enumerate private repos (#97). The caller DID is
@@ -27,7 +27,7 @@ impl QueryRoot {
let rules_by_repo = db
.list_visibility_rules_for_repos(&ids)
.await
.map_err(crate::graphql::graphql_db_err)?;
.map_err(|e| async_graphql::Error::new(e.to_string()))?;
Ok(repos
.into_iter()
@@ -71,27 +71,11 @@ impl QueryRoot {
let updates =
crate::api::events::collect_visible_ref_updates(db, repo.as_deref(), limit, caller)
.await
.map_err(crate::graphql::graphql_app_err)?;
.map_err(|e| async_graphql::Error::new(e.to_string()))?;
// Resolve the trusted display owner_did per row, identical to the REST
// feed: the stored wire value is untrusted, so it is echoed only when it
// matches the canonical owner of the local repo the slug names (#P1);
// legacy None rows are attributed via an exact unique local match (#P3).
// The batch resolver issues at most one query per distinct local repo
// rather than one per event row (#P2).
let pairs: Vec<(&str, Option<&str>)> = updates
.iter()
.map(|u| (u.repo.as_str(), u.owner_did.as_deref()))
.collect();
let owner_dids = db
.resolve_ref_update_owner_dids(&pairs)
.await
.map_err(crate::graphql::graphql_db_err)?;
let resolved: Vec<RefUpdateType> = updates
Ok(updates
.into_iter()
.zip(owner_dids)
.map(|(u, owner_did)| RefUpdateType {
.map(|u| RefUpdateType {
repo: u.repo,
ref_name: u.ref_name,
old_sha: u.old_sha,
@@ -99,10 +83,8 @@ impl QueryRoot {
pusher_did: u.pusher_did,
node_did: u.node_did,
timestamp: u.timestamp,
owner_did,
})
.collect();
Ok(resolved)
.collect())
}
async fn tasks(
@@ -110,21 +92,13 @@ impl QueryRoot {
ctx: &Context<'_>,
status: Option<String>,
assignee_did: Option<String>,
#[graphql(
default = 50,
desc = "Max 200; larger requests are clamped to 200 (no error). Negative values clamp to 0."
)]
limit: i64,
#[graphql(default = 50)] limit: i64,
) -> Result<Vec<AgentTaskType>> {
let db = ctx.data_unchecked::<Arc<Db>>();
// Clamp before SQL: a negative LIMIT is a client fault that Postgres
// rejects with 2201W, which would otherwise trip the opaque DB path
// and write an error-level log on every probe (#250 review).
let limit = limit.clamp(0, 200);
let tasks = db
.list_tasks(status.as_deref(), assignee_did.as_deref(), limit)
.await
.map_err(crate::graphql::graphql_db_err)?;
.map_err(|e| async_graphql::Error::new(e.to_string()))?;
Ok(tasks.into_iter().map(AgentTaskType::from).collect())
}
@@ -133,7 +107,7 @@ impl QueryRoot {
let t = db
.get_task(&id)
.await
.map_err(crate::graphql::graphql_db_err)?;
.map_err(|e| async_graphql::Error::new(e.to_string()))?;
Ok(t.map(AgentTaskType::from))
}
}
@@ -188,7 +162,6 @@ mod tests {
cert_id: None,
received_at: Utc::now().to_rfc3339(),
from_peer: "peer1".into(),
owner_did: None,
}
}
@@ -272,10 +245,10 @@ mod tests {
.await
.unwrap();
let schema = schema(db);
let q = r#"{ refUpdates { repo refName ownerDid } }"#;
let q = r#"{ refUpdates { repo refName } }"#;
let resp = anon(&schema, q).await;
assert_eq!(count(&resp), 1);
// The one row returned must be the public repo's with owner_did echoed.
// The one row returned must be the public repo's.
let async_graphql::Value::Object(obj) = &resp.data else {
unreachable!()
};
@@ -289,11 +262,6 @@ mod tests {
row.get("repo").unwrap(),
&async_graphql::Value::from("z6MkOwner/openrepo")
);
assert_eq!(
row.get("ownerDid").unwrap(),
&async_graphql::Value::from(OWNER),
"ownerDid must fall back to the local record's owner for legacy rows"
);
}
// Scenario 4 — alias fail-closed: private repo's row stored full-DID form.
@@ -427,91 +395,4 @@ mod tests {
let q = r#"{ refUpdates { repo } }"#;
assert_eq!(count(&authed(&schema, q, "did:key:z6MkQuar").await), 0);
}
/// #250: anonymous GraphQL query DB failures must not leak sqlx/schema text.
#[sqlx::test]
async fn repos_query_db_error_message_is_opaque(pool: PgPool) {
let db = db(pool.clone()).await;
db.create_repo(&repo("r1", OWNER, "widget", true))
.await
.unwrap();
sqlx::query("ALTER TABLE repos DROP COLUMN is_public")
.execute(&pool)
.await
.unwrap();
let schema = schema(db);
let resp = anon(&schema, "{ repos { name ownerDid } }").await;
assert!(
!resp.errors.is_empty(),
"DB failure must surface as a GraphQL error"
);
for err in &resp.errors {
assert_eq!(
err.message,
crate::graphql::GRAPHQL_DB_ERROR_MESSAGE,
"raw DB detail leaked into GraphQL error: {}",
err.message
);
assert!(
!err.message.contains("is_public") && !err.message.contains("column"),
"schema text leaked: {}",
err.message
);
}
}
/// #250: negative tasks(limit) must not hit Postgres (and must not 500-log).
#[sqlx::test]
async fn tasks_negative_limit_clamped(pool: PgPool) {
let db = db(pool).await;
let schema = schema(db);
let resp = anon(&schema, "{ tasks(limit: -1) { id } }").await;
assert!(
resp.errors.is_empty(),
"negative limit must clamp, not fail: {:?}",
resp.errors
);
assert_eq!(count_tasks(&resp), 0);
}
/// #255: ceiling of the tasks(limit) clamp must be held (schema promises Max 200).
#[sqlx::test]
async fn tasks_limit_ceiling_clamped_to_200(pool: PgPool) {
let db = db(pool).await;
let now = Utc::now().to_rfc3339();
for i in 0..201 {
db.create_task(&crate::db::AgentTask {
id: format!("task-ceil-{i}"),
repo_id: None,
kind: "build".into(),
status: "pending".into(),
delegator_did: OWNER.into(),
assignee_did: None,
capability: "repo:write".into(),
ucan_token: None,
payload: None,
result: None,
created_at: now.clone(),
updated_at: now.clone(),
deadline: None,
})
.await
.unwrap();
}
let schema = schema(db);
let resp = anon(&schema, "{ tasks(limit: 5000) { id } }").await;
assert_eq!(count_tasks(&resp), 200, "limit above 200 must clamp to 200");
}
fn count_tasks(resp: &async_graphql::Response) -> usize {
assert!(resp.errors.is_empty(), "graphql errors: {:?}", resp.errors);
let async_graphql::Value::Object(obj) = &resp.data else {
panic!("data not an object: {:?}", resp.data);
};
let async_graphql::Value::List(rows) = obj.get("tasks").expect("tasks key") else {
panic!("tasks not a list");
};
rows.len()
}
}
@@ -39,7 +39,6 @@ impl SubscriptionRoot {
pusher_did: ev.pusher_did,
node_did: ev.node_did,
timestamp: ev.timestamp,
owner_did: Some(ev.owner_did),
}),
_ => None,
}
-1
View File
@@ -57,7 +57,6 @@ pub struct RefUpdateType {
pub pusher_did: String,
pub node_did: String,
pub timestamp: String,
pub owner_did: Option<String>,
}
#[derive(SimpleObject, Clone)]
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
+13 -177
View File
@@ -1,4 +1,4 @@
//! Prometheus metrics for the Twigpine node.
//! Prometheus metrics for the gitlawb node.
//!
//! Exposes a small, deliberately conservative set of counters and histograms
//! that cover the questions an operator actually asks of a live node:
@@ -11,11 +11,6 @@
//! `gitlawb_sync_queue_processed_total{status}`
//! * are webhooks reaching their endpoints? —
//! `gitlawb_webhook_deliveries_total{result}`
//! * is inbound gossip being admitted or shed, and for which reason?
//! `gitlawb_gossip_ingest_events_total{outcome}`
//! * is the gossip replay seen-set full, and therefore admitting events it
//! cannot deduplicate?
//! `gitlawb_gossip_replay_guard_saturated_total`
//! * how big are the packs we're sending and receiving? —
//! `gitlawb_pack_size_bytes`
//! * a single `gitlawb_info{version, did}` gauge = 1, for joins/dashboards
@@ -38,8 +33,8 @@
use std::sync::OnceLock;
use prometheus::{
Encoder, Histogram, HistogramOpts, IntCounter, IntCounterVec, IntGauge, IntGaugeVec, Opts,
Registry, TextEncoder,
Encoder, Histogram, HistogramOpts, IntCounterVec, IntGauge, IntGaugeVec, Opts, Registry,
TextEncoder,
};
/// The single, process-wide metrics registry. Initialized by [`init`].
@@ -54,30 +49,18 @@ static AUTH_SUCCESSES: OnceLock<IntCounterVec> = OnceLock::new();
static AUTH_FAILURES: OnceLock<IntCounterVec> = OnceLock::new();
static SYNC_PROCESSED: OnceLock<IntCounterVec> = OnceLock::new();
static WEBHOOK_DELIVERIES: OnceLock<IntCounterVec> = OnceLock::new();
static GOSSIP_INGEST: OnceLock<IntCounterVec> = OnceLock::new();
static GOSSIP_REPLAY_GUARD_SATURATED: OnceLock<IntCounter> = OnceLock::new();
static PACK_SIZE: OnceLock<Histogram> = OnceLock::new();
static PEERS_CONNECTED: OnceLock<IntGauge> = OnceLock::new();
/// 1 when the node started without its p2p identity because the key storage
/// could not be used. The node keeps serving HTTP in that state and /health
/// stays green, so without this the outage is visible only to whoever reads
/// the startup log.
static P2P_KEY_LOAD_FAILED: OnceLock<IntGauge> = OnceLock::new();
/// One-time initializer. Builds the registry, registers every metric,
/// and sets the constant `gitlawb_info` gauge. Idempotent — calling
/// more than once is a silent no-op. MUST be called from `main()` after
/// the node DID is known.
pub fn init(version: &str, node_did: &str) {
// Guard with Once so two concurrent callers cannot both pass an unsynchronized
// `REGISTRY.get()` check and then race on `OnceLock::set(...).expect(...)`,
// panicking the loser (#192 F4). Once runs the body exactly once, so the
// `.expect("set X once")` calls below can never observe an already-set slot.
static INIT: std::sync::Once = std::sync::Once::new();
INIT.call_once(|| init_inner(version, node_did));
}
if REGISTRY.get().is_some() {
return;
}
fn init_inner(version: &str, node_did: &str) {
let registry = Registry::new();
let info = IntGaugeVec::new(
@@ -180,33 +163,6 @@ fn init_inner(version: &str, node_did: &str) {
.set(webhook_deliveries)
.expect("set WEBHOOK_DELIVERIES once");
let gossip_ingest = IntCounterVec::new(
Opts::new(
"gitlawb_gossip_ingest_events_total",
"Total inbound gossip ref-update events and what the ingest path decided about each",
),
&["outcome"],
)
.expect("gitlawb_gossip_ingest_events_total definition");
registry
.register(Box::new(gossip_ingest.clone()))
.expect("register gitlawb_gossip_ingest_events_total");
GOSSIP_INGEST
.set(gossip_ingest)
.expect("set GOSSIP_INGEST once");
let gossip_replay_guard_saturated = IntCounter::with_opts(Opts::new(
"gitlawb_gossip_replay_guard_saturated_total",
"Inbound gossip ref-updates admitted without being recorded because the replay seen-set was at capacity",
))
.expect("gitlawb_gossip_replay_guard_saturated_total definition");
registry
.register(Box::new(gossip_replay_guard_saturated.clone()))
.expect("register gitlawb_gossip_replay_guard_saturated_total");
GOSSIP_REPLAY_GUARD_SATURATED
.set(gossip_replay_guard_saturated)
.expect("set GOSSIP_REPLAY_GUARD_SATURATED once");
let pack_size = Histogram::with_opts(
HistogramOpts::new(
"gitlawb_pack_size_bytes",
@@ -241,18 +197,6 @@ fn init_inner(version: &str, node_did: &str) {
.set(peers_connected)
.expect("set PEERS_CONNECTED once");
let p2p_key_load_failed = IntGauge::with_opts(Opts::new(
"gitlawb_p2p_identity_key_load_failed",
"1 if the node is running without p2p because its identity key could not be loaded",
))
.expect("gitlawb_p2p_identity_key_load_failed definition");
registry
.register(Box::new(p2p_key_load_failed.clone()))
.expect("register gitlawb_p2p_identity_key_load_failed");
P2P_KEY_LOAD_FAILED
.set(p2p_key_load_failed)
.expect("set P2P_KEY_LOAD_FAILED once");
REGISTRY
.set(registry)
.expect("set REGISTRY once (init must be called exactly once)");
@@ -274,17 +218,6 @@ pub fn record_fetch(repo: &str) {
}
}
/// Test-only: current `gitlawb_fetches_total` value for `repo` (0 if the registry
/// is not initialized). Lets api-layer tests assert the completed-fetch count with
/// a unique label instead of scraping the encoded text.
#[cfg(test)]
pub fn fetch_count_for_test(repo: &str) -> u64 {
FETCHES
.get()
.map(|c| c.with_label_values(&[repo]).get())
.unwrap_or(0)
}
/// Record one HTTP signature check that passed.
#[allow(dead_code)] // wired in a follow-up; helpers are part of the public metrics surface
pub fn record_auth_success(route: &str) {
@@ -302,11 +235,7 @@ pub fn record_auth_failure(route: &str, reason: &str) {
}
}
/// Record one sync_queue item outcome. `status` ∈ {done, failed, skipped,
/// rejected, deferred}. `rejected` is terminal and caused by a guard (a
/// malformed slug, or a path resolving outside repos_dir); `deferred` leaves
/// the row pending for a later pass, so a queue stalled on an operator
/// condition is visible rather than looking idle.
/// Record one sync_queue item outcome. `status` ∈ {done, failed, skipped}.
pub fn record_sync_processed(status: &str) {
if let Some(c) = SYNC_PROCESSED.get() {
c.with_label_values(&[status]).inc();
@@ -314,75 +243,13 @@ pub fn record_sync_processed(status: &str) {
}
/// Record one webhook delivery attempt outcome. `result` ∈
/// {ok, http_error, network_error, skipped, invalid_url}.
/// `invalid_url` is a policy skip: the stored hook URL failed the public-URL
/// check at delivery time and the POST was never sent.
/// {ok, http_error, network_error, skipped}.
pub fn record_webhook_delivery(result: &str) {
if let Some(c) = WEBHOOK_DELIVERIES.get() {
c.with_label_values(&[result]).inc();
}
}
/// Record what the gossip ingest path decided about one inbound ref-update.
/// `outcome` ∈ {accepted, unsigned_admitted, write_failed, rejected,
/// source_rate_limited, author_rate_limited, unsigned_source_rate_limited,
/// replayed, stale_timestamp}.
///
/// `accepted` is reserved for signature-verified events. An unsigned event that
/// survives the rolling-upgrade window is `unsigned_admitted`, so the
/// authenticated-admission rate is not silently padded by legacy compatibility
/// traffic; an operator can tell whether the fleet still relies on the
/// compatibility allowance.
///
/// The three shed reasons stay separate labels rather than one `rate_limited`
/// because they answer different operator questions: `source_rate_limited` is a
/// forwarding peer over the pre-parse brake, `unsigned_source_rate_limited` is
/// that same forwarder relaying unproven traffic, and `author_rate_limited`
/// names a principal a signature actually proved. Collapsing them would make a
/// flood of unauthenticated garbage indistinguishable from one registered peer
/// pushing too hard, which is exactly the distinction an alert needs.
///
/// The label is the variant, never the free-form reason string that
/// `Rejected`/`WriteFailed` carry: those are attacker-influenced and would blow
/// up the label cardinality of a process-wide registry.
pub fn record_gossip_ingest(outcome: &str) {
if let Some(c) = GOSSIP_INGEST.get() {
c.with_label_values(&[outcome]).inc();
}
}
/// Record one inbound gossip ref-update that the replay seen-set could not
/// record because it was at capacity.
///
/// Label-less on purpose: there is one degradation mode and nothing about it
/// varies per event, so a label would only add cardinality.
///
/// This is an alert-worthy signal, not a debug counter, and it is the ONLY way
/// the degraded state is visible. At capacity the guard fails open, which means
/// the event is admitted and `gitlawb_gossip_ingest_events_total{outcome=
/// "accepted"}` counts it exactly as it counts a healthy admission. Fail-open is
/// the right policy (a saturated set that dropped all fresh gossip would convert
/// a loud resource attack into quiet mesh-wide censorship), but it is not a free
/// one: while this counter is moving, a captured signature can again be replayed
/// against the freshness window's full 10 minutes, spending a `peer_exists`
/// round trip and a debit from the victim author's budget on every replay.
pub fn record_gossip_replay_guard_saturated() {
if let Some(c) = GOSSIP_REPLAY_GUARD_SATURATED.get() {
c.inc();
}
}
/// Test-only: current `gitlawb_gossip_replay_guard_saturated_total` value (0 if
/// the registry is not initialized). Lets the seen-set tests assert the
/// saturation signal fires without scraping the encoded text.
#[cfg(test)]
pub fn replay_guard_saturated_count_for_test() -> u64 {
GOSSIP_REPLAY_GUARD_SATURATED
.get()
.map(|c| c.get())
.unwrap_or(0)
}
/// Record a pack body size observation (bytes).
pub fn observe_pack_size(bytes: f64) {
if let Some(h) = PACK_SIZE.get() {
@@ -397,13 +264,6 @@ pub fn set_peers_connected(count: i64) {
}
}
/// Record whether the node is running without its p2p identity.
pub fn set_p2p_key_load_failed(failed: bool) {
if let Some(g) = P2P_KEY_LOAD_FAILED.get() {
g.set(i64::from(failed));
}
}
/// Encode the registry as the standard Prometheus text exposition format.
/// Returns an error if `init` was never called.
pub fn encode() -> Result<String, prometheus::Error> {
@@ -422,11 +282,11 @@ pub fn encode() -> Result<String, prometheus::Error> {
mod tests {
use super::*;
// Note: cargo runs tests in a binary in parallel via threads, and several
// tests here call init(). init() is guarded by std::sync::Once, so concurrent
// callers run the body exactly once and never race on the OnceLock setters
// (#192 F4). The encode test below is structured so it's safe to run alongside
// other tests in the same binary regardless of ordering.
// Note: these tests are not run in parallel by default (cargo runs
// tests in a binary in parallel via threads but the OnceLock guard
// means only the first init() call succeeds; subsequent ones are
// no-ops). The encode test below is structured so it's safe to run
// alongside other tests in the same binary.
#[test]
fn encode_after_init_returns_prometheus_text() {
@@ -457,24 +317,6 @@ mod tests {
);
}
/// #192 F4: `init` is idempotent and safe to call repeatedly. The panic that
/// hit concurrent callers (both passing the old unsynchronized `REGISTRY.get()`
/// check, then racing on `OnceLock::set(...).expect(...)`) is fixed by the
/// `Once` guard. That race is inherently non-deterministic to assert in the
/// shared-binary suite — a 32-thread barrier stress-test reproduces it only in
/// isolation (verified: RED before the `Once` fix, GREEN after). This guard
/// covers the deterministic half of the contract: a repeat call is a no-op,
/// never a panic, and the registry stays usable.
#[test]
fn init_is_idempotent_no_panic_on_repeat() {
init("0.0.0-a", "did:key:a");
init("0.0.0-b", "did:key:b");
assert!(
encode().is_ok(),
"registry must stay usable after a repeated init"
);
}
#[test]
fn record_helpers_are_noops_before_init() {
// These don't panic. They also don't show up in encode() output
@@ -485,12 +327,6 @@ mod tests {
record_auth_failure("test/route", "test_reason");
record_sync_processed("done");
record_webhook_delivery("ok");
// `record_gossip_replay_guard_saturated` is deliberately NOT called
// here. It is a label-less process-wide counter and the seen-set test
// that asserts it asserts a before/after delta, so an increment from a
// second test running concurrently in this binary would be a flake with
// no diagnostic value.
record_gossip_ingest("accepted");
observe_pack_size(1024.0);
set_peers_connected(0);
}
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
+1 -33
View File
@@ -143,23 +143,6 @@ impl RateLimiter {
false
}
/// The configured key ceiling. Tests use it to assert that a limiter a
/// production factory builds is actually bounded, which flooding the real
/// ceiling in distinct keys would be far too slow to show.
#[cfg(test)]
pub fn max_keys(&self) -> usize {
self.max_keys
}
/// The configured per-key request cap. Tests use it to assert that a
/// limiter a production factory builds carries the cap it is documented
/// with; re-asserting the constant on its own cannot tell whether the
/// factory passed that constant or a different one.
#[cfg(test)]
pub fn max_requests(&self) -> usize {
self.max_requests
}
pub async fn cleanup(&self) {
let now = Instant::now();
let mut state = self.state.lock().await;
@@ -171,28 +154,13 @@ impl RateLimiter {
}
/// Number of distinct keys currently tracked. Test-only introspection so a
/// cross-module test can assert that a sweep actually evicted expired entries
/// and observe what it reclaimed. There is no production reader.
/// cross-module test can assert that a sweep actually evicted expired entries.
#[cfg(test)]
pub(crate) async fn tracked_keys(&self) -> usize {
self.state.lock().await.len()
}
}
/// Per-source concurrency cap derived from the write-pool size: one resolved client
/// key (see [`client_key`]) may hold at most an eighth of the pool, so saturating it
/// takes ~8 distinct keys. Real for an IPv4 or single-address caller; a caller with a
/// routed IPv6 /64 has 2^64 keys, because `client_key` returns the full address with
/// no prefix folding. Narrowing that keying is a deferred decision, so this cap is a
/// bound per key, not a bound per operator.
///
/// Floored at 1 because the value feeds [`PerCallerConcurrency`], where a cap of 0
/// would shed EVERY receive-pack advertisement and break all pushes. The floor is
/// load-bearing at the minimum write-pool size (1), which integer-divides to 0.
pub(crate) fn per_source_push_cap(max_concurrent_git_pushes: usize) -> usize {
(max_concurrent_git_pushes / 8).max(1)
}
/// A bounded per-caller CONCURRENCY limiter — distinct from [`RateLimiter`], which
/// caps request RATE. Each caller key may hold at most `per_caller` in-flight
/// permits at once; beyond that [`try_acquire`](Self::try_acquire) returns `None`
+22 -766
View File
@@ -42,37 +42,17 @@ async fn graphql_playground() -> impl IntoResponse {
))
}
/// Pre-auth body-buffer ceiling for every signed route group except the git write
/// routes. Deliberately equal to axum's `DefaultBodyLimit` (2 MB), which is what
/// these groups' `Bytes`/`Json` extractors already enforce: that default works
/// through a request extension the extractors consult, and `require_signature`
/// collects the raw body itself, so it never saw the cap. Matching the number
/// keeps this change behavior-preserving on the response path (nothing that used
/// to be accepted starts failing) and purely a memory fix.
pub const SIGNED_BODY_LIMIT: usize = 2 * 1024 * 1024;
/// Applies the standard auth middleware pair to a router: HTTP Signature verification
/// followed by UCAN chain validation. The two layers run in this order for every
/// matched request: `require_signature` first (sets `AuthenticatedDid`), then
/// `require_ucan_chain` (reads it).
///
/// `body_limit` bounds the request body `require_signature` buffers before it has
/// authenticated anything (see [`auth::SignatureBodyLimit`]): `max_pack_bytes` for
/// the git write group, [`SIGNED_BODY_LIMIT`] for every other group.
fn add_auth_layers(
router: Router<AppState>,
state: AppState,
body_limit: usize,
) -> Router<AppState> {
fn add_auth_layers(router: Router<AppState>, state: AppState) -> Router<AppState> {
router
.layer(middleware::from_fn_with_state(
state,
auth::require_ucan_chain,
))
.layer(middleware::from_fn_with_state(
auth::SignatureBodyLimit(body_limit),
auth::require_signature,
))
.layer(middleware::from_fn(auth::require_signature))
}
pub fn build_router(state: AppState) -> Router {
@@ -83,10 +63,7 @@ pub fn build_router(state: AppState) -> Router {
// Attach the verified DID to /graphql when a signature is present. The
// layer covers only routes added before it, so /graphql/ws (added after,
// read-only subscriptions) stays open.
.layer(middleware::from_fn_with_state(
auth::SignatureBodyLimit(SIGNED_BODY_LIMIT),
auth::optional_signature,
))
.layer(middleware::from_fn(auth::optional_signature))
.route_service("/graphql/ws", GraphQLSubscription::new(schema));
// ── Task routes (write — require HTTP Signature) ───────────────────────
@@ -97,7 +74,6 @@ pub fn build_router(state: AppState) -> Router {
.route("/api/v1/tasks/{id}/complete", post(tasks::complete_task))
.route("/api/v1/tasks/{id}/fail", post(tasks::fail_task)),
state.clone(),
SIGNED_BODY_LIMIT,
);
// ── Task routes (read — open) ──────────────────────────────────────────
@@ -131,7 +107,6 @@ pub fn build_router(state: AppState) -> Router {
.layer(middleware::from_fn(rate_limit::rate_limit_by_did))
.layer(axum::Extension(limiter)),
state.clone(),
SIGNED_BODY_LIMIT,
)
.layer(middleware::from_fn(rate_limit::rate_limit_by_ip))
.layer(axum::Extension(create_ip_limiter));
@@ -206,7 +181,6 @@ pub fn build_router(state: AppState) -> Router {
axum::routing::delete(agents::deregister_agent),
),
state.clone(),
SIGNED_BODY_LIMIT,
);
// Body limit is raised to GITLAWB_MAX_PACK_BYTES (default 2 GB) for git
@@ -231,68 +205,33 @@ pub fn build_router(state: AppState) -> Router {
.layer(DefaultBodyLimit::disable())
.layer(RequestBodyLimitLayer::new(pack_limit)),
state.clone(),
pack_limit,
)
.layer(middleware::from_fn(rate_limit::rate_limit_by_ip))
.layer(axum::Extension(push_limiter));
// ── IPFS content-addressed retrieval and pin listing ──────────────────
// Two independent sub-routers, then merged. They share a URL prefix family
// but have separate rate-limit policies and must not share a bucket.
//
// `/ipfs/{cid}` (CID resolver): carries `optional_signature` so `get_by_cid`
// sees the caller identity and can apply per-repo visibility (#110); anon
// callers stay anonymous and still read genuinely public content. The
// per-IP flood brake is layered on because the resolver is anon-reachable
// and each request can drive a full-history git walk — the brake is the
// outermost layer (rejects a flood before the walk-admission work), mirroring
// the push/create routers. The `IpRateLimiter` extension MUST be attached
// or `rate_limit_by_ip` is a silent no-op.
//
// `/api/v1/ipfs/pins` (pin listing): now carries `optional_signature` only
// (#121). The handler rejects requests without a verified `AuthenticatedDid`
// with 401. It does NOT carry the CID flood brake — `list_pins` is a single
// `list_pinned_cids()` call, no walk, and routing pins through the resolver's
// bucket would let `/ipfs/{cid}` traffic exhaust the bucket and 429 the
// pins endpoint, or let signed pin polling exhaust the bucket for legitimate
// CID reads. The two surfaces have separate availability contracts.
//
// Both sub-routers are built first with their own layer sets, then merged.
// This is the structure the prior routing guidance called for and the
// earlier 429 test for `/ipfs/{cid}` (test_support.rs) assumes.
// `/ipfs/{cid}` carries `optional_signature` so `get_by_cid` sees the caller
// identity and can apply per-repo visibility (#110); anonymous callers stay
// anonymous and still read genuinely public content. `/api/v1/ipfs/pins`
// stays unsigned — gating the pin index is tracked separately (#121).
// `/ipfs/{cid}` also carries a per-IP flood brake: it is anon-reachable and each
// request can drive a full-history git walk, so the per-IP rate limiter is the
// outermost layer (rejects a flood before the walk-admission work), mirroring the
// push/create routers. The extension MUST be attached or rate_limit_by_ip is a
// silent no-op. `/api/v1/ipfs/pins` (no walk) is merged in unbraked, as before.
let ipfs_limiter = rate_limit::IpRateLimiter {
limiter: state.ipfs_rate_limiter.clone(),
trust: state.push_limiter_trust,
};
let ipfs_cid_routes = Router::new()
let ipfs_routes = Router::new()
.route("/ipfs/{cid}", get(ipfs::get_by_cid))
.layer(middleware::from_fn_with_state(
auth::SignatureBodyLimit(SIGNED_BODY_LIMIT),
auth::optional_signature,
))
.layer(middleware::from_fn(auth::optional_signature))
.layer(middleware::from_fn(rate_limit::rate_limit_by_ip))
.layer(axum::Extension(ipfs_limiter));
let ipfs_pins_routes = Router::new()
.route("/api/v1/ipfs/pins", get(ipfs::list_pins))
.layer(middleware::from_fn_with_state(
auth::SignatureBodyLimit(SIGNED_BODY_LIMIT),
auth::optional_signature,
));
let ipfs_routes = ipfs_cid_routes.merge(ipfs_pins_routes);
.layer(axum::Extension(ipfs_limiter))
.merge(Router::new().route("/api/v1/ipfs/pins", get(ipfs::list_pins)));
// ── Arweave permanent anchors ──────────────────────────────────────────
// `list_anchors` rejects callers without a verified `AuthenticatedDid`, so
// unsigned enumeration is denied. The same `optional_signature` layer used
// on the other read surfaces is applied here, closing the anonymous
// anchor-listing gap alongside `/api/v1/ipfs/pins`. The verify endpoint
// and its probe are deferred to the vertical slice with the uploader and
// a real provider contract.
let arweave_routes = Router::new()
.route("/api/v1/arweave/anchors", get(arweave::list_anchors))
.layer(middleware::from_fn_with_state(
auth::SignatureBodyLimit(SIGNED_BODY_LIMIT),
auth::optional_signature,
));
let arweave_routes = Router::new().route("/api/v1/arweave/anchors", get(arweave::list_anchors));
// ── Bounty routes (write — require HTTP Signature) ─────────────────
let bounty_write_routes = add_auth_layers(
@@ -319,7 +258,6 @@ pub fn build_router(state: AppState) -> Router {
post(bounties::dispute_bounty),
),
state.clone(),
SIGNED_BODY_LIMIT,
);
// ── Bounty routes (read — open) ──────────────────────────────────────
@@ -335,16 +273,12 @@ pub fn build_router(state: AppState) -> Router {
"/api/v1/agents/{did}/bounties",
get(bounties::agent_bounty_stats),
)
.layer(middleware::from_fn_with_state(
auth::SignatureBodyLimit(SIGNED_BODY_LIMIT),
auth::optional_signature,
));
.layer(middleware::from_fn(auth::optional_signature));
// ── Profile routes (write — require HTTP Signature) ─────────────────
let profile_write_routes = add_auth_layers(
Router::new().route("/api/v1/profile", axum::routing::put(profiles::set_profile)),
state.clone(),
SIGNED_BODY_LIMIT,
);
// ── Issue routes (write — require HTTP Signature, no rate limit) ─────
@@ -359,7 +293,6 @@ pub fn build_router(state: AppState) -> Router {
post(issues::create_issue_comment),
),
state.clone(),
SIGNED_BODY_LIMIT,
);
// ── Peer discovery routes ─────────────────────────────────────────────
@@ -378,7 +311,6 @@ pub fn build_router(state: AppState) -> Router {
let sync_trigger_routes = add_auth_layers(
Router::new().route("/api/v1/sync/trigger", post(peers::trigger_sync)),
state.clone(),
SIGNED_BODY_LIMIT,
)
.layer(middleware::from_fn(rate_limit::rate_limit_by_ip))
.layer(axum::Extension(rate_limit::IpRateLimiter {
@@ -395,12 +327,9 @@ pub fn build_router(state: AppState) -> Router {
.route("/api/v1/peers/announce", post(peers::announce))
.route("/api/v1/sync/notify", post(peers::notify_sync));
peer_write_routes = if state.config.require_signed_peer_writes {
add_auth_layers(peer_write_routes, state.clone(), SIGNED_BODY_LIMIT)
add_auth_layers(peer_write_routes, state.clone())
} else {
peer_write_routes.layer(middleware::from_fn_with_state(
auth::SignatureBodyLimit(SIGNED_BODY_LIMIT),
auth::optional_signature,
))
peer_write_routes.layer(middleware::from_fn(auth::optional_signature))
};
let peer_write_routes = peer_write_routes
.layer(middleware::from_fn(rate_limit::rate_limit_by_ip))
@@ -499,10 +428,7 @@ pub fn build_router(state: AppState) -> Router {
"/api/v1/repos/{owner}/{repo}/replicas",
get(replicas::list_replicas),
)
.layer(middleware::from_fn_with_state(
auth::SignatureBodyLimit(SIGNED_BODY_LIMIT),
auth::optional_signature,
));
.layer(middleware::from_fn(auth::optional_signature));
// git-upload-pack (clone/fetch) — same raised body limit as receive-pack so
// large pack responses from the server don't get truncated on the client side.
@@ -534,13 +460,7 @@ pub fn build_router(state: AppState) -> Router {
)
.layer(DefaultBodyLimit::disable())
.layer(RequestBodyLimitLayer::new(pack_limit))
// `pack_limit`, not SIGNED_BODY_LIMIT: git-upload-pack POSTs live here and
// the group has deliberately raised its ceiling, so the pre-auth buffer for
// a signed fetch negotiation must match rather than narrow it.
.layer(middleware::from_fn_with_state(
auth::SignatureBodyLimit(pack_limit),
auth::optional_signature,
));
.layer(middleware::from_fn(auth::optional_signature));
// ── Meta ──────────────────────────────────────────────────────────────
let meta_routes = Router::new()
@@ -699,667 +619,3 @@ async fn p2p_info(State(state): State<AppState>) -> Json<serde_json::Value> {
None => Json(json!({ "enabled": false })),
}
}
#[cfg(test)]
mod tests {
use super::*;
use axum::body::Body;
use axum::http::{Request, StatusCode};
use sqlx::PgPool;
use tower::ServiceExt;
use crate::db::RepoRecord;
use crate::test_support::test_state;
/// Regression: anonymous callers must not see the pin/anchor index (#121, #134).
#[sqlx::test]
async fn unsigned_get_pins_and_anchors_is_401_through_build_router(pool: PgPool) {
let state = test_state(pool).await;
// Seed identifiable rows so the denial oracle proves the 401 body omits
// them: a status-only check would stay green if a later change served
// the listing before the auth layer rejected.
state
.db
.record_pinned_cid("seeded-pin-oid", "bafyPIN-SECRET-DO-NOT-LEAK", None)
.await
.expect("seed pin");
let owner_did = "did:key:zSEEDOWNERAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA";
let repo = seed_repo_inline(owner_did, "seeded-repo");
state.db.create_repo(&repo).await.expect("seed repo");
state
.db
.record_arweave_anchor(&crate::db::RecordAnchorInput {
repo: "zSEEDOWNERAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA/seeded-repo",
owner_did,
ref_name: "refs/heads/secret",
old_sha: "0".repeat(64).as_str(),
new_sha: "1".repeat(64).as_str(),
cid: Some("bafytest"),
irys_tx_id: "irys-secret-tx-DO-NOT-LEAK",
arweave_url: "https://arweave.net/secret-tx",
node_did: "did:key:zNODE",
})
.await
.expect("seed anchor");
let router = build_router(state);
let pins = Request::builder()
.method("GET")
.uri("/api/v1/ipfs/pins?limit=50")
.body(Body::empty())
.unwrap();
let pins_resp = router.clone().oneshot(pins).await.unwrap();
assert_eq!(
pins_resp.status(),
StatusCode::UNAUTHORIZED,
"anonymous pin listing must be rejected"
);
let bytes = axum::body::to_bytes(pins_resp.into_body(), usize::MAX)
.await
.unwrap();
let body = String::from_utf8_lossy(&bytes);
assert!(
!body.contains("bafyPIN-SECRET-DO-NOT-LEAK"),
"the 401 body must not leak the seeded pin cid; body was: {body}"
);
let anchors = Request::builder()
.method("GET")
.uri("/api/v1/arweave/anchors?limit=50")
.body(Body::empty())
.unwrap();
let anchors_resp = router.oneshot(anchors).await.unwrap();
assert_eq!(
anchors_resp.status(),
StatusCode::UNAUTHORIZED,
"anonymous anchors listing must be rejected"
);
let bytes = axum::body::to_bytes(anchors_resp.into_body(), usize::MAX)
.await
.unwrap();
let body = String::from_utf8_lossy(&bytes);
assert!(
!body.contains("irys-secret-tx-DO-NOT-LEAK"),
"the 401 body must not leak the anchor's irys tx id; body was: {body}"
);
}
/// Regression: a real RFC-9421 signature produced exactly as `gl` does — built
/// with `gitlawb_core::http_sig::sign_request` over a GET, headers attached,
/// and sent through the actual `build_router` — is verified by the
/// `optional_signature` layer that wraps the pin/anchor routes, and the
/// handler returns 200. Pairs with the anonymous-denial test above; one
/// proves headers are required, the other proves a valid header is honored.
/// Without this test the unsigned-denial test would stay green even if the
/// `optional_signature` layer were never wired onto these routes, because
/// signed and unsigned requests would fail identically (#134 review).
#[sqlx::test]
async fn signed_get_pins_and_anchors_succeeds_through_build_router(pool: PgPool) {
use gitlawb_core::http_sig::sign_request;
use gitlawb_core::identity::Keypair;
let state = test_state(pool).await;
let router = build_router(state);
let kp = Keypair::generate();
let pins_path = "/api/v1/ipfs/pins";
let signed = sign_request(&kp, "GET", pins_path, b"");
let pins = Request::builder()
.method("GET")
.uri(pins_path)
.header("content-digest", signed.content_digest)
.header("signature-input", signed.signature_input)
.header("signature", signed.signature)
.body(Body::empty())
.unwrap();
let pins_resp = router.clone().oneshot(pins).await.unwrap();
assert_eq!(
pins_resp.status(),
StatusCode::OK,
"a valid signature on /api/v1/ipfs/pins must be honored through build_router"
);
let anchors_path = "/api/v1/arweave/anchors";
let signed = sign_request(&kp, "GET", anchors_path, b"");
let anchors = Request::builder()
.method("GET")
.uri(anchors_path)
.header("content-digest", signed.content_digest)
.header("signature-input", signed.signature_input)
.header("signature", signed.signature)
.body(Body::empty())
.unwrap();
let anchors_resp = router.oneshot(anchors).await.unwrap();
assert_eq!(
anchors_resp.status(),
StatusCode::OK,
"a valid signature on /api/v1/arweave/anchors must be honored through build_router"
);
}
/// Companion to the two regressions above: a request that *carries* signature
/// headers but whose signature does not verify (here, garbled) must be denied
/// with 401, not silently treated as anonymous and re-checked by the handler.
/// This pins the failure mode of the `optional_signature` layer: when the
/// caller claims to be signed, the layer must commit to verifying — there is
/// no fall-through path that lets a bad signature bypass auth.
#[sqlx::test]
async fn malformed_signature_on_pins_is_401_through_build_router(pool: PgPool) {
use gitlawb_core::http_sig::sign_request;
use gitlawb_core::identity::Keypair;
let state = test_state(pool).await;
let router = build_router(state);
let kp = Keypair::generate();
let path = "/api/v1/ipfs/pins";
let mut signed = sign_request(&kp, "GET", path, b"");
// Flip a character well inside the signature value (base64) so the header
// still parses but does not verify against the key.
let mut tampered = signed.signature.clone();
let mid = tampered.len() / 2;
let flipped = if tampered.as_bytes()[mid] == b'A' {
'B'
} else {
'A'
};
tampered.replace_range(mid..mid + 1, &flipped.to_string());
signed.signature = tampered;
let req = Request::builder()
.method("GET")
.uri(path)
.header("content-digest", signed.content_digest)
.header("signature-input", signed.signature_input)
.header("signature", signed.signature)
.body(Body::empty())
.unwrap();
let resp = router.oneshot(req).await.unwrap();
assert_eq!(
resp.status(),
StatusCode::UNAUTHORIZED,
"a malformed signature must be rejected by the auth layer, not silently accepted"
);
}
/// The same malformed-signature rejection must hold on the anchors route,
/// which carries its own `optional_signature` layer beside pins: a later
/// edit that dropped the layer here (but not on pins) would otherwise stay
/// green.
#[sqlx::test]
async fn malformed_signature_on_anchors_is_401_through_build_router(pool: PgPool) {
use gitlawb_core::http_sig::sign_request;
use gitlawb_core::identity::Keypair;
let state = test_state(pool).await;
let router = build_router(state);
let kp = Keypair::generate();
let path = "/api/v1/arweave/anchors";
let mut signed = sign_request(&kp, "GET", path, b"");
let mut tampered = signed.signature.clone();
let mid = tampered.len() / 2;
let flipped = if tampered.as_bytes()[mid] == b'A' {
'B'
} else {
'A'
};
tampered.replace_range(mid..mid + 1, &flipped.to_string());
signed.signature = tampered;
let req = Request::builder()
.method("GET")
.uri(path)
.header("content-digest", signed.content_digest)
.header("signature-input", signed.signature_input)
.header("signature", signed.signature)
.body(Body::empty())
.unwrap();
let resp = router.oneshot(req).await.unwrap();
assert_eq!(
resp.status(),
StatusCode::UNAUTHORIZED,
"a malformed signature must be rejected by the auth layer, not silently accepted"
);
}
// ── Scoped anchor contract (P1 follow-up: ?repo= must authorize_repo_read) ──
//
// The follow-up review (after the auth-layer wiring landed) found that a
// signed but unauthorized caller could still obtain scoped anchor metadata
// because the `?repo=` branch handed the user-supplied string straight to
// the SQL filter. These tests exercise the full production contract end to
// end: real RFC-9421 signature → `optional_signature` layer →
// `authorize_repo_read` → SQL. They fail closed if the authz call is moved
// after the query or removed entirely.
/// Inline repo seed for the scoped-anchor tests. Mirrors the shape in
/// `test_support::tests::seed_repo` without taking a cross-module private
/// helper as a dependency.
fn seed_repo_inline(owner_did: &str, name: &str) -> RepoRecord {
let now = chrono::Utc::now();
RepoRecord {
id: uuid::Uuid::new_v4().to_string(),
name: name.to_string(),
owner_did: owner_did.to_string(),
description: None,
is_public: true,
default_branch: "main".to_string(),
created_at: now,
updated_at: now,
disk_path: format!("/tmp/{name}"),
forked_from: None,
machine_id: None,
}
}
/// Unsigned scoped anchor request → 401, identical to the global anchor
/// 401 test. Proves the auth layer fires before any scope decision (no
/// existence oracle via the `?repo=` path either).
#[sqlx::test]
async fn unsigned_scoped_anchors_is_401_through_build_router(pool: PgPool) {
let state = test_state(pool).await;
// Seed an anchor under a repo the request claims to scope to, so the
// denial oracle proves the 401 body omits it rather than merely
// returning the right status over an empty table.
let owner_did = "did:key:zSCOPEDAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA";
let repo = seed_repo_inline(owner_did, "priv");
state.db.create_repo(&repo).await.expect("seed repo");
state
.db
.record_arweave_anchor(&crate::db::RecordAnchorInput {
repo: "zSCOPEDAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA/priv",
owner_did,
ref_name: "refs/heads/secret",
old_sha: "0".repeat(64).as_str(),
new_sha: "1".repeat(64).as_str(),
cid: Some("bafytest"),
irys_tx_id: "irys-scoped-secret-DO-NOT-LEAK",
arweave_url: "https://arweave.net/scoped-secret",
node_did: "did:key:zNODE",
})
.await
.expect("seed anchor");
let router = build_router(state);
let resp = router
.oneshot(
Request::builder()
.method("GET")
.uri("/api/v1/arweave/anchors?repo=did:key:zSCOPED%2Fpriv")
.body(Body::empty())
.unwrap(),
)
.await
.unwrap();
assert_eq!(
resp.status(),
StatusCode::UNAUTHORIZED,
"unsigned scoped anchor request must be 401, before any scope lookup"
);
let bytes = axum::body::to_bytes(resp.into_body(), usize::MAX)
.await
.unwrap();
let body = String::from_utf8_lossy(&bytes);
assert!(
!body.contains("irys-scoped-secret-DO-NOT-LEAK"),
"the 401 body must not leak the anchor's irys tx id; body was: {body}"
);
}
/// Signed owner on a private repo → 200, anchor metadata returned. Mirrors
/// the existing `list_webhooks_accepts_a_real_gl_signature_e2e` shape: real
/// signature, real middleware, real `authorize_repo_read`.
#[sqlx::test]
async fn signed_scoped_anchors_owner_succeeds_through_build_router(pool: PgPool) {
use gitlawb_core::http_sig::sign_request;
use gitlawb_core::identity::Keypair;
let kp = Keypair::generate();
let owner_did = kp.did().to_string();
let state = test_state(pool).await;
let mut repo = seed_repo_inline(&owner_did, "scoped-priv");
repo.is_public = false;
state.db.create_repo(&repo).await.expect("seed repo");
let short = owner_did.split(':').next_back().unwrap().to_string();
state
.db
.record_arweave_anchor(&crate::db::RecordAnchorInput {
repo: &format!("{short}/scoped-priv"),
owner_did: &owner_did,
ref_name: "refs/heads/main",
old_sha: "0".repeat(64).as_str(),
new_sha: "1".repeat(64).as_str(),
cid: Some("bafytest"),
irys_tx_id: "irys-owner-tx",
arweave_url: "https://arweave.net/owner-tx",
node_did: "did:key:zNODE",
})
.await
.expect("seed anchor");
let path = format!("/api/v1/arweave/anchors?repo={short}/scoped-priv");
let signed = sign_request(&kp, "GET", &path, b"");
let req = Request::builder()
.method("GET")
.uri(&path)
.header("content-digest", signed.content_digest)
.header("signature-input", signed.signature_input)
.header("signature", signed.signature)
.body(Body::empty())
.unwrap();
let router = build_router(state);
let resp = router.oneshot(req).await.unwrap();
assert_eq!(
resp.status(),
StatusCode::OK,
"the owner of a private repo must see their scoped anchors"
);
let bytes = axum::body::to_bytes(resp.into_body(), usize::MAX)
.await
.unwrap();
let body = String::from_utf8_lossy(&bytes);
assert!(
body.contains("irys-owner-tx"),
"owner must see the anchor's irys tx id; body was: {body}"
);
// Owner must see the ref name + new SHA so this is not just a count oracle.
assert!(
body.contains("refs/heads/main"),
"owner must see the ref name; body was: {body}"
);
// Sanity: a second test would need a different repo name to avoid
// colliding on `did:key:zSCOPED/scoped-priv` in the anchors table.
// Distinct repo names per test keep the rows queryable in isolation.
let _ = repo;
}
/// #121 R3: the SQL filter must use the canonical stored slug, not the
/// user-supplied `?repo=` string. Anchor rows are stored as
/// `{normalize_owner_key(owner_did)}/{name}` (the short form). Authz
/// passes for the full-DID form, but a literal `WHERE repo=$1` against
/// the full DID would match zero rows and return a false empty page,
/// breaking callers that legitimately use the full `did:key:…/name`
/// form in `?repo=`. RED before the fix: handler built the filter from
/// the raw `?repo=` string, so this query returned `anchors: []`.
#[sqlx::test]
async fn signed_scoped_anchors_full_did_form_matches_stored_slug_through_build_router(
pool: PgPool,
) {
use gitlawb_core::http_sig::sign_request;
use gitlawb_core::identity::Keypair;
let kp = Keypair::generate();
let owner_did = kp.did().to_string();
let state = test_state(pool).await;
let mut repo = seed_repo_inline(&owner_did, "scoped-did-form");
repo.is_public = false;
state.db.create_repo(&repo).await.expect("seed repo");
// Use the same slug-construction the writer uses
// (`db::normalize_owner_key`) so the test reader and the production
// writer can't drift if a future change moves the slug off the
// short-form. P3 (reviewer-3): the previous `split(':').next_back()`
// hand-rolled a third convention that agreed with the writer for
// `did:key` owners and diverged for every other method.
let short = crate::db::normalize_owner_key(&owner_did).to_string();
state
.db
.record_arweave_anchor(&crate::db::RecordAnchorInput {
repo: &format!("{short}/scoped-did-form"),
owner_did: &owner_did,
ref_name: "refs/heads/main",
old_sha: "0".repeat(64).as_str(),
new_sha: "1".repeat(64).as_str(),
cid: Some("bafytest"),
irys_tx_id: "irys-did-form-tx",
arweave_url: "https://arweave.net/did-form-tx",
node_did: "did:key:zNODE",
})
.await
.expect("seed anchor");
// P2 (reviewer-3): the previous suite only proved the authorized
// row is present. A correct filter must ALSO prove that anchors
// for OTHER repos do not leak into the response — the failure
// mode of the original bug was a cross-repo leak, not a
// fail-safe empty page. Seed a second repo (also under the
// same owner, so the authz test stays scoped) with its own
// irys_tx_id and assert it is absent from the body.
let other_owner = {
let kp2 = Keypair::generate();
kp2.did().to_string()
};
let mut other_repo = seed_repo_inline(&other_owner, "other-repo");
other_repo.is_public = true;
state
.db
.create_repo(&other_repo)
.await
.expect("seed other repo");
state
.db
.record_arweave_anchor(&crate::db::RecordAnchorInput {
repo: &format!(
"{}/other-repo",
crate::db::normalize_owner_key(&other_owner)
),
owner_did: &other_owner,
ref_name: "refs/heads/main",
old_sha: "0".repeat(64).as_str(),
new_sha: "2".repeat(64).as_str(),
cid: Some("bafyother"),
irys_tx_id: "irys-other-repo-tx",
arweave_url: "https://arweave.net/other-tx",
node_did: "did:key:zNODE",
})
.await
.expect("seed other anchor");
// The caller passes the FULL DID form in ?repo=. validate_repo_slug
// accepts it, authorize_repo_read resolves it, and the handler MUST
// then translate to the canonical stored slug (short form) before
// running the SQL — otherwise this returns 200 with anchors: [].
let path = format!("/api/v1/arweave/anchors?repo={owner_did}/scoped-did-form");
let signed = sign_request(&kp, "GET", &path, b"");
let req = Request::builder()
.method("GET")
.uri(&path)
.header("content-digest", signed.content_digest)
.header("signature-input", signed.signature_input)
.header("signature", signed.signature)
.body(Body::empty())
.unwrap();
let router = build_router(state);
let resp = router.oneshot(req).await.unwrap();
assert_eq!(
resp.status(),
StatusCode::OK,
"authz passed via the full DID form, so the response must be 200"
);
let bytes = axum::body::to_bytes(resp.into_body(), usize::MAX)
.await
.unwrap();
let body = String::from_utf8_lossy(&bytes);
assert!(
body.contains("irys-did-form-tx"),
"the SQL filter must use the stored short slug, not the full DID — \
a false-empty page would break callers that pass ?repo=did:key:…/name. \
body was: {body}"
);
assert!(
!body.contains("irys-other-repo-tx"),
"the scoped anchor filter must exclude other repos' anchors — \
a cross-repo leak would expose metadata of repos the caller \
did not authorize. body was: {body}"
);
let _ = repo;
}
/// Signed non-reader on a private repo → 404, anchor metadata MUST NOT
/// leak. This is the test that catches the "auth-but-no-authz" bug if
/// the gate is moved after the SQL query or removed entirely. The 404
/// is the standard `repo_not_found` shape — indistinguishable from the
/// missing-repo case below.
#[sqlx::test]
async fn signed_scoped_anchors_non_reader_is_404_no_leak_through_build_router(pool: PgPool) {
use gitlawb_core::http_sig::sign_request;
use gitlawb_core::identity::Keypair;
let owner_kp = Keypair::generate();
let stranger_kp = Keypair::generate();
let owner_did = owner_kp.did().to_string();
let stranger_did = stranger_kp.did().to_string();
let state = test_state(pool).await;
let mut repo = seed_repo_inline(&owner_did, "scoped-priv-nr");
repo.is_public = false;
state.db.create_repo(&repo).await.expect("seed repo");
let short_owner = owner_did.split(':').next_back().unwrap().to_string();
state
.db
.record_arweave_anchor(&crate::db::RecordAnchorInput {
repo: &format!("{short_owner}/scoped-priv-nr"),
owner_did: &owner_did,
ref_name: "refs/heads/secret",
old_sha: "2".repeat(64).as_str(),
new_sha: "3".repeat(64).as_str(),
cid: Some("bafytest"),
irys_tx_id: "irys-secret-tx-DO-NOT-LEAK",
arweave_url: "https://arweave.net/secret-tx",
node_did: "did:key:zNODE",
})
.await
.expect("seed anchor");
let _ = stranger_did; // signature carries the DID; this is just documentation
let path = format!("/api/v1/arweave/anchors?repo={short_owner}/scoped-priv-nr");
let signed = sign_request(&stranger_kp, "GET", &path, b"");
let req = Request::builder()
.method("GET")
.uri(&path)
.header("content-digest", signed.content_digest)
.header("signature-input", signed.signature_input)
.header("signature", signed.signature)
.body(Body::empty())
.unwrap();
let router = build_router(state);
let resp = router.oneshot(req).await.unwrap();
assert_eq!(
resp.status(),
StatusCode::NOT_FOUND,
"a signed non-reader must be 404 on a private repo's anchors"
);
let bytes = axum::body::to_bytes(resp.into_body(), usize::MAX)
.await
.unwrap();
let body = String::from_utf8_lossy(&bytes);
assert!(
!body.contains("irys-secret-tx-DO-NOT-LEAK"),
"the 404 body must not leak the anchor's irys tx id; body was: {body}"
);
assert!(
!body.contains("refs/heads/secret"),
"the 404 body must not leak the ref name; body was: {body}"
);
let v: serde_json::Value = serde_json::from_slice(&bytes).expect("json body");
assert_eq!(
v["error"], "repo_not_found",
"non-reader denial must carry the same error code as a missing repo"
);
}
/// Signed caller on a missing repo → 404, same shape as the non-reader
/// case. This is the indistinguishability half: a probe cannot tell
/// "private" from "absent" from the response.
#[sqlx::test]
async fn signed_scoped_anchors_missing_repo_is_404_through_build_router(pool: PgPool) {
use gitlawb_core::http_sig::sign_request;
use gitlawb_core::identity::Keypair;
let kp = Keypair::generate();
let state = test_state(pool).await;
let short = kp
.did()
.to_string()
.split(':')
.next_back()
.unwrap()
.to_string();
let path = format!("/api/v1/arweave/anchors?repo={short}/does-not-exist");
let signed = sign_request(&kp, "GET", &path, b"");
let req = Request::builder()
.method("GET")
.uri(&path)
.header("content-digest", signed.content_digest)
.header("signature-input", signed.signature_input)
.header("signature", signed.signature)
.body(Body::empty())
.unwrap();
let router = build_router(state);
let resp = router.oneshot(req).await.unwrap();
assert_eq!(
resp.status(),
StatusCode::NOT_FOUND,
"a missing repo must 404 indistinguishably from a non-readable private repo"
);
let bytes = axum::body::to_bytes(resp.into_body(), usize::MAX)
.await
.unwrap();
let v: serde_json::Value = serde_json::from_slice(&bytes).expect("json body");
assert_eq!(
v["error"], "repo_not_found",
"the 404 body must carry the standard repo_not_found error code"
);
}
/// `?limit=-1` must not crash as `LIMIT -1` (Postgres 500). It clamps to
/// zero and returns 200 with an empty list — the same shape as a valid
/// listing that happens to have no rows in the configured range.
#[sqlx::test]
async fn signed_anchors_negative_limit_clamps_to_zero_through_build_router(pool: PgPool) {
use gitlawb_core::http_sig::sign_request;
use gitlawb_core::identity::Keypair;
let kp = Keypair::generate();
let state = test_state(pool).await;
let path = "/api/v1/arweave/anchors?limit=-1";
let signed = sign_request(&kp, "GET", path, b"");
let req = Request::builder()
.method("GET")
.uri(path)
.header("content-digest", signed.content_digest)
.header("signature-input", signed.signature_input)
.header("signature", signed.signature)
.body(Body::empty())
.unwrap();
let router = build_router(state);
let resp = router.oneshot(req).await.unwrap();
assert_eq!(
resp.status(),
StatusCode::OK,
"?limit=-1 must clamp, not 500"
);
let bytes = axum::body::to_bytes(resp.into_body(), usize::MAX)
.await
.unwrap();
let v: serde_json::Value = serde_json::from_slice(&bytes).expect("json body");
assert_eq!(v["count"], 0, "clamped limit yields an empty list");
assert_eq!(v["anchors"].as_array().map(|a| a.len()), Some(0));
}
}
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
-13
View File
@@ -79,19 +79,6 @@ async fn fire_event_async(
let delivery_id = uuid::Uuid::new_v4().to_string();
tokio::spawn(async move {
// Policy skip, not a delivery failure: the stored URL predates the
// create-time gate. The "invalid_url" label is documented on
// record_webhook_delivery in metrics.rs.
if !crate::api::peers::is_public_http_url(&hook.url) {
crate::metrics::record_webhook_delivery("invalid_url");
tracing::warn!(
url = %hook.url,
event = %event_name,
"webhook delivery skipped: stored URL is not a public http(s) target"
);
return;
}
let signature = hook.secret.as_deref().map(|s| sign_payload(s, &bytes));
let mut req = client
+14 -449
View File
@@ -85,24 +85,21 @@ fn inv22_concurrency_gates_present_and_not_bypassed() {
is reaped on disconnect (the path-scoped half of #174 P1-a)"
);
// U2 / R1: a cancelled or timed-out `GET /ipfs/{cid}` must release admission only
// after the blocking work it admitted has finished, not the instant the handler
// future drops (the /ipfs half of #174 P1-a). The mechanism is the shared
// `Arc<WalkAdmission>`: both walk permits are moved into it once per request and a
// clone rides every `spawn_blocking`, so the permits release when the LAST holder
// drops. Reverting to handler-local permits trips this; the per-site clones are
// bound separately by `inv22_ipfs_walk_admission_reaches_every_blocking_site`.
//
// This previously required the permits to be owned by a detached `tokio::spawn`
// running the whole pipeline. That closed the same bypass but kept an abandoned
// request's full legacy scan running against a held slot, so the merge of #173 and
// #174 kept the Arc and dropped the detached task.
// U2 / R1 (#173 round-10): the `GET /ipfs/{cid}` serve pipeline must run in a
// DETACHED tokio task that OWNS the AdmissionGuard built from BOTH walk permits, so a
// cancelled or timed-out request releases admission only after the spawned (bounded)
// work completes — not the instant the handler future drops (the /ipfs half of #174
// P1-a). Two load-bearing markers: the guard is constructed from the two named
// permits, and the whole pipeline is moved into a `tokio::spawn`. Reverting to
// handler-local permits (dropping the guard/spawn) trips this.
assert!(
ipfs.contains("struct WalkAdmission")
&& ipfs.contains("let admission = std::sync::Arc::new(WalkAdmission {"),
"U2/R1 gate missing: get_by_cid must move both /ipfs admission permits into a \
shared Arc<WalkAdmission> whose clones ride the blocking work, so admission is \
released only once that work completes (the /ipfs half of #174 P1-a)"
ipfs.contains("AdmissionGuard::new(ipfs_walk_permit, ipfs_caller_permit)")
&& ipfs.contains(
"let serve: tokio::task::JoinHandle<Result<Response>> = tokio::spawn(async move {"
),
"U2/R1 gate missing: get_by_cid must move both /ipfs admission permits into an \
AdmissionGuard owned by a detached tokio::spawn task so admission is released \
only after the spawned serve work completes (the /ipfs half of #174 P1-a)"
);
// U2 / KTD2 (#173 round-10): the probe/read children on the /ipfs path must be the
@@ -139,436 +136,4 @@ fn inv22_concurrency_gates_present_and_not_bypassed() {
"U4/P2-2 gate missing: the detached post-push encryption spawn must consult \
encrypt_inflight.try_begin to coalesce per repo (bound the outstanding-task set)"
);
// F5: coalescing must REQUEUE, not shed. The in-flight task pins only its own
// pre-spawn snapshot, so a coalesced push's tip pairs are recorded on its key and
// the task must loop-drain them (`finish_or_take_pending`) before releasing it.
// Removing the drain reverts to the silent loss: a coalesced push's pins and
// recovery copies are absent until an unrelated later push. Scan only the
// production half of the file — the u5 tests in its `mod tests` also name the
// drain call, and matching them would make this check vacuous.
// Split at the TEST MODULE, not at the first `#[cfg(test)]`. api/repos.rs carries
// test-only items (the drain fault seam, the task entry point) ABOVE the production
// code these gates scan for, so splitting on the attribute truncated the production
// half above every line being checked and the gate went vacuously blind.
let repos_production = repos
.split("\nmod tests {")
.next()
.expect("split always yields a first chunk");
assert!(
repos_production.contains("finish_or_take_pending"),
"F5 gate missing: the post-push encryption task must loop-drain coalesced \
pushes via finish_or_take_pending before releasing its repo key"
);
// F4: every post-receive scan helper admits itself to the shared scan pool via
// `crate::state::acquire_scan_permit` BEFORE its spawn_blocking git walk, so a
// push burst cannot accumulate unbounded concurrent scans once the write permit
// is released. Two halves, both load-bearing: the helper body must actually
// acquire the pool (state.rs sits the helper at the end of the file, so the
// definition tail contains no other `acquire_owned` to match vacuously), and
// within each scan helper the first qualified call precedes the first
// `spawn_blocking`. Severing a call site pushes the next occurrence past the
// helper's own `spawn_blocking` (or off the end of the file), turning the
// assertion red; comments name the helper without the `crate::state::` prefix,
// so this targets the real call sites.
let state_rs = src("state.rs");
let helper_def = state_rs
.find("fn acquire_scan_permit")
.expect("F4 gate missing: state.rs no longer defines acquire_scan_permit");
assert!(
state_rs[helper_def..].contains("acquire_owned"),
"F4 gate gutted: acquire_scan_permit must acquire the scan pool via acquire_owned"
);
let push_delta = src("git/push_delta.rs");
for (file_src, file, helper) in [
(&repos, "api/repos.rs", "fn replication_withheld_set"),
(&repos, "api/repos.rs", "fn fail_closed_full_scan_objects"),
(
&push_delta,
"git/push_delta.rs",
"fn resolve_candidates_for_push",
),
] {
let start = file_src
.find(helper)
.unwrap_or_else(|| panic!("{file}: `{helper}` not found"));
let tail = &file_src[start..];
let acquire = tail
.find("crate::state::acquire_scan_permit(")
.unwrap_or_else(|| {
panic!("F4 gate missing: {file} `{helper}` no longer acquires a scan permit")
});
let spawn = tail.find("spawn_blocking").unwrap_or_else(|| {
panic!("{file}: `{helper}` lost its spawn_blocking walk — update this guard")
});
assert!(
acquire < spawn,
"F4 gate bypassed: {file} `{helper}` must acquire its git_encrypt_semaphore \
permit BEFORE dispatching the blocking git scan"
);
}
}
/// F4 (repo_store advisory-unlock cancellation safety): `RepoWriteGuard::release`
/// must await `pg_advisory_unlock` while `self` still owns the pooled connection,
/// and must not mark itself `released` until that await resolves. Either shape,
/// reintroduced, re-opens the mid-unlock cancellation leak: taking the connection
/// early leaves `Drop` with `conn == None`, and setting `released = true` early
/// leaves the `Drop` backstop inert — both strand the session lock on cancellation.
///
/// Scoped to the `release` fn body: the `Drop` impl legitimately takes the
/// connection and unlocks, so a whole-file scan would match it and read as a false
/// pass. Reverting either ordering turns this red (proven load-bearing).
#[test]
fn f4_release_keeps_conn_owned_until_unlock_resolves() {
let repo_store = src("git/repo_store.rs");
let rel_start = repo_store
.find("pub async fn release(self")
.or_else(|| repo_store.find("pub async fn release(mut self"))
.expect("F4 gate: repo_store.rs no longer defines RepoWriteGuard::release");
let rel_end = repo_store[rel_start..]
.find("impl Drop for RepoWriteGuard")
.map(|off| rel_start + off)
.expect("F4 gate: release fn / Drop impl markers moved — update this guard");
let release_body = &repo_store[rel_start..rel_end];
let unlock = release_body
.find("pg_advisory_unlock")
.expect("F4 gate: release must still issue pg_advisory_unlock");
let before_unlock = &release_body[..unlock];
// (a) the connection must still be owned by `self` at the unlock await.
assert!(
!before_unlock.contains("self.conn.take()"),
"F4 regression: RepoWriteGuard::release takes self.conn BEFORE awaiting \
pg_advisory_unlock. A cancellation during the unlock await then strands the \
session advisory lock (Drop sees conn == None and skips its backstop). \
Unlock through the still-owned connection instead."
);
// (b) `released` must not be set before the unlock await — the other
// reintroduction shape a single-reorder check on (a) alone is blind to.
assert!(
!before_unlock.contains("released = true"),
"F4 regression: RepoWriteGuard::release sets `released = true` BEFORE awaiting \
pg_advisory_unlock. A cancellation during the await then leaves the Drop \
backstop inert (it early-returns on released). Set released only AFTER the \
unlock await resolves."
);
}
/// F6/KTD-5 (IPFS metadata queries deadline-wrapped): `get_by_cid` acquires
/// the scarce walk permits (RAII, held for the whole request) BEFORE its metadata
/// queries, and the per-repo loop's first budget gate runs only later. So
/// both `list_repos_page_for_scan` and `list_visibility_rules_for_repos` must be clamped to the
/// remaining request budget — otherwise a query blocked in Postgres pins the walk slot
/// for the whole stall, past the budget. This scans the PRODUCTION half of `api/ipfs.rs`
/// (the `mod tests` half names the same calls in its own harness and would make the
/// check vacuous) and asserts each query call is immediately preceded by a
/// `tokio::time::timeout(` wrapper. Removing either wrapper turns this red (proven
/// load-bearing).
#[test]
fn f6_ipfs_metadata_queries_are_deadline_wrapped() {
let ipfs = src("api/ipfs.rs");
// Split at the TEST MODULE, not at the first `#[cfg(test)]`: the handler carries
// in-line `#[cfg(test)]` query counters, so splitting on the attribute cut the
// production half off above the calls this guard exists to check and the scan went
// vacuously quiet (found 0 of each rather than failing on a missing wrapper).
let production = ipfs
.split("\nmod tests {")
.next()
.expect("split always yields a first chunk");
// EVERY occurrence must be wrapped, not just the first. The handler now reaches
// these queries from two places (the provenance fast path, one repo at a time, and
// the legacy scan's preload), and an exact-count check would have to be relaxed
// every time a call site is added, which is how a guard quietly stops covering the
// site that matters. Requiring all of them scales with the code instead.
for call in [
".list_repos_page_for_scan(",
".list_visibility_rules_for_repos(",
".get_repo_by_id(",
".is_repo_quarantined(",
] {
let occurrences = production.matches(call).count();
assert!(
occurrences >= 1,
"F6 guard stale: api/ipfs.rs no longer calls `{call}` — update this guard"
);
for (n, (idx, _)) in production.match_indices(call).enumerate() {
// The wrapper opens a few lines above the call (match tokio::time::timeout(
// ... remaining budget ..., <query> )); a 240-char lookback covers it
// without reaching the previous statement.
let window = &production[idx.saturating_sub(240)..idx];
assert!(
window.contains("tokio::time::timeout("),
"F6 gate missing: occurrence {n} of `{call}` (of {occurrences}) is not \
wrapped in tokio::time::timeout(...) clamped to the remaining request \
budget. An unwrapped await pins the held walk permit for the whole DB \
stall, past GITLAWB_IPFS_REQUEST_BUDGET_SECS."
);
}
}
}
/// #174 F2 / KTD-3: the detached post-receive Pinata replication task must enqueue
/// only the push's `(ref, old, new)` tuples and RE-DERIVE its object set inside the
/// worker once a pin slot frees — it must NOT move the full per-push object list into
/// the closure and hold it across the `pin_semaphore` acquire. Retaining the list makes
/// every parked task (under a slow Pinata backend) hold an MB-scale OID list, so
/// outstanding memory grows O(pushes x object-list). Coalescing/shedding the task is
/// forbidden (its per-ref effects are non-idempotent), so the fix bounds the retained
/// data, not the task count.
///
/// Two load-bearing checks, both against the PRODUCTION half of `api/repos.rs` (the
/// `mod tests` half names the same identifiers in its own harness and would make the
/// scan vacuous): (a) the closure-local `object_list_pinata` binding — the retain form —
/// must be gone; reintroducing `let object_list_pinata = object_list;` turns this red.
/// (b) the re-derivation (`pinata_object_list_for_refs`) must appear AFTER the Pinata
/// pin permit is acquired, so the object list is materialized only inside the pin-bounded
/// section and a parked task holds O(ref tuples).
#[test]
fn f2_pinata_enqueues_refs_not_retained_object_lists() {
let repos = src("api/repos.rs");
// Split at the TEST MODULE, not at the first `#[cfg(test)]`. api/repos.rs carries
// test-only items (the drain fault seam, the task entry point) ABOVE the production
// code these gates scan for, so splitting on the attribute truncated the production
// half above every line being checked and the gate went vacuously blind.
let production = repos
.split("\nmod tests {")
.next()
.expect("split always yields a first chunk");
// (a) the retained-list form must be gone from production.
assert!(
!production.contains("object_list_pinata"),
"F2/KTD-3 regression: the Pinata task retains a full per-push object list \
(`object_list_pinata`) across the pin-permit acquire. Enqueue only the ref \
tuples and re-derive the object set inside the worker (pinata_object_list_for_refs)."
);
// (b) the re-derivation runs AFTER the pin permit is acquired. Anchor on the
// Pinata pin-admission clone so the window is the Pinata task, not the sibling
// IPFS/encrypt spawn (which shares `pin_semaphore` but never re-derives).
let anchor = production
.find("let pin_sem_pinata")
.expect("F2 gate stale: the Pinata pin-admission clone (pin_sem_pinata) moved");
let tail = &production[anchor..];
let acquire = tail
.find(".acquire_owned()")
.expect("F2 gate: the Pinata task no longer acquires a pin permit (acquire_owned)");
let rederive = tail.find("pinata_object_list_for_refs(").expect(
"F2 gate missing: the Pinata task must re-derive its object set via \
pinata_object_list_for_refs; it can no longer move a pre-resolved list into the closure",
);
assert!(
acquire < rederive,
"F2 gate bypassed: the Pinata object-set re-derivation must run AFTER the pin \
permit is acquired, so a parked task never holds the MB-scale object list"
);
// The re-derivation is driven by the push's ref tuples (the enqueued unit), not a
// retained object list — tie "enqueue ref_updates" to the call explicitly.
assert!(
tail[rederive..].starts_with("pinata_object_list_for_refs(")
&& tail[rederive..]
.get(..400)
.map(|w| w.contains("ref_updates_clone"))
.unwrap_or(false),
"F2 gate: pinata_object_list_for_refs must re-derive from the push's ref tuples \
(ref_updates_clone), the small unit the parked task retains"
);
}
/// #174 U2 / F3 (second same-repo writer serialized until a disconnected first push's
/// git group is reaped): `git_receive_pack` must take the per-repo in-process write
/// lease and CARRY it on the write-path `AdmissionGuard` (via `.with_lease`), so the
/// lease rides `KillGroupOnDrop`'s detached reaper and frees only after the group is
/// reaped — supplementing the pg advisory lock, which `RepoWriteGuard::Drop` releases at
/// the disconnect instant. Tying the lease to `RepoWriteGuard` instead (or dropping the
/// `.with_lease` carry) reopens the race; the behavioral RED/GREEN test
/// (`f3_second_push_serialized_until_disconnected_group_reaped`) is the real bar, this
/// is the completeness tripwire. Scanned against the PRODUCTION half of `api/repos.rs`
/// (the `mod tests` half names these identifiers in its own harness).
#[test]
fn f3_second_writer_leased_until_reap() {
let repos = src("api/repos.rs");
let smart_http = src("git/smart_http.rs");
// Split at the TEST MODULE, not at the first `#[cfg(test)]`. api/repos.rs carries
// test-only items (the drain fault seam, the task entry point) ABOVE the production
// code these gates scan for, so splitting on the attribute truncated the production
// half above every line being checked and the gate went vacuously blind.
let repos_production = repos
.split("\nmod tests {")
.next()
.expect("split always yields a first chunk");
// The lease is acquired, then carried by the AdmissionGuard (.with_lease), before
// receive_pack runs the write. Severing any of the three turns this red.
let lease_acquire = repos_production.find("repo_write_leases").expect(
"F3 gate missing: git_receive_pack no longer takes the per-repo write lease \
(state.repo_write_leases)",
);
let with_lease = repos_production.find(".with_lease(").expect(
"F3 gate missing: the write-path AdmissionGuard no longer carries the lease \
(.with_lease). The lease must ride the disconnect reaper via the AdmissionGuard, \
NOT RepoWriteGuard (which drops at the disconnect instant, reopening F3).",
);
let receive = repos_production
.find("smart_http::receive_pack(")
.expect("F3 gate stale: git_receive_pack no longer calls smart_http::receive_pack");
assert!(
lease_acquire < with_lease && with_lease < receive,
"F3 gate bypassed: the write lease must be acquired, then carried by the \
AdmissionGuard (.with_lease), BEFORE receive_pack runs the write"
);
// The AdmissionGuard must actually hold the lease (Option<RepoWriteLease>) via a
// with_lease setter, so it travels into the detached reaper. Removing the field or
// method turns this red.
assert!(
smart_http.contains("_lease: Option<crate::state::RepoWriteLease>")
&& smart_http.contains("pub fn with_lease("),
"F3 gate missing: AdmissionGuard must hold an Option<RepoWriteLease> set via \
with_lease, so the lease rides the guard into KillGroupOnDrop's reaper"
);
}
/// #174 U1 — every blocking walk in the `/ipfs` scan carries the request's walk
/// admission.
///
/// The admission is an `Arc<WalkAdmission>` cloned into each `spawn_blocking`
/// closure, so the global + per-source permits release only when the last holder
/// drops: a client disconnect leaves the abandoned closure holding the slot, and a
/// panicking closure leaves the handler holding it.
///
/// Only the walk site runs under `state.git_bin`, so only it can be pinned by the
/// fake-git harness and mutation-verified dynamically (see
/// `get_by_cid_walk_permit_held_through_blocking_walk`). The probe and the content
/// read deliberately shell to the real `git`, which is why this structural check
/// exists: it binds all three sites, and any blocking site added to this loop
/// later, without reversing that deliberate independence.
///
/// MUTATION (RED): delete any one `Arc::clone(ctx.admission)` binding, or drop the
/// clone from inside its closure, and the count falls below three.
#[test]
fn inv22_ipfs_walk_admission_reaches_every_blocking_site() {
let ipfs = src("api/ipfs.rs");
// The shared owner must exist and be built once per request.
assert!(
ipfs.contains("struct WalkAdmission") && ipfs.contains("Arc::new(WalkAdmission {"),
"U1 gate missing: the /ipfs walk admission must be a shared WalkAdmission, \
not a handler-local permit pair"
);
// Every blocking site in the scan takes its own clone...
// `Arc::clone(ctx.admission)` in the gate, `Arc::clone(&admission)` if a site is
// ever added back in the handler body itself; count both spellings.
let clones = ipfs.matches("Arc::clone(ctx.admission)").count()
+ ipfs.matches("Arc::clone(&admission)").count();
assert!(
clones >= 3,
"U1 gate bypassed: expected an admission clone for each of the three \
/ipfs spawn_blocking sites (probe, walk, read); found {clones}. A blocking \
walk that does not hold the admission lets a disconnect or a panic free the \
slot while its git child is still running."
);
// ...and each clone is actually moved INTO the blocking closure, not merely
// created in the async frame (which would hold nothing across the join).
let held = ipfs.matches("let _admission = ").count();
assert!(
held >= 3,
"U1 gate bypassed: each admission clone must be bound inside its \
spawn_blocking closure so the blocking work owns it; found {held} of 3."
);
// The count of blocking sites is itself the thing being covered: if a fourth
// appears, it needs an admission clone too and this gate must be revisited.
let sites = ipfs.matches("spawn_blocking(move ||").count();
assert_eq!(
sites, 3,
"the /ipfs scan grew or lost a spawn_blocking site ({sites} found, expected 3); \
give any new blocking walk its own Arc::clone(&admission) and update this gate"
);
}
/// #174 U5: the post-receive replication tail is spawned at the DURABILITY BOUNDARY,
/// which is the moment receive-pack returns success, not the end of the handler and
/// not after `guard.release()`.
///
/// The tail owes this push its pins, recovery copy, and announcements. Everything
/// below the spawn stays in the cancellable request future, so anything the tail is
/// spawned after is a window where a client disconnect drops that work while the pack
/// is already durable on disk. `guard.release()` is such a window: on success it
/// awaits the Tigris upload and then the advisory unlock.
///
/// The lower bound matters just as much as the upper one: `release` runs on failure
/// too, so an ungated spawn would fire for a push git rejected, pinning and announcing
/// a half-applied repo. Above `release` the `?` on `receive_result` can no longer be
/// what gates it, so the success check is explicit and this gate binds it: the spawn
/// must sit inside `if push_succeeded`, and `release` must consume the same flag so
/// the two cannot drift apart.
///
/// This is an ordering check rather than a cancellation-race test on purpose: it is
/// the companion to `receive_pack_tail_survives_a_disconnect_during_release`, which
/// drives the actual disconnect through a parked `release`. Same instrument the F3
/// gate above uses.
///
/// MUTATION (RED): move the `tokio::spawn(post_receive_replication_tail` call below
/// `guard.release(` and the ordering assertion fails; take it out of the
/// `if push_succeeded` block and the failed-push assertion fails.
#[test]
fn inv22_replication_tail_spawns_at_the_durability_boundary() {
let repos = src("api/repos.rs");
// Production half only — the tests below name these identifiers too.
// Split at the TEST MODULE, not at the first `#[cfg(test)]`. api/repos.rs carries
// test-only items (the drain fault seam, the task entry point) ABOVE the production
// code these gates scan for, so splitting on the attribute truncated the production
// half above every line being checked and the gate went vacuously blind.
let production = repos
.split("\nmod tests {")
.next()
.expect("split always yields a first chunk");
let success_flag = production
.find("let push_succeeded = receive_result.is_ok();")
.expect(
"U5 gate missing: the tail's success gate must be bound from receive_result.is_ok()",
);
let gate_open = production
.find("if push_succeeded {")
.expect("U5 gate missing: the tail spawn must be gated on the push having succeeded");
let spawn = production
.find("tokio::spawn(post_receive_replication_tail(")
.expect("U5 gate missing: the replication tail must be spawned by git_receive_pack");
let release = production
.find(".release(push_succeeded)")
.expect("U5 gate stale: release must consume the same success flag as the tail gate");
let touch = production
.find("state.db.touch_repo(")
.expect("U5 gate stale: git_receive_pack no longer calls touch_repo");
let webhook = production
.find("webhooks::fire_event(")
.expect("U5 gate stale: git_receive_pack no longer fires push webhooks");
assert!(
success_flag < gate_open && gate_open < spawn,
"U5 gate bypassed: the tail must be spawned inside `if push_succeeded`, or a \
rejected push spawns a tail that pins and announces a half-applied repo"
);
// Still inside that block: no `}` may close it between the gate and the spawn.
assert!(
!production[gate_open + "if push_succeeded {".len()..spawn].contains('}'),
"U5 gate bypassed: the tail spawn left the `if push_succeeded` block, so a \
rejected push now spawns a tail"
);
assert!(
spawn < release && spawn < touch && spawn < webhook,
"U5 gate bypassed: the tail must be spawned BEFORE guard.release, touch_repo \
and the webhook fan-out, so a disconnect in any of those windows cannot drop \
this push's pins, recovery copy, and announcements"
);
}
@@ -1,177 +0,0 @@
//! #26 Split PR 4 — `GITLAWB_ENFORCE_OWNER_PUSH` default is consistent across surfaces.
//!
//! Reviewer 2 (2026-08-28) closed PR #224 with a P2 finding: SECURITY.md and
//! README.md both stated the default was `false`, while `Config::enforce_owner_push`
//! declared `default_value_t = true` and the runtime, .env.example, and the env
//! var table on the README all agreed with the code. Two operator-facing
//! surfaces lied about the authorization policy. PR #330 flipped the code
//! default and updated .env.example / README env table / docs/RUN-A-NODE.md,
//! but missed these two prose mentions.
//!
//! This guard pins the documentation and example surfaces together. Each one
//! is read off the working tree at test time, not a copy in the binary, so a
//! revert of the doc edit turns it red:
//!
//! 1. `SECURITY.md` says the default is `true`.
//! 2. `README.md` says the default is `true` in the known-limitations prose.
//! 3. `README.md` says the default is `true` in the env-var table row.
//! 4. `.env.example` sets `GITLAWB_ENFORCE_OWNER_PUSH=true`.
//!
//! The runtime declared default (`Config::enforce_owner_push`'s
//! `default_value_t = true`) is protected by the
//! `enforce_owner_push_is_declared_true_independent_of_the_environment`
//! unit test in `config.rs`, which reads the parser directly and is
//! env-independent. The two checks together cover the surfaces that
//! produced the original doc-vs-code drift; this guard cannot reach into
//! the binary crate's `Config` type (it is private to the bin target,
//! which has no `lib.rs`) without a larger refactor, so the contract is
//! stated as the four documentation/example surfaces above.
//!
//! The four are checked independently. A check that just verified "at least
//! one surface says true" would pass while a stale SECURITY.md lied; a check
//! that only verified the binary would not catch the doc drift that
//! produced this finding in the first place.
use std::path::Path;
fn read(rel: &str) -> String {
let p = Path::new(env!("CARGO_MANIFEST_DIR"))
.join("..")
.join("..")
.join(rel);
std::fs::read_to_string(&p).unwrap_or_else(|e| panic!("read {}: {e}", p.display()))
}
/// Pin the prose mentions of `GITLAWB_ENFORCE_OWNER_PUSH` so a future doc
/// edit cannot reintroduce the "defaults to false" wording the reviewer
/// flagged. The wording in the test is what the docs MUST say about the
/// default; absence of any prose mention is also a fail because operators
/// rely on the docs to learn the default.
#[test]
fn security_md_states_owner_push_default_is_true() {
let security = read("SECURITY.md");
let section = section_after(&security, "### Repository write authorization defaults")
.unwrap_or_else(|| panic!("SECURITY.md is missing the owner-push section"));
assert!(
section.contains("defaults to `true`"),
"SECURITY.md owner-push section must state the default is true. \
Found:\n{section}"
);
assert!(
!section_claims_default_is_false(&section),
"SECURITY.md owner-push section must not claim the default is false. \
Found:\n{section}"
);
}
#[test]
fn readme_known_limitations_states_owner_push_default_is_true() {
let readme = read("README.md");
let limitations = section_after(&readme, "Known limitations:")
.unwrap_or_else(|| panic!("README.md is missing the Known limitations: section"));
// The owner-push line is one bullet in the limitations list. Pull the
// single line that mentions the env var so the assertion is local to
// the claim, not a substring of unrelated text.
let line = limitations
.lines()
.find(|l| l.contains("GITLAWB_ENFORCE_OWNER_PUSH"))
.unwrap_or_else(|| {
panic!(
"README.md Known limitations must mention GITLAWB_ENFORCE_OWNER_PUSH.\n\
Got:\n{limitations}"
)
});
assert!(
line.contains("defaults to `true`") || line.contains("defaults to true"),
"README Known-limitations owner-push bullet must state the default is true. \
Found:\n{line}"
);
assert!(
!section_claims_default_is_false(line),
"README Known-limitations owner-push bullet must not claim the default is false. \
Found:\n{line}"
);
}
#[test]
fn readme_env_var_table_states_owner_push_default_is_true() {
let readme = read("README.md");
let line = readme
.lines()
.find(|l| l.contains("GITLAWB_ENFORCE_OWNER_PUSH") && l.contains("|"))
.unwrap_or_else(|| {
panic!("README.md is missing the env-var-table row for GITLAWB_ENFORCE_OWNER_PUSH")
});
assert!(
line.contains("Defaults to `true`") || line.contains("defaults to `true`"),
"README env-var-table row for GITLAWB_ENFORCE_OWNER_PUSH must state the default is true. \
Found:\n{line}"
);
assert!(
!section_claims_default_is_false(line),
"README env-var-table row for GITLAWB_ENFORCE_OWNER_PUSH must not claim the default is false. \
Found:\n{line}"
);
}
#[test]
fn env_example_sets_owner_push_true() {
let env = read(".env.example");
let line = env
.lines()
.find(|l| l.trim_start().starts_with("GITLAWB_ENFORCE_OWNER_PUSH"))
.unwrap_or_else(|| panic!(".env.example is missing GITLAWB_ENFORCE_OWNER_PUSH"));
assert!(
line.contains("=true"),
".env.example GITLAWB_ENFORCE_OWNER_PUSH must be set to true. Found:\n{line}"
);
}
/// A claim that the default is false can hide in many phrasings. Match the
/// forms that have appeared in the wild on this branch: "defaults to `false`",
/// "defaults to false", "default is false", "is `false` for compatibility",
/// "is false for compatibility". The match is intentionally narrow — false
/// positives would suppress a real doc edit.
fn section_claims_default_is_false(section: &str) -> bool {
let lower = section.to_ascii_lowercase();
lower.contains("defaults to `false`")
|| lower.contains("defaults to false")
|| lower.contains("default is `false`")
|| lower.contains("default is false")
|| (lower.contains("`false`") && lower.contains("for compatibility"))
}
/// Return the prose after a marker up to the next heading boundary. The
/// marker is normally a markdown heading (e.g. `### Section`); if it has
/// no `#` prefix it is treated as a plain-text marker and the slice is
/// bounded by the next heading of any level. This matters because a
/// plain-text marker would otherwise produce `header_level = 0`, making
/// the same-or-higher-level stop condition unsatisfiable and letting the
/// slice run to EOF.
fn section_after(doc: &str, heading: &str) -> Option<String> {
let start = doc.find(heading)?;
let after = &doc[start..];
let header_level = heading.chars().take_while(|c| *c == '#').count();
// Skip past the heading line itself.
let body_start = after.find('\n')? + 1;
let body = &after[body_start..];
// Stop at the next heading of the same or higher level, or at the next
// heading of any level when the marker had no `#` prefix.
let mut end = body.len();
for line in body.lines() {
let level = line.chars().take_while(|c| *c == '#').count();
if level > 0 && (header_level == 0 || level <= header_level) {
end = line.as_ptr() as usize - body.as_ptr() as usize;
break;
}
}
Some(body[..end].to_string())
}
@@ -1,637 +0,0 @@
//! Which p2p key-storage failures stop the node, and which leave HTTP up.
//!
//! The two domains are decided by different code at different phases, so the
//! only honest proof is at the process boundary: spawn the real binary and
//! watch what it does. A helper's returned error says nothing about whether
//! `main` treated it as fatal.
//!
//! Every row also asserts the key tree is byte-identical afterwards. A refusal
//! that mutates storage on its way out is its own defect, and a returned error
//! cannot show that.
//!
//! No database: `DATABASE_URL` points at a closed port on purpose, so a row
//! that reaches the degraded server proves the outcome is observable without
//! Postgres. That is what makes the port-zero no-I/O guarantee testable at all.
#![cfg(unix)]
use std::io::{BufRead, BufReader, Read, Write};
use std::os::unix::fs::PermissionsExt;
use std::path::{Path, PathBuf};
use std::process::{Child, Command, Stdio};
use std::time::{Duration, Instant};
/// Wall clock, never an iteration count: a fixed loop count starves on a slow
/// or loaded machine and turns a real pass into a flake.
const DEADLINE: Duration = Duration::from_secs(15);
/// Kills the child on every exit path, including a panicking assertion.
///
/// Without this a failing row leaves a node behind that retries its database
/// connection forever, holding the row's tempdir open. The first run of this
/// file did exactly that, and the leak is invisible until something else in
/// the suite gets slow.
struct ChildGuard(Child);
impl ChildGuard {
/// The tracing subscriber writes to STDOUT; only anyhow's final error
/// print goes to stderr. A row that watches one stream sees half the
/// story, so both are read.
fn take_stdout(&mut self) -> std::process::ChildStdout {
self.0.stdout.take().expect("stdout piped")
}
fn take_stderr(&mut self) -> std::process::ChildStderr {
self.0.stderr.take().expect("stderr piped")
}
}
impl Drop for ChildGuard {
fn drop(&mut self) {
let _ = self.0.kill();
let _ = self.0.wait();
}
}
/// `(path, is_dir, is_symlink, mode, len)` for everything under `root`, sorted.
/// Taken before and after each row so a refusal that creates, chmods, or
/// deletes anything fails the row.
fn snapshot(root: &Path) -> Vec<String> {
fn walk(dir: &Path, out: &mut Vec<String>) {
let Ok(entries) = std::fs::read_dir(dir) else {
return;
};
for e in entries.flatten() {
let p = e.path();
let Ok(md) = std::fs::symlink_metadata(&p) else {
continue;
};
out.push(format!(
"{} dir={} link={} mode={:04o} len={}",
p.display(),
md.is_dir(),
md.is_symlink(),
md.permissions().mode() & 0o7777,
if md.is_file() { md.len() } else { 0 }
));
if md.is_dir() && !md.is_symlink() {
walk(&p, out);
}
}
}
let mut out = Vec::new();
walk(root, &mut out);
out.sort();
out
}
struct Row {
/// Sandbox the row owns: HOME, repos, cwd and the key tree all live here.
home: tempfile::TempDir,
}
impl Row {
/// The only directory the no-mutation assertions watch.
fn tree(&self) -> PathBuf {
self.home.path().join("keytree")
}
}
impl Row {
fn new() -> Row {
let home = tempfile::tempdir().expect("tempdir");
std::fs::set_permissions(home.path(), std::fs::Permissions::from_mode(0o700))
.expect("chmod home");
// Created here, before any row takes its `before` snapshot, so the
// no-mutation assertions measure what the NODE did and not what this
// harness set up.
std::fs::create_dir_all(home.path().join("repos")).expect("repos dir");
// The p2p key tree lives in its own subdirectory so the no-mutation
// assertions can watch exactly it. HOME also holds `.gitlawb`, which
// the node creates for its NODE identity on every start; measuring the
// whole home would read that legitimate write as a p2p storage
// mutation.
let tree = home.path().join("keytree");
std::fs::create_dir(&tree).expect("key tree");
std::fs::set_permissions(&tree, std::fs::Permissions::from_mode(0o700))
.expect("chmod key tree");
Row { home }
}
fn spawn(&self, p2p_key: &str, p2p_port: &str, cwd: &Path) -> ChildGuard {
self.spawn_env(p2p_key, p2p_port, cwd, &[])
}
fn spawn_env(
&self,
p2p_key: &str,
p2p_port: &str,
cwd: &Path,
extra: &[(&str, &str)],
) -> ChildGuard {
let repos = self.home.path().join("repos");
let mut cmd = Command::new(env!("CARGO_BIN_EXE_gitlawb-node"));
cmd.env_clear()
.env("PATH", std::env::var("PATH").unwrap_or_default())
// The subscriber is built from the default env filter, so with a
// cleared environment the node logs nothing at all and every
// assertion here reads as a timeout with empty output. The rows
// are about which log lines appear, so the filter is part of the
// fixture, not incidental setup.
.env("RUST_LOG", "info")
.env("HOME", self.home.path())
.env("GITLAWB_REPOS_DIR", &repos)
.env("GITLAWB_HOST", "127.0.0.1")
// Ephemeral: the real port is read back from the ready log.
.env("GITLAWB_PORT", "0")
// A closed port, so the node stays on the degraded path for the
// whole row instead of ever reaching a database.
.env("DATABASE_URL", "postgres://127.0.0.1:1/nonexistent")
.env("GITLAWB_P2P_PORT", p2p_port)
.env("GITLAWB_P2P_KEY", p2p_key)
.env("GITLAWB_METRICS_ADDR", "")
.current_dir(cwd)
.stdout(Stdio::piped())
.stderr(Stdio::piped());
for (k, v) in extra {
cmd.env(k, v);
}
cmd.spawn().map(ChildGuard).expect("spawn gitlawb-node")
}
}
/// Read the child's stderr until `needle` appears, the stream ends, or the
/// deadline passes. Returns everything read.
///
/// The read runs on its own thread feeding a channel, and the deadline is
/// enforced with `recv_timeout`. Checking elapsed time between blocking
/// `read_line` calls does NOT work: a child that goes quiet blocks the read
/// forever and the check never runs, so the deadline looks present and cannot
/// fire. The first version of this file did that and hung for minutes on a row
/// that was supposed to fail in fifteen seconds.
fn read_until(child: &mut ChildGuard, needle: &str) -> (bool, String) {
read_until_all(child, &[needle])
}
/// Read until EVERY needle has appeared, the stream ends, or the deadline
/// passes.
///
/// Waiting on a single line is a race here: the degraded server logs "ready"
/// from a spawned task while the p2p gate logs its verdict from the main task,
/// so whichever the reader stops at first can leave the other uncaptured. That
/// made this file pass or fail depending on scheduling.
fn read_until_all(child: &mut ChildGuard, needles: &[&str]) -> (bool, String) {
let stdout = child.take_stdout();
let (tx, rx) = std::sync::mpsc::channel::<String>();
std::thread::spawn(move || {
let mut reader = BufReader::new(stdout);
loop {
let mut line = String::new();
match reader.read_line(&mut line) {
Ok(0) | Err(_) => break,
Ok(_) => {
if tx.send(line).is_err() {
break;
}
}
}
}
});
let mut acc = String::new();
let start = Instant::now();
loop {
if needles.iter().all(|n| acc.contains(n)) {
return (true, acc);
}
let left = match DEADLINE.checked_sub(start.elapsed()) {
Some(d) if !d.is_zero() => d,
_ => return (false, acc),
};
match rx.recv_timeout(left) {
Ok(line) => acc.push_str(&line),
// Sender gone: the stream ended. One last check, then give up.
Err(std::sync::mpsc::RecvTimeoutError::Disconnected) => {
return (needles.iter().all(|n| acc.contains(n)), acc)
}
Err(std::sync::mpsc::RecvTimeoutError::Timeout) => return (false, acc),
}
}
}
fn wait_for_exit(child: &mut ChildGuard) -> (std::process::ExitStatus, String) {
// Both streams: the refusal reason is anyhow's stderr print, while
// "binding HTTP listener" is a tracing line on stdout, and the fatal rows
// assert on one of each.
let mut out = String::new();
let mut o = child.take_stdout();
let mut e = child.take_stderr();
let reader = std::thread::spawn(move || {
let mut buf = String::new();
let _ = o.read_to_string(&mut buf);
buf
});
let _ = e.read_to_string(&mut out);
let status = child.0.wait().expect("wait");
out.push_str(&reader.join().unwrap_or_default());
(status, out)
}
/// Strip ANSI escapes: the subscriber colourises even when piped, which would
/// otherwise sit between `addr=` and the value.
fn strip_ansi(s: &str) -> String {
let mut out = String::with_capacity(s.len());
let mut chars = s.chars();
while let Some(c) = chars.next() {
if c == '\u{1b}' {
for c2 in chars.by_ref() {
if c2.is_ascii_alphabetic() {
break;
}
}
} else {
out.push(c);
}
}
out
}
/// A lexically invalid value must stop the process before the listener binds.
fn assert_fatal_before_bind(row: &Row, key: &str, cwd: &Path) {
let before = snapshot(&row.tree());
let mut child = row.spawn(key, "7546", cwd);
let (status, err) = wait_for_exit(&mut child);
assert!(
!status.success(),
"key={key:?} must exit non-zero\n--- stderr ---\n{err}"
);
assert!(
err.contains("invalid configuration"),
"key={key:?} must fail as invalid configuration\n--- stderr ---\n{err}"
);
assert!(
!err.contains("binding HTTP listener"),
"key={key:?} must be refused BEFORE the listener binds\n--- stderr ---\n{err}"
);
assert_eq!(
snapshot(&row.tree()),
before,
"key={key:?}: a configuration refusal must not touch the key tree"
);
}
/// A live storage fault must leave HTTP serving with p2p off, and must not
/// mutate the tree on its way to that verdict.
fn assert_degrades_with_http_up(row: &Row, key: &str, cwd: &Path, label: &str) {
let before = snapshot(&row.tree());
let mut child = row.spawn(key, "7546", cwd);
let (found, log) = read_until_all(
&mut child,
&[
"degraded HTTP server ready",
"failed to load p2p identity key",
],
);
assert!(
found,
"{label}: the node must bind and serve while p2p is off\n--- stderr ---\n{log}"
);
// The operator docs promise a specific signal, so pin it here: the level,
// the stable event name, and the message. A degrade that logs at warn or
// renames the event silently breaks the alert those docs tell operators to
// build, and asserting only the prose would not notice.
let clean_log = strip_ansi(&log);
let signal = clean_log
.lines()
.find(|l| l.contains("failed to load p2p identity key"))
.unwrap_or_else(|| panic!("{label}: the p2p failure must be logged\n{clean_log}"));
assert!(
signal.contains("ERROR"),
"{label}: the p2p failure must be logged at error level, got: {signal}"
);
assert!(
signal.contains("p2p_identity_key_load_failed"),
"{label}: the p2p failure must carry the documented event name, got: {signal}"
);
let clean = strip_ansi(&log);
let addr = clean
.lines()
.find(|l| l.contains("degraded HTTP server ready"))
.and_then(|l| l.rsplit("addr=").next())
.map(|a| {
a.trim()
.trim_start_matches("Some(")
.trim_matches(|c| c == '"' || c == ')')
})
.and_then(|a| a.parse::<std::net::SocketAddr>().ok())
.unwrap_or_else(|| panic!("{label}: could not parse the bound address from {clean}"));
// A served response, any status: the degraded server answers 503, which is
// still proof the port is up and answering.
let mut sock = std::net::TcpStream::connect(addr).expect("connect to the degraded server");
sock.set_read_timeout(Some(DEADLINE)).unwrap();
sock.write_all(b"GET /health HTTP/1.1\r\nHost: localhost\r\nConnection: close\r\n\r\n")
.expect("send request");
let mut resp = String::new();
let _ = sock.read_to_string(&mut resp);
assert!(
resp.starts_with("HTTP/1.1 "),
"{label}: the degraded server must answer, got {resp:?}"
);
drop(child);
assert_eq!(
snapshot(&row.tree()),
before,
"{label}: a storage refusal must not mutate the key tree"
);
}
#[test]
fn lexically_invalid_key_paths_stop_the_node_before_it_binds() {
for key in [
"p2p.key",
"./p2p.key",
"a/../p2p.key",
"../p2p.key",
"/p2p.key",
"keys/",
"~/",
"~//etc/p2p.key",
"~/../x/p2p.key",
] {
let row = Row::new();
let cwd = row.home.path().to_path_buf();
assert_fatal_before_bind(&row, key, &cwd);
}
}
#[test]
fn live_storage_faults_leave_http_up_with_p2p_off() {
// Symlinked final parent.
{
let row = Row::new();
let real = row.tree().join("real");
std::fs::create_dir(&real).unwrap();
std::fs::set_permissions(&real, std::fs::Permissions::from_mode(0o700)).unwrap();
let link = row.tree().join("keys");
std::os::unix::fs::symlink(&real, &link).unwrap();
let key = link.join("p2p.key");
let cwd = row.home.path().to_path_buf();
assert_degrades_with_http_up(&row, key.to_str().unwrap(), &cwd, "symlinked parent");
}
// Regular file where the key directory should be.
{
let row = Row::new();
let file = row.tree().join("keys");
std::fs::write(&file, b"not a directory").unwrap();
let key = file.join("p2p.key");
let cwd = row.home.path().to_path_buf();
assert_degrades_with_http_up(&row, key.to_str().unwrap(), &cwd, "regular-file parent");
}
// Unreadable parent: an inspection error, the purest instance of a live
// fault that used to be reported as invalid configuration.
{
let row = Row::new();
let parent = row.tree().join("locked");
std::fs::create_dir(&parent).unwrap();
std::fs::set_permissions(&parent, std::fs::Permissions::from_mode(0o000)).unwrap();
let key = parent.join("keys").join("p2p.key");
let cwd = row.home.path().to_path_buf();
assert_degrades_with_http_up(&row, key.to_str().unwrap(), &cwd, "unreadable parent");
// Restore so the tempdir can be cleaned up.
let _ = std::fs::set_permissions(&parent, std::fs::Permissions::from_mode(0o700));
}
// World-writable non-sticky ancestor: already the degrade class today, kept
// so the row set covers both sides of the policy.
{
let row = Row::new();
let anc = row.tree().join("open");
std::fs::create_dir(&anc).unwrap();
std::fs::set_permissions(&anc, std::fs::Permissions::from_mode(0o777)).unwrap();
let key = anc.join("keys").join("p2p.key");
let cwd = row.home.path().to_path_buf();
assert_degrades_with_http_up(&row, key.to_str().unwrap(), &cwd, "world-writable ancestor");
}
}
#[test]
fn port_zero_does_no_key_storage_io_at_all() {
// A lexically invalid path that would be fatal with p2p enabled.
{
let row = Row::new();
let before = snapshot(&row.tree());
let cwd = row.home.path().to_path_buf();
let mut child = row.spawn("p2p.key", "0", &cwd);
let (found, log) = read_until(&mut child, "p2p disabled");
assert!(
found,
"port zero must bypass key handling entirely\n--- stderr ---\n{log}"
);
drop(child);
assert_eq!(
snapshot(&row.tree()),
before,
"port zero must not touch the key tree"
);
}
// A hostile tree that would degrade with p2p enabled.
{
let row = Row::new();
let real = row.tree().join("real");
std::fs::create_dir(&real).unwrap();
let link = row.tree().join("keys");
std::os::unix::fs::symlink(&real, &link).unwrap();
let before = snapshot(&row.tree());
let cwd = row.home.path().to_path_buf();
let key: PathBuf = link.join("p2p.key");
let mut child = row.spawn(key.to_str().unwrap(), "0", &cwd);
let (found, log) = read_until(&mut child, "p2p disabled");
assert!(
found,
"port zero must bypass a hostile tree too\n--- stderr ---\n{log}"
);
drop(child);
assert_eq!(
snapshot(&row.tree()),
before,
"port zero must not touch a hostile key tree"
);
}
}
/// A terminal `.` is a directory-valued spelling. Path drops it, so the load
/// path would otherwise chmod the parent and publish a file named `keys`.
#[test]
fn terminal_dot_key_path_is_fatal_and_does_not_mutate() {
// Absent leaf: `/data/keys/.` must not create `keys` as a file or chmod `/data`.
{
let row = Row::new();
let data = row.tree().join("data");
std::fs::create_dir(&data).unwrap();
std::fs::set_permissions(&data, std::fs::Permissions::from_mode(0o755)).unwrap();
let key = format!("{}/keys/.", data.display());
let cwd = row.home.path().to_path_buf();
assert_fatal_before_bind(&row, &key, &cwd);
let mode = std::fs::metadata(&data).unwrap().permissions().mode() & 0o7777;
assert_eq!(mode, 0o755, "rejection must not chmod the parent");
assert!(
!data.join("keys").exists(),
"rejection must not create a file named keys"
);
}
// Existing directory at the would-be leaf: still boot-fatal, directory untouched.
{
let row = Row::new();
let data = row.tree().join("data");
let keys = data.join("keys");
std::fs::create_dir_all(&keys).unwrap();
std::fs::set_permissions(&data, std::fs::Permissions::from_mode(0o755)).unwrap();
std::fs::set_permissions(&keys, std::fs::Permissions::from_mode(0o700)).unwrap();
let key = format!("{}/.", keys.display());
let cwd = row.home.path().to_path_buf();
assert_fatal_before_bind(&row, &key, &cwd);
assert!(
keys.is_dir(),
"the existing keys directory must remain a directory"
);
let data_mode = std::fs::metadata(&data).unwrap().permissions().mode() & 0o7777;
let keys_mode = std::fs::metadata(&keys).unwrap().permissions().mode() & 0o7777;
assert_eq!(data_mode, 0o755, "rejection must not chmod the parent");
assert_eq!(
keys_mode, 0o700,
"rejection must not chmod the existing keys dir"
);
}
}
/// GITLAWB_KEY has the same Path-retarget hole as GITLAWB_P2P_KEY: a terminal
/// `.` would publish a file named `keys` under `data`. Fatal before bind, and
/// the parent tree is unchanged. p2p is off so it cannot chmod a shared dir.
#[test]
fn identity_terminal_dot_is_fatal_and_does_not_mutate() {
let row = Row::new();
let data = row.home.path().join("data");
std::fs::create_dir(&data).unwrap();
std::fs::set_permissions(&data, std::fs::Permissions::from_mode(0o755)).unwrap();
let key = format!("{}/keys/.", data.display());
let cwd = row.home.path().to_path_buf();
let mut child = row.spawn_env("p2p.key", "0", &cwd, &[("GITLAWB_KEY", key.as_str())]);
let (status, err) = wait_for_exit(&mut child);
assert!(
!status.success(),
"GITLAWB_KEY={key:?} must exit non-zero\n--- stderr ---\n{err}"
);
assert!(
err.contains("invalid configuration"),
"GITLAWB_KEY={key:?} must fail as invalid configuration\n--- stderr ---\n{err}"
);
assert!(
!err.contains("binding HTTP listener"),
"GITLAWB_KEY={key:?} must be refused BEFORE the listener binds\n--- stderr ---\n{err}"
);
let mode = std::fs::metadata(&data).unwrap().permissions().mode() & 0o7777;
assert_eq!(mode, 0o755, "rejection must not chmod the parent");
assert!(
!data.join("keys").exists(),
"rejection must not create a file named keys"
);
}
/// An existing 0755 identity parent is usable and must not be chmodded. p2p is
/// off so a default `~/.gitlawb/p2p.key` cannot tighten the same directory.
#[test]
fn identity_existing_0755_parent_is_not_chmodded_on_boot() {
let row = Row::new();
let iddir = row.home.path().join("idparent");
std::fs::create_dir(&iddir).unwrap();
std::fs::set_permissions(&iddir, std::fs::Permissions::from_mode(0o755)).unwrap();
let key = iddir.join("identity.pem");
let cwd = row.home.path().to_path_buf();
let mut child = row.spawn_env(
"p2p.key",
"0",
&cwd,
&[("GITLAWB_KEY", key.to_str().unwrap())],
);
let (found, log) = read_until(&mut child, "degraded HTTP server ready");
assert!(
found,
"a 0755 identity parent must still boot\n--- stderr ---\n{log}"
);
drop(child);
let mode = std::fs::metadata(&iddir).unwrap().permissions().mode() & 0o7777;
assert_eq!(
mode, 0o755,
"boot must not chmod the existing identity parent"
);
assert_eq!(
std::fs::metadata(&key).unwrap().permissions().mode() & 0o7777,
0o600,
"the identity key must be owner-only"
);
}
/// Creating into a world-writable identity parent is refused before bind, and
/// the directory is left untouched.
#[test]
fn identity_writable_parent_is_fatal_and_unchanged() {
let row = Row::new();
let iddir = row.home.path().join("idparent");
std::fs::create_dir(&iddir).unwrap();
std::fs::set_permissions(&iddir, std::fs::Permissions::from_mode(0o777)).unwrap();
let key = iddir.join("identity.pem");
let cwd = row.home.path().to_path_buf();
let mut child = row.spawn_env(
"p2p.key",
"0",
&cwd,
&[("GITLAWB_KEY", key.to_str().unwrap())],
);
let (status, err) = wait_for_exit(&mut child);
assert!(
!status.success(),
"a writable identity parent must exit non-zero\n--- stderr ---\n{err}"
);
assert!(
!err.contains("binding HTTP listener"),
"a writable identity parent must be refused BEFORE the listener binds\n--- stderr ---\n{err}"
);
let mode = std::fs::metadata(&iddir).unwrap().permissions().mode() & 0o7777;
assert_eq!(mode, 0o777, "refusal must not chmod the parent");
assert!(!key.exists(), "refusal must not publish the identity key");
}
/// A 2700 (setgid, owner rwx) p2p key directory is repairable: boot, tighten
/// to 0700, HTTP still comes up.
#[test]
fn p2p_setgid_key_directory_is_tightened_and_http_comes_up() {
let row = Row::new();
let keys = row.tree().join("keys");
std::fs::create_dir(&keys).unwrap();
std::fs::set_permissions(&keys, std::fs::Permissions::from_mode(0o2700)).unwrap();
let key = keys.join("p2p.key");
let cwd = row.home.path().to_path_buf();
let mut child = row.spawn(key.to_str().unwrap(), "7546", &cwd);
let (found, log) = read_until_all(
&mut child,
&["degraded HTTP server ready", "generated new p2p identity"],
);
assert!(
found,
"a 2700 key directory must boot and generate a key\n--- stderr ---\n{log}"
);
drop(child);
let mode = std::fs::metadata(&keys).unwrap().permissions().mode() & 0o7777;
assert_eq!(
mode, 0o700,
"2700 must be normalized to 0700, found {mode:04o}"
);
}
+3 -5
View File
@@ -1,7 +1,7 @@
[package]
name = "gl"
description = "The Twigpine CLI — identity management, node control, MCP server"
version = "0.7.1" # x-release-please-version
description = "The gitlawb CLI — identity management, node control, MCP server"
version = "0.5.1" # x-release-please-version
edition.workspace = true
rust-version.workspace = true
license.workspace = true
@@ -11,9 +11,8 @@ name = "gl"
path = "src/main.rs"
[dependencies]
gitlawb-core = { path = "../gitlawb-core", features = ["redirect"] }
gitlawb-core = { path = "../gitlawb-core" }
icaptcha-client = { path = "../icaptcha-client" }
base64 = { workspace = true }
tokio = { workspace = true }
serde = { workspace = true }
serde_json = { workspace = true }
@@ -42,4 +41,3 @@ alloy = { version = "1", default-features = false, features = [
mockito = "1"
tempfile = "3"
tokio = { workspace = true }
ed25519-dalek = { workspace = true }
+28 -27
View File
@@ -2,6 +2,7 @@
use anyhow::{Context, Result};
use clap::{Args, Subcommand};
use serde_json::Value;
use std::path::PathBuf;
use crate::http::NodeClient;
@@ -16,11 +17,7 @@ pub struct AgentArgs {
pub enum AgentCmd {
/// List agents registered on a node
List {
#[arg(
long,
default_value = "https://node.twigpine.com",
env = "GITLAWB_NODE"
)]
#[arg(long, default_value = "https://node.gitlawb.com", env = "GITLAWB_NODE")]
node: String,
/// Filter by capability (e.g. git:push)
#[arg(long)]
@@ -32,11 +29,7 @@ pub enum AgentCmd {
Show {
/// Agent DID or short key segment
did: String,
#[arg(
long,
default_value = "https://node.twigpine.com",
env = "GITLAWB_NODE"
)]
#[arg(long, default_value = "https://node.gitlawb.com", env = "GITLAWB_NODE")]
node: String,
},
}
@@ -64,16 +57,24 @@ async fn cmd_list(node: String, capability: Option<String>) -> Result<()> {
.get(&path)
.await
.context("failed to connect to node")?;
// Keep the bespoke 404 hint (older node without the agents API) ahead of
// read_json; every other status routes through it (capped + sanitized error).
if resp.status() == reqwest::StatusCode::NOT_FOUND {
let status = resp.status();
if status == reqwest::StatusCode::NOT_FOUND {
anyhow::bail!(
"this node does not yet support the agents API (v0.3+)\n\
upgrade the node or check GITLAWB_NODE is pointing to the right server"
);
}
let body = crate::http::read_json(resp, "list agents").await?;
let body: Value = resp
.json()
.await
.context("invalid JSON from node — is GITLAWB_NODE correct?")?;
if !status.is_success() {
let msg = body["message"].as_str().unwrap_or("unknown error");
anyhow::bail!("list agents failed ({status}): {msg}");
}
let agents = body["agents"].as_array().cloned().unwrap_or_default();
@@ -88,7 +89,7 @@ async fn cmd_list(node: String, capability: Option<String>) -> Result<()> {
let short = did
.split(':')
.next_back()
.map(|s| crate::text::truncate(s, 16))
.map(|s| &s[..s.len().min(16)])
.unwrap_or("?");
let trust = agent["trust_score"].as_f64().unwrap_or(0.0);
let caps = agent["capabilities"]
@@ -116,15 +117,24 @@ async fn cmd_show(did: String, node: String) -> Result<()> {
.get(&format!("/api/v1/agents/{did}"))
.await
.context("failed to connect to node")?;
// Keep the bespoke 404 hint ahead of read_json; other statuses route through it.
if resp.status() == reqwest::StatusCode::NOT_FOUND {
let status = resp.status();
if status == reqwest::StatusCode::NOT_FOUND {
anyhow::bail!(
"agent not found, or this node does not yet support the agents API (v0.3+)\n\
upgrade the node or check GITLAWB_NODE is pointing to the right server"
);
}
let agent = crate::http::read_json(resp, "agent").await?;
let agent: Value = resp
.json()
.await
.context("invalid JSON from node — is GITLAWB_NODE correct?")?;
if !status.is_success() {
let msg = agent["message"].as_str().unwrap_or("unknown error");
anyhow::bail!("agent not found ({status}): {msg}");
}
let full_did = agent["did"].as_str().unwrap_or("?");
let trust = agent["trust_score"].as_f64().unwrap_or(0.0);
@@ -205,14 +215,11 @@ mod tests {
.with_status(404)
.with_header("content-type", "application/json")
.with_body(r#"{"message":"not found"}"#)
.expect(1)
.create_async()
.await;
let err = cmd_list(server.url(), None).await.unwrap_err();
assert!(err.to_string().contains("agents API"));
// Prove the mocked route was actually requested; a non-matching request (mockito's 501, also non-2xx) would otherwise satisfy the error assertion vacuously.
_m.assert_async().await;
}
#[tokio::test]
@@ -223,14 +230,11 @@ mod tests {
.with_status(500)
.with_header("content-type", "application/json")
.with_body(r#"{"message":"internal error"}"#)
.expect(1)
.create_async()
.await;
let err = cmd_list(server.url(), None).await.unwrap_err();
assert!(err.to_string().contains("list agents failed"));
// Prove the mocked route was actually requested; a non-matching request (mockito's 501, also non-2xx) would otherwise satisfy the error assertion vacuously.
_m.assert_async().await;
}
#[tokio::test]
@@ -257,7 +261,6 @@ mod tests {
.with_status(404)
.with_header("content-type", "application/json")
.with_body(r#"{"message":"agent not found"}"#)
.expect(1)
.create_async()
.await;
@@ -265,8 +268,6 @@ mod tests {
.await
.unwrap_err();
assert!(err.to_string().contains("agents API") || err.to_string().contains("not found"));
// Prove the mocked route was actually requested; a non-matching request (mockito's 501, also non-2xx) would otherwise satisfy the error assertion vacuously.
_m.assert_async().await;
}
#[tokio::test]
+60 -146
View File
@@ -2,6 +2,7 @@
use anyhow::{Context, Result};
use clap::{Args, Subcommand};
use serde_json::Value;
use std::path::PathBuf;
use crate::http::NodeClient;
@@ -38,11 +39,7 @@ pub enum BountyCmd {
/// Deadline in seconds (default 7 days)
#[arg(long)]
deadline: Option<i64>,
#[arg(
long,
default_value = "https://node.twigpine.com",
env = "GITLAWB_NODE"
)]
#[arg(long, default_value = "https://node.gitlawb.com", env = "GITLAWB_NODE")]
node: String,
#[arg(long)]
dir: Option<PathBuf>,
@@ -54,11 +51,7 @@ pub enum BountyCmd {
/// Filter by status: open, claimed, submitted, completed
#[arg(long)]
status: Option<String>,
#[arg(
long,
default_value = "https://node.twigpine.com",
env = "GITLAWB_NODE"
)]
#[arg(long, default_value = "https://node.gitlawb.com", env = "GITLAWB_NODE")]
node: String,
#[arg(long)]
dir: Option<PathBuf>,
@@ -67,11 +60,7 @@ pub enum BountyCmd {
Show {
/// Bounty ID
id: String,
#[arg(
long,
default_value = "https://node.twigpine.com",
env = "GITLAWB_NODE"
)]
#[arg(long, default_value = "https://node.gitlawb.com", env = "GITLAWB_NODE")]
node: String,
#[arg(long)]
dir: Option<PathBuf>,
@@ -83,11 +72,7 @@ pub enum BountyCmd {
/// Your wallet address for payout
#[arg(long)]
wallet: Option<String>,
#[arg(
long,
default_value = "https://node.twigpine.com",
env = "GITLAWB_NODE"
)]
#[arg(long, default_value = "https://node.gitlawb.com", env = "GITLAWB_NODE")]
node: String,
#[arg(long)]
dir: Option<PathBuf>,
@@ -99,11 +84,7 @@ pub enum BountyCmd {
/// Pull request ID
#[arg(long)]
pr: String,
#[arg(
long,
default_value = "https://node.twigpine.com",
env = "GITLAWB_NODE"
)]
#[arg(long, default_value = "https://node.gitlawb.com", env = "GITLAWB_NODE")]
node: String,
#[arg(long)]
dir: Option<PathBuf>,
@@ -115,11 +96,7 @@ pub enum BountyCmd {
/// On-chain payout tx hash
#[arg(long)]
tx_hash: Option<String>,
#[arg(
long,
default_value = "https://node.twigpine.com",
env = "GITLAWB_NODE"
)]
#[arg(long, default_value = "https://node.gitlawb.com", env = "GITLAWB_NODE")]
node: String,
#[arg(long)]
dir: Option<PathBuf>,
@@ -128,22 +105,14 @@ pub enum BountyCmd {
Cancel {
/// Bounty ID
id: String,
#[arg(
long,
default_value = "https://node.twigpine.com",
env = "GITLAWB_NODE"
)]
#[arg(long, default_value = "https://node.gitlawb.com", env = "GITLAWB_NODE")]
node: String,
#[arg(long)]
dir: Option<PathBuf>,
},
/// Show bounty stats and leaderboard
Stats {
#[arg(
long,
default_value = "https://node.twigpine.com",
env = "GITLAWB_NODE"
)]
#[arg(long, default_value = "https://node.gitlawb.com", env = "GITLAWB_NODE")]
node: String,
},
}
@@ -220,7 +189,13 @@ async fn cmd_create(
.await
.context("failed to connect to node")?;
let body = crate::http::read_json(resp, "create bounty").await?;
let status = resp.status();
let body: Value = resp.json().await.unwrap_or_default();
if !status.is_success() {
let msg = body["message"].as_str().unwrap_or("unknown error");
anyhow::bail!("create bounty failed ({status}): {msg}");
}
let id = body["id"].as_str().unwrap_or("?");
println!("Bounty created: {id}");
@@ -256,14 +231,11 @@ async fn cmd_list(
u
};
let body = crate::http::read_json(
client
.get_authed(&url)
.await
.context("failed to connect to node")?,
"bounties",
)
.await?;
let resp = client
.get_authed(&url)
.await
.context("failed to connect to node")?;
let body: Value = resp.json().await.unwrap_or_default();
let bounties = body["bounties"].as_array();
if let Some(arr) = bounties {
@@ -275,7 +247,7 @@ async fn cmd_list(
let title = b["title"].as_str().unwrap_or("?");
let amount = b["amount"].as_i64().unwrap_or(0);
let st = b["status"].as_str().unwrap_or("?");
let short_id = crate::text::truncate(id, 8);
let short_id = &id[..8.min(id.len())];
println!("{short_id} {st:<10} {amount:>12} $GITLAWB {title}");
}
}
@@ -287,7 +259,13 @@ async fn cmd_show(id: String, node: String, dir: Option<PathBuf>) -> Result<()>
let client = signed_client(&node, dir.as_deref());
let resp = client.get_authed(&format!("/api/v1/bounties/{id}")).await?;
let body = crate::http::read_json(resp, "show bounty").await?;
let status = resp.status();
let body: Value = resp.json().await.unwrap_or_default();
if !status.is_success() {
let msg = body["message"].as_str().unwrap_or("not found");
anyhow::bail!("bounty not found: {msg}");
}
println!("{}", serde_json::to_string_pretty(&body)?);
Ok(())
@@ -311,7 +289,13 @@ async fn cmd_claim(
)
.await?;
let _ = crate::http::read_json(resp, "claim").await?;
let status = resp.status();
let body: Value = resp.json().await.unwrap_or_default();
if !status.is_success() {
let msg = body["message"].as_str().unwrap_or("unknown error");
anyhow::bail!("claim failed ({status}): {msg}");
}
println!("Bounty {id} claimed. Deadline starts now.");
Ok(())
@@ -330,7 +314,13 @@ async fn cmd_submit(id: String, pr: String, node: String, dir: Option<PathBuf>)
)
.await?;
let _ = crate::http::read_json(resp, "submit").await?;
let status = resp.status();
let body: Value = resp.json().await.unwrap_or_default();
if !status.is_success() {
let msg = body["message"].as_str().unwrap_or("unknown error");
anyhow::bail!("submit failed ({status}): {msg}");
}
println!("Bounty {id} submitted with PR {pr}. Awaiting creator approval.");
Ok(())
@@ -354,7 +344,13 @@ async fn cmd_approve(
)
.await?;
let _ = crate::http::read_json(resp, "approve").await?;
let status = resp.status();
let body: Value = resp.json().await.unwrap_or_default();
if !status.is_success() {
let msg = body["message"].as_str().unwrap_or("unknown error");
anyhow::bail!("approve failed ({status}): {msg}");
}
println!("Bounty {id} approved! Payout released to agent.");
Ok(())
@@ -373,7 +369,13 @@ async fn cmd_cancel(id: String, node: String, dir: Option<PathBuf>) -> Result<()
)
.await?;
let _ = crate::http::read_json(resp, "cancel").await?;
let status = resp.status();
let body: Value = resp.json().await.unwrap_or_default();
if !status.is_success() {
let msg = body["message"].as_str().unwrap_or("unknown error");
anyhow::bail!("cancel failed ({status}): {msg}");
}
println!("Bounty {id} cancelled. Tokens refunded.");
Ok(())
@@ -381,11 +383,8 @@ async fn cmd_cancel(id: String, node: String, dir: Option<PathBuf>) -> Result<()
async fn cmd_stats(node: String) -> Result<()> {
let client = NodeClient::new(&node, None);
// Route through read_json so a node denial (403/404/5xx, incl. a non-JSON
// body) surfaces as an error instead of a fake `Bounty Stats` with every
// counter zeroed (#186, the denial-as-success class #123 fixes elsewhere).
let body =
crate::http::read_json(client.get("/api/v1/bounties/stats").await?, "bounty stats").await?;
let resp = client.get("/api/v1/bounties/stats").await?;
let body: Value = resp.json().await.unwrap_or_default();
let open = body["open"].as_i64().unwrap_or(0);
let claimed = body["claimed"].as_i64().unwrap_or(0);
@@ -403,7 +402,7 @@ async fn cmd_stats(node: String) -> Result<()> {
let did = entry["did"].as_str().unwrap_or("?");
let count = entry["completed"].as_i64().unwrap_or(0);
let earned = entry["total_earned"].as_i64().unwrap_or(0);
let short = crate::text::truncate_suffix(did, 8);
let short = &did[did.len().saturating_sub(8)..];
println!(
" {}. ...{short} {count} bounties {earned} $GITLAWB",
i + 1
@@ -522,7 +521,6 @@ mod tests {
.with_status(404)
.with_header("content-type", "application/json")
.with_body(r#"{"message":"not found"}"#)
.expect(1)
.create_async()
.await;
@@ -530,8 +528,6 @@ mod tests {
.await
.unwrap_err();
assert!(err.to_string().contains("not found"));
// Prove the mocked route was actually requested; a non-matching request (mockito's 501, also non-2xx) would otherwise satisfy the error assertion vacuously.
_m.assert_async().await;
}
#[tokio::test]
@@ -548,86 +544,4 @@ mod tests {
cmd_stats(server.url()).await.unwrap();
}
/// #186 (F1): a node denial on the stats endpoint must surface as an error,
/// not print a fake `Bounty Stats` with every counter zeroed. RED before the
/// fix (unwrap_or_default + no status check -> Ok with zeros), GREEN after
/// (read_json bails on the non-2xx).
#[tokio::test]
async fn cmd_stats_surfaces_denial_not_fake_result() {
let mut server = mockito::Server::new_async().await;
let m = server
.mock("GET", mockito::Matcher::Regex(r"/stats$".to_string()))
.with_status(403)
.with_header("content-type", "application/json")
.with_body(r#"{"message":"forbidden"}"#)
.expect(1)
.create_async()
.await;
let err = cmd_stats(server.url())
.await
.expect_err("a 403 must be an error, not a zeroed fake result");
assert!(err.to_string().contains("403"), "err={err}");
// Pin the mock hit explicitly. The status assertion above already fails
// when the route does not match (mockito answers an unmatched route with
// 501, so "403" is absent), but that proof is incidental to mockito's
// fallback status; assert the hit directly so it survives a reworded
// assertion.
m.assert_async().await;
}
/// #186 (F1): the same denial-as-error behavior across the non-2xx response
/// classes, not just 403 — a 404 and a 500 (incl. a non-JSON body) must each
/// surface as an error with the status, never a zeroed fake result.
#[tokio::test]
async fn cmd_stats_surfaces_denial_across_status_classes() {
for (status, body, json) in [
(404, r#"{"message":"not found"}"#, true),
(500, "internal boom", false),
] {
let mut server = mockito::Server::new_async().await;
let mut m = server
.mock("GET", mockito::Matcher::Regex(r"/stats$".to_string()))
.with_status(status)
.with_body(body);
if json {
m = m.with_header("content-type", "application/json");
}
let m = m.expect(1).create_async().await;
let err = cmd_stats(server.url()).await.unwrap_err().to_string();
assert!(
err.contains(&status.to_string()),
"status {status} must surface as an error, got: {err}"
);
m.assert_async().await;
}
}
#[tokio::test]
async fn cmd_list_repo_scoped_surfaces_denial_not_empty() {
// `gl bounty list --repo owner/name` hits the gated /repos/{o}/{n}/bounties;
// a 404 must Err, not silently print nothing.
let mut server = mockito::Server::new_async().await;
let _m = server
.mock(
"GET",
mockito::Matcher::Regex(r"/repos/alice/secret/bounties".to_string()),
)
.with_status(404)
.with_header("content-type", "application/json")
.with_body(r#"{"message":"repository 'alice/secret' not found"}"#)
.expect(1)
.create_async()
.await;
let result = cmd_list(Some("alice/secret".to_string()), None, server.url(), None).await;
assert!(
result.is_err(),
"bounty list --repo must Err on a gated 404"
);
// Prove the gated repo-scoped path was actually requested: without this,
// an unmatched route (mockito's 501, also non-2xx) would satisfy is_err().
_m.assert_async().await;
}
}
+58 -823
View File
@@ -4,6 +4,7 @@
use anyhow::{Context, Result};
use clap::{Args, Subcommand};
use serde_json::Value;
use std::path::PathBuf;
use crate::http::NodeClient;
@@ -25,11 +26,7 @@ pub enum CertCmd {
List {
/// Repository in <owner>/<repo> or <repo> format
repo: String,
#[arg(
long,
default_value = "https://node.twigpine.com",
env = "GITLAWB_NODE"
)]
#[arg(long, default_value = "https://node.gitlawb.com", env = "GITLAWB_NODE")]
node: String,
#[arg(long)]
dir: Option<PathBuf>,
@@ -40,24 +37,10 @@ pub enum CertCmd {
repo: String,
/// Certificate ID
id: String,
#[arg(
long,
default_value = "https://node.twigpine.com",
env = "GITLAWB_NODE"
)]
#[arg(long, default_value = "https://node.gitlawb.com", env = "GITLAWB_NODE")]
node: String,
#[arg(long)]
dir: Option<PathBuf>,
/// Exit non-zero unless the Ed25519 signature verifies AND the
/// issuing node matches the queried node (or --expect-node)
#[arg(long)]
verify: bool,
/// Expected issuing node DID for --verify. A valid signature alone
/// only proves the cert is internally consistent — signed by whatever
/// key it names — so --verify also anchors the issuer to a DID you
/// trust: this value when given, else the queried node's DID.
#[arg(long, requires = "verify")]
expect_node: Option<String>,
},
}
@@ -69,9 +52,7 @@ pub async fn run(args: CertArgs) -> Result<()> {
id,
node,
dir,
verify,
expect_node,
} => cmd_show(repo, id, node, dir, verify, expect_node).await,
} => cmd_show(repo, id, node, dir).await,
}
}
@@ -89,7 +70,12 @@ async fn resolve_repo(
did.split(':').next_back().unwrap_or(&did).to_string()
} else {
let client = signed_client(node, dir);
let info = crate::http::read_json(client.get_authed("/").await?, "node info").await?;
let info: Value = client
.get_authed("/")
.await?
.json()
.await
.context("failed to fetch node info")?;
let did = info["did"].as_str().context("node info missing 'did'")?;
did.split(':').next_back().unwrap_or(did).to_string()
};
@@ -102,7 +88,12 @@ async fn cmd_list(repo: String, node: String, dir: Option<PathBuf>) -> Result<()
let client = signed_client(&node, dir.as_deref());
let path = format!("/api/v1/repos/{owner}/{name}/certs");
let resp = crate::http::read_json(client.get_authed(&path).await?, "certificates").await?;
let resp: Value = client
.get_authed(&path)
.await?
.json()
.await
.context("failed to list certificates")?;
let certs = resp["certificates"].as_array().cloned().unwrap_or_default();
@@ -117,32 +108,26 @@ async fn cmd_list(repo: String, node: String, dir: Option<PathBuf>) -> Result<()
let id = cert["id"].as_str().unwrap_or("?");
let ref_name = cert["ref_name"].as_str().unwrap_or("?");
let new_sha = cert["new_sha"].as_str().unwrap_or("?");
let issued_at = cert["issued_at"]
.as_str()
.map(|s| crate::text::truncate(s, 19))
.unwrap_or("?");
let issued_at = cert["issued_at"].as_str().map(|s| &s[..19]).unwrap_or("?");
println!(" {id:.8} {issued_at} {ref_name} {new_sha:.12}");
}
Ok(())
}
async fn cmd_show(
repo: String,
id: String,
node: String,
dir: Option<PathBuf>,
require_valid: bool,
expect_node: Option<String>,
) -> Result<()> {
async fn cmd_show(repo: String, id: String, node: String, dir: Option<PathBuf>) -> Result<()> {
let (owner, name) = resolve_repo(&repo, &node, dir.as_deref()).await?;
let client = signed_client(&node, dir.as_deref());
let id = resolve_cert_id(&client, &owner, &name, &id).await?;
// Fetch the certificate. read_json checks status first and surfaces the node's
// capped+sanitized message on a non-2xx (a bounded error read, not the whole body).
// Fetch the certificate
let path = format!("/api/v1/repos/{owner}/{name}/certs/{id}");
let cert = crate::http::read_json(client.get_authed(&path).await?, "certificate").await?;
let resp = client
.get_authed(&path)
.await?
.error_for_status()
.context("certificate not found")?;
let cert: Value = resp.json().await.context("certificate not found")?;
let cert_id = cert["id"].as_str().unwrap_or("?");
let ref_name = cert["ref_name"].as_str().unwrap_or("?");
@@ -163,164 +148,53 @@ async fn cmd_show(
println!(" Signature: {signature}");
println!();
// Verify the Ed25519 signature: rebuild the exact canonical payload the
// node signed (see gitlawb-node/src/cert.rs::issue_ref_certificate) and
// check it against the public key embedded in the certificate's node DID.
// This proves the cert is internally authentic — signed by the key it
// names; the node-DID comparison below covers *which* node that is.
let repo_id = cert["repo_id"].as_str().unwrap_or("");
let verdict = verify_signature(
repo_id, ref_name, old_sha, new_sha, pusher, node_did, issued_at, signature,
);
// Reconstruct the signing payload and verify
// Fetch the node's current public key to verify
let info: Value = client
.get("/")
.await?
.json()
.await
.context("failed to fetch node info")?;
let current_node_did = info["did"].as_str().unwrap_or("");
println!("Signature verification:");
match &verdict {
Ok(()) => {
println!(
" VALID — Ed25519 signature verified against the key the certificate names ({node_did})"
);
}
Err(reason) => {
println!(" INVALID — {reason}");
}
println!(" Signing payload would be:");
println!(" {{\"repo_id\": ..., \"ref\": \"{ref_name}\", \"old\": \"{old_sha}\",");
println!(" \"new\": \"{new_sha}\", \"pusher\": \"{pusher}\",");
println!(" \"node\": \"{node_did}\", \"ts\": \"{issued_at}\"}}");
println!();
if current_node_did == node_did {
println!(" Node DID matches current node. Signature is an Ed25519/base64url value.");
println!(" To verify offline, use the node's Ed25519 public key derived from:");
println!(" did:key → {node_did}");
} else {
println!(" WARNING: Certificate node DID ({node_did}) does not match");
println!(" current node DID ({current_node_did}).");
println!(" This certificate was issued by a different node.");
}
// Contextual only — the verdict above stands on its own, so a node-info
// hiccup here must not turn a successfully displayed certificate into an
// error exit. Route the lookup through read_json so a denial or a capped
// error body yields a reportable reason instead of an opaque None, and keep
// "carried no DID" distinct from "the lookup failed": an empty DID must not
// reach the comparison and fabricate a mismatch warning.
let current: std::result::Result<String, String> = match client.get("/").await {
Ok(resp) => match crate::http::read_json(resp, "node info").await {
Ok(info) => did_from_node_info(&info),
Err(e) => Err(e.to_string()),
},
Err(e) => Err(e.to_string()),
};
let current_node_did = current.as_ref().ok().cloned();
for line in did_check_report(&current, node_did) {
println!("{line}");
}
if require_valid {
if let Err(reason) = verdict {
anyhow::bail!("certificate signature did not verify: {reason}");
}
// A valid signature proves internal consistency only: the payload was
// signed by whatever key the certificate itself names. A hostile
// source can mint a keypair, put its DID in node_did, and self-sign.
// --verify therefore also anchors the issuer to a trusted DID:
// --expect-node when given, else the DID of the node being queried.
let expected = expect_node.as_deref().or(current_node_did.as_deref());
match expected {
Some(expected) if expected == node_did => {}
Some(expected) => anyhow::bail!(
"certificate is signed by {node_did}, but the expected issuer is {expected} — \
a valid signature alone proves internal consistency, not a trusted issuer"
),
None => anyhow::bail!(
"cannot anchor the issuer: node info is unreachable and no --expect-node was given"
),
}
}
println!();
println!(" Signature (base64url): {signature}");
Ok(())
}
/// Pull the node's DID out of a `GET /` body for the comparison in `cmd_show`.
///
/// A missing OR empty `did` is a failure, not a value: letting `""` through
/// would reach the comparison and print a mismatch WARNING against a DID the
/// node never claimed, which reads as "issued by a different node" when the
/// truth is that the lookup told us nothing.
fn did_from_node_info(info: &serde_json::Value) -> std::result::Result<String, String> {
match info["did"].as_str() {
Some(did) if !did.is_empty() => Ok(did.to_string()),
_ => Err("node info response carried no DID".to_string()),
}
}
/// Select the report lines for the node-DID comparison in `cmd_show`.
///
/// `current` is the current node's DID, or the reason it could not be
/// determined. A comparison verdict (match or WARNING) is only produced when a
/// real DID was obtained; otherwise the report degrades to a could-not-compare
/// hint naming the reason, plus the offline-verification guidance. The signature
/// verdict printed above stands on its own either way — this block only answers
/// *which* node issued the certificate.
fn did_check_report(current: &std::result::Result<String, String>, node_did: &str) -> Vec<String> {
match current {
Ok(current) if current == node_did => {
vec![" Issuing node DID matches the node being queried.".to_string()]
}
Ok(current) => vec![
format!(" WARNING: Certificate node DID ({node_did}) does not match"),
format!(" current node DID ({current})."),
" This certificate was issued by a different node.".to_string(),
],
Err(reason) => vec![
format!(" Could not fetch the current node's DID ({reason}), so the comparison"),
" with the certificate's node DID is unavailable.".to_string(),
" To verify offline, use the node's Ed25519 public key derived from:".to_string(),
format!(" did:key → {node_did}"),
],
}
}
/// Rebuild the node's canonical signing payload (field order must match
/// gitlawb-node/src/cert.rs::issue_ref_certificate exactly) and verify the
/// certificate's Ed25519 signature against the key embedded in `node_did`.
#[allow(clippy::too_many_arguments)]
fn verify_signature(
repo_id: &str,
ref_name: &str,
old_sha: &str,
new_sha: &str,
pusher: &str,
node_did: &str,
issued_at: &str,
signature_b64: &str,
) -> std::result::Result<(), String> {
use base64::{engine::general_purpose::URL_SAFE_NO_PAD, Engine};
use std::str::FromStr;
let payload = serde_json::json!({
"repo_id": repo_id,
"ref": ref_name,
"old": old_sha,
"new": new_sha,
"pusher": pusher,
"node": node_did,
"ts": issued_at,
});
let payload_bytes =
serde_json::to_vec(&payload).map_err(|e| format!("could not serialize payload: {e}"))?;
let did =
gitlawb_core::did::Did::from_str(node_did).map_err(|e| format!("bad node DID: {e}"))?;
let verifying_key = did
.to_verifying_key()
.map_err(|e| format!("cannot derive a public key from {node_did}: {e}"))?;
let sig_vec = URL_SAFE_NO_PAD
.decode(signature_b64)
.map_err(|e| format!("signature is not valid base64url: {e}"))?;
let sig_bytes: [u8; 64] = sig_vec
.try_into()
.map_err(|_| "signature is not 64 bytes".to_string())?;
gitlawb_core::identity::verify(&verifying_key, &payload_bytes, &sig_bytes)
.map_err(|_| "Ed25519 signature does not match the signed payload".to_string())
}
async fn resolve_cert_id(client: &NodeClient, owner: &str, name: &str, id: &str) -> Result<String> {
if id.len() >= 36 {
return Ok(id.to_string());
}
let path = format!("/api/v1/repos/{owner}/{name}/certs?prefix={id}");
let resp = crate::http::read_json(client.get_authed(&path).await?, "certificates").await?;
let resp: Value = client
.get_authed(&path)
.await?
.error_for_status()
.context("failed to list certificates")?
.json()
.await
.context("failed to list certificates")?;
let certs = resp["certificates"].as_array().cloned().unwrap_or_default();
let matches: Vec<String> = certs
@@ -335,642 +209,3 @@ async fn resolve_cert_id(client: &NodeClient, owner: &str, name: &str, id: &str)
_ => anyhow::bail!("certificate prefix {id} matches multiple certificates"),
}
}
#[cfg(test)]
mod tests {
use super::*;
/// Pins gl's payload reconstruction to the frozen canonical byte form the
/// node signs (default serde_json maps = alphabetically ordered keys). If
/// serialization drifts — a field added, or a preserve_order feature
/// landing anywhere in the workspace (feature unification flips every
/// crate at once) — this literal stops matching and the test fails,
/// instead of every real certificate silently rendering INVALID.
#[test]
fn payload_serialization_matches_frozen_canonical_form() {
let payload = serde_json::json!({
"repo_id": "repo-1",
"ref": "refs/heads/main",
"old": "oldsha",
"new": "newsha",
"pusher": "did:key:z6MkPusher",
"node": "did:key:z6MkNode",
"ts": "2026-07-22T00:00:00+00:00",
});
let frozen = concat!(
r#"{"new":"newsha","node":"did:key:z6MkNode","old":"oldsha","#,
r#""pusher":"did:key:z6MkPusher","ref":"refs/heads/main","#,
r#""repo_id":"repo-1","ts":"2026-07-22T00:00:00+00:00"}"#,
);
assert_eq!(serde_json::to_string(&payload).unwrap(), frozen);
}
/// Signing exactly as the node does must round-trip through
/// verify_signature; any field tampering must fail it.
#[test]
fn verify_signature_round_trip_and_tamper() {
let kp = gitlawb_core::identity::Keypair::generate();
let node_did = kp.did().as_str().to_string();
let payload = serde_json::json!({
"repo_id": "repo-1",
"ref": "refs/heads/main",
"old": "0".repeat(40),
"new": "a".repeat(40),
"pusher": "did:key:z6MkPusher",
"node": node_did,
"ts": "2026-07-22T00:00:00+00:00",
});
let sig = kp.sign_b64(&serde_json::to_vec(&payload).unwrap());
let ok = verify_signature(
"repo-1",
"refs/heads/main",
&"0".repeat(40),
&"a".repeat(40),
"did:key:z6MkPusher",
&node_did,
"2026-07-22T00:00:00+00:00",
&sig,
);
assert!(ok.is_ok(), "expected valid signature, got: {ok:?}");
let tampered = verify_signature(
"repo-1",
"refs/heads/main",
&"0".repeat(40),
&"b".repeat(40), // new_sha changed after signing
"did:key:z6MkPusher",
&node_did,
"2026-07-22T00:00:00+00:00",
&sig,
);
assert!(tampered.is_err(), "tampered payload must not verify");
let garbage = verify_signature(
"repo-1",
"refs/heads/main",
&"0".repeat(40),
&"a".repeat(40),
"did:key:z6MkPusher",
&node_did,
"2026-07-22T00:00:00+00:00",
"not-base64url!!!",
);
assert!(garbage.is_err(), "malformed signature must not verify");
}
#[tokio::test]
async fn cmd_list_surfaces_denial_not_empty() {
// A gated 404 on the repo-scoped certs read must Err, not print "No certificates".
let mut server = mockito::Server::new_async().await;
let _m = server
.mock(
"GET",
mockito::Matcher::Regex(r"^/api/v1/repos/alice/secret/certs$".to_string()),
)
.with_status(404)
.with_header("content-type", "application/json")
.with_body(r#"{"message":"repository 'alice/secret' not found"}"#)
.expect(1)
.create_async()
.await;
let result = cmd_list("alice/secret".to_string(), server.url(), None).await;
assert!(result.is_err(), "cert list must Err on a gated 404");
// Prove the gated certs path was actually requested: without this, an
// unmatched route (mockito's 501, also non-2xx) would satisfy is_err().
_m.assert_async().await;
}
#[tokio::test]
async fn resolve_repo_surfaces_denial() {
// A slash-free repo with an empty identity dir forces the GET / node-info
// fetch. A gated 404 there must Err (surfacing the status), proving the
// read_json conversion is load-bearing rather than silently ignored.
let mut server = mockito::Server::new_async().await;
let dir = tempfile::TempDir::new().unwrap(); // empty, no identity.pem, forces the GET / branch
let _m = server
.mock("GET", "/")
.with_status(404)
.with_header("content-type", "application/json")
.with_body(r#"{"message":"denied"}"#)
.expect(1)
.create_async()
.await;
let err = resolve_repo("noslash", &server.url(), Some(dir.path()))
.await
.unwrap_err();
assert!(err.to_string().contains("404"), "got: {err}");
_m.assert_async().await;
}
// The `GET /` node-info lookup after the cert loads is a fail-soft diagnostic:
// a response-level failure degrades to a could-not-compare hint and the command
// completes Ok, never a fabricated mismatch warning and never a fatal Err. The
// cert fetch itself stays fail-closed. A >=36-char id skips resolve_cert_id so
// only two mocks are needed.
#[tokio::test]
async fn cmd_show_completes_with_degraded_hint_when_node_info_denied() {
let mut server = mockito::Server::new_async().await;
let long_id = "a".repeat(36);
let _cert = server
.mock(
"GET",
format!("/api/v1/repos/alice/secret/certs/{long_id}").as_str(),
)
.with_status(200)
.with_header("content-type", "application/json")
.with_body(
r#"{"id":"c1","ref_name":"refs/heads/main","old_sha":"0","new_sha":"1","pusher_did":"p","node_did":"n","signature":"s","issued_at":"2026-01-01T00:00:00Z"}"#,
)
.create_async()
.await;
let _root = server
.mock("GET", "/")
.with_status(403)
.with_header("content-type", "application/json")
.with_body(r#"{"message":"denied"}"#)
.expect(1)
.create_async()
.await;
let result = cmd_show(
"alice/secret".to_string(),
long_id,
server.url(),
None,
false,
None,
)
.await;
assert!(
result.is_ok(),
"cert show must complete despite a denied node-info lookup"
);
_cert.assert_async().await;
_root.assert_async().await;
}
#[tokio::test]
async fn cmd_show_degrades_on_malformed_node_info() {
// A 2xx node-info body that fails to parse degrades the same way a denial
// does: hint printed, command completes Ok.
let mut server = mockito::Server::new_async().await;
let long_id = "a".repeat(36);
let _cert = server
.mock(
"GET",
format!("/api/v1/repos/alice/secret/certs/{long_id}").as_str(),
)
.with_status(200)
.with_header("content-type", "application/json")
.with_body(
r#"{"id":"c1","ref_name":"refs/heads/main","old_sha":"0","new_sha":"1","pusher_did":"p","node_did":"n","signature":"s","issued_at":"2026-01-01T00:00:00Z"}"#,
)
.create_async()
.await;
let _root = server
.mock("GET", "/")
.with_status(200)
.with_body("not json")
.expect(1)
.create_async()
.await;
let result = cmd_show(
"alice/secret".to_string(),
long_id,
server.url(),
None,
false,
None,
)
.await;
assert!(
result.is_ok(),
"cert show must complete despite malformed node info"
);
_cert.assert_async().await;
_root.assert_async().await;
}
#[tokio::test]
async fn cmd_show_degrades_when_node_info_lacks_did() {
// A 2xx node-info body with no `did` routes to the degraded hint, not a
// fabricated empty-DID mismatch warning; the command completes Ok.
let mut server = mockito::Server::new_async().await;
let long_id = "a".repeat(36);
let _cert = server
.mock(
"GET",
format!("/api/v1/repos/alice/secret/certs/{long_id}").as_str(),
)
.with_status(200)
.with_header("content-type", "application/json")
.with_body(
r#"{"id":"c1","ref_name":"refs/heads/main","old_sha":"0","new_sha":"1","pusher_did":"p","node_did":"n","signature":"s","issued_at":"2026-01-01T00:00:00Z"}"#,
)
.create_async()
.await;
let _root = server
.mock("GET", "/")
.with_status(200)
.with_header("content-type", "application/json")
.with_body("{}")
.expect(1)
.create_async()
.await;
let result = cmd_show(
"alice/secret".to_string(),
long_id,
server.url(),
None,
false,
None,
)
.await;
assert!(
result.is_ok(),
"cert show must complete when node info lacks a DID"
);
_cert.assert_async().await;
_root.assert_async().await;
}
// Must-not case: the certificate fetch itself stays fail-closed. A gated 404
// aborts the command with the status surfaced, and the node-info lookup is
// never reached (the expect(0) assert proves it never ran).
#[tokio::test]
async fn cmd_show_surfaces_denied_certificate() {
let mut server = mockito::Server::new_async().await;
let long_id = "a".repeat(36);
let _cert = server
.mock(
"GET",
format!("/api/v1/repos/alice/secret/certs/{long_id}").as_str(),
)
.with_status(404)
.with_header("content-type", "application/json")
.with_body(r#"{"message":"repository not found"}"#)
.expect(1)
.create_async()
.await;
let _root = server
.mock("GET", "/")
.with_status(200)
.with_header("content-type", "application/json")
.with_body(r#"{"did":"n"}"#)
.expect(0)
.create_async()
.await;
let err = cmd_show(
"alice/secret".to_string(),
long_id,
server.url(),
None,
false,
None,
)
.await
.unwrap_err();
assert!(err.to_string().contains("404"), "got: {err}");
_cert.assert_async().await;
_root.assert_async().await;
}
#[tokio::test]
async fn cmd_show_reports_matching_node_did() {
// Pins the unchanged success path: node info fetched, DIDs compared, Ok.
let mut server = mockito::Server::new_async().await;
let long_id = "a".repeat(36);
let _cert = server
.mock(
"GET",
format!("/api/v1/repos/alice/secret/certs/{long_id}").as_str(),
)
.with_status(200)
.with_header("content-type", "application/json")
.with_body(
r#"{"id":"c1","ref_name":"refs/heads/main","old_sha":"0","new_sha":"1","pusher_did":"p","node_did":"n","signature":"s","issued_at":"2026-01-01T00:00:00Z"}"#,
)
.create_async()
.await;
let _root = server
.mock("GET", "/")
.with_status(200)
.with_header("content-type", "application/json")
.with_body(r#"{"did":"n"}"#)
.expect(1)
.create_async()
.await;
let result = cmd_show(
"alice/secret".to_string(),
long_id,
server.url(),
None,
false,
None,
)
.await;
assert!(result.is_ok(), "cert show must report a matching node DID");
_cert.assert_async().await;
_root.assert_async().await;
}
#[tokio::test]
async fn cmd_show_warns_on_mismatching_node_did() {
// A real, differing node DID drives the WARNING branch end to end; the
// command still completes Ok.
let mut server = mockito::Server::new_async().await;
let long_id = "a".repeat(36);
let _cert = server
.mock(
"GET",
format!("/api/v1/repos/alice/secret/certs/{long_id}").as_str(),
)
.with_status(200)
.with_header("content-type", "application/json")
.with_body(
r#"{"id":"c1","ref_name":"refs/heads/main","old_sha":"0","new_sha":"1","pusher_did":"p","node_did":"n","signature":"s","issued_at":"2026-01-01T00:00:00Z"}"#,
)
.create_async()
.await;
let _root = server
.mock("GET", "/")
.with_status(200)
.with_header("content-type", "application/json")
.with_body(r#"{"did":"did:key:other"}"#)
.expect(1)
.create_async()
.await;
let result = cmd_show(
"alice/secret".to_string(),
long_id,
server.url(),
None,
false,
None,
)
.await;
assert!(
result.is_ok(),
"cert show must warn, not fail, on a mismatching node DID"
);
_cert.assert_async().await;
_root.assert_async().await;
}
// ── --verify issuer anchoring ────────────────────────────────────────────
//
// Every other cmd_show test passes require_valid=false, so the whole
// `if require_valid` block went unexecuted. It is the security-bearing half of
// the command: a valid signature only proves the certificate is internally
// consistent (a hostile node can mint a keypair, name it in node_did, and
// self-sign), so --verify must additionally anchor the issuer to a DID the
// caller trusts. These drive all four outcomes plus the must-not case.
//
// The certificate must carry a REAL signature over the canonical payload:
// with a bogus one, --verify bails on the signature and never reaches the
// anchoring, which would make every assertion below vacuous.
fn signed_cert(node_kp: &gitlawb_core::identity::Keypair) -> (String, String) {
let node_did = node_kp.did().as_str().to_string();
let id = "a".repeat(36); // >= 36 chars skips resolve_cert_id's prefix lookup
let payload = serde_json::json!({
"repo_id": "repo-1",
"ref": "refs/heads/main",
"old": "0".repeat(40),
"new": "b".repeat(40),
"pusher": "did:key:z6MkPusher",
"node": node_did,
"ts": "2026-07-22T00:00:00+00:00",
});
let sig = node_kp.sign_b64(&serde_json::to_vec(&payload).unwrap());
let body = serde_json::json!({
"id": id,
"repo_id": "repo-1",
"ref_name": "refs/heads/main",
"old_sha": "0".repeat(40),
"new_sha": "b".repeat(40),
"pusher_did": "did:key:z6MkPusher",
"node_did": node_did,
"signature": sig,
"issued_at": "2026-07-22T00:00:00+00:00",
})
.to_string();
(id, body)
}
/// Mount the cert fetch, plus a `GET /` answering with `node_info` (a JSON
/// body) or a denial when `None`.
async fn cert_server(
server: &mut mockito::Server,
id: &str,
cert_body: &str,
node_info: Option<&str>,
) -> (mockito::Mock, mockito::Mock) {
let cert = server
.mock("GET", format!("/api/v1/repos/alice/r/certs/{id}").as_str())
.with_status(200)
.with_header("content-type", "application/json")
.with_body(cert_body)
.expect(1)
.create_async()
.await;
let root = match node_info {
Some(b) => {
server
.mock("GET", "/")
.with_status(200)
.with_header("content-type", "application/json")
.with_body(b)
.create_async()
.await
}
None => {
server
.mock("GET", "/")
.with_status(403)
.with_header("content-type", "application/json")
.with_body(r#"{"message":"denied"}"#)
.create_async()
.await
}
};
(cert, root)
}
#[tokio::test]
async fn verify_ok_when_issuing_node_is_the_queried_node() {
let kp = gitlawb_core::identity::Keypair::generate();
let (id, body) = signed_cert(&kp);
let info = format!(r#"{{"did":"{}"}}"#, kp.did().as_str());
let mut server = mockito::Server::new_async().await;
let (cert, _root) = cert_server(&mut server, &id, &body, Some(&info)).await;
let got = cmd_show("alice/r".to_string(), id, server.url(), None, true, None).await;
assert!(got.is_ok(), "valid sig + matching issuer must pass");
cert.assert_async().await;
}
#[tokio::test]
async fn verify_bails_when_issuer_is_a_different_node() {
// The self-signing-hostile-node case: the signature verifies against the
// key the cert names, but that key is not the node we queried.
let kp = gitlawb_core::identity::Keypair::generate();
let (id, body) = signed_cert(&kp);
let other = gitlawb_core::identity::Keypair::generate();
let info = format!(r#"{{"did":"{}"}}"#, other.did().as_str());
let mut server = mockito::Server::new_async().await;
let (cert, _root) = cert_server(&mut server, &id, &body, Some(&info)).await;
let err = cmd_show("alice/r".to_string(), id, server.url(), None, true, None)
.await
.unwrap_err()
.to_string();
assert!(err.contains("expected issuer"), "got: {err}");
cert.assert_async().await;
}
#[tokio::test]
async fn verify_bails_when_node_info_denied_and_no_expect_node() {
// Fail closed: with no trusted DID to anchor against, --verify must NOT
// fall through to a pass just because the signature checked out.
let kp = gitlawb_core::identity::Keypair::generate();
let (id, body) = signed_cert(&kp);
let mut server = mockito::Server::new_async().await;
let (cert, _root) = cert_server(&mut server, &id, &body, None).await;
let err = cmd_show("alice/r".to_string(), id, server.url(), None, true, None)
.await
.unwrap_err()
.to_string();
assert!(err.contains("cannot anchor the issuer"), "got: {err}");
cert.assert_async().await;
}
#[tokio::test]
async fn verify_ok_when_expect_node_anchors_a_denied_lookup() {
// --expect-node supplies the trust anchor the denied lookup could not, so
// the same denial that fails the case above now passes.
let kp = gitlawb_core::identity::Keypair::generate();
let (id, body) = signed_cert(&kp);
let mut server = mockito::Server::new_async().await;
let (cert, _root) = cert_server(&mut server, &id, &body, None).await;
let got = cmd_show(
"alice/r".to_string(),
id,
server.url(),
None,
true,
Some(kp.did().as_str().to_string()),
)
.await;
assert!(got.is_ok(), "explicit anchor must pass");
cert.assert_async().await;
}
#[tokio::test]
async fn verify_bails_on_a_bad_signature_before_anchoring() {
// The must-not: a forged certificate naming the queried node must fail on
// the signature, even though its issuer would otherwise anchor cleanly.
let kp = gitlawb_core::identity::Keypair::generate();
let (id, body) = signed_cert(&kp);
let forged = body.replace(&"b".repeat(40), &"c".repeat(40));
let info = format!(r#"{{"did":"{}"}}"#, kp.did().as_str());
let mut server = mockito::Server::new_async().await;
let (cert, _root) = cert_server(&mut server, &id, &forged, Some(&info)).await;
let err = cmd_show("alice/r".to_string(), id, server.url(), None, true, None)
.await
.unwrap_err()
.to_string();
assert!(err.contains("signature did not verify"), "got: {err}");
cert.assert_async().await;
}
// The must-not case for the DID extraction: an empty or missing `did` has to
// become a could-not-compare reason. If it leaks through as a value, the
// comparison below fabricates a mismatch WARNING against a DID the node
// never claimed.
#[test]
fn did_from_node_info_rejects_empty_and_missing() {
assert!(did_from_node_info(&serde_json::json!({"did": "n"})).is_ok());
for body in [
serde_json::json!({"did": ""}),
serde_json::json!({}),
serde_json::json!({"did": 7}),
] {
let got = did_from_node_info(&body);
assert!(
got.is_err(),
"must not yield a comparable DID: {body} -> {got:?}"
);
}
// And the reason it produces must route to the degraded hint, never a verdict.
let report =
did_check_report(&did_from_node_info(&serde_json::json!({"did": ""})), "n").join("\n");
assert!(
!report.contains("WARNING"),
"fabricated a mismatch: {report}"
);
assert!(report.contains("Could not fetch"), "got: {report}");
}
// did_check_report is the three-way selector between the match text, the
// mismatch WARNING, and the degraded could-not-compare hint. Substring
// asserts (not full-line equality) so cosmetic wording edits don't break them.
#[test]
fn did_check_report_match() {
let report = did_check_report(&Ok("n".to_string()), "n").join("\n");
assert!(
report.contains("matches the node being queried"),
"got: {report}"
);
assert!(!report.contains("WARNING"), "got: {report}");
assert!(!report.contains("Could not fetch"), "got: {report}");
}
#[test]
fn did_check_report_mismatch() {
let report = did_check_report(&Ok("did:key:other".to_string()), "n").join("\n");
assert!(report.contains("WARNING"), "got: {report}");
assert!(report.contains("does not match"), "got: {report}");
assert!(report.contains("did:key:other"), "got: {report}");
assert!(!report.contains("Could not fetch"), "got: {report}");
}
#[test]
fn did_check_report_missing_did_reason() {
let report =
did_check_report(&Err("node info response carried no DID".to_string()), "n").join("\n");
assert!(report.contains("Could not fetch"), "got: {report}");
assert!(
report.contains("node info response carried no DID"),
"got: {report}"
);
assert!(report.contains("verify offline"), "got: {report}");
// The must-not case: no real DID was obtained, so no comparison verdict
// may be claimed in either direction.
assert!(!report.contains("WARNING"), "got: {report}");
assert!(
!report.contains("matches the node being queried"),
"got: {report}"
);
}
#[test]
fn did_check_report_lookup_error_reason() {
let report =
did_check_report(&Err("node info failed (403): denied".to_string()), "n").join("\n");
assert!(report.contains("Could not fetch"), "got: {report}");
assert!(report.contains("403"), "got: {report}");
assert!(report.contains("verify offline"), "got: {report}");
assert!(!report.contains("WARNING"), "got: {report}");
}
}
+18 -40
View File
@@ -2,6 +2,7 @@
use anyhow::{Context, Result};
use clap::Args;
use serde_json::Value;
use std::path::PathBuf;
use crate::http::NodeClient;
@@ -14,11 +15,7 @@ pub struct ChangelogArgs {
/// Maximum number of events to show
#[arg(long, default_value = "20")]
pub limit: usize,
#[arg(
long,
default_value = "https://node.twigpine.com",
env = "GITLAWB_NODE"
)]
#[arg(long, default_value = "https://node.gitlawb.com", env = "GITLAWB_NODE")]
pub node: String,
#[arg(long)]
pub dir: Option<PathBuf>,
@@ -34,7 +31,7 @@ pub async fn run(args: ChangelogArgs) -> Result<()> {
};
if repo.is_empty() {
anyhow::bail!("no repo specified — pass <repo> or run from inside a Twigpine repo");
anyhow::bail!("no repo specified — pass <repo> or run from inside a gitlawb repo");
}
let (owner, name) = if let Some((o, n)) = repo.split_once('/') {
@@ -45,7 +42,12 @@ pub async fn run(args: ChangelogArgs) -> Result<()> {
did.split(':').next_back().unwrap_or(&did).to_string()
} else {
let client = NodeClient::new(&args.node, None);
let info = crate::http::read_json(client.get("/").await?, "node info").await?;
let info: Value = client
.get("/")
.await?
.json()
.await
.context("failed to fetch node info")?;
let did = info["did"].as_str().context("node missing DID")?;
did.split(':').next_back().unwrap_or(did).to_string()
};
@@ -62,7 +64,13 @@ pub async fn run(args: ChangelogArgs) -> Result<()> {
.await
.context("failed to connect to node")?;
let body = crate::http::read_json(resp, "changelog").await?;
let status = resp.status();
let body: Value = resp.json().await.unwrap_or_default();
if !status.is_success() {
let msg = body["message"].as_str().unwrap_or("unknown error");
anyhow::bail!("changelog failed ({status}): {msg}");
}
let events = body["events"].as_array().cloned().unwrap_or_default();
@@ -75,14 +83,14 @@ pub async fn run(args: ChangelogArgs) -> Result<()> {
for event in &events {
let ts = event["timestamp"].as_str().unwrap_or("?");
let date = crate::text::truncate(ts, 10);
let date = &ts[..ts.len().min(10)];
match event["type"].as_str().unwrap_or("") {
"commit" => {
let sha = event["sha"].as_str().unwrap_or("?");
let msg = event["message"].as_str().unwrap_or("?");
let first_line = msg.lines().next().unwrap_or(msg);
let short_sha = crate::text::truncate(sha, 8);
let short_sha = &sha[..sha.len().min(8)];
println!(" {date} commit {short_sha} {first_line}");
}
"pr_merged" => {
@@ -204,7 +212,6 @@ mod tests {
.with_status(404)
.with_header("content-type", "application/json")
.with_body(r#"{"message":"repo not found"}"#)
.expect(1)
.create_async()
.await;
@@ -216,8 +223,6 @@ mod tests {
};
let err = run(args).await.unwrap_err();
assert!(err.to_string().contains("changelog failed"));
// Prove the mocked route was actually requested; a non-matching request (mockito's 501, also non-2xx) would otherwise satisfy the error assertion vacuously.
_m.assert_async().await;
}
#[tokio::test]
@@ -265,31 +270,4 @@ mod tests {
let err = rt.block_on(run(args)).unwrap_err();
assert!(err.to_string().contains("no repo specified"));
}
#[tokio::test]
async fn resolve_via_run_surfaces_denial() {
// A slash-free repo with an empty identity dir forces the inline GET /
// node-info fetch during resolution. A gated 404 there must Err (surfacing
// the status), proving the read_json conversion is load-bearing.
let mut server = mockito::Server::new_async().await;
let dir = tempfile::TempDir::new().unwrap(); // empty, no identity.pem, forces the GET / branch
let _m = server
.mock("GET", "/")
.with_status(404)
.with_header("content-type", "application/json")
.with_body(r#"{"message":"denied"}"#)
.expect(1)
.create_async()
.await;
let err = run(ChangelogArgs {
repo: Some("noslash".to_string()),
limit: 20,
node: server.url(),
dir: Some(dir.path().to_path_buf()),
})
.await
.unwrap_err();
assert!(err.to_string().contains("404"), "got: {err}");
_m.assert_async().await;
}
}
+16 -355
View File
@@ -1,4 +1,4 @@
//! `gl clone`: clean partial clone of a Twigpine repo with private subtrees.
//! `gl clone`: clean partial clone of a gitlawb repo with private subtrees.
//!
//! A repo may withhold blob content under some path globs from the caller
//! (Phase 3). The resulting pack is not closed under reachability, so a stock
@@ -12,58 +12,9 @@ use serde::Deserialize;
use std::path::Path;
use std::process::Command;
use crate::http::{sanitize_node_msg, NodeClient};
use crate::http::NodeClient;
use crate::identity::load_keypair_from_dir;
/// Report why one candidate blob could not be recovered, in the register of the
/// recovery loop's other warnings.
///
/// Both halves are gateway-supplied text on their way to a terminal: the oid comes
/// out of a manifest fetched from a caller-chosen Arweave gateway, and a transport
/// error carries the URL the same manifest chose. So the whole line is defanged and
/// length-capped, like every other node/gateway string this crate prints.
///
/// Under `cfg(test)` the line is also mirrored into a per-thread buffer, so a test
/// can assert the loop actually says what it skipped rather than only that it
/// returned nothing.
fn warn_skip(oid: &str, why: &str) {
emit_warning(&format!(
"warning: could not fetch encrypted blob {oid}: {why}; skipping"
));
}
/// The same report for a manifest rather than a blob. The manifest is one step
/// earlier in the same recovery: losing it silently means every blob it would have
/// named is missing with no reason given anywhere.
fn warn_skip_manifest(id: &str, why: &str) {
emit_warning(&format!(
"warning: could not fetch blob manifest {id}: {why}; skipping"
));
}
/// Sanitize a warning and write it, once.
///
/// The write goes to [`warn_sink`], which is stderr in a normal build and the tests'
/// per-thread mirror under `cfg(test)`. That indirection is the point: the mirror
/// used to sit BESIDE an `eprintln!`, so deleting the user-visible write left every
/// assertion on the mirror green and the shipped behaviour could be removed with the
/// tests unchanged. There is one write now, and the tests observe it.
fn emit_warning(line: &str) {
use std::io::Write;
let line = sanitize_node_msg(line);
let _ = writeln!(warn_sink(), "{line}");
}
#[cfg(not(test))]
fn warn_sink() -> impl std::io::Write {
std::io::stderr()
}
#[cfg(test)]
fn warn_sink() -> impl std::io::Write {
tests::WarnMirror
}
#[derive(Args)]
pub struct CloneArgs {
/// Repo to clone: gitlawb://<owner_did>/<name> or <owner>/<name>.
@@ -76,11 +27,7 @@ pub struct CloneArgs {
#[arg(long)]
pub branch: Option<String>,
#[arg(
long,
default_value = "https://node.twigpine.com",
env = "GITLAWB_NODE"
)]
#[arg(long, default_value = "https://node.gitlawb.com", env = "GITLAWB_NODE")]
pub node: String,
/// Arweave gateway for B3 manifest discovery/fetch when a node cannot supply
@@ -163,8 +110,8 @@ pub fn setup_partial_clone(
if withheld_globs.is_empty() {
match branch {
Some(b) => git_global(&["clone", "-q", "--branch", b, "--", remote_url, dest_str])?,
None => git_global(&["clone", "-q", "--", remote_url, dest_str])?,
Some(b) => git_global(&["clone", "-q", "--branch", b, remote_url, dest_str])?,
None => git_global(&["clone", "-q", remote_url, dest_str])?,
}
return Ok(());
}
@@ -174,7 +121,6 @@ pub fn setup_partial_clone(
"-q",
"--filter=blob:none",
"--no-checkout",
"--",
remote_url,
dest_str,
])?;
@@ -366,30 +312,15 @@ async fn recover_encrypted_blobs(
.await
{
Ok(r) if r.status().is_success() => r,
// The node path has the same silent exit the gateway path had: a 403, a
// 404, or a dead connection all reached the caller as "blob not
// recoverable" with no reason attached. One unreachable blob still must
// not end the recovery of the rest, so it warns and moves on.
Ok(r) => {
warn_skip(oid, &format!("node returned {}", r.status()));
continue;
}
Err(e) => {
warn_skip(oid, &format!("node request failed: {e}"));
continue;
}
_ => continue,
};
let envelope = match env_resp.bytes().await {
Ok(b) => b,
Err(e) => {
warn_skip(oid, &format!("reading the envelope failed: {e}"));
continue;
}
let Ok(envelope) = env_resp.bytes().await else {
continue;
};
let plaintext = match open_blob(&envelope, keypair) {
Ok(p) => p,
Err(e) => {
emit_warning(&format!("warning: could not decrypt {oid}: {e}"));
eprintln!("warning: could not decrypt {oid}: {e}");
continue;
}
};
@@ -407,9 +338,7 @@ async fn recover_encrypted_blobs(
recovered.push(p.clone());
}
} else {
emit_warning(&format!(
"warning: recovered blob {oid} hashed to {written}; discarding"
));
eprintln!("warning: recovered blob {oid} hashed to {written}; discarding");
}
}
Ok(recovered)
@@ -581,7 +510,7 @@ fn merge_manifests(
.collect()
}
/// Option B3 fallback recovery, with no dependency on a Twigpine node API. Query
/// Option B3 fallback recovery, with no dependency on a gitlawb node API. Query
/// the Arweave gateway for this repo's encrypted manifests, merge them, and for
/// each blob still missing locally that the caller can decrypt, pull the envelope
/// from a public IPFS gateway, decrypt, and install it as a loose object. Returns
@@ -713,14 +642,7 @@ async fn recover_from_arweave(
for r in refs {
let m = match client.get(format!("{ag}/{}", r.id)).send().await {
Ok(resp) if resp.status().is_success() => resp,
Ok(resp) => {
warn_skip_manifest(&r.id, &format!("gateway returned {}", resp.status()));
continue;
}
Err(e) => {
warn_skip_manifest(&r.id, &format!("gateway request failed: {e}"));
continue;
}
_ => continue,
};
if let Ok(parsed) = m.json::<Manifest>().await {
manifests.push((parsed, r.height));
@@ -762,31 +684,10 @@ async fn recover_from_arweave(
}
let env_resp = match client.get(format!("{ig}/ipfs/{cid}")).send().await {
Ok(r) if r.status().is_success() => r,
// Every other outcome used to leave through a bare `continue`, so a
// gateway that answered 503, 404, or nothing at all was reported to the
// caller as "blob not recoverable" with no reason attached. This is the
// second in-repo client of GET /ipfs/{cid} and it has no resume ladder,
// so saying what happened is all the recourse there is. It still skips to
// the next candidate either way: one unreachable blob must not end the
// recovery of the rest.
Ok(r) => {
warn_skip(&oid, &format!("gateway returned {}", r.status()));
continue;
}
Err(e) => {
warn_skip(&oid, &format!("gateway request failed: {e}"));
continue;
}
_ => continue,
};
// A body that dies part-way through is the same mid-read failure the capped
// read exists to stop rendering as silence, and it sits INSIDE the loop whose
// status arms were already fixed.
let envelope = match env_resp.bytes().await {
Ok(b) => b,
Err(e) => {
warn_skip(&oid, &format!("reading the envelope failed: {e}"));
continue;
}
let Ok(envelope) = env_resp.bytes().await else {
continue;
};
// open_blob succeeds only if this caller is a recipient: this is the
// authorization gate (no node, no DID check needed).
@@ -807,9 +708,7 @@ async fn recover_from_arweave(
recovered.push(p.clone());
}
} else {
emit_warning(&format!(
"warning: recovered blob {oid} hashed to {written}; discarding"
));
eprintln!("warning: recovered blob {oid} hashed to {written}; discarding");
}
}
Ok(recovered)
@@ -902,42 +801,9 @@ pub async fn run(args: CloneArgs) -> Result<()> {
#[cfg(test)]
mod tests {
use super::*;
use std::cell::RefCell;
use std::process::Command;
use tempfile::TempDir;
thread_local! {
/// Test-visible copy of the stderr warnings `warn_skip` emits.
/// `#[tokio::test]` runs the future on the test's own thread, so a
/// thread-local is enough.
static WARNINGS: RefCell<String> = const { RefCell::new(String::new()) };
}
/// The `cfg(test)` warning sink. `emit_warning` writes here instead of to
/// stderr, so the assertions below observe the shipped write rather than a copy
/// made beside it: delete that write and they go red.
pub(super) struct WarnMirror;
impl std::io::Write for WarnMirror {
fn write(&mut self, buf: &[u8]) -> std::io::Result<usize> {
let text = String::from_utf8_lossy(buf).into_owned();
WARNINGS.with(|w| w.borrow_mut().push_str(&text));
Ok(buf.len())
}
fn flush(&mut self) -> std::io::Result<()> {
Ok(())
}
}
fn reset_warnings() {
WARNINGS.with(|w| w.borrow_mut().clear());
}
fn warnings() -> String {
WARNINGS.with(|w| w.borrow().clone())
}
fn g(args: &[&str], dir: &Path) {
assert!(Command::new("git")
.args(args)
@@ -1095,85 +961,6 @@ mod tests {
);
}
#[test]
fn setup_partial_clone_plain_without_withheld_paths() {
let (td, url) = bare_remote(&[("file.txt", b"hello\n")]);
let dest = td.path().join("dest");
setup_partial_clone(&dest, &url, &[], &[], None).unwrap();
assert!(dest.join("file.txt").exists());
}
#[test]
fn setup_partial_clone_with_branch_without_withheld_paths() {
let (td, url) = bare_remote(&[("file.txt", b"hello\n")]);
let branch_out = Command::new("git")
.args([
"-C",
td.path().join("origin").to_str().unwrap(),
"branch",
"--show-current",
])
.output()
.unwrap();
let branch = String::from_utf8(branch_out.stdout)
.unwrap()
.trim()
.to_string();
let dest = td.path().join("dest");
setup_partial_clone(&dest, &url, &[], &[], Some(&branch)).unwrap();
assert!(dest.join("file.txt").exists());
}
/// #374: every `setup_partial_clone` shape interpolates `remote_url` into a
/// `git clone` argv. The `--` delimiter is what forces an option-shaped
/// remote to be read as a repository rather than parsed as an option.
///
/// The assertion is on git's own words — `repository '<value>'`, single-quoted
/// — and not merely on the value appearing somewhere in the error. `git_global`
/// formats its failure as `git {args:?} failed: {stderr}`, so the injected
/// string is already present in the debug-printed argv whether or not the
/// delimiter is there; asserting on containment alone would pass with `--`
/// deleted. Undelimited, git consumes the value as an option, leaving the
/// destination as the only positional, and names *that* instead
/// (`repository '<dest>' does not exist`) — so this goes red without the fix.
///
/// The payload carries no `:` or space: a colon would make git read the
/// argument as an ssh-style `host:path` URL and report a hostname, which is a
/// different parse.
fn assert_option_shaped_remote_is_a_repository(withheld: &[String], branch: Option<&str>) {
let td = TempDir::new().unwrap();
let dest = td.path().join("dest");
let injected = "--upload-pack=false";
let err = setup_partial_clone(&dest, injected, withheld, &[], branch)
.expect_err("an option-shaped remote is not a repository and must fail")
.to_string();
assert!(
err.contains(&format!("repository '{injected}'")),
"git must name the argument as the repository, not consume it as an option: {err}"
);
}
#[test]
fn plain_clone_never_parses_a_remote_as_a_git_option() {
assert_option_shaped_remote_is_a_repository(&[], None);
}
#[test]
fn branch_clone_never_parses_a_remote_as_a_git_option() {
// Pins that `--branch <b>` stays ahead of the delimiter: if `--` were moved
// before `--branch`, git would read the branch name as a positional instead.
assert_option_shaped_remote_is_a_repository(&[], Some("main"));
}
#[test]
fn sparse_clone_never_parses_a_remote_as_a_git_option() {
// The withheld-globs arm is a separate argv construction
// (`--filter=blob:none --no-checkout`) and needs its own proof.
assert_option_shaped_remote_is_a_repository(&["/secret/**".to_string()], None);
}
#[test]
fn sparse_patterns_subtree_and_exact() {
assert_eq!(sparse_patterns("/secret/**"), vec!["/secret/".to_string()]);
@@ -1468,132 +1255,6 @@ mod tests {
assert_eq!(a.get("o1").map(String::as_str), Some("cidTS"));
}
/// A gateway that refuses one blob must SAY so, and must not take the rest of the
/// recovery down with it.
///
/// The IPFS fetch used to leave through a bare `_ => continue` on every
/// non-success, so a 503 (the truncation status a tuned node makes more likely),
/// a 404, and a dead connection all reached the caller as the same silent "blob
/// not recoverable". Two withheld blobs here: the gateway answers 503 for the
/// first and serves the second. The recovered path proves the loop continued; the
/// warning proves the skip was reported and names both the object and the status.
#[tokio::test]
async fn recover_from_arweave_reports_a_refusing_gateway_and_continues() {
use gitlawb_core::encrypt::seal_blob;
use gitlawb_core::identity::Keypair;
reset_warnings();
let (td, url) = bare_remote(&[
("public/a.txt", b"pub\n"),
("secret/b.txt", b"SECRET B\n"),
("secret/c.txt", b"SECRET C\n"),
]);
let dest = td.path().join("dest");
let bare = url.strip_prefix("file://").unwrap();
assert!(Command::new("git")
.args(["-C", bare, "config", "uploadpack.allowFilter", "true"])
.status()
.unwrap()
.success());
setup_partial_clone(&dest, &url, &["/secret/**".to_string()], &[], None).unwrap();
let oid_of = |path: &str| {
let out = Command::new("git")
.args([
"-C",
dest.to_str().unwrap(),
"rev-parse",
&format!("HEAD:{path}"),
])
.output()
.unwrap();
String::from_utf8_lossy(&out.stdout).trim().to_string()
};
let refused_oid = oid_of("secret/b.txt");
let served_oid = oid_of("secret/c.txt");
// Origin death, so recovery has to go through the gateways.
std::fs::remove_dir_all(bare).unwrap();
let reader = Keypair::generate();
let envelope = seal_blob(b"SECRET C\n", &[reader.verifying_key()]).unwrap();
let refused_cid = "cidrefused";
let served_cid = "cidserved";
let mut server = mockito::Server::new_async().await;
let _gql = server
.mock("POST", "/graphql")
.with_status(200)
.with_header("content-type", "application/json")
.with_body(r#"{"data":{"transactions":{"edges":[{"node":{"id":"TX1"}}]}}}"#)
.create_async()
.await;
let manifest_body = serde_json::json!({
"timestamp": "2026-06-11T00:00:00Z",
"blobs": [
{ "oid": refused_oid, "cid": refused_cid },
{ "oid": served_oid, "cid": served_cid },
],
})
.to_string();
let _tx = server
.mock("GET", "/TX1")
.with_status(200)
.with_header("content-type", "application/json")
.with_body(manifest_body)
.create_async()
.await;
let refusal = server
.mock("GET", format!("/ipfs/{refused_cid}").as_str())
.with_status(503)
.with_header("content-type", "application/json")
.with_body(r#"{"error":"search_incomplete","message":"scan truncated"}"#)
.expect(1)
.create_async()
.await;
let ok = server
.mock("GET", format!("/ipfs/{served_cid}").as_str())
.with_status(200)
.with_body(envelope)
.expect(1)
.create_async()
.await;
let paths = recover_from_arweave(
&server.url(),
&server.url(),
"alice",
"myrepo",
&dest,
&reader,
)
.await
.unwrap();
refusal.assert_async().await;
ok.assert_async().await;
assert_eq!(
paths,
vec!["secret/c.txt".to_string()],
"a refused candidate must not stop the loop reaching the next one"
);
let warnings = warnings();
assert!(
warnings.contains(&refused_oid),
"the skip must name the object it could not fetch, got: {warnings}"
);
assert!(
warnings.contains("503"),
"the skip must name the gateway's status rather than failing silently, \
got: {warnings}"
);
assert!(
!warnings.contains(&served_oid),
"the blob that was served must not be reported as skipped, got: {warnings}"
);
}
/// Read-path end-to-end over a mocked Arweave + IPFS gateway: discover the
/// manifest via GraphQL, fetch it, fetch the envelope, decrypt with the
/// caller's key, and install the previously-withheld blob.
+19 -246
View File
@@ -1,10 +1,10 @@
//! `gl doctor` — check your Twigpine installation and connectivity.
//! `gl doctor` — check your gitlawb installation and connectivity.
//!
//! Checks:
//! 1. Identity — ~/.gitlawb/identity.pem exists, DID parseable
//! 2. Registration — ~/.gitlawb/ucan.json exists (registered with a node)
//! 3. GITLAWB_NODE — env var is set to a non-localhost URL
//! 4. Node — GITLAWB_NODE is reachable (default: https://node.twigpine.com)
//! 4. Node — GITLAWB_NODE is reachable (default: https://node.gitlawb.com)
//! 5. git helper — git-remote-gitlawb is in PATH
//! 6. git — git is installed
//! 7. version — gl is up to date with latest GitHub release
@@ -15,8 +15,7 @@ use std::path::PathBuf;
use crate::http::NodeClient;
const PUBLIC_NODE: &str = "https://node.twigpine.com";
const GITHUB_API_BASE: &str = "https://api.github.com";
const PUBLIC_NODE: &str = "https://node.gitlawb.com";
#[derive(Args)]
pub struct DoctorArgs {
@@ -70,7 +69,7 @@ impl Check {
}
pub async fn run(args: DoctorArgs) -> Result<()> {
println!("gl doctor — checking your Twigpine setup");
println!("gl doctor — checking your gitlawb setup");
println!();
let dir = args.dir.unwrap_or_else(|| {
@@ -142,25 +141,23 @@ pub async fn run(args: DoctorArgs) -> Result<()> {
// ── 3. GITLAWB_NODE env var ───────────────────────────────────────────
match std::env::var("GITLAWB_NODE") {
// A loopback host is a legitimate setup (self-hosted node, dev
// harness) — the connectivity check below fails loudly if it is not
// actually reachable, so don't red-flag the configuration itself.
Ok(v) if is_loopback_url(&v) => {
checks.push(Check::pass(
Ok(v) if !v.is_empty() && !v.contains("127.0.0.1") && !v.contains("localhost") => {
checks.push(Check::pass("GITLAWB_NODE", v.to_string()));
}
Ok(v) if v.contains("127.0.0.1") || v.contains("localhost") => {
checks.push(Check::fail(
"GITLAWB_NODE",
format!(
"{v} (local node — intentional for self-hosting/dev; unset to target the public network)"
"set to local address ({v}) — git push/clone will fail against remote nodes"
),
"export GITLAWB_NODE=https://node.gitlawb.com",
));
}
Ok(v) if !v.is_empty() => {
checks.push(Check::pass("GITLAWB_NODE", v.to_string()));
}
_ => {
checks.push(Check::fail(
"GITLAWB_NODE",
"not set — git-remote-gitlawb will fall back to http://127.0.0.1:7545",
"export GITLAWB_NODE=https://node.twigpine.com",
"export GITLAWB_NODE=https://node.gitlawb.com",
));
}
}
@@ -187,7 +184,7 @@ pub async fn run(args: DoctorArgs) -> Result<()> {
"node is v{version} but gl is v{gl_ver} — your CLI may be missing \
features (signing / iCaptcha) this node requires"
),
"upgrade gl: curl -sSf https://twigpine.com/install.sh | sh",
"upgrade gl: curl -sSf https://gitlawb.com/install.sh | sh",
));
}
}
@@ -241,19 +238,10 @@ pub async fn run(args: DoctorArgs) -> Result<()> {
checks.push(Check::fail(
"git-remote-gitlawb",
"not found in PATH — gitlawb:// clone/push will not work",
"curl -sSf https://twigpine.com/install.sh | sh",
"curl -sSf https://gitlawb.com/install.sh | sh",
));
}
// ── 5b. shell alias shadowing ─────────────────────────────────────────
// oh-my-zsh's default `git` plugin aliases gl='git pull', which silently
// shadows this binary in every interactive zsh — the classic symptom is
// `gl` printing "fatal: not a git repository". Aliases beat PATH, so no
// install method can fix it; the user's rc file has to unalias.
if let Some(check) = check_shell_alias_shadowing(dirs::home_dir()) {
checks.push(check);
}
// ── 6. git ────────────────────────────────────────────────────────────
match std::process::Command::new("git")
.arg("--version")
@@ -275,7 +263,7 @@ pub async fn run(args: DoctorArgs) -> Result<()> {
// ── 7. Version up to date ─────────────────────────────────────────────
let current = env!("CARGO_PKG_VERSION");
checks.push(check_version(current, GITHUB_API_BASE).await);
checks.push(check_version(current).await);
// ── Render ────────────────────────────────────────────────────────────
for check in &checks {
@@ -320,83 +308,6 @@ pub async fn run(args: DoctorArgs) -> Result<()> {
}
/// Check if a binary name exists anywhere on PATH.
/// True when the rc file contains a real `unalias` command naming `gl` —
/// not a comment, and not a longer word like `unalias global`. Ordering
/// relative to oh-my-zsh loading is not modeled (an unalias placed before the
/// plugin loads would be ineffective); acceptable slack for a warning check.
fn rc_unaliases_gl(rc: &str) -> bool {
rc.lines().any(|line| {
let line = line.trim_start();
if line.starts_with('#') {
return false;
}
line.split([';', '&', '|']).any(|cmd| {
// A trailing comment must not count: `unalias foo # gl`
let cmd = cmd.split('#').next().unwrap_or("");
let mut tokens = cmd.split_whitespace();
tokens.next() == Some("unalias") && tokens.any(|t| t == "gl")
})
})
}
/// True when the URL's host is exactly a loopback address. Substring checks
/// misclassify hosts like `localhost.example` or URLs with "localhost" in the
/// path, so parse and compare the actual host.
fn is_loopback_url(value: &str) -> bool {
reqwest::Url::parse(value)
.ok()
.and_then(|u| {
u.host_str()
.map(|h| h == "localhost" || h == "127.0.0.1" || h == "[::1]" || h == "::1")
})
.unwrap_or(false)
}
/// Detect interactive-shell setups where an alias shadows `gl` (aliases beat
/// PATH, so no install method can fix this). Best-effort heuristic over
/// ~/.zshrc: an explicit `alias gl=`, or oh-my-zsh's `git` plugin (which
/// ships gl='git pull'). Returns None when nothing suspicious is found or the
/// rc file already contains an `unalias gl`.
fn check_shell_alias_shadowing(home: Option<PathBuf>) -> Option<Check> {
let home = home?;
let rc = std::fs::read_to_string(home.join(".zshrc")).ok()?;
if rc_unaliases_gl(&rc) {
return None;
}
let explicit_alias = rc.lines().any(|l| l.trim_start().starts_with("alias gl="));
// Single-line `plugins=(git ...)` is the overwhelmingly common form; a
// multi-line plugins array slips past this heuristic, which is acceptable
// for a warning-level check.
let omz_git_plugin = home.join(".oh-my-zsh/plugins/git").exists()
&& rc.lines().any(|l| {
let l = l.trim_start();
l.starts_with("plugins=")
&& l.trim_start_matches("plugins=(")
.trim_end_matches(')')
.split_whitespace()
.any(|p| p == "git")
});
if explicit_alias || omz_git_plugin {
let source = if explicit_alias {
"~/.zshrc defines `alias gl=`"
} else {
"oh-my-zsh's git plugin aliases gl='git pull'"
};
Some(Check::warn(
"shell alias",
format!(
"{source} — interactive shells run that instead of this binary \
(symptom: `gl` prints \"fatal: not a git repository\")"
),
"echo 'unalias gl 2>/dev/null' >> ~/.zshrc && source ~/.zshrc (must come after oh-my-zsh loads)",
))
} else {
None
}
}
fn which_in_path(name: &str) -> bool {
std::env::var_os("PATH")
.map(|paths| {
@@ -408,9 +319,8 @@ fn which_in_path(name: &str) -> bool {
.unwrap_or(false)
}
/// Fetch the latest release tag from Twigpine/node (the actual release repo — see install.sh's
/// `GITLAWB_RELEASE_REPO` default) and compare to current version.
async fn check_version(current: &'static str, github_api_base: &str) -> Check {
/// Fetch the latest release tag from gitlawb/releases and compare to current version.
async fn check_version(current: &'static str) -> Check {
let client = match reqwest::Client::builder()
.user_agent(format!("gl/{current}"))
.timeout(std::time::Duration::from_secs(5))
@@ -421,9 +331,7 @@ async fn check_version(current: &'static str, github_api_base: &str) -> Check {
};
let resp = match client
.get(format!(
"{github_api_base}/repos/Twigpine/node/releases/latest"
))
.get("https://api.github.com/repos/gitlawb/releases/releases/latest")
.send()
.await
{
@@ -431,13 +339,6 @@ async fn check_version(current: &'static str, github_api_base: &str) -> Check {
Err(_) => return Check::pass("version", format!("v{current} (offline — could not check)")),
};
if !resp.status().is_success() {
return Check::pass(
"version",
format!("v{current} (GitHub API returned HTTP {})", resp.status()),
);
}
let body: serde_json::Value = match resp.json().await {
Ok(v) => v,
Err(_) => return Check::pass("version", format!("v{current} (invalid response)")),
@@ -457,7 +358,7 @@ async fn check_version(current: &'static str, github_api_base: &str) -> Check {
Check::warn(
"version",
format!("v{current} → v{tag} available"),
"curl -sSf https://twigpine.com/install.sh | sh",
"curl -sSf https://gitlawb.com/install.sh | sh",
)
} else {
Check::pass("version", format!("v{current} (up to date)"))
@@ -481,88 +382,6 @@ fn is_newer(latest: &str, current: &str) -> bool {
mod tests {
use super::*;
fn fake_home(zshrc: Option<&str>, with_omz_git: bool) -> tempfile::TempDir {
let home = tempfile::TempDir::new().unwrap();
if let Some(rc) = zshrc {
std::fs::write(home.path().join(".zshrc"), rc).unwrap();
}
if with_omz_git {
std::fs::create_dir_all(home.path().join(".oh-my-zsh/plugins/git")).unwrap();
}
home
}
#[test]
fn alias_shadowing_flags_omz_git_plugin() {
let home = fake_home(Some("plugins=(git z)\nsource $ZSH/oh-my-zsh.sh\n"), true);
let check = check_shell_alias_shadowing(Some(home.path().to_path_buf()));
assert!(check.is_some(), "omz git plugin without unalias must warn");
}
#[test]
fn alias_shadowing_flags_explicit_alias() {
let home = fake_home(Some("alias gl='git pull'\n"), false);
assert!(check_shell_alias_shadowing(Some(home.path().to_path_buf())).is_some());
}
#[test]
fn alias_shadowing_silent_when_unaliased() {
let home = fake_home(
Some("plugins=(git z)\nsource $ZSH/oh-my-zsh.sh\nunalias gl 2>/dev/null\n"),
true,
);
assert!(check_shell_alias_shadowing(Some(home.path().to_path_buf())).is_none());
}
#[test]
fn alias_shadowing_ignores_fake_unaliases() {
// None of these actually free `gl` — the warning must survive them.
for rc in [
"plugins=(git)\n# unalias gl\n", // commented out
"plugins=(git)\nunalias global\n", // longer word
"plugins=(git)\nunalias foo # gl\n", // gl only in a comment
"plugins=(git)\necho unalias-gl-later\n", // not an unalias command
] {
let home = fake_home(Some(rc), true);
assert!(
check_shell_alias_shadowing(Some(home.path().to_path_buf())).is_some(),
"must still warn for rc: {rc:?}"
);
}
// Real unalias forms that do free it — all must silence the warning.
for rc in [
"plugins=(git)\nunalias gl\n",
"plugins=(git)\nunalias gl 2>/dev/null\n",
"plugins=(git)\ntrue; unalias gl\n",
"plugins=(git)\nunalias glog gl gst\n",
] {
let home = fake_home(Some(rc), true);
assert!(
check_shell_alias_shadowing(Some(home.path().to_path_buf())).is_none(),
"must be silent for rc: {rc:?}"
);
}
}
#[test]
fn loopback_url_detection_is_host_exact() {
assert!(is_loopback_url("http://127.0.0.1:7545"));
assert!(is_loopback_url("http://localhost:7545"));
assert!(is_loopback_url("http://[::1]:7545"));
assert!(!is_loopback_url("https://localhost.example"));
assert!(!is_loopback_url("https://node.twigpine.com/localhost"));
assert!(!is_loopback_url("not a url"));
}
#[test]
fn alias_shadowing_silent_without_signals() {
let home = fake_home(Some("plugins=(z)\nexport EDITOR=vim\n"), false);
assert!(check_shell_alias_shadowing(Some(home.path().to_path_buf())).is_none());
let no_rc = fake_home(None, false);
assert!(check_shell_alias_shadowing(Some(no_rc.path().to_path_buf())).is_none());
assert!(check_shell_alias_shadowing(None).is_none());
}
#[test]
fn test_is_newer_minor_bump() {
assert!(is_newer("0.2.0", "0.1.0"));
@@ -598,50 +417,4 @@ mod tests {
fn test_which_in_path_missing_binary() {
assert!(!which_in_path("this-binary-does-not-exist-gl-test-12345"));
}
#[tokio::test]
async fn test_check_version_queries_gitlawb_node_releases() {
let mut server = mockito::Server::new_async().await;
let _m = server
.mock("GET", "/repos/Twigpine/node/releases/latest")
.with_status(200)
.with_header("content-type", "application/json")
.with_body(r#"{"tag_name":"v9.9.9"}"#)
.create_async()
.await;
let check = check_version("0.1.0", &server.url()).await;
assert!(matches!(check.state, CheckState::Warn));
assert!(check.detail.contains("v9.9.9"));
}
#[tokio::test]
async fn test_check_version_up_to_date() {
let mut server = mockito::Server::new_async().await;
let _m = server
.mock("GET", "/repos/Twigpine/node/releases/latest")
.with_status(200)
.with_header("content-type", "application/json")
.with_body(r#"{"tag_name":"v0.1.0"}"#)
.create_async()
.await;
let check = check_version("0.1.0", &server.url()).await;
assert!(matches!(check.state, CheckState::Ok));
assert!(check.detail.contains("up to date"));
}
#[tokio::test]
async fn test_check_version_http_error() {
let mut server = mockito::Server::new_async().await;
let _m = server
.mock("GET", "/repos/Twigpine/node/releases/latest")
.with_status(403)
.create_async()
.await;
let check = check_version("0.1.0", &server.url()).await;
assert!(matches!(check.state, CheckState::Ok));
assert!(check.detail.contains("GitHub API returned HTTP 403"));
}
}
+2 -968
View File
File diff suppressed because it is too large. Load diff
Loaded 100 of 176 files, more files were not shown because too many files have changed in this diff. Show more