Compare commits

..
Author SHA1 Message Date
Kevin Codex 6b0763ac2a fix(api): make the push-event cursor opaque and bound to its repository
The cursor was the table-global bigserial, handed to clients verbatim and
accepted back as any non-negative integer, then passed straight into a
per-repo seq > $1. Two things followed from that, and both were silent.

A value issued by a DIFFERENT repository — or by this one before a
restore — is an ordinary larger number here, so the query matched
nothing, the surface answered 200 with an empty page and echoed the value
back, and the subscriber sat permanently past history it had never
received with nothing anywhere reading as an error. And because
repo_push_events is written by every repository on the node, the gaps
between one repository's cursors measured how much the others pushed in
between, private ones included.

The cursor is now an opaque token carrying a version, a binding to
(node DID, repository), and the id of the last event served. The binding
is a hash and not a MAC, because there is nothing to forge: the position
inside is a row id resolved against this repository's own rows anyway.
What it buys is that a foreign cursor is refused by shape, with a message
naming which mistake was made, instead of being applied to a repository
it was never issued for. An id that resolves to nothing — a token kept
across a restore that lost the row — is a 400 that names the recovery,
not an empty page the poller reads as being up to date.

The global sequence still orders the walk internally, which is what keeps
the page cheap, but it no longer leaves the node. The position is the row
id, which this surface already publishes in every event, so the token
discloses nothing new.

The two properties an earlier round established are kept, in their new
shape: next_cursor is never null and never moves backwards, and the start
of history is an expressible position rather than an absent one, so a
first poll of a repository with no events still hands back a token the
endpoint accepts back rather than a value a persisting poller would
misread.
2026-08-31 21:56:23 +08:00
Kevin Codex 8652b86d33 fix(api): apply a push's side effects only to the refs git accepted
receive-pack exits zero when the process ran, not when the push applied.
A non-fast-forward, a hook rejection or a failed update comes back as an
ng pkt-line inside a response this handler forwarded without reading, so
every side effect keyed on the refs parsed from the REQUEST fired for
refs that were refused: the pull request on that branch had its stored
head moved to a commit this node does not have, and the catch-up poll
surface handed subscribers a SHA that resolves to nothing.

The report is now parsed and only the refs it names with ok are used.
unpack fail accepts nothing, since the pack never landed. A report that
does not parse at all is inconclusive and keeps every declared ref: a
client that never requested report-status is told nothing about
individual refs, and reading that silence as rejection would permanently
drop the events and the head update for a push git really did accept,
which is the failure the poll surface exists to prevent.

The parser is ported from PR #384, which introduced it for this same
problem, so the two branches converge on one definition of "accepted"
rather than growing two. Whichever lands second should drop its copy. It
diverges by three lines, and they are load-bearing: git double wraps the
report when the client negotiates side-band-64k, which git push over
smart HTTP does. The observed framing from git 2.50.1 is an outer
pkt-line carrying a band byte whose payload is itself a pkt-line stream,
so one pass of strip_sideband leaves "000eunpack ok" as the first line,
the parse fails, and every ref falls back to inconclusive — the parser
answering "cannot tell" for exactly the pushes it exists to classify. A
second pass peels the inner layer and is a no-op on the single-wrapped
shape. The tests carry the real captured byte layout, both wrappings, so
this is pinned rather than assumed.

Both writes also moved inside the repository write lock. That lock orders
the ref updates of two concurrent pushes; released before these ran, the
pair was an unordered race, and B's unconditional UPDATE landing before
A's leaves the stored head — which the rollup prefers over its fallback —
pinned to a commit the branch has moved past. record_push_events has the
same inversion by a different route, since latest_push_sha_for_ref reads
the highest seq and seq order is insert order. There is no per-row
version to make the UPDATE conditional on, so the ordering has to come
from the lock. Holding it across two batched statements on an already
open pool, on a path that just ran a subprocess over the whole pack, is
the cheaper half of that trade. A source guard pins the position, because
this is a property of where the calls sit.

Separately, the 10,000-ref bound was checked only after the parser had
scanned the entire request and kept three heap strings per valid
pkt-line. The git routes raise the body limit to GITLAWB_MAX_PACK_BYTES —
2 GB by default — so a signed caller could force the whole scan and the
whole allocation and only then be told 400: the work the cap exists to
refuse, done in full before the refusal. The parser stops one past the
cap, which is still enough for bound_declared_refs to refuse on, and the
message no longer reports a count that would now be a floor rather than
the request's real one.
2026-08-31 21:56:07 +08:00
Kevin Codex a4e2b9f3f3 fix(status): bind a stored claim to the body its signature covered
A claim is provenance, so the signature stored beside it has to reach the
bytes stored beside it. It did not. The signing string covers the body
only through a content-digest line, and require_signature rebuilds the
covered values from the request: with no Content-Digest header the
covered digest is the empty string on the signing side and the verifying
side alike, the Ed25519 check passes over method, path and nothing else,
and the digest-versus-body comparison was skipped for want of a header.
An owner could sign that once and post any body under it. The row that
produced carried a valid signature, a signing string that verifies under
the owner's key, and a request_body the signing string says nothing
about — history nobody can re-verify, which is the one thing the row is
for.

The write path now recomputes the digest of the bytes it is about to
persist and requires the signing string to carry exactly that line, which
is the write-time form of the check re_verify already ran against a
stored row. Line-exact rather than a substring search: build_signing_string
emits one component per line, and a contains would also be satisfied by
the digest appearing inside another component's value.

This is deliberately not the only line of defence, and deliberately not
the middleware's. PR #306 requires the header in require_signature and
closes the same hole one layer earlier for every signed route; that
belongs there, because no other route's correctness should depend on this
handler. Neither substitutes for the other. Without #306 the empty digest
is still accepted everywhere else; without this, a later relaxation of the
middleware resumes writing unverifiable history in silence. Nothing here
touches require_signature, so the two do not collide.

The route's advertised 8 KiB body bound was also a bound on what gets
stored rather than on what gets buffered: require_signature collects and
hashes the whole body before any handler runs, so by the time the handler
measured it the allocation had happened, and concurrent requests
multiplied exactly the cost the number was written to prevent. A
RequestBodyLimitLayer now carries the same figure at the transport,
outside the auth layers and inside the rate limiters, matching the shape
the git routes already use. The handler keeps its own check: it is the one
that holds for a handler reached by any other route table.

The hand-injected test material had to become coherent with the request
it rides on, because a stand-in built over other bytes now takes every
bare-router test down the refusal branch. It is built from the body it
accompanies and carries a real content-digest line.
2026-08-31 21:55:46 +08:00
beardthelion 61033af261 fix(auth): carry the request body only where a route persists it
The signature middleware handed every signed request a second handle to
the buffered body, and only the status write reads it. The field is
optional now and populated behind a marker the persisting route group
applies.

Be accurate about what that saves, because the first reading of this
finding overstated it and the plan was corrected before implementation.
The push handler takes its body as an extractor, so axum consumes the
request and drops every extension at that point; the extra handle lived
across the remaining middleware chain, not the handler's lifetime. The
whole pack really is pinned for the whole receive-pack, but by the
handler's own parameter, which is pre-existing and untouched here.

Layer order is the failure mode worth guarding. A marker applied inside
the auth layers is never seen by the middleware that reads it, so the
body silently stops being captured while every test stays green. The
mutation that proves this leaves the layer present on the same group and
only moves it inside, and the production-router test still reddens. The
handler also refuses an absent body rather than storing an empty column,
so a lost marker fails loudly at the one place it matters.

Four comments claimed a timestamp or a uuid tiebreak decided ordering
when the code orders on the sequence. Three were named in review; the
fourth turned up next to them, calling the timestamp format load-bearing
for a cursor that no longer reads it. The mirror-dedup comments that
correctly document timestamp ordering were checked and left alone.

The shared scrape helper advanced by one byte to stop an end anchor
matching at position zero, which reports a missing anchor for any region
starting on a multibyte character. It advances by a character now. Not
reachable today since every anchor is ASCII, but five guards share this
helper and one that silently finds nothing is worse than one that fails.
2026-08-08 11:07:10 -05:00
beardthelion 5445dda3f5 fix(db): stop the push cursor skipping a concurrently committed row
The sequence is a bigserial, so Postgres allocates it at INSERT and makes
it visible at COMMIT, and nextval does not roll back. Two overlapping
writes to one repo could therefore commit out of sequence order: one
allocates, another allocates and commits, a poller reads the later row
and advances, and the earlier row then becomes visible behind a cursor
that has already passed it. Never delivered, no error, no gap signal,
which is the one thing the catch-up surface exists to prevent.

This is the second time this branch has traded one ordering hazard for
another. The cursor moved off a wall clock because stamping and
committing disagree; it moved onto a sequence where allocating and
committing disagree.

The insert now runs in an explicit transaction that takes a per-repo
advisory lock, so the lock releases at commit and no writer can allocate
until the previous row is visible. The wait is bounded, because the lock
sits on a pooled connection on the inline push path and an unbounded
wait would turn one busy repo into pool exhaustion for unrelated
requests. Three attempts at a one second timeout leaves headroom under
the pool's own acquire timeout.

That bound needs the caller to stop swallowing failures. It warned and
continued, so any error the lock introduced would silently drop those
events forever while the push had already returned success, reaching the
same outcome by another route. It now retries once and then logs at
error level naming the repo, the refs, and the SHAs that were lost.

The defect is pinned by a test that owns both transaction boundaries
rather than a barrier. A barrier cannot reproduce it: before this change
the write was a single autocommit statement, so allocate and commit were
one round trip from the client and the interleaving window was never
open to a caller of the public API.
2026-08-08 10:07:47 -05:00
beardthelion d3074a6c86 fix(status): make a replayed write idempotent and stop dropping refs
Both defects came from an independent cross-model review, and neither
was visible to the reviewers that share this session's model.

A signed write was protected only by clock skew: no nonce, no
idempotency key, no record of signatures already seen. So a captured
request replayed inside the skew window inserted a new row with a new
sequence number, and because the projection takes the latest claim per
producer and context, a stale success outranked the failure that
superseded it. The append-only design is what made this reversible
rather than merely duplicative: a replay that only duplicated a row
would be harmless, a replay that earns a fresh sequence flips the
answer. An earlier reviewer saw the replay window and concluded
append-only bounded it, which had the mechanism exactly backwards.

An exact replay is now idempotent instead of an error. The row carries a
digest over the signature, its input, and the body, a unique index
enforces it in the database rather than a check-then-insert race, and a
repeat returns the original claim. Erroring would have punished the
legitimate case this design otherwise handles worst: a client whose
response was lost retrying the same signed request and concluding its
report failed when it had succeeded.

The fan-out cap was mine, added while fixing unbounded work, and it
truncated at 256 refs after receive-pack had already accepted them. A
push whose pull request source branch sat past that point left the head
permanently stale with no push event, announced only by a log line git
had already contradicted. The writers chunk now, so nothing accepted is
dropped, and the bound that remains refuses before the accept rather
than after: chunking bounds a statement, not a request, and the work
that scales with ref count sits upstream anyway, one protection query
per ref before the service runs and one certificate and webhook per ref
after, with body limits disabled on the git routes.

Removing the cap also exposed a quadratic branch dedupe it had been
keeping cheap; that is a map now.
2026-08-08 07:21:54 -05:00
beardthelion 612d9d2dbe fix(api): order the poll cursor on a sequence and batch the push writes
The push-event cursor paged on an application-stamped wall clock, so a
row stamped later could commit earlier and a poller past that point
would never see the earlier row. A clock step backwards widens it. It
now pages on a database-assigned sequence, the same decision
status_claims already made, so the two surfaces do not disagree about
what ordering means. The rollup's branch resolve had the same flaw and
is fixed with it.

The receive-pack path issued one round trip per ref for each of two
writers, sequentially, on the user's push, with nothing bounding the ref
count. Both are single statements now and the per-push fan-out is
capped, with a warning when it truncates rather than silent loss. The
head update batches through a VALUES join, with last-write-wins dedup on
duplicate branches: git will not produce them, but a join over
duplicates picks arbitrarily where the loop it replaced was
deterministic.

The cursor was unvalidated. A malformed value produced a wrong page
instead of a refusal, limit zero returned a page with a null cursor, and
a poller persisting that cursor silently restarted from the beginning of
history. It is validated, the limit is clamped, and the returned cursor
never moves backwards. This changes the query parameter names, which is
a breaking wire change on an endpoint that has never shipped; every
consumer was checked and only the route registration reads them.

Three copies of the source-scraping parser became one shared helper.
That refactor exposed a real gap: the empty-region mutation reddened
four of the five guards, but the gossip-containment guard passed on an
empty region because it only asserts a must-not. It now asserts its scan
covered what it claims to cover, so a helper bug cannot make it
vacuously green.

status.rs is split into status/mod.rs and status/tests.rs, production
logic unchanged. Four things read the old path and all were retargeted,
including 22 mutation targets across four specs, each re-confirmed to
still match exactly once.
2026-08-08 07:21:54 -05:00
beardthelion 806f9027c0 fix(status): make the caps hold and the projection mean one identity
Five reviewers independently flagged the capped insert. It counted then
inserted inside one transaction with a docstring promising a concurrent
writer could not slip past, which READ COMMITTED does not deliver, and
every cap test was sequential so deleting the transaction would have
kept them green. A per-repo advisory lock now serializes writers on the
bound they are both testing. The new test runs eight writers through a
barrier at cap minus one: without the lock it left 4, 5, and 5 rows
against a cap of 3 on consecutive runs, so the window was real.

The projection compared producer_did as a raw column while the handler
stored whatever spelling the signer used, so one owner writing as
did:key:X and as X produced two entries for one context and the
superseded claim kept voting. Identity is now canonicalized at write
time, and the read filter is a single equality rather than a set.

That fix had a trap. Applying normalize_owner_key directly, as the
finding suggested, collapses to the bare key, which is not a parseable
DID, and would have quietly undone the re-verifiability the previous
commit established. canonical_did canonicalizes to the full did:key
form instead and is written as a function of normalize_owner_key so the
equivalence with did_matches still holds exactly, including the case
where a naive prefix-add would merge two distinct identities.

With that in place the ported copy of the DID collapse in the status
module is gone rather than relocated, and did_matches itself now
delegates, so the Rust gate, the stored column, and the SQL all trace to
one definition.

The projection also selected the signature columns it never renders,
which the previous commit made worse by adding the request body. It has
its own narrow type now, enforced by a source-read test.

The per-repo cap counted for all time with nothing pruning the table, so
a repo that reached it was permanently closed while 429 told clients to
keep retrying. It is a rolling window now, which still bounds the
fan-out a caller-chosen SHA allows, and makes the refusal true.

Also folded two count queries that scanned the same rows into one.
2026-08-08 07:21:54 -05:00
beardthelion ad71f79e6d fix(status): make a stored claim actually re-verifiable
The row stored the RFC 9421 signing string under a column called
signed_payload. That string covers the body only through the content
digest, so it proved someone signed a request with some digest but not
that this claim row was what they signed. The whole append-only argument
rests on a claim staying verifiable after the request is gone, so the
row now carries the request body as well, in a column that says so, and
a test re-verifies a claim from the row alone: body to digest, digest
into the signing string, signature over that string under the producer's
key. Its negative mutates the stored state and shows the same procedure
refuses.

Migration v24 is amended in place rather than stacked, since it has
never shipped.

The persisted material had no size bound and is caller-influenced, so
each field is now bounded. The write response returned the whole claim,
so the wire carried the stored bytes as a JSON integer array; it now
returns a client-facing type and StatusClaim no longer derives
Serialize, so the leak cannot reappear by accident.

Carrying the body as Bytes rather than a Vec matters: the middleware
runs on every signed request including receive-pack POSTs bounded at
2 GB, so copying would have doubled peak memory on every push to serve
a field only the status write reads.
2026-08-08 07:21:54 -05:00
beardthelion 90b8294c2b fix(api): resolve the rollup head from push events, not pinned CIDs
Two review findings.

The fallback read branch_cids, whose only production writer sits inside
a Pinata pin-success branch. With no Pinata configured that table is
never written, so on a default node the fallback could never resolve and
every open pull request without a stored head answered head_resolved
false forever. It now reads repo_push_events, which the receive-pack
path writes unconditionally for every ref update, with both predicates
and the ordering in SQL against the existing keyset index. Still one
database read per request, still no repository acquire.

This is better, not complete: repo_push_events only carries pushes that
land after this ships, so a pull request whose branch was last pushed
before deployment still will not resolve until it is pushed again. There
is no backfill.

The best-effort persist propagated its error, so a transient database
failure turned a rollup that had already resolved its answer into a 500.
It now catches and logs, matching the sibling writes on the push path.
The test asserts the persist genuinely fails before driving the read, so
it cannot pass vacuously.

Also corrected a doc comment that claimed the resolve branch is never
taken twice for one pull request. Only the persist is once-per-pull-
request; the resolve attempt repeats on every read until it succeeds.
The accompanying test pins the corrected wording rather than a behavior
change, since the old code counted the same way.

The resolve counter is process-global while sqlx test databases are
per-test, so unguarded tests were inflating the counts guarded tests
assert on. The guard now covers every test that triggers a resolve.
2026-08-07 18:16:53 -05:00
beardthelion db3681e48d feat(api): add the catch-up poll surface for missed push webhooks
Webhook delivery is spawned, fire-and-forget, and never retried, so a
checker whose endpoint was unreachable during a push simply never learns
about the commit. Rather than add retry machinery and an unbounded queue
pointed at an owner-supplied URL, delivery reliability becomes a
read-side property: a push records an event, and a subscriber walks
forward from a cursor to find every commit a webhook would have told it
about.

Containment is the load-bearing constraint. The row goes into
repo_push_events, written only by the receive-pack path and read only by
this repo-scoped, gate-checked handler. It must never go into
received_ref_updates, which the unauthenticated global feed also reads;
writing local pushes there would publish private-repo pushes on an
anonymous surface. A test pushes to a private repo and asserts nothing
appears on that feed, and retargeting the write reddens it.

The cursor emits a canonical Z-form timestamp with fixed sub-second
width rather than the default rfc3339. The default emits a +00:00
offset, and + decodes to a space in a query string, so a poller echoing
the cursor back re-read the same page forever. The fixed width also
makes the TEXT column's lexicographic order match time order for the
keyset predicate. Found by seeding a timestamp collision, not by
reading.

The scenario tests drive the recorder directly, so a source-scrape test
pins its call site in the push handler, the same way the stored-head
update is pinned. Removing that call was verified to turn it red.
2026-08-07 16:29:48 -05:00
beardthelion 7be9f08b3d feat(api): add the pull request head rollup
GET /api/v1/repos/{owner}/{repo}/pulls/{number}/status answers for the
pull request's head. Both read surfaces now call one extracted
projection, so "the rollup uses the same projection as the commit read"
holds by construction rather than by two implementations agreeing today.

A pull request row stores branch names, so the head has to come from
somewhere. Preference order is the stored head, then a database-backed
branch lookup for an open pull request that has none. No path in this
module acquires the repository or lists refs from disk: that would
download the whole repository from object storage on a cold node, and
this read group carries no rate limiter. A source-read test enforces
that, and asserts its own scan still covers the handler so it cannot
shrink to vacuously passing.

The read-side persist is self-limiting by SQL, not by handler ordering:
the update is conditioned on the head being absent and the pull request
open, so a push landing mid-request cannot be rolled back to a staler
tip and a closed pull request is never back-filled. Reading the same
pull request twice resolves the branch once.

That property is invisible in the response. A variant that re-resolved
on every call and then still preferred the stored head would return
byte-identical bodies across the whole suite, so the test counts the
work done rather than the results emitted, and injecting exactly that
variant reddens only the counter assertion.

An unresolvable head is not a fifth wire state. The state field stays
inside the four values and the condition rides in head_resolved beside
the pull request's own state, so the two cases differ on that boolean
alone and both are tested.
2026-08-07 15:46:00 -05:00
beardthelion 6663fed064 feat(api): serve the combined commit status behind the visibility gate
GET /api/v1/repos/{owner}/{repo}/commits/{sha}/status returns the
projection over the claim history, computed per read rather than
materialized. A materialized index gated only at write time is how this
codebase previously kept serving a repo's slug, owner, branches, and
SHAs after it was made private, so the regression test writes a claim
while public, tightens visibility, and asserts an anonymous read finds
no trace of it.

Three outcomes stay distinct, which is the point of the unit. A caller
who cannot read the repo gets the repo's own not-found, compared byte
for byte against a missing repo so the endpoint is not an existence
oracle. A commit nobody has reported on gets 200 with the explicit
pending zero-count body, asserted exactly so no client can render it as
green. A failed lookup propagates as a 500 carrying the db_error code,
never as an empty success; nothing on this path uses ok-or-default.

Latest-per-context is decided by the database-assigned sequence, not by
the timestamp and not by the uuid. The ordering test seeds the winning
row with the lexically smaller id and a far older timestamp, so it fails
under either wrong key rather than passing by luck.

The projection filters to claims authorized by the current owner, so an
ownership transfer drops the prior owner's claims from the answer while
the history keeps them. That filter is a set-membership test inside the
query, which is the constraint the delegated-capability follow-on
inherits. It expands the owner DID into its equivalent forms, and a test
runs that expansion against did_matches over a ten-case matrix so the
two cannot drift apart silently.
2026-08-07 14:53:53 -05:00
beardthelion ac31868205 feat(api): add the owner-gated commit status write path
POST /api/v1/repos/{owner}/{repo}/statuses/{sha} appends a status claim.
Authorization runs read-visibility first, then owner, and the order is
the security property: authorize_repo_read denies a quarantined repo
before the visibility gate and answers with the repo's own not-found, so
a caller who cannot read the repo cannot learn it exists. Loading the
repo and comparing the owner would answer 403 there and turn the
endpoint into an existence oracle. That case is tested directly, since
it is what separates a correct implementation from a plausible one.

Three caps, all evaluated in the insert transaction and all refusing
with 429: per producer and context, distinct contexts per commit, and
total rows per repo. The per-tuple cap alone bounds nothing, because the
context string and the SHA are both caller-chosen and the SHA is never
existence-checked. The route group also carries the per-DID and per-IP
limiters that the creation routes use; the plain write group carries
neither and this endpoint appends on every call.

Wider than the endpoint: require_signature now attaches the verified
RFC 9421 material as a request extension, so a handler can persist what
the signature actually covered. Rebuilding the canonical string in the
handler would store a string nobody verified, and none of it survives
the request. The write fails closed with a 500 when that material is
absent, rather than storing an empty payload, because a claim nobody can
re-verify is not history the later attestation substrate can adopt.

Nine injected-defect mutations confirm the gates are load-bearing rather
than decorative, including that removing the route group's merge leaves
a group that compiles, lints, and does not exist at runtime.
2026-08-07 14:16:38 -05:00
beardthelion af17831429 feat(pulls): track the source branch head on open pull requests
The rollup needs an honest target commit, and a pull request row stores
only branch names. Resolving the branch at read time is racy (the SHA
moves between the resolve and the rollup) and gives a merged or
deleted-branch pull request no answer at all, since the merge commit is
computed and never persisted.

The receive-pack path now sets head_commit for open pull requests whose
source branch matches an updated ref, one statement keyed on repo,
branch, and open status. Excluding closed and merged rows from the WHERE
clause is what freezes the value. The merge path stamps the source head
it actually consumed, which covers a push landing between the handler
reading the row and committing the merge.

A ref arriving on the push path is a full refs/heads/ name while
source_branch stores the bare branch, so the normalisation is tested
directly; a silently non-matching WHERE clause would make this a no-op
that still reads green. Branch deletions are skipped rather than stored,
since 40 zeros is a target no commit resolves to.

The scenario tests drive the update helper directly, because a real
receive-pack POST needs a pack file. That proves the mechanism and says
nothing about the sink, so a source-scrape test pins the call site in
the push handler. Removing that call was verified to turn it red.
2026-08-07 13:17:32 -05:00
beardthelion 04263dbd49 feat(db): add migration v24 for status claims, PR head, and push events
Schema for the commit-status surface, all three objects in one versioned
entry rather than three: an append-only status_claims table, a nullable
head_commit column on pull_requests, and repo_push_events for the
catch-up poll surface.

Version 24 rather than 18 because origin/pr-173 and
origin/fix/issue-135-ipfs-cid-tree-gate both claim through 23, and the
migration runner keys only on version without comparing the recorded
name, so a collision is a silent full skip rather than an error.

status_claims orders on a database-assigned bigserial seq, not the uuid
id. A uuid v4 is random, so using it as the timestamp tiebreak would let
a retried pending claim beat a success roughly half the time when both
land in the same rfc3339 tick.

Each claim stores the producer's RFC 9421 signature headers and the
canonical signed bytes. Append-only preserves rows, but only stored
signature material preserves provenance, and a claim nobody can verify
after the request is gone is not history a signed-attestation substrate
can adopt later.

The table is repo_push_events, not push_events: the latter already
exists in v1 for agent trust scoring, so a second CREATE TABLE under
IF NOT EXISTS is a silent no-op whose follow-on index then fails.

The upgrade-path test is the load-bearing one. A sqlx::test provisions
an empty database and runs the whole array, so the fresh-chain tests
stay green even when DDL is appended to an already-applied entry.
2026-08-07 12:07:35 -05:00
Kevin Codex fdf716dd25 Merge pull request #281 from Gitlawb/release-please--branches--main--components--gitlawb-node
chore(main): release 0.7.1
2026-08-05 13:06:15 +08:00
Kevin Codex c926e1ede5 Merge pull request #292 from Gitlawb/chore/bump-ruint-rustsec-2026-0220
chore(deps): bump ruint to 1.20.0 for RUSTSEC-2026-0220
2026-08-03 10:44:46 +08:00
github-actions[bot] 5f445152a8 chore: sync Cargo.lock with the release version bump 2026-08-03 02:44:15 +00:00
github-actions[bot] cd810b3375 chore(main): release 0.7.1 2026-08-03 02:44:00 +00:00
Kevin Codex acb47145be Merge pull request #298 from euxaristia/fix/get-profile-strict-did-matching
fix: strict DID matching in get_profile
2026-08-03 10:43:29 +08:00
euxaristia b4016dce9d fix(db): pin profile DID matching with lone non-key fixture and boundary matrix
Add a lone non-key DID fixture check to get_profile_does_not_match_non_key_did to verify that a bare short query returns None when only a non-key profile exists. Add profile_did_case_sql_matches_normalize_owner_key to pin PROFILE_DID_CASE_SQL against Rust normalize_owner_key across the full owner-key boundary matrix. Add cfg(unix) target gates to Unix-specific symlink and permission test helpers.

Refs #298
2026-08-01 10:20:22 -04:00
euxaristia ee9bc93198 chore(deps): bump ruint to 1.20.0 for RUSTSEC-2026-0220
cargo audit fails on ruint 1.17.2 (incorrect overflow flags / truncated
shift amounts). Upgrade the lockfile to the patched 1.20.0 release.
2026-08-01 05:47:26 -04:00
euxaristia 9836c3a21b fix: target stored did on profile update aliases
get_profile equates bare short ids with did:key forms. upsert_profile
must UPDATE by existing.did so a did🔑 alias against a bare-stored
profile actually persists. set_profile_cid uses the same CASE match.
Add a regression test covering both write paths.
2026-08-01 05:41:31 -04:00
euxaristia 13b565bcac fix: strict DID matching in get_profile
Replace the broad LIKE suffix match in profile resolution with the same
normalize_owner_key / CASE pattern already used by get_repo, so non-key
DID methods that share a short-id suffix cannot resolve the wrong profile.

Add a sqlx test locking in bare, did:key, and non-key DID behavior.
2026-08-01 05:27:58 -04:00
t 0e42eb3343 chore(deps): bump ruint to 1.20.0 for RUSTSEC-2026-0220
cargo audit fails on main, so every open PR inherits a red check for a
problem that is not in its diff.

ruint 1.17.2 carries RUSTSEC-2026-0220: Uint shift operations report
incorrect overflow flags and truncate shift amounts. It reaches us
transitively through alloy-primitives, which alloy pulls into both
gitlawb-node and gl for the on-chain operator client. Our own U256 use in
operator.rs is pow, mul, div, and comparison rather than shifts, so the
direct exposure looks limited, but alloy's internals are not audited here
and the fix is a semver-compatible bump.

Lockfile only; no manifest change. The bump adds six arkworks crates
(ark-ff, ark-ff-asm, ark-ff-macros, ark-serialize, ark-serialize-derive,
ark-std), taking the tree from 715 to 721 dependencies. That surface
increase is the cost of the fix and is worth noting rather than passing
over: it arrives through ruint's own feature set, not through anything we
asked for.

None of it reaches gitlawb-core, so the dependency purity check still
passes at 83 allowlisted normal deps.

Verified: cargo audit exits 0, cargo build --locked --release, 564 node
tests and 83 core tests pass.
2026-07-30 16:05:23 -05:00
Kevin Codex c83cbc56e0 Merge pull request #243 from Gitlawb/ci/242-lock-sync-release
ci: keep Cargo.lock in sync through releases and build --locked (#242)
2026-07-30 09:35:55 +08:00
Kevin Codex 1964f88378 Merge pull request #280 from Gitlawb/fix/270-reset-peer-reachability-on-url-change
fix(node): reset a peer's reachability gate when an announce changes its http_url (#270)
2026-07-30 09:15:01 +08:00
Kevin Codex 99b2b9a166 Merge pull request #267 from Gitlawb/fix/docker-build-cache-and-msrv
fix(ci): make the docker dependency cache work and unbreak Dockerfile.bins
2026-07-30 09:14:22 +08:00
Kevin Codex 9418e315bf Merge pull request #213 from Gravirei/fix/issue-212-icaptcha-client-api-key-gate-test
test(icaptcha-client): add test that new() honors key_trusted gate (#212)
2026-07-30 09:13:24 +08:00
Kevin Codex c70b1df42d Merge pull request #274 from Gitlawb/fix/272-sync-slug-path-escape
fix(node): stop a peer-supplied repo slug from escaping repos_dir (#272)
2026-07-30 09:12:23 +08:00
t 811ad504bc fix(review): correct two comment claims and guard two more wrong shapes (#270)
Review of the previous two commits found the SQL correct and the comment
wrong in two places, plus two implementations of the intent that all four
tests accepted.

Comment fixes:

The claim that four http_url consumers "never read the flag at all" named
the public resolve route, which does read it: api/resolve.rs republishes it
as `reachable`, as does the peer listing. The accurate split is that only
the federated repo fan-out in api/repos.rs GATES on the flag, four consumers
act on a repointed URL regardless of it, and two surfaces republish it.
Three reviewers caught this independently.

The atomicity sentence implied the conflict row lock protects the flag. It
orders announces against each other and nothing else: mark_peer_ping writes
by DID with no http_url predicate, so a probe of the previous URL that lands
after a reset can still re-grant the flag until the next round. Said so, and
qualified the self-healing claim on cosmetic differences, which holds only
if no further announce lands first.

Two tests added, each proven load-bearing against a shape the original four
accept:

  cosmetic_url_difference_counts_as_a_change
    Pins the exact-comparison decision. Normalizing the comparison
    (rtrim(peers.http_url, '/') vs rtrim($2, '/')) passes the other five and
    fails only this one. The original four only ever compare identical or
    wholly different hosts, so none of them constrain the comparison.

  same_url_reannounce_still_advances_last_seen
    The reset must ride the existing UPDATE, not gate it. Hoisting the
    condition to a statement-level WHERE passes the other five while
    skipping the update entirely on a same-URL re-announce, so liveness
    stops advancing and the peer ages out on last_seen. Observed with both
    timestamps identical.

Gates: cargo test -p gitlawb-node 516 passed 0 failed; clippy --all-targets
-D warnings clean workspace-wide; fmt --check clean.

Refs #270
2026-07-29 17:25:05 -05:00
t e2560ee46c fix(node): reset a peer's reachability gate when an announce changes its http_url (#270)
The ON CONFLICT branch rewrote http_url and left last_ping_ok alone, so an
unsigned announce could repoint an existing peer's URL and inherit the
reachable=true that the previous host earned. api/repos.rs filters the
federated fan-out on that flag and then stamps each entry from the peer's
response body with the peer's DID, so the inheritance is what made the
injection instant rather than probe-delayed.

Clear the flag when the URL actually changes, and leave it alone when it does
not. In the DO UPDATE branch peers.http_url is the existing pre-update row and
the comparison runs under the conflict row lock, so this needs no second round
trip and does not race a read-then-write. Both writers funnel through
upsert_peer, so the announce handler and the bootstrap announce-back are both
covered.

Observed transition on the tests from the previous commit:

  before  url_change_clears_reachability ... FAILED (last_ping_ok assertion)
  after   url_change_clears_reachability ... ok
          the other three ... ok throughout

Both guards were confirmed load-bearing by reverting, not by reading a green
suite. Removing the CASE clause entirely turns url_change_clears_reachability
RED again. Flattening it to last_ping_ok = (peers.http_url IS NOT DISTINCT FROM
$2) still passes three of the four cases and is caught only by
same_url_reannounce_does_not_grant_reachability, which is why that case exists.

This bounds the automatic inheritance; it does not close the rewrite. Four
other http_url consumers never read this flag, and until #248 lands the
unauthenticated ping route can write it back from the attacker's own host.
#273 is the closure.

Closes #270
2026-07-29 17:02:21 -05:00
t 75e3e03c09 test(node): pin peer reachability across a URL change, a repeat, and a fresh insert (#270)
upsert_peer's ON CONFLICT branch rewrites http_url and says nothing about
last_ping_ok, so a repointed peer keeps the reachability the previous host
earned. These four cases state the behavior the fix has to produce, and they
land before it so the failure is observed rather than assumed.

Pre-fix results:

  url_change_clears_reachability ... FAILED
  same_url_reannounce_keeps_reachability ... ok
  fresh_peer_inserts_unreachable ... ok
  same_url_reannounce_does_not_grant_reachability ... ok

  panicked at crates/gitlawb-node/src/db/mod.rs:5737:
  a repointed peer must re-earn reachability, not inherit it

The failure is on the last_ping_ok assertion; the http_url assertion above it
passes, so the row is being rewritten and only the gate is wrong.

The three that already pass are regression pins, not filler.
same_url_reannounce_keeps_reachability guards against clearing the flag on
every conflict rather than only on a change.
same_url_reannounce_does_not_grant_reachability guards the other direction and
is the only case that fails if the conditional is flattened to
last_ping_ok = (peers.http_url IS NOT DISTINCT FROM $2), which would let any
unsigned same-URL re-announce set the flag TRUE with no probe.
fresh_peer_inserts_unreachable guards the INSERT branch.

Refs #270
2026-07-29 16:56:42 -05:00
beardthelion fbc5868bd1 fix(node): move the attempted_at migration to v17, clear of two open branches
v12 was main's current_max + 1, which INV-7a says is necessary and not
sufficient: the runner keys the applied set on the integer alone, so a
version another in-flight branch also claims is skipped in full on
whichever side merges second. No error, no warning, schema_migrations
still reading healthy, and the column simply absent, after which
dequeue_pending_syncs fails on every poll and sync ingestion stalls.

Two open branches already claim into that range. #135/#173 holds
through 14 (15 once it rebases past main's v11) and #253 took 16. 17
clears both, and the reservation is recorded in a comment so the next
author can see it. Gaps are harmless; the runner iterates the array and
never requires contiguity.

Not doing the other half here. INV-7a's load-bearing rule is the name
check that makes the silent skip impossible rather than merely
unlikely, and that lands with #253 on test/replay-guards-253; adding it
on this branch would collide with that work.
2026-07-29 13:16:41 -05:00
beardthelion 457be11d72 fix(review): stop the dequeue overclaiming, and pin the starvation premise
Three review fixes on the two commits above.

The docstring claimed more than the code delivers. UPDATE ... RETURNING
does not order its output, so the batch is the right set in no
particular order (nothing in process_batch depends on the order); and
nothing is claimed, since the rows stay pending with no lock held past
the statement, so two workers against one database can still be handed
the same batch. Single-worker is the existing assumption and
FOR UPDATE SKIP LOCKED is what would change it. It also implied the
caller handles a failed stamp robustly; the caller logs and skips the
poll, which is worth stating plainly now that this writes and can fail
for reasons a plain SELECT could not.

The outer UPDATE now repeats status = 'pending'. Between the subquery
and the update a concurrent worker can settle a row, and without the
predicate the statement would stamp and return a row that had already
left the pending set.

Both starvation tests now assert the healthy row is still pending
before the poll that frees it. Without that they would keep passing if
the batch size ever grew past the stuck set, having silently stopped
exercising head-of-line yield, and the larger test now also asserts the
stuck rows rotated rather than settled.
2026-07-29 12:45:03 -05:00
beardthelion 6e64d0d0a7 fix(node): stamp every dequeued sync row so a stuck one yields the batch
A row the worker cannot make progress on stays pending, and while its
ordering key never moved it stayed among the oldest rows forever,
holding the fixed-size window against every healthy repo behind it.
Classifying more error kinds as terminal does not close this: an owner
directory that exists but denies traversal lets create_dir_all succeed
and defers at path_within_root instead, per-repo rather than as a
whole-repos_dir outage.

dequeue_pending_syncs now selects and stamps in one statement, ordering
by COALESCE(attempted_at, enqueued_at), so the key means least recently
handed out. Doing it in the dequeue rather than at each deferral branch
makes it hold by construction: no call site can forget it, a batch that
dies mid-loop still leaves its rows stamped, and a failed stamp is a
failed dequeue the caller already handles. enqueued_at keeps meaning
enqueue time so backlog age stays measurable.

The column lands as migration v12, not a v1 append: v1 never re-runs on
an existing install, so appending there would leave every deployed node
dequeueing against a column that does not exist. The upgrade-path test
simulates a v11 node and is the only test that catches that placement,
since every #[sqlx::test] otherwise starts from a fresh database.

N stuck rows still cost N slots per rotation, delaying a healthy row by
up to ceil(N/10) polls, so the attempts column and a retry cap remain
worth doing; this removes the starvation, not the residual cost.
2026-07-29 12:27:47 -05:00
beardthelion 2e664b7885 fix(node): fail a sync row terminally when a non-directory holds the owner path
create_dir_all reports AlreadyExists when a regular file, dangling
symlink, or link loop occupies repos_dir/<owner>, and none of those
clear on a retry. Classifying them as transient left the row pending,
and since dequeue_pending_syncs is oldest-first over a fixed batch of
10, a handful of them held the window and starved every healthy repo
behind them. This restores the pre-#272 behavior, where the same state
failed the clone and hit the terminal Err arm.

A concurrent mkdir resolves to Ok rather than AlreadyExists, because
create_dir_all falls back to is_dir() on EEXIST, so the race is not
caught by this classification.
2026-07-29 12:15:44 -05:00
beardthelion 9aa4c17aef fix(node): make a stalled sync queue visible, and fix two test claims
Two follow-ups from code review.

The pending-on-IoError arm had no operator-visible signal: the only
record_sync_processed calls were the done and failed arms, so a queue
stalled on an unmounted repos_dir looked identical to an idle one. Both
deferring arms now record "deferred" and the guard rejections record
"rejected", which also distinguishes a retryable stall from a terminal
refusal. There is still no attempt cap; bounding retries needs an attempts
column on sync_queue, which is a migration and belongs in its own change.

The two escape tests were asserting that repos_dir stays empty, but the
owner directory is created before containment has a parent to canonicalize,
so that assertion was detecting the create_dir_all side effect rather than
an escape. They now assert no mirror exists anywhere under repos_dir.

Their comments were also wrong about which layer holds. Mutation testing
says the slug rule and containment are each independently sufficient for
that input: removing either alone leaves the tests green, and only removing
both turns them red, with the mirror observed written outside repos_dir.
The comments now say that, and point at the per-layer witnesses instead:
./hello isolates the slug rule because it resolves back inside repos_dir
and containment approves it, and the two symlink tests isolate containment
because no character rule can see a symlink.
2026-07-29 04:14:07 -05:00
beardthelion 27b0495a6f fix(node): bound the slug owner by NAME_MAX, not the DID limit (#272)
Code review caught an availability regression introduced by the guard added
earlier on this branch. validate_owner_did bails only above 256 chars, but
the owner half becomes a single path component and Linux NAME_MAX is 255, so
a 256-char owner passed validation and then failed create_dir_all with
ENAMETOOLONG on every attempt. The worker left such a row pending, and
dequeue_pending_syncs is oldest-first with a fixed batch of ten, so ten
unsigned notify calls held the entire window and starved every healthy repo
behind them. Before this branch the same row reached git clone, failed, and
was retired.

Two changes. Reject the length in the shared validator, so an undeliverable
slug never enters the queue from either boundary. Then split the
create_dir_all arm by whether the error can ever clear: a transient one
still leaves the row pending to be retried, a permanently uncreatable path
is marked failed rather than re-picked forever.

The earlier test missed this by asserting 257, one past the bound. It now
pins both sides: 256 rejected, 255 still accepted.

Also make the two permission-bit tests fail loudly instead of returning
early. One probed with symlink_metadata on a path that was never created,
which is Err for every user, so its root guard could never fire; the other
returned green after an eprintln that cargo swallows on a passing test, so
a root runner would have reported ok while exercising nothing.
2026-07-29 03:41:55 -05:00
beardthelion 6456bfb0ba test(node): commit the #272 probes as regression tests
The two throwaway probes from the investigation, in attack form, asserting
the closed outcome. The router-level one drives the full router with no auth
extension, because the point it pins is reachability: the attack needs no
signature and no operator config change, only an unsigned announce followed
by an unsigned notify.

The worker-level one seeds the peer row directly, since process_batch
resolves the peer URL before it looks at the slug and an unseeded row dies
at the no-peer arm, which would leave the test green with the guard gone.
It asserts on the parent directory too, because git clone removes the
destination on failure but leaves the parent.

Note for reviewers: reverting the worker slug check alone does not recreate
the absolute-path escape, since containment catches it and leaves the row
pending. That check is independently load-bearing for ./hello, which
canonicalizes back inside repos_dir and so passes containment.
2026-07-29 03:16:55 -05:00
beardthelion 3ca94739ed fix(node): prove the mirror path is inside repos_dir before running git (#272)
The character rules on the slug cannot see a symlink. Two cases get through
them: a symlinked owner directory, and a symlinked <name>.git leaf. The leaf
is the one a parent-only check misses, since the parent canonicalizes clean,
exists() follows the link, and the fetch branch then runs git -C through it
with the mirror refspec, force-overwriting refs outside repos_dir.

So the predicate canonicalizes the candidate when symlink_metadata says it
exists, and its parent only when it does not. Canonicalizing the candidate
unconditionally would reject every first clone, which is total loss of
mirroring, and the must-not-break test covers that direction.

Three-valued rather than a bool because the worker has to tell a hostile
path from a filesystem that could not answer. Outside is terminal; an I/O
error leaves the row pending, since dequeue only selects pending and a
transient EACCES would otherwise retire a legitimate repo permanently.

The predicate is pure. create_dir_all lives at the call site so the admin
purge path can ask a containment question without mutating anything.
2026-07-29 03:03:30 -05:00
beardthelion c587a40a50 fix(node): reject a malformed slug at the notify boundary (#272)
/api/v1/sync/notify accepts unsigned callers by default, and its only gate
was that node_did appear in the peer table, which one unsigned announce of
the caller's own DID satisfies. req.repo then reached enqueue_sync verbatim.

Validate after the peer check so an unknown peer still gets the unknown-peer
message, and before enqueue_sync so nothing is written. The 400 names the
field and carries no path: the validator's text never interpolates the slug,
so a hostile string is not echoed back either.

NOTIFY_BODY used "demo", a slug with no separator, so the three brake tests
would have started asserting against the new 400 instead of the brake.
2026-07-29 02:40:48 -05:00
beardthelion 9f00e24aec fix(node): validate the queued slug before the worker derives a path (#272)
process_batch split item.repo on the first '/' and joined both halves onto
repos_dir. Since PathBuf::join does not normalize and an absolute second
component replaces the path, a queued `a//tmp/x` cloned an attacker's
origin to /tmp/x.git with repos_dir left empty.

The worker is the real choke point rather than defence in depth: three
writers reach sync_queue (the notify handler, trigger_sync, and the
gossipsub ref-update handler at p2p/mod.rs), and rows queued before this
change are already in the table.

The tests serve the composed remote URL for every hostile slug, not just
the escaping one. With an unreachable remote those rows end failed anyway
because the clone fails, so the scenarios would have passed before the
guard existed and proven nothing.
2026-07-29 02:27:35 -05:00
beardthelion 51d82ded57 fix(node): add a shared owner/name slug validator (#272)
The sync queue carries a peer-supplied `repo` string that the worker turns
into a filesystem path, and `PathBuf::join` does not normalize, so an
absolute second component replaces the accumulated path. `a//tmp/x`
resolved to `/tmp/x.git`, outside repos_dir entirely.

Validate the slug once, in the module that already owns the owner and name
rules, and return the two halves so callers cannot re-split and drift. The
only rule added here is the leading `.`/`-` check on the owner half, which
`validate_owner_did` lacks because it also serves full DIDs.

Each guard is covered by a case that isolates it: the earlier rules shadow
the obvious candidates, so `../hello` does not actually exercise the
owner delegation and `a//tmp/x` does not exercise the separator count.
2026-07-29 02:27:35 -05:00
t a49e04ddcd fix(ci): make the docker dependency cache work and unbreak Dockerfile.bins
The Dockerfile cache layer primed from four crate manifests while the
workspace has six members, and crates/gl takes a path dependency on
icaptcha-client. Cargo loads the whole workspace before resolving, so the
layer aborted on the missing manifest and || true swallowed it. Every
image build has been a full cold build since icaptcha-client landed,
including the docker-build smoke test on every PR. Copy all six
manifests, and drop the || true: the layer is expected to succeed, and
tolerating its failure is what let this sit dead unnoticed.

Dockerfile.bins pinned rust:1.85-bookworm against a workspace
rust-version of 1.91. An unmet MSRV is a hard cargo error, so both Linux
musl targets of scripts/build-bins.sh have been unbuildable. Nothing in
CI builds that file, which is why nothing reported it.

Closes #265
Closes #266
2026-07-28 17:26:24 -05:00
t ebfe89d1c9 ci: declare the lock-sync toolchain and extend --locked to the shipping builds
sync-release-lock ran cargo update --workspace on whatever Rust the
ubuntu-latest image shipped, so the lock it wrote for the release PR came
off an undeclared input. Install the same stable toolchain the other
cargo jobs use. This does not make resolution identical (stable moves,
and the MSRV gate checks the same lock on 1.91), it just removes the
runner image from the equation.

The release-binaries build, both Dockerfiles and the darwin path of
build-bins.sh also resolved without --locked, so a lockfile disagreeing
with the manifests was caught on the PR gates but not where artifacts
are actually produced. Add --locked to all five.

Two of those five cannot reach resolution today and are hardened for
when they can: the Dockerfile dep-cache layer at line 24 copies four of
the workspace's six member manifests, so cargo dies at workspace load and
|| true swallows it, and Dockerfile.bins pins rust:1.85 against a 1.91
MSRV. Both are pre-existing and left alone here. The live coverage comes
from release.yml, Dockerfile:37 and build-bins.sh.

The gitlawb-core dependency-purity gate still resolves without --locked
on purpose, but its comment justified that by saying the build and test
jobs do the same, which stopped being true when those jobs gained
--locked. Restate the real reason: a stale lock would red that gate for
something unrelated to the dependency closure it checks.
2026-07-28 16:28:27 -05:00
t 5091f2dba0 ci: fail the release-lock job on a failed dispatch, and scope release-PR discovery
Two review findings on the sync-release-lock job.

Fail on every dispatch failure. The step tolerated the 422 a release branch
cut before pr-checks gained its workflow_dispatch trigger returns, on the
theory that release-please would rewrite the branch and heal it. A rewrite
only happens when a later releasable commit lands, so an open release PR
nobody touches is never rewritten, and the tolerated path left the lock-sync
commit this job had just pushed on the release head with no --locked
validation. That is the exact state the job exists to prevent. No release PR
is open today, and after this lands every release branch is cut from a main
that already carries the trigger, so the tolerance protects nothing.

Scope the fallback discovery. `gh pr list` includes fork PRs and a fork's
headRefName is bare, so matching on the `release-please--` prefix alone let
any fork nominate the branch this job checks out, commits to, and dispatches
on: actions/checkout then cannot resolve that fork-only ref here, failing the
job and leaving the real release PR unsynced and unchecked. Discovery now
requires the head repo to be this repository and the author to be the
release-please app, and selects the highest PR number instead of relying on
gh's undocumented default ordering.

Verified the new filter against the hostile inputs, using the jq program as
extracted from the parsed YAML: a fork lookalike, a fork PR claiming the
app author, and a same-repo human branch named `release-please--*` are all
rejected; the real release PR is still selected when listed after a fork and
when a second component release PR is present; an empty listing yields empty.
The previous expression selected the fork's branch on all three hostile cases.
2026-07-27 18:34:28 -05:00
t f2f99c6684 chore: regenerate Cargo.lock for the 0.7.0 manifests after rebase
The branch forked before v0.7.0 merged, so the rebase left main's 0.7.0
manifests against workspace lock records still at 0.6.0. That state fails
the gate this PR adds: `cargo metadata --locked` exits 101 with "cannot
update the lock file because --locked was passed".

Regenerated with `cargo update --workspace`, the same command the new
sync-release-lock job runs. Only the 5 workspace crates move 0.6.0 ->
0.7.0; all 252 external dependencies are unchanged, so this is a version
resync and not a dependency bump. `cargo metadata --locked` now exits 0.
2026-07-27 18:28:00 -05:00
t b605b2269a ci: dispatch pr-checks on the release branch so GITHUB_TOKEN pushes still get validated 2026-07-27 18:27:13 -05:00
t 52a8f010b7 ci: lock-sync the release PR and build --locked so lock drift fails loudly (#242) 2026-07-27 18:27:13 -05:00
t b8eff30fc2 chore: sync Cargo.lock with the 0.6.0 manifests 2026-07-27 18:26:45 -05:00
Kevin CodexandClaude Fable 5 111cff7e20 infra: add production fly configs for gitlawb-node-2 (sjc) and gitlawb-node-3 (nrt)
Fleet configs from the 2026-07-22 multi-node deploy: iCaptcha enforce
(level 3), cross-node bootstrap peers, 500/400 connection limits.

Co-Authored-By: Claude Fable 5 <[email protected]>
2026-07-23 09:55:22 +08:00
Kevin Codex e9da517fde Merge pull request #240 from Gitlawb/release-please--branches--main--components--gitlawb-node
chore(main): release 0.7.0
2026-07-23 08:27:35 +08:00
Kevin Codex 9ca94c4d47 Merge pull request #241 from Gitlawb/ci/gitlawb-core-dep-purity
ci(core): add dependency-purity allowlist gate for gitlawb-core
2026-07-23 07:06:43 +08:00
beardthelion c0ad500b55 ci(core): allowlist curve25519-dalek-derive
Proc-macro helper for curve25519-dalek (already allowlisted). Newer cargo
versions list it under `cargo tree --edges normal` while older ones do not;
the allowlist is a superset so it passes on both. Caught by core-deps-purity
on the first CI run of this branch.
2026-07-22 12:36:25 -05:00
beardthelion e9d6aadd58 ci(core): add dependency-purity allowlist gate for gitlawb-core
gitlawb-core is embedded by gl, git-remote-gitlawb, and the node daemon,
so it must stay lean. Add a CI gate that fails if its normal (non-dev,
non-build) transitive dependency tree gains any crate not on an explicit
allowlist.

The allowlist is exhaustive-by-construction: a new heavy dependency reds
CI whether or not anyone thought to ban it, which a denylist cannot do.
The checker resolves like the build/test jobs (no --locked, since the
committed lockfile can lag the manifests) and snapshots/restores
Cargo.lock so a local run leaves no working-tree side effects.

- ci/gitlawb-core-allowed-deps.txt: the 83 current normal deps, plus the
  regeneration command and rationale in the header
- scripts/check-gitlawb-core-deps.sh: recompute and diff; hard-fail on a
  non-allowlisted crate, note-only on a stale (removed) allowlist entry
- pr-checks.yml: new hard-fail core-deps-purity job, matching the pinned
  action SHAs and style of the sibling jobs
2026-07-22 12:04:34 -05:00
github-actions[bot] 32006b26a2 chore(main): release 0.7.0 2026-07-22 15:28:32 +00:00
Gravirei 1ba9ee505a test(icaptcha-client): use is_some() instead of != None 2026-07-22 19:18:03 +06:00
Gravirei e1e6e1cf83 test(icaptcha-client): cover https port isolation through new(), collect failures 2026-07-22 18:54:33 +06:00
Gravirei 82fec97a97 test(icaptcha-client): cover operator+hostile-advert and no-operator+no-advert arms 2026-07-20 14:26:18 +06:00
Gravirei 9be5eac3a2 fix(icaptcha-client): update dependencies to version 0.5.1 and enhance operator fallback handling in tests 2026-07-19 11:28:41 +06:00
Gravirei 6b2ec93de1 test(icaptcha-client): add test that new() honors key_trusted gate (#212) 2026-07-19 01:55:57 +06:00
45 changed files with 12607 additions and 10648 deletions

No files matched your search

-30
View File
@@ -13,16 +13,9 @@ GITLAWB_PUBLIC_URL=https://your-node.example.com
# ── Server ────────────────────────────────────────────────────────────────
GITLAWB_HOST=0.0.0.0
GITLAWB_PORT=7545
# Optional address to bind a Prometheus /metrics exposition endpoint on (e.g.
# 127.0.0.1:9091). Leave empty (default) to disable. Bind to localhost or a
# private interface — the metrics endpoint is unauthenticated.
GITLAWB_METRICS_ADDR=
# ── Storage ───────────────────────────────────────────────────────────────
GITLAWB_REPOS_DIR=/data/repos
# Tigris (S3-compatible) bucket for repo storage. Leave empty (default) to
# disable Tigris and use local-only storage.
GITLAWB_TIGRIS_BUCKET=
# PostgreSQL connection URL. Required.
# When using the bundled docker-compose, this is wired automatically.
@@ -33,12 +26,6 @@ DATABASE_URL=postgresql://gitlawb:changeme@localhost:5432/gitlawb
# connections open lazily. Size against the DB server's max_connections,
# remembering admin tooling opens its own pool.
GITLAWB_DB_MAX_CONNECTIONS=20
# Maximum connections in the dedicated advisory-lock pool (separate from the
# pool above). Each in-flight repo write pins one connection here for its whole
# lifetime, so size it to peak concurrent distinct-repo writers — keeping it
# separate is what stops a push burst from starving request handlers. Keep
# (main pool + lock pool) within the DB server's max_connections.
GITLAWB_DB_LOCK_POOL_MAX_CONNECTIONS=32
# Seconds a request waits for a pool connection before failing with 503.
GITLAWB_DB_ACQUIRE_TIMEOUT_SECS=5
# Upper bound on each startup connect+migrate attempt, in seconds. Keep it
@@ -52,9 +39,6 @@ GITLAWB_DB_RETRY_INITIAL_SECS=5
GITLAWB_DB_RETRY_MAX_SECS=60
# ── IPFS pinning (Pinata) ─────────────────────────────────────────────────
# URL of a local IPFS/Kubo node HTTP API (e.g. http://127.0.0.1:5001). Leave
# empty (default) to disable local IPFS.
GITLAWB_IPFS_API=
# Get a JWT at https://app.pinata.cloud/developers/api-keys
GITLAWB_PINATA_JWT=
GITLAWB_PINATA_UPLOAD_URL=https://uploads.pinata.cloud/v3/files
@@ -143,20 +127,6 @@ GITLAWB_PUSH_RATE_LIMIT=600
# the client IP. 0 disables. Default 120.
GITLAWB_CREATE_RATE_LIMIT=120
# ── Write rate limiting (non-creation authenticated writes) ───────────────
# Max non-creation write requests per client IP per hour: issue/PR comments,
# labels, stars, merges, protect/unprotect, replicas, visibility, tasks,
# bounties, profile, and all GraphQL HTTP requests. The brake wraps the whole
# /graphql route, so queries and the playground GET consume this bucket too, not
# only mutations (GraphQL WebSocket subscriptions are excluded). Its own bucket,
# separate from the creation and push brakes. Uses GITLAWB_TRUSTED_PROXY to
# resolve the client IP.
# NOTE: this is a per-IP aggregate across ALL those write actions, so behind a
# shared NAT/egress IP (or with GITLAWB_TRUSTED_PROXY unset) many users collapse
# onto one bucket — raise this for automation-heavy or multi-user single-IP
# deployments. 0 disables. Default 600.
GITLAWB_WRITE_RATE_LIMIT=600
# ── Peer-sync rate limiting (per client IP, uses GITLAWB_TRUSTED_PROXY below) ─
# /api/v1/peers/announce and /api/v1/sync/notify accept unsigned requests from
# known peers and run at higher frequency, so a generous bucket. Separate from
+41 -5
View File
@@ -6,6 +6,13 @@ on:
# result before it lands. Guards against semantic merge races where two
# individually-green PRs break main in combination (#113 + #145 → #229).
merge_group:
# Release-branch heads are created and pushed by GITHUB_TOKEN (release-please
# and the lock-sync job), and GitHub's recursion guard means those pushes
# trigger no workflow runs. workflow_dispatch is the documented carve-out that
# GITHUB_TOKEN CAN trigger, so the release flow dispatches this workflow at
# the release branch to attach check runs to its head. Read-only workflow
# (contents: read), so the dispatch surface adds no privilege.
workflow_dispatch:
push:
branches:
- main
@@ -46,7 +53,7 @@ jobs:
run: cargo fmt --all -- --check
- name: cargo clippy
run: cargo clippy --workspace --all-targets -- -D warnings
run: cargo clippy --locked --workspace --all-targets -- -D warnings
test:
name: test (${{ matrix.toolchain }})
@@ -94,7 +101,7 @@ jobs:
- name: cargo test
env:
DATABASE_URL: postgres://postgres:postgres@localhost:5432/gitlawb_test
run: cargo test --workspace
run: cargo test --locked --workspace
build-release:
name: build --release
@@ -117,7 +124,7 @@ jobs:
key: release
- name: cargo build --release
run: cargo build --release --workspace
run: cargo build --locked --release --workspace
audit:
name: cargo audit
@@ -199,7 +206,7 @@ jobs:
key: msrv
- name: cargo check on MSRV
run: cargo check --workspace --all-targets
run: cargo check --locked --workspace --all-targets
docker-build:
name: Docker build smoke test
@@ -246,4 +253,33 @@ jobs:
key: windows
- name: cargo test (shipped Windows crates)
run: cargo test -p gl -p git-remote-gitlawb
run: cargo test --locked -p gl -p git-remote-gitlawb
# gitlawb-core is embedded by every consumer (gl, git-remote-gitlawb, the node
# daemon), so it must stay lean. This gate fails if gitlawb-core's normal
# (non-dev, non-build) dependency tree gains any crate not on the allowlist in
# ci/gitlawb-core-allowed-deps.txt. Exhaustive-by-construction: a new heavy
# dependency reds CI whether or not anyone thought to ban it, which a denylist
# cannot do. Regen instructions live in the allowlist header.
core-deps-purity:
name: gitlawb-core dependency purity
runs-on: ubuntu-latest
timeout-minutes: 15
steps:
- name: Check out repository
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
persist-credentials: false
- name: Set up Rust toolchain
uses: dtolnay/rust-toolchain@b3b07ba8b418998c39fb20f53e8b695cdcc8de1b # stable
with:
toolchain: stable
- name: Cache cargo
uses: Swatinem/rust-cache@98c8021b550208e191a6a3145459bfc9fb29c4c0 # v2.8.0
with:
key: core-deps-purity
- name: Check gitlawb-core dependency allowlist
run: bash scripts/check-gitlawb-core-deps.sh
+139 -1
View File
@@ -41,6 +41,7 @@ jobs:
release_created: ${{ steps.release.outputs.release_created }}
tag_name: ${{ steps.release.outputs.tag_name }}
version: ${{ steps.release.outputs.version }}
pr: ${{ steps.release.outputs.pr }}
steps:
- name: Run release-please
id: release
@@ -48,6 +49,143 @@ jobs:
with:
token: ${{ secrets.GITHUB_TOKEN }}
# The release PR bumps the workspace crate versions in the Cargo.tomls but
# release-please cannot update Cargo.lock, so without this job every release
# merge ships a tag whose lockfile disagrees with its manifests (a --locked
# build from the tag fails, and pr-checks builds --locked). Sync the lock on
# the release branch so the tagged tree is internally consistent.
#
# Runs on every push-triggered release run, not only when release-please
# reports the PR: the `pr` output is set only on runs that CREATE or UPDATE
# the release PR, so gating on it would leave an already-open release PR
# (opened before this job existed, or untouched by a non-releasable push)
# permanently without the sync and without check runs. The resolve step
# discovers any open release-please PR itself and no-ops when there is none.
sync-release-lock:
name: Sync Cargo.lock on the release PR
needs: release-please
if: ${{ !cancelled() && github.repository == 'Gitlawb/node' && github.event_name != 'workflow_dispatch' }}
runs-on: ubuntu-latest
permissions:
contents: write
pull-requests: read
# actions: write lets the final step dispatch pr-checks on the release
# branch. Note this grant is repo-wide dispatch rights (GitHub cannot
# scope it to one workflow); acceptable here because reaching this job's
# steps already requires write to main. Needed because every push to the
# release branch (release-please's and the lock sync below) is made with
# GITHUB_TOKEN, which triggers no workflow runs, so without an explicit
# dispatch the release PR's head carries no check runs and the --locked
# gate never validates it.
actions: write
steps:
# Prefer the fresh `pr` output when this run just created/updated the
# release PR; otherwise fall back to querying for an open release-please
# branch (prefix match: the branch carries `--components--` suffixes).
- name: Resolve the open release PR branch
id: resolve
env:
GH_TOKEN: ${{ github.token }}
OWNER: ${{ github.repository_owner }}
BRANCH_FROM_OUTPUT: ${{ needs.release-please.outputs.pr && fromJSON(needs.release-please.outputs.pr).headBranchName || '' }}
run: |
branch="$BRANCH_FROM_OUTPUT"
if [ -z "$branch" ]; then
# The branch name alone is not a trust signal. `gh pr list` includes
# fork PRs, and a fork's headRefName is bare (no owner prefix), so a
# prefix match by itself lets any fork nominate the branch this job
# checks out, commits to, and dispatches checks on. Require the head
# repo to be THIS repository and the author to be the release-please
# app, so neither a fork nor a same-repo human branch named
# `release-please--*` can be selected. Then take the highest PR
# number: `[0]` depended on gh's default ordering, which is not a
# documented guarantee, and picking a stable one keeps a second
# (component) release PR from making the choice flap between runs.
branch=$(gh pr list --repo "$GITHUB_REPOSITORY" --state open \
--json headRefName,headRepositoryOwner,number,author \
--jq '[ .[]
| select(.headRepositoryOwner.login == env.OWNER)
| select(.author.login == "app/github-actions")
| select(.headRefName | startswith("release-please--"))
] | max_by(.number).headRefName // empty')
fi
if [ -z "$branch" ]; then
echo "no open release PR; nothing to sync"
fi
echo "branch=$branch" >> "$GITHUB_OUTPUT"
- name: Checkout release branch
if: ${{ steps.resolve.outputs.branch != '' }}
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
ref: ${{ steps.resolve.outputs.branch }}
# Resolve on a declared toolchain instead of whatever Rust the runner
# image happens to ship, so the lock this job writes comes off the same
# stable channel the other cargo jobs use. Not a claim of identical
# resolution: `stable` still moves between runs, and the MSRV gate
# checks this same lock on 1.91 (pr-checks.yml). It only removes the
# runner image as an undeclared input.
- name: Install Rust toolchain
if: ${{ steps.resolve.outputs.branch != '' }}
uses: dtolnay/rust-toolchain@b3b07ba8b418998c39fb20f53e8b695cdcc8de1b # stable
with:
toolchain: stable
- name: Sync workspace crate versions into Cargo.lock
if: ${{ steps.resolve.outputs.branch != '' }}
run: cargo update --workspace
- name: Commit and push when the lock changed
if: ${{ steps.resolve.outputs.branch != '' }}
run: |
if git diff --quiet Cargo.lock; then
echo "Cargo.lock already in sync"
exit 0
fi
git config user.name "github-actions[bot]"
git config user.email "41898282+github-actions[bot]@users.noreply.github.com"
git add Cargo.lock
git commit -m "chore: sync Cargo.lock with the release version bump"
git push
# Every push to the release branch is made with GITHUB_TOKEN (release-
# please's own pushes and the lock sync above), and events created by
# GITHUB_TOKEN start no workflow runs. workflow_dispatch is the documented
# exception GITHUB_TOKEN may trigger, so dispatch pr-checks at the release
# branch to attach check runs to its current head. Unconditional (also on
# the already-in-sync path) because the branch's ORIGINAL head has the
# same no-runs problem. Loud on failure by design: a silent fallback
# would recreate the checkless-head problem, and a job rerun is safe
# (sync no-ops, dispatch retries). Loop-safe: pr-checks is read-only and
# dispatches nothing. Runs after the push ack, so the dispatched ref tip
# is the sync commit. Merge-queue note: this validates the branch tip;
# the enforced merge queue's merge_group run still re-validates the true
# merged result before landing.
- name: Dispatch PR checks on the release branch
if: ${{ steps.resolve.outputs.branch != '' }}
env:
GH_TOKEN: ${{ github.token }}
BRANCH: ${{ steps.resolve.outputs.branch }}
run: |
# Every dispatch failure fails the job, including the 422 a release
# branch cut before pr-checks gained its workflow_dispatch trigger
# returns ("does not have a workflow_dispatch trigger", reported
# against the dispatched ref's copy of the file). An earlier revision
# tolerated exactly that 422 on the theory that release-please would
# rewrite the branch and heal it, but a rewrite only happens when a
# later releasable commit lands: an open release PR nobody touches is
# never rewritten. Tolerating it therefore left the lock-sync commit
# this job had just pushed sitting on the release head with no
# --locked validation, which is the precise state this workflow exists
# to prevent. The tolerance is also no longer needed: once this lands,
# every release branch is cut from a main that already carries the
# trigger, so the only branch that could 422 is one open at merge time
# (there is none), and a job rerun after a manual rebase is safe (the
# sync no-ops, the dispatch retries).
gh api "repos/${GITHUB_REPOSITORY}/actions/workflows/pr-checks.yml/dispatches" \
-f "ref=$BRANCH"
# Each architecture builds NATIVELY (amd64 on ubuntu-latest, arm64 on
# ubuntu-24.04-arm) and pushes by digest; docker-manifest below stitches the
# digests into the tagged multi-arch image. No QEMU: emulating the arm64
@@ -308,7 +446,7 @@ jobs:
set -euo pipefail
args=""
for b in $BINS; do args="$args -p $b"; done
cargo build --release --target ${{ matrix.target }} $args
cargo build --release --locked --target ${{ matrix.target }} $args
# Run each packaged binary's --version so a broken release artifact fails the
# build instead of shipping. Every target builds on a matching-arch runner,
+1 -1
View File
@@ -1,3 +1,3 @@
{
".": "0.6.0"
".": "0.7.1"
}
+44
View File
@@ -1,5 +1,49 @@
# Changelog
## [0.7.1](https://github.com/Gitlawb/node/compare/v0.7.0...v0.7.1) (2026-08-03)
### Bug Fixes
* **ci:** make the docker dependency cache work and unbreak Dockerfile.bins ([99b2b9a](https://github.com/Gitlawb/node/commit/99b2b9a16662761a3959a2941798e6823621bda5))
* **ci:** make the docker dependency cache work and unbreak Dockerfile.bins ([a49e04d](https://github.com/Gitlawb/node/commit/a49e04ddcd23f3b666421486ae60c3c4329e2242)), closes [#265](https://github.com/Gitlawb/node/issues/265) [#266](https://github.com/Gitlawb/node/issues/266)
* **db:** pin profile DID matching with lone non-key fixture and boundary matrix ([b4016dc](https://github.com/Gitlawb/node/commit/b4016dce9d1fe450603173f9d92bd880127620fa)), closes [#298](https://github.com/Gitlawb/node/issues/298)
* **node:** add a shared owner/name slug validator ([#272](https://github.com/Gitlawb/node/issues/272)) ([51d82de](https://github.com/Gitlawb/node/commit/51d82ded57a776d9c7e2fe65895efdbf9e227dc8))
* **node:** bound the slug owner by NAME_MAX, not the DID limit ([#272](https://github.com/Gitlawb/node/issues/272)) ([27b0495](https://github.com/Gitlawb/node/commit/27b0495a6f7edeecddbe62b0f075a7e3b1ef5053))
* **node:** fail a sync row terminally when a non-directory holds the owner path ([2e664b7](https://github.com/Gitlawb/node/commit/2e664b7885036ee72ce175af95bb567b18222583))
* **node:** make a stalled sync queue visible, and fix two test claims ([9aa4c17](https://github.com/Gitlawb/node/commit/9aa4c17aefd1885d134b1334b54551366cdd8ef0))
* **node:** move the attempted_at migration to v17, clear of two open branches ([fbc5868](https://github.com/Gitlawb/node/commit/fbc5868bd16946ea184dad26abcdb0e89d68e99a))
* **node:** prove the mirror path is inside repos_dir before running git ([#272](https://github.com/Gitlawb/node/issues/272)) ([3ca9473](https://github.com/Gitlawb/node/commit/3ca94739ed7b90e4c265a0947025dc3b50e7583b))
* **node:** reject a malformed slug at the notify boundary ([#272](https://github.com/Gitlawb/node/issues/272)) ([c587a40](https://github.com/Gitlawb/node/commit/c587a40a508acea9f74c3468d42ff037397c75d0))
* **node:** reset a peer's reachability gate when an announce changes its http_url ([#270](https://github.com/Gitlawb/node/issues/270)) ([1964f88](https://github.com/Gitlawb/node/commit/1964f88378fe6942ffca5836de3a092b5b8f67b4))
* **node:** reset a peer's reachability gate when an announce changes its http_url ([#270](https://github.com/Gitlawb/node/issues/270)) ([e2560ee](https://github.com/Gitlawb/node/commit/e2560ee46c28e4e966d87867a86dc4bad4ca7a3a))
* **node:** stamp every dequeued sync row so a stuck one yields the batch ([6e64d0d](https://github.com/Gitlawb/node/commit/6e64d0d0a7fdd38ef3a56820c1808dd7c9bbd658))
* **node:** stop a peer-supplied repo slug from escaping repos_dir ([#272](https://github.com/Gitlawb/node/issues/272)) ([c70b1df](https://github.com/Gitlawb/node/commit/c70b1df42d5c3f70ec4c272886c85fcc16ed92d1))
* **node:** validate the queued slug before the worker derives a path ([#272](https://github.com/Gitlawb/node/issues/272)) ([9f00e24](https://github.com/Gitlawb/node/commit/9f00e24aec217e4b8256dd437cff4741b1478b96))
* **review:** correct two comment claims and guard two more wrong shapes ([#270](https://github.com/Gitlawb/node/issues/270)) ([811ad50](https://github.com/Gitlawb/node/commit/811ad504bc268c8e9cddac93822adcd00bfe9ff8))
* **review:** stop the dequeue overclaiming, and pin the starvation premise ([457be11](https://github.com/Gitlawb/node/commit/457be11d728a2a34775cf96be2bc5af9a58cb250))
* strict DID matching in get_profile ([acb4714](https://github.com/Gitlawb/node/commit/acb47145beec8ce7b20b7a2479f7a55ed68e2a89))
* strict DID matching in get_profile ([13b565b](https://github.com/Gitlawb/node/commit/13b565bcac09f075a7fd555a6e9acdd27de6c7bb))
* target stored did on profile update aliases ([9836c3a](https://github.com/Gitlawb/node/commit/9836c3a21bc96a2ce95cb1fd12bd84b22142787d))
## [0.7.0](https://github.com/Gitlawb/node/compare/v0.6.0...v0.7.0) (2026-07-22)
### Features
* **gl:** actually verify certificate signatures in gl cert show ([b0a2ed8](https://github.com/Gitlawb/node/commit/b0a2ed830821b5a77b0dafdcf5ce4b1ad904ec25))
* **gl:** doctor warns when a shell alias shadows the gl binary ([0765c42](https://github.com/Gitlawb/node/commit/0765c42feff47b29054b9a277881a7cd5bfd9135))
### Bug Fixes
* **gl:** address review findings — non-fatal DID check, exact-host loopback, detached HEAD, multi-URL remotes, command-aware unalias ([3fa38dc](https://github.com/Gitlawb/node/commit/3fa38dce4e435ef017f607d355266c2892161f8d))
* **gl:** anchor cert --verify to a trusted issuer; pin canonical payload form ([c681af7](https://github.com/Gitlawb/node/commit/c681af70dba3a76b11795d324f2ca0fecf7e42fd))
* **gl:** detect gitlawb remotes beyond origin in gl status ([2d88937](https://github.com/Gitlawb/node/commit/2d88937fbabf21013f28debb4d4f93270a4bb8cd))
* **gl:** doctor treats a reachable local node as configuration, not failure ([94c61f7](https://github.com/Gitlawb/node/commit/94c61f79421fa26ecc1bdf7c255e0c48e66a2048))
* **gl:** make gl init's push guidance match the repo's actual state ([cb42a48](https://github.com/Gitlawb/node/commit/cb42a4830d0d070165e2212f316d009ec716dbee))
* warn about the oh-my-zsh gl alias at install time ([e3df51d](https://github.com/Gitlawb/node/commit/e3df51d76c0275fde33954ff0af0aab95fd0cb7d))
## [0.6.0](https://github.com/Gitlawb/node/compare/v0.5.1...v0.6.0) (2026-07-22)
Generated
+86 -12
View File
@@ -770,6 +770,23 @@ dependencies = [
"zeroize",
]
[[package]]
name = "ark-ff"
version = "0.6.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f7a806ac6c8307b929df4645776290a50ee2aac754ad09d8bdf73391309e43af"
dependencies = [
"ark-ff-asm 0.6.0",
"ark-ff-macros 0.6.0",
"ark-serialize 0.6.0",
"ark-std 0.6.0",
"digest 0.10.7",
"educe",
"num-bigint",
"num-traits",
"zeroize",
]
[[package]]
name = "ark-ff-asm"
version = "0.3.0"
@@ -800,6 +817,16 @@ dependencies = [
"syn 2.0.117",
]
[[package]]
name = "ark-ff-asm"
version = "0.6.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1479009684adc073dff49a1025d3a7065b317a9ead25aaaca38cdc70058ba8a2"
dependencies = [
"quote",
"syn 2.0.117",
]
[[package]]
name = "ark-ff-macros"
version = "0.3.0"
@@ -838,6 +865,19 @@ dependencies = [
"syn 2.0.117",
]
[[package]]
name = "ark-ff-macros"
version = "0.6.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "4a0691ed21ef00ef89c1e9bda832eba493dda3ec2f8d892fb25b705f73f06bb8"
dependencies = [
"num-bigint",
"num-traits",
"proc-macro2",
"quote",
"syn 2.0.117",
]
[[package]]
name = "ark-serialize"
version = "0.3.0"
@@ -871,6 +911,30 @@ dependencies = [
"num-bigint",
]
[[package]]
name = "ark-serialize"
version = "0.6.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a74dd304fd536fb95d0a328e72be759209cc496a9da094c5bc56e5fea4f9e86b"
dependencies = [
"ark-serialize-derive",
"ark-std 0.6.0",
"digest 0.10.7",
"num-bigint",
"serde_with",
]
[[package]]
name = "ark-serialize-derive"
version = "0.6.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "4f153690697a2b91e5e1251ff98411ee5371500a111a0fd317a70e588eb300f9"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.117",
]
[[package]]
name = "ark-std"
version = "0.3.0"
@@ -901,6 +965,16 @@ dependencies = [
"rand 0.8.6",
]
[[package]]
name = "ark-std"
version = "0.6.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "367c9c827ed431bff6868b7aa926e05b16eb46603cc8b6e768e4a5553fa1d155"
dependencies = [
"num-traits",
"rand 0.8.6",
]
[[package]]
name = "arrayref"
version = "0.3.9"
@@ -2556,7 +2630,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7ab67060fc6b8ef687992d439ca0fa36e7ed17e9a0b16b25b601e8757df720de"
dependencies = [
"data-encoding",
"syn 1.0.109",
"syn 2.0.117",
]
[[package]]
@@ -2928,7 +3002,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "39cab71617ae0d63f51a36d69f866391735b51691dbda63cf6f96d042b63efeb"
dependencies = [
"libc",
"windows-sys 0.52.0",
"windows-sys 0.61.2",
]
[[package]]
@@ -3300,7 +3374,7 @@ dependencies = [
[[package]]
name = "git-remote-gitlawb"
version = "0.6.0"
version = "0.7.1"
dependencies = [
"anyhow",
"gitlawb-core",
@@ -3312,7 +3386,7 @@ dependencies = [
[[package]]
name = "gitlawb-attest"
version = "0.6.0"
version = "0.7.1"
dependencies = [
"base64",
"ed25519-dalek",
@@ -3329,7 +3403,7 @@ dependencies = [
[[package]]
name = "gitlawb-core"
version = "0.6.0"
version = "0.7.1"
dependencies = [
"anyhow",
"base64",
@@ -3356,14 +3430,13 @@ dependencies = [
[[package]]
name = "gitlawb-node"
version = "0.6.0"
version = "0.7.1"
dependencies = [
"alloy",
"anyhow",
"async-compression",
"async-graphql",
"async-graphql-axum",
"async-trait",
"aws-config",
"aws-sdk-s3",
"axum",
@@ -3413,7 +3486,7 @@ dependencies = [
[[package]]
name = "gl"
version = "0.6.0"
version = "0.7.1"
dependencies = [
"alloy",
"anyhow",
@@ -5742,14 +5815,15 @@ dependencies = [
[[package]]
name = "ruint"
version = "1.17.2"
version = "1.20.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c141e807189ad38a07276942c6623032d3753c8859c146104ac2e4d68865945a"
checksum = "f5e99bff0393163bb25029a6af25d3d8d202ba5b5438a74d1bd8789f5c822970"
dependencies = [
"alloy-rlp",
"ark-ff 0.3.0",
"ark-ff 0.4.2",
"ark-ff 0.5.0",
"ark-ff 0.6.0",
"bytes",
"fastrlp 0.3.1",
"fastrlp 0.4.0",
@@ -5823,7 +5897,7 @@ dependencies = [
"errno",
"libc",
"linux-raw-sys",
"windows-sys 0.52.0",
"windows-sys 0.61.2",
]
[[package]]
@@ -6717,7 +6791,7 @@ dependencies = [
"getrandom 0.4.2",
"once_cell",
"rustix",
"windows-sys 0.52.0",
"windows-sys 0.61.2",
]
[[package]]
+18 -5
View File
@@ -8,20 +8,33 @@ FROM rust:1.91-bookworm AS builder
WORKDIR /build
# Cache dependencies first for faster rebuilds
# Cache dependencies first for faster rebuilds.
# EVERY workspace member's manifest must be copied here, not just the ones this
# image ships: cargo loads the whole workspace before it resolves anything, so a
# missing member's Cargo.toml (or a path dependency pointing at one) aborts the
# layer before a single crate is fetched.
COPY Cargo.toml Cargo.lock ./
COPY crates/gitlawb-core/Cargo.toml crates/gitlawb-core/
COPY crates/gitlawb-node/Cargo.toml crates/gitlawb-node/
COPY crates/gl/Cargo.toml crates/gl/
COPY crates/git-remote-gitlawb/Cargo.toml crates/git-remote-gitlawb/
COPY crates/gitlawb-attest/Cargo.toml crates/gitlawb-attest/
COPY crates/icaptcha-client/Cargo.toml crates/icaptcha-client/
# Fetch deps (this layer caches until Cargo.{toml,lock} change)
RUN mkdir -p crates/gitlawb-core/src crates/gitlawb-node/src crates/gl/src crates/git-remote-gitlawb/src && \
# Fetch deps (this layer caches until Cargo.{toml,lock} change).
# No `|| true`: this layer is expected to succeed, and swallowing its exit code
# is what let it sit dead from the moment gl gained a path dependency on
# icaptcha-client. A failure here should stop the build, not silently turn every
# image build into a cold one.
RUN mkdir -p crates/gitlawb-core/src crates/gitlawb-node/src crates/gl/src \
crates/git-remote-gitlawb/src crates/gitlawb-attest/src crates/icaptcha-client/src && \
echo 'fn main() {}' > crates/gitlawb-node/src/main.rs && \
echo 'fn main() {}' > crates/gl/src/main.rs && \
echo 'fn main() {}' > crates/git-remote-gitlawb/src/main.rs && \
echo '' > crates/gitlawb-core/src/lib.rs && \
cargo build --release -p gitlawb-node -p gl -p git-remote-gitlawb || true
echo '' > crates/gitlawb-attest/src/lib.rs && \
echo '' > crates/icaptcha-client/src/lib.rs && \
cargo build --release --locked -p gitlawb-node -p gl -p git-remote-gitlawb
# Now copy real sources and build for real.
# Force-bump mtimes so cargo's fingerprint check rebuilds — without this,
@@ -34,7 +47,7 @@ RUN find crates -name "*.rs" -exec touch {} + && \
rm -rf target/release/.fingerprint/gitlawb-node-* \
target/release/.fingerprint/gl-* \
target/release/.fingerprint/git-remote-gitlawb-* && \
cargo build --release -p gitlawb-node -p gl -p git-remote-gitlawb && \
cargo build --release --locked -p gitlawb-node -p gl -p git-remote-gitlawb && \
strip target/release/gitlawb-node target/release/gl target/release/git-remote-gitlawb
# ── Runtime stage ───────────────────────────────────────────────────────────
+5 -2
View File
@@ -3,7 +3,10 @@
# Used by scripts/build-bins.sh — not used for node deployment.
ARG TARGET=x86_64-unknown-linux-musl
FROM rust:1.85-bookworm AS builder
# Must stay at or above the workspace `rust-version` in Cargo.toml. An unmet
# MSRV is a hard cargo error, and nothing in CI builds this file, so a stale
# base image here fails only when someone runs scripts/build-bins.sh.
FROM rust:1.91-bookworm AS builder
ARG TARGET
WORKDIR /build
@@ -23,7 +26,7 @@ COPY crates/ crates/
# Configure cross-linker for aarch64
ENV CARGO_TARGET_AARCH64_UNKNOWN_LINUX_MUSL_LINKER=aarch64-linux-gnu-gcc
RUN cargo build --release --target "$TARGET" -p gl -p git-remote-gitlawb
RUN cargo build --release --locked --target "$TARGET" -p gl -p git-remote-gitlawb
FROM debian:bookworm-slim
ARG TARGET
+112
View File
@@ -0,0 +1,112 @@
# Dependency allowlist for the `gitlawb-core` crate.
#
# gitlawb-core is the shared-primitives crate every consumer embeds (the `gl`
# CLI, git-remote-gitlawb, the node daemon). It must stay lean so those
# consumers do not inherit daemon-weight dependencies. This file is the source
# of truth for the crates gitlawb-core's NORMAL (non-dev, non-build) transitive
# dependency closure is allowed to contain. CI (`scripts/check-gitlawb-core-deps.sh`,
# wired into the `core-deps-purity` job in .github/workflows/pr-checks.yml)
# fails if gitlawb-core's tree contains any crate not listed here.
#
# The allowlist is exhaustive-by-construction: a NEW dependency reds CI whether
# or not anyone thought to ban it, which a denylist cannot do.
#
# Measured in-workspace, so feature unification across the workspace can, in
# rare cases, surface a crate here that gitlawb-core would not pull standalone.
# An unexpected new entry is a prompt to check what pulled it in, not noise to
# rubber-stamp.
#
# Regenerate (from repo root, after an INTENTIONAL dependency change):
# cargo tree -p gitlawb-core --edges normal --prefix none \
# | sed -E 's/ v[0-9].*$//' \
# | grep -v '^gitlawb-core$' \
# | sort -u
# then paste the result below this header block. (The checker snapshots and
# restores Cargo.lock; a manual regen may refresh it — `git checkout Cargo.lock`
# afterward if you did not intend that.)
#
# Lines starting with `#` and blank lines are ignored by the checker.
aead
anyhow
base-x
base256emoji
base64
base64ct
block-buffer
cfg-if
chacha20
chacha20poly1305
chrono
cid
cipher
const-oid
const-str
core2
cpufeatures
crypto-common
crypto_box
crypto_secretbox
curve25519-dalek
curve25519-dalek-derive
data-encoding
data-encoding-macro
data-encoding-macro-internal
der
digest
ed25519
ed25519-dalek
equivalent
generic-array
getrandom
hashbrown
hex
iana-time-zone
indexmap
inout
itoa
libc
match-lookup
memchr
multibase
multihash
multihash-codetable
multihash-derive
multihash-derive-impl
num-traits
opaque-debug
pem-rfc7468
pkcs8
poly1305
ppv-lite86
proc-macro-crate
proc-macro2
quote
rand
rand_chacha
rand_core
salsa20
serde
serde_core
serde_derive
serde_json
sha2
signature
spki
subtle
syn
synstructure
thiserror
thiserror-impl
toml_datetime
toml_edit
toml_parser
typenum
unicode-ident
universal-hash
unsigned-varint
uuid
winnow
zerocopy
zeroize
zeroize_derive
zmij
+1 -1
View File
@@ -1,7 +1,7 @@
[package]
name = "git-remote-gitlawb"
description = "Git remote helper — enables 'git clone gitlawb://did:gitlawb:...'"
version = "0.6.0" # x-release-please-version
version = "0.7.1" # x-release-please-version
edition.workspace = true
rust-version.workspace = true
license.workspace = true
+1 -1
View File
@@ -1,7 +1,7 @@
[package]
name = "gitlawb-attest"
description = "External Attestation v1: pluggable provenance attachments for gitlawb ref-update certs"
version = "0.6.0" # x-release-please-version
version = "0.7.1" # x-release-please-version
edition.workspace = true
rust-version.workspace = true
license.workspace = true
+1 -1
View File
@@ -1,7 +1,7 @@
[package]
name = "gitlawb-core"
description = "Core cryptographic primitives for the gitlawb network: DIDs, CIDs, UCAN, HTTP Signatures, ref-update certificates"
version = "0.6.0" # x-release-please-version
version = "0.7.1" # x-release-please-version
edition.workspace = true
rust-version.workspace = true
license.workspace = true
+1 -2
View File
@@ -1,7 +1,7 @@
[package]
name = "gitlawb-node"
description = "The gitlawb node daemon — git hosting over HTTP with DID auth"
version = "0.6.0" # x-release-please-version
version = "0.7.1" # x-release-please-version
edition.workspace = true
rust-version.workspace = true
license.workspace = true
@@ -12,7 +12,6 @@ path = "src/main.rs"
[dependencies]
gitlawb-core = { path = "../gitlawb-core" }
async-trait = "0.1"
ed25519-dalek = { workspace = true }
base64 = { workspace = true }
tokio = { workspace = true }
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
+7 -490
View File
@@ -70,37 +70,11 @@ pub async fn create_issue(
let create_result = git_issues::create_issue(&disk_path, &issue_id, &json_str);
// Always release the advisory lock, even on error; upload to Tigris only on
// success. A false return means the durable upload failed/timed out: the
// issue ref applied to local disk but never reached object storage, so the
// next acquire_write would revert it from the stale archive. Report 5xx so
// the client retries rather than trusting an issue that never landed durably,
// and skip the trust bump below. (Same P1 data-loss guard as receive-pack.)
// On that failure, roll back the local ref BEFORE the lock releases — an
// orphan ref would make the retry a duplicate, and in an unlock-to-delete
// window a concurrent same-repo write could upload an archive still
// carrying it. Best-effort: a failed delete logs and the 5xx still returns.
let upload_ok = guard
.release_with_failure_cleanup(create_result.is_ok(), |path| {
if let Err(e) = git_issues::delete_issue_ref(path, &issue_id) {
tracing::warn!(issue = %issue_id, err = %e,
"failed to roll back local issue ref after failed durable upload; \
a retry may duplicate this issue");
}
})
.await;
// Always release the advisory lock — even on error; upload to Tigris only on success.
guard.release(create_result.is_ok()).await;
create_result.map_err(|e| AppError::Git(e.to_string()))?;
if !upload_ok {
tracing::error!(repo = %record.name,
"durable upload failed after issue create; reporting failure so the client retries");
return Err(AppError::Git(format!(
"durable storage upload failed for {}",
record.name
)));
}
// Bump trust score for the issue author — increment current score by 0.05
// (avoids the push_count=0 stuck-at-0.05 bug for agents who only file issues)
if let Some(ref author_did) = issue.author {
@@ -270,14 +244,11 @@ pub async fn close_issue(
.ok()
.and_then(|i| i.author),
Ok(None) => {
// release(false): no upload attempted, so the bool is always true
// (nothing to fail on). Ignore it on this pre-write error path.
let _ = guard.release(false).await;
guard.release(false).await;
return Err(AppError::NotFound(format!("issue {issue_id} not found")));
}
Err(e) => {
// release(false): no upload attempted, nothing to fail on.
let _ = guard.release(false).await;
guard.release(false).await;
return Err(AppError::Git(e.to_string()));
}
};
@@ -286,57 +257,21 @@ pub async fn close_issue(
.as_deref()
.is_some_and(|a| crate::api::did_matches(&auth.0, a));
if !is_owner && !is_author {
// release(false): no upload attempted, nothing to fail on.
let _ = guard.release(false).await;
guard.release(false).await;
return Err(AppError::Forbidden(
"only the repo owner or the issue author can close this issue".into(),
));
}
// Snapshot the issue ref's pre-close OID so a failed durable upload can roll
// the close back on local disk, mirroring create_issue. Best-effort: if the
// snapshot read fails we proceed without a rollback closure (the close still
// applies and a failed upload still surfaces as a 5xx below).
let prior_ref = git_issues::issue_ref_oid(&disk_path, &issue_id)
.ok()
.flatten();
let close_result = git_issues::close_issue(&disk_path, &issue_id);
// Always release the advisory lock, even on error; upload to Tigris only on
// success. A false return means the durable upload failed: the close applied
// to local disk but never reached object storage, so a later acquire_write
// reverts it. Report 5xx so the client retries. (P1 data-loss guard.)
// On that failure, roll the local close BACK to the captured open ref BEFORE
// the lock releases — otherwise the local fast path in RepoStore::acquire
// serves the "closed" state until a later write refreshes the archive, even
// though the client got a 5xx. The cleanup runs only when an upload was
// attempted and failed, while the advisory lock is still held.
let upload_ok = guard
.release_with_failure_cleanup(close_result.is_ok(), |path| {
if let Some((full_id, oid)) = &prior_ref {
if let Err(e) = git_issues::restore_issue_ref(path, full_id, oid) {
tracing::warn!(issue = %issue_id, err = %e,
"failed to roll back local issue close after failed durable upload; \
a local-fast-path read may serve a stale closed state");
}
}
})
.await;
// Always release the advisory lock — even on error; upload to Tigris only on success.
guard.release(close_result.is_ok()).await;
let updated = close_result
.map_err(|e| AppError::Git(e.to_string()))?
.ok_or_else(|| AppError::RepoNotFound(format!("issue {issue_id} not found")))?;
if !upload_ok {
tracing::error!(repo = %repo,
"durable upload failed after issue close; reporting failure so the client retries");
return Err(AppError::Git(format!(
"durable storage upload failed for {}",
record.name
)));
}
let issue: serde_json::Value = serde_json::from_str(&updated)
.map_err(|e| AppError::BadRequest(format!("invalid issue data: {e}")))?;
@@ -344,421 +279,3 @@ pub async fn close_issue(
Ok(Json(issue))
}
#[cfg(test)]
mod tests {
use super::*;
use axum::extract::{Path as AxPath, State};
use axum::Extension;
use std::path::Path as StdPath;
use std::time::Duration;
// Object store whose upload() parks forever, so release()'s bounded timeout
// is the only thing that completes it, modeling a stalled durable PUT. exists()
// is false so acquire_write never downloads over the fresh local bare repo.
struct StallStore;
#[async_trait::async_trait]
impl crate::git::tigris::ObjectStore for StallStore {
async fn exists(&self, _o: &str, _r: &str) -> anyhow::Result<bool> {
Ok(false)
}
async fn upload(&self, _o: &str, _r: &str, _p: &StdPath) -> anyhow::Result<()> {
std::future::pending::<()>().await;
Ok(())
}
async fn download(&self, _o: &str, _r: &str, _p: &StdPath) -> anyhow::Result<()> {
Ok(())
}
async fn delete(&self, _o: &str, _r: &str) -> anyhow::Result<()> {
Ok(())
}
}
// Object store whose upload succeeds immediately. exists() is false so
// acquire_write never performs a reverting refresh over the local dir —
// the retry test below pins that contract explicitly.
struct OkStore;
#[async_trait::async_trait]
impl crate::git::tigris::ObjectStore for OkStore {
async fn exists(&self, _o: &str, _r: &str) -> anyhow::Result<bool> {
Ok(false)
}
async fn upload(&self, _o: &str, _r: &str, _p: &StdPath) -> anyhow::Result<()> {
Ok(())
}
async fn download(&self, _o: &str, _r: &str, _p: &StdPath) -> anyhow::Result<()> {
Ok(())
}
async fn delete(&self, _o: &str, _r: &str) -> anyhow::Result<()> {
Ok(())
}
}
// Shared setup for the failed-upload rollback tests: a state whose repo
// store stalls uploads (200ms release timeout), a public repo row for
// `owner/name`, and an initialized bare repo at the returned path.
async fn stall_state_with_repo(
pool: sqlx::PgPool,
repos_dir: &tempfile::TempDir,
owner: &str,
name: &str,
) -> (crate::state::AppState, std::path::PathBuf) {
let mut state = crate::test_support::test_state(pool.clone()).await;
state.repo_store = crate::git::repo_store::RepoStore::new(
repos_dir.path().to_path_buf(),
Some(std::sync::Arc::new(StallStore)),
pool.clone(),
)
.with_release_upload_timeout(Duration::from_millis(200));
let owner_slug = owner.replace([':', '/'], "_");
let bare = repos_dir
.path()
.join(&owner_slug)
.join(format!("{name}.git"));
state
.db
.upsert_mirror_repo(owner, name, &bare.to_string_lossy(), None, false)
.await
.unwrap();
std::fs::create_dir_all(&bare).unwrap();
let out = std::process::Command::new("git")
.args(["init", "--bare", "-q", &bare.to_string_lossy()])
.output()
.unwrap();
assert!(out.status.success(), "git init --bare failed");
(state, bare)
}
async fn create_issue_status(
state: &crate::state::AppState,
owner: &str,
name: &str,
) -> axum::http::StatusCode {
use axum::response::IntoResponse;
let resp = super::create_issue(
State(state.clone()),
Extension(crate::auth::AuthenticatedDid(owner.to_string())),
AxPath((owner.to_string(), name.to_string())),
axum::Json(CreateIssueRequest {
title: "t".into(),
body: None,
signed_payload: None,
}),
)
.await;
match resp {
Ok((code, _)) => code,
Err(e) => e.into_response().status(),
}
}
async fn close_issue_status(
state: &crate::state::AppState,
owner: &str,
name: &str,
issue_id: &str,
) -> axum::http::StatusCode {
use axum::response::IntoResponse;
let resp = super::close_issue(
State(state.clone()),
Extension(crate::auth::AuthenticatedDid(owner.to_string())),
AxPath((owner.to_string(), name.to_string(), issue_id.to_string())),
)
.await;
match resp {
Ok(_) => axum::http::StatusCode::OK,
Err(e) => e.into_response().status(),
}
}
// Seed an OPEN issue directly on local disk, bypassing the (stalling) upload
// path so the close test starts from a durably-consistent open issue.
fn seed_open_issue(bare: &StdPath, owner: &str, issue_id: &str) {
let issue = IssueRecord {
id: issue_id.to_string(),
title: "t".into(),
body: None,
author: Some(owner.to_string()),
created_at: Utc::now().to_rfc3339(),
status: "open".to_string(),
signed_payload: None,
};
let json = serde_json::to_string(&issue).unwrap();
git_issues::create_issue(bare, issue_id, &json).unwrap();
}
fn issue_status(bare: &StdPath, issue_id: &str) -> String {
let raw = git_issues::get_issue(bare, issue_id).unwrap().unwrap();
let v: serde_json::Value = serde_json::from_str(&raw).unwrap();
v["status"].as_str().unwrap().to_string()
}
// U2 [F2, P1] — the failed-upload rollback on the close path. close_issue
// applies the "closed" ref to LOCAL disk, then release() uploads to durable
// storage. When that upload times out the local close is NOT persisted, so a
// later acquire_write reverts it from the stale archive — but until then, a
// local-fast-path read via RepoStore::acquire serves the stale "closed"
// state even though the client got a 5xx. The fix rolls the close back to the
// captured open ref while the advisory lock is held. RED with the cleanup
// closure reverted to `|_| {}`: the read sees "closed".
#[sqlx::test]
async fn issue_close_failed_upload_rolls_back_to_open(pool: sqlx::PgPool) {
let repos_dir = tempfile::TempDir::new().unwrap();
let owner = "z6issuecloserollbackowner";
let name = "issuecloserollback";
let (state, bare) = stall_state_with_repo(pool, &repos_dir, owner, name).await;
let issue_id = "aaaa1111-0000-0000-0000-000000000000";
seed_open_issue(&bare, owner, issue_id);
assert_eq!(issue_status(&bare, issue_id), "open", "seed must be open");
// Close via the handler — the durable upload stalls and times out (5xx).
let status = close_issue_status(&state, owner, name, issue_id).await;
assert!(
status.is_server_error(),
"a stalled durable upload on issue-close must 500, got {status}"
);
// The local fast path must still see the issue OPEN: the failed upload
// rolled the close back while the lock was held.
assert_eq!(
issue_status(&bare, issue_id),
"open",
"a failed durable upload must roll the local close back to open; a \
local-fast-path read served the stale closed state"
);
}
// Success path: a working store closes the issue (2xx) and the read reflects
// the close. Confirms the rollback wiring does not fire on success.
#[sqlx::test]
async fn issue_close_success_persists_close(pool: sqlx::PgPool) {
let repos_dir = tempfile::TempDir::new().unwrap();
let owner = "z6issueclosesuccessowner";
let name = "issueclosesuccess";
let (mut state, bare) = stall_state_with_repo(pool.clone(), &repos_dir, owner, name).await;
// Swap the stalling store for one whose upload succeeds immediately.
state.repo_store = crate::git::repo_store::RepoStore::new(
repos_dir.path().to_path_buf(),
Some(std::sync::Arc::new(OkStore)),
pool.clone(),
)
.with_release_upload_timeout(Duration::from_millis(200));
let issue_id = "bbbb2222-0000-0000-0000-000000000000";
seed_open_issue(&bare, owner, issue_id);
let status = close_issue_status(&state, owner, name, issue_id).await;
assert_eq!(status, StatusCode::OK, "working store must close (200)");
assert_eq!(
issue_status(&bare, issue_id),
"closed",
"a successful close must persist as closed"
);
}
// Pre-write error path: closing a nonexistent issue returns 404 and never
// attempts an upload (release(false)), so the stalling store cannot hang it.
#[sqlx::test]
async fn issue_close_not_found_releases_without_upload(pool: sqlx::PgPool) {
let repos_dir = tempfile::TempDir::new().unwrap();
let owner = "z6issueclosemissingowner";
let name = "issueclosemissing";
let (state, _bare) = stall_state_with_repo(pool, &repos_dir, owner, name).await;
let status =
close_issue_status(&state, owner, name, "cccc3333-0000-0000-0000-000000000000").await;
assert_eq!(
status,
StatusCode::NOT_FOUND,
"closing a missing issue must 404 without attempting an upload"
);
}
fn issue_refs(bare: &StdPath) -> Vec<String> {
let out = std::process::Command::new("git")
.args([
"for-each-ref",
"--format=%(refname)",
"refs/gitlawb/issues/",
])
.current_dir(bare)
.output()
.unwrap();
assert!(out.status.success(), "git for-each-ref failed");
String::from_utf8_lossy(&out.stdout)
.lines()
.filter(|l| !l.trim().is_empty())
.map(|l| l.trim().to_string())
.collect()
}
// Rollback half of the #196 round-four issue-create fix. When the durable
// upload fails/times out, the handler 500s (covered below) — but the issue
// ref it wrote to LOCAL disk must also be rolled back, while the advisory
// lock is still held. An orphan ref makes the 500 dishonest: a retry
// re-creates the issue under a new uuid while the orphan is still listed,
// and a concurrent same-repo write can upload an archive carrying it.
// RED pre-fix: the orphan ref survives the failed create.
#[sqlx::test]
async fn issue_create_failed_upload_rolls_back_local_ref(pool: sqlx::PgPool) {
let repos_dir = tempfile::TempDir::new().unwrap();
let owner = "z6issuerollbackowner";
let name = "issuerollback";
let (state, bare) = stall_state_with_repo(pool, &repos_dir, owner, name).await;
let status = create_issue_status(&state, owner, name).await;
assert!(
status.is_server_error(),
"stalled upload must 500, got {status}"
);
let refs = issue_refs(&bare);
assert!(
refs.is_empty(),
"a failed durable upload must roll back the local issue ref while \
the advisory lock is held; orphan ref(s) survived: {refs:?}"
);
}
// Retry half: after the failed-upload 500, a retry with a WORKING store
// must yield exactly one issue. Pre-fix the orphan ref from the failed
// attempt makes the retry a duplicate (two issues in list_issues).
#[sqlx::test]
async fn issue_create_failed_upload_retry_does_not_duplicate(pool: sqlx::PgPool) {
use crate::git::tigris::ObjectStore as _;
let repos_dir = tempfile::TempDir::new().unwrap();
let owner = "z6issueretryowner";
let name = "issueretry";
let (state, bare) = stall_state_with_repo(pool.clone(), &repos_dir, owner, name).await;
let status = create_issue_status(&state, owner, name).await;
assert!(
status.is_server_error(),
"stalled upload must 500, got {status}"
);
// PIN the double's contract: exists() stays false after the failed
// upload, so the retry's acquire_write performs no reverting refresh
// from a stale archive. Without this pin, a download could revert the
// orphan ref and the assertion below would pass without the rollback
// fix — it must rest on the rollback, not a stale-archive revert.
let owner_slug = owner.replace([':', '/'], "_");
assert!(
!OkStore.exists(&owner_slug, name).await.unwrap(),
"test double contract broken: exists() must stay false"
);
// Retry with a store whose upload succeeds.
let mut retry_state = state.clone();
retry_state.repo_store = crate::git::repo_store::RepoStore::new(
repos_dir.path().to_path_buf(),
Some(std::sync::Arc::new(OkStore)),
pool.clone(),
)
.with_release_upload_timeout(Duration::from_millis(200));
let status = create_issue_status(&retry_state, owner, name).await;
assert_eq!(
status,
StatusCode::CREATED,
"retry with working store must 201"
);
let issues = crate::git::issues::list_issues(&bare).unwrap();
assert_eq!(
issues.len(),
1,
"retry after a failed-upload 500 must yield exactly ONE issue; the \
failed attempt's orphan ref duplicated it (got {} issues)",
issues.len()
);
}
// P1-class data-loss regression, issue-create path. create_issue writes the
// issue ref to local disk, then release(true) uploads the repo to durable
// storage. When that upload times out the local write is NOT persisted, so
// the next acquire_write reverts it from the stale archive. The pre-fix
// handler ignored release()'s bool and still returned 201 AND bumped the
// author's trust score, so the client trusted an issue that never landed
// durably. The fix surfaces a failed/timed-out upload as a 5xx and skips the
// success tail (the trust bump). Stall the upload past a tiny release timeout
// and assert (a) the handler returns 5xx and (b) the trust bump did not run.
// RED pre-fix: 201 + trust score bumped to 0.05.
#[sqlx::test]
async fn issue_create_durable_upload_timeout_fails_and_skips_side_effects(pool: sqlx::PgPool) {
use axum::response::IntoResponse;
let mut state = crate::test_support::test_state(pool.clone()).await;
let repos_dir = tempfile::TempDir::new().unwrap();
state.repo_store = crate::git::repo_store::RepoStore::new(
repos_dir.path().to_path_buf(),
Some(std::sync::Arc::new(StallStore)),
pool.clone(),
)
.with_release_upload_timeout(Duration::from_millis(200));
let owner = "z6issueownerdurablefail";
let name = "issuedurablefail";
let owner_slug = owner.replace([':', '/'], "_");
let bare = repos_dir
.path()
.join(&owner_slug)
.join(format!("{name}.git"));
// Public mirror repo so the owner clears authorize_repo_read, and a real
// agent row so the post-write trust bump is an observable durable side
// effect (register_agent seeds trust_score = 0.0).
state
.db
.upsert_mirror_repo(owner, name, &bare.to_string_lossy(), None, false)
.await
.unwrap();
state.db.register_agent(owner, &[]).await.unwrap();
std::fs::create_dir_all(&bare).unwrap();
let out = std::process::Command::new("git")
.args(["init", "--bare", "-q", &bare.to_string_lossy()])
.output()
.unwrap();
assert!(out.status.success(), "git init --bare failed");
// acquire_write → git_issues::create_issue (writes the ref) → release
// (upload stalls, times out after 200ms). The client MUST see a failure.
let resp = super::create_issue(
State(state.clone()),
Extension(crate::auth::AuthenticatedDid(owner.to_string())),
AxPath((owner.to_string(), name.to_string())),
axum::Json(CreateIssueRequest {
title: "t".into(),
body: None,
signed_payload: None,
}),
)
.await;
let status = match resp {
Ok((code, _)) => code,
Err(e) => e.into_response().status(),
};
assert!(
status.is_server_error(),
"a timed-out durable upload on issue-create must fail (5xx) so the \
client retries (got {status}), which the client trusts as a landed \
issue that a later acquire_write would silently revert"
);
// The success tail (trust bump) must NOT have run: the issue was never
// durably accepted. register_agent seeded 0.0; a bump would make it 0.05.
let score = state.db.get_trust_score(owner).await.unwrap();
assert_eq!(
score, 0.0,
"the trust bump must NOT run when the durable upload failed; the \
issue was not durably accepted (score bumped to {score})"
);
}
}
+30 -10
View File
@@ -22,6 +22,7 @@ pub mod replicas;
pub mod repos;
pub mod resolve;
pub mod stars;
pub mod status;
pub mod tasks;
pub mod visibility;
pub mod webhooks;
@@ -67,17 +68,14 @@ pub(crate) async fn authorize_repo_read(
/// representation only within `did:key`; never let a bare id match across methods —
/// `did:web` / `did:gitlawb` share the base58 space with `did:key`, so a
/// trailing-segment compare would treat `did:key:X` and `did:gitlawb:X` as equal.
///
/// The collapse itself lives in exactly one place, [`crate::db::normalize_owner_key`],
/// which is also the function the stored `owner_did` / `authorizing_did` columns
/// are normalized through and which `OWNER_KEY_CASE_SQL` mirrors byte for byte.
/// Two identities match when they normalize to the same key; a second copy of the
/// rule here is how the Rust gate and the SQL filters would drift apart.
pub(crate) fn did_matches(a: &str, b: &str) -> bool {
if a == b {
return true;
}
fn key_id(d: &str) -> &str {
d.strip_prefix("did:key:").unwrap_or(d)
}
let (ka, kb) = (key_id(a), key_id(b));
// After stripping `did:key:`, a value still containing ':' is a non-key full
// DID — do not let it match a bare `did:key` id.
!ka.contains(':') && !kb.contains(':') && ka == kb
crate::db::normalize_owner_key(a) == crate::db::normalize_owner_key(b)
}
/// 403 unless `caller` is the repo owner. Uses [`did_matches`] so the owner check
@@ -175,6 +173,7 @@ mod authz_guard {
let events = include_str!("events.rs");
let tasks = include_str!("tasks.rs");
let stars = include_str!("stars.rs");
let status = include_str!("status/mod.rs");
let protect = include_str!("protect.rs");
let visibility = include_str!("visibility.rs");
let profiles = include_str!("profiles.rs");
@@ -195,6 +194,13 @@ mod authz_guard {
// (existence hiding) and require_repo_owner guards the owner half.
(webhooks, "list_webhooks", "authorize_repo_read("),
(webhooks, "list_webhooks", "require_repo_owner("),
// Same two-half shape as list_webhooks: authorize_repo_read runs
// first (a quarantined or unreadable repo gets the missing-repo
// not-found, never a 403 that would confirm existence), then
// require_repo_owner 403s a non-owner of a readable repo. Both halves
// are pinned, because dropping either changes what a stranger learns.
(status, "create_status", "authorize_repo_read("),
(status, "create_status", "require_repo_owner("),
(labels, "add_label", "require_repo_owner("),
(labels, "remove_label", "require_repo_owner("),
// Bucket A' — owner OR author (did_matches against the author)
@@ -217,6 +223,19 @@ mod authz_guard {
(protect, "list_protected_branches", "authorize_repo_read("),
(labels, "list_labels", "authorize_repo_read("),
(events, "list_repo_events", "authorize_repo_read("),
// The catch-up poll surface gates before any push-event row is read,
// so a stranger cannot use it to learn that a private repo exists or
// is being pushed to.
(events, "list_repo_push_events", "authorize_repo_read("),
// The status read gates before any claim data is loaded, so its deny
// is the repo's own not-found rather than an existence oracle over
// which commits were reported on.
(status, "commit_status", "authorize_repo_read("),
// The rollup gates on the same helper before the pull request row is
// loaded, so a caller who cannot read the repo cannot learn which
// pull request numbers exist on it — and the fallback's branch
// resolve and head persist sit behind that same gate.
(status, "pull_request_status", "authorize_repo_read("),
// Bucket C — signer-self: the acting DID is matched/bound to auth.0
(tasks, "create_task", "did_matches("),
(tasks, "claim_task", "did_matches("),
@@ -490,6 +509,7 @@ mod authz_guard {
(include_str!("replicas.rs"), "replicas.rs"),
(include_str!("repos.rs"), "repos.rs"),
(include_str!("stars.rs"), "stars.rs"),
(include_str!("status/mod.rs"), "status/mod.rs"),
(include_str!("visibility.rs"), "visibility.rs"),
(include_str!("webhooks.rs"), "webhooks.rs"),
];
+270 -1
View File
@@ -381,6 +381,14 @@ pub async fn notify_sync(
)));
}
// #272: the slug reaches a PathBuf::join in the sync worker, where an extra
// separator or a traversal segment places the mirror outside repos_dir. Reject
// it here, before the queue row and the ref-update row are written. The message
// names the field and nothing about where repos live on disk.
if let Err(e) = crate::git::repo_store::validate_repo_slug(&req.repo) {
return Err(AppError::BadRequest(format!("invalid repo field: {e}")));
}
state
.db
.enqueue_sync(&req.repo, &req.node_did, &req.ref_name, &req.new_sha, None)
@@ -584,7 +592,9 @@ mod tests {
state.config = Arc::new(cfg);
}
const NOTIFY_BODY: &str = r#"{"repo":"demo","ref_name":"refs/heads/main","new_sha":"0000000000000000000000000000000000000000","node_did":"PEER_DID"}"#;
// The slug must be a well-formed owner/name (#272); these brake tests are about
// the rate limiter, so the body carries a valid slug and reaches the brake.
const NOTIFY_BODY: &str = r#"{"repo":"z6Mkfoo/hello","ref_name":"refs/heads/main","new_sha":"0000000000000000000000000000000000000000","node_did":"PEER_DID"}"#;
// ── trigger: mandatory signature ──────────────────────────────────────────
@@ -866,6 +876,265 @@ mod tests {
);
}
// ── notify: repo slug validation (#272) ───────────────────────────────────
//
// An unsigned caller who has announced their own DID is a "known peer", so
// the slug they send reaches the sync worker's path join. `a//tmp/probe`
// resolved to /tmp/probe.git, outside repos_dir entirely. notify must reject
// anything that is not a well-formed owner/name slug before it is queued.
/// Every slug shape the boundary must refuse. `a//tmp/gitlawb-probe` is the
/// verified escape; the rest are the traversal and empty-half neighbours.
const MALFORMED_SLUGS: [&str; 9] = [
"a//tmp/gitlawb-probe",
"../hello",
"./hello",
"../../etc/evil",
"a/../../x",
"..",
"demo",
"/hello",
"z6Mkfoo/",
];
fn notify_body(repo: &str, peer_did: &str) -> String {
serde_json::json!({
"repo": repo,
"ref_name": "refs/heads/main",
"new_sha": "0000000000000000000000000000000000000000",
"node_did": peer_did,
})
.to_string()
}
#[sqlx::test]
async fn sync_notify_rejects_malformed_repo_slugs(pool: PgPool) {
let state = test_state(pool).await;
let db = state.db.clone();
let peer_did = seed_peer(&state).await;
let router = crate::server::build_router(state);
for (i, slug) in MALFORMED_SLUGS.iter().enumerate() {
// Distinct source IPs so the per-IP brake never masks the 400.
let ip = format!("198.51.100.{}:5000", i + 1);
let resp = router
.clone()
.oneshot(unsigned_post(
"/api/v1/sync/notify",
&notify_body(slug, &peer_did),
&ip,
))
.await
.unwrap();
assert_eq!(
resp.status(),
StatusCode::BAD_REQUEST,
"slug {slug:?} must be rejected"
);
}
let queued = db.dequeue_pending_syncs(100).await.unwrap();
assert!(
queued.is_empty(),
"no malformed slug may be queued, got {:?}",
queued.iter().map(|q| &q.repo).collect::<Vec<_>>()
);
}
#[sqlx::test]
async fn sync_notify_rejection_body_names_field_not_path(pool: PgPool) {
// R3: the client learns which field was bad, never where repos live on
// disk. A message carrying the resolved path would hand an unsigned
// caller the server's layout.
let state = test_state(pool).await;
let repos_dir = state.config.repos_dir.display().to_string();
let peer_did = seed_peer(&state).await;
let router = crate::server::build_router(state);
let resp = router
.oneshot(unsigned_post(
"/api/v1/sync/notify",
&notify_body("a//tmp/gitlawb-probe", &peer_did),
"198.51.100.20:5000",
))
.await
.unwrap();
assert_eq!(resp.status(), StatusCode::BAD_REQUEST);
let bytes = axum::body::to_bytes(resp.into_body(), usize::MAX)
.await
.unwrap();
let body = String::from_utf8(bytes.to_vec()).unwrap();
assert!(body.contains("repo"), "body must name the field: {body}");
assert!(
!body.contains(&repos_dir),
"body must not leak repos_dir ({repos_dir}): {body}"
);
for trimmed in repos_dir.trim_start_matches("./").split('/') {
assert!(
trimmed.is_empty() || !body.contains(trimmed),
"body must not leak a repos_dir segment ({trimmed}): {body}"
);
}
// A rooted path is a leading '/' with something after it. A bare quoted
// '/' is the separator the message is allowed to talk about.
assert!(
!body
.split(|c: char| c.is_whitespace() || c == '"' || c == '\'')
.any(|tok| tok.starts_with('/') && tok.len() > 1),
"body must not contain a rooted filesystem path: {body}"
);
}
#[sqlx::test]
async fn sync_notify_rejection_writes_no_ref_update(pool: PgPool) {
// The second half of R1: the ref-update feed row sits after enqueue_sync,
// so the early return must skip it too. Asserted, not argued.
let state = test_state(pool.clone()).await;
let peer_did = seed_peer(&state).await;
let router = crate::server::build_router(state);
let resp = router
.oneshot(unsigned_post(
"/api/v1/sync/notify",
&notify_body("a//tmp/gitlawb-probe", &peer_did),
"198.51.100.21:5000",
))
.await
.unwrap();
assert_eq!(resp.status(), StatusCode::BAD_REQUEST);
let count: i64 = sqlx::query_scalar("SELECT COUNT(*) FROM received_ref_updates")
.fetch_one(&pool)
.await
.unwrap();
assert_eq!(
count, 0,
"a rejected notify must not seed the ref-update feed"
);
}
#[sqlx::test]
async fn sync_notify_accepts_valid_slug_and_records_ref_update(pool: PgPool) {
// must-not-break, and the control that keeps the two negative assertions
// above from being vacuous: a well-formed slug still queues and still
// writes its ref-update row.
let state = test_state(pool.clone()).await;
let db = state.db.clone();
let peer_did = seed_peer(&state).await;
let router = crate::server::build_router(state);
let resp = router
.oneshot(unsigned_post(
"/api/v1/sync/notify",
&notify_body("z6Mkfoo/hello", &peer_did),
"198.51.100.22:5000",
))
.await
.unwrap();
assert_eq!(resp.status(), StatusCode::OK);
let queued = db.dequeue_pending_syncs(100).await.unwrap();
assert_eq!(queued.len(), 1);
assert_eq!(queued[0].repo, "z6Mkfoo/hello");
let count: i64 =
sqlx::query_scalar("SELECT COUNT(*) FROM received_ref_updates WHERE repo = $1")
.bind("z6Mkfoo/hello")
.fetch_one(&pool)
.await
.unwrap();
assert_eq!(
count, 1,
"the valid path must still write the ref-update row"
);
}
#[sqlx::test]
async fn sync_notify_unknown_peer_wins_over_malformed_slug(pool: PgPool) {
// Ordering: the slug check sits after the peer gate, so an unknown peer
// still gets the unknown-peer message rather than a slug complaint.
let state = test_state(pool).await;
let stranger = Keypair::generate().did().to_string();
let router = crate::server::build_router(state);
let resp = router
.oneshot(unsigned_post(
"/api/v1/sync/notify",
&notify_body("a//tmp/gitlawb-probe", &stranger),
"198.51.100.23:5000",
))
.await
.unwrap();
assert_eq!(resp.status(), StatusCode::BAD_REQUEST);
let bytes = axum::body::to_bytes(resp.into_body(), usize::MAX)
.await
.unwrap();
let body = String::from_utf8(bytes.to_vec()).unwrap();
assert!(
body.contains("unknown peer DID"),
"peer gate must still fire first: {body}"
);
}
#[sqlx::test]
async fn unsigned_announce_then_notify_cannot_queue_a_repos_dir_escape(pool: PgPool) {
// The reachability case for #272, committed in its attack form. Two
// unsigned requests through the shipped router, no signature and no
// config change: announce the attacker's own DID (which is what makes
// them a "known peer"), then notify with `a/<abs>/gitlawb-probe`. That
// slug used to reach PathBuf::join in the sync worker, where the
// absolute second component discarded repos_dir and planted a mirror at
// /<abs>/gitlawb-probe.git. The notify must now be refused outright.
//
// This has to run over crate::server::build_router rather than a
// directly mounted handler: the point of the test is that the default
// layer stack (optional_signature, not require_signature) leaves the
// route open, so mounting the handler alone would assert nothing about
// reachability.
let state = test_state(pool).await;
let db = state.db.clone();
let attacker = Keypair::generate().did().to_string();
let router = crate::server::build_router(state);
let announce =
format!(r#"{{"did":"{attacker}","http_url":"https://attacker.example.com"}}"#);
let resp = router
.clone()
.oneshot(unsigned_post(
"/api/v1/peers/announce",
&announce,
"198.51.100.40:5000",
))
.await
.unwrap();
// Asserted, not assumed: this plan does not change announce, and the
// whole attack rests on an unsigned announce still succeeding.
assert_eq!(
resp.status(),
StatusCode::OK,
"unsigned announce still admits an attacker-generated DID"
);
let outside = tempfile::TempDir::new().unwrap();
let escape_dir = outside.path().join("gitlawb-probe");
let slug = format!("a/{}", escape_dir.display());
let resp = router
.oneshot(unsigned_post(
"/api/v1/sync/notify",
&notify_body(&slug, &attacker),
"198.51.100.41:5000",
))
.await
.unwrap();
assert_eq!(
resp.status(),
StatusCode::BAD_REQUEST,
"notify must refuse the escaping slug {slug:?}"
);
let queued = db.dequeue_pending_syncs(100).await.unwrap();
assert!(
queued.is_empty(),
"sync_queue must stay empty, got {:?}",
queued.iter().map(|q| &q.repo).collect::<Vec<_>>()
);
assert!(
!escape_dir.with_extension("git").exists(),
"nothing may be written at the escape target"
);
}
#[sqlx::test]
async fn announce_still_accepts_unsigned_in_default_mode(pool: PgPool) {
// must-not-over-reach: adding the brake must not tighten announce's
+150 -177
View File
@@ -65,6 +65,7 @@ pub async fn create_pr(
status: "open".to_string(),
merged_by_did: None,
merged_at: None,
head_commit: None,
created_at: now.clone(),
updated_at: now,
};
@@ -217,12 +218,17 @@ pub async fn merge_pr(
let disk_path = guard.path().to_path_buf();
let merger_did = auth.0;
// Snapshot the target branch's pre-merge tip so a failed durable upload can
// roll the merge commit back on local disk (mirrors create_issue). Without
// this the local fast path in RepoStore::acquire serves the merged tree until
// a later write refreshes the archive, even though the client got a 5xx.
let target_ref = format!("refs/heads/{}", pr.target_branch);
let prior_target = store::ref_oid(&disk_path, &target_ref).ok().flatten();
// The source head this merge is about to consume, read under the write lock.
// The PR row above was loaded before the lock, so a push landing in between
// already moved the branch and the row's own `head_commit` is stale; the
// value frozen on the merged PR has to be what was merged. `None` when the
// ref cannot be resolved, which leaves whatever the push path last stored.
let merged_source_head = store::list_refs(&disk_path).ok().and_then(|refs| {
let want = format!("refs/heads/{}", pr.source_branch);
refs.into_iter()
.find(|(name, _)| *name == want)
.map(|(_, sha)| sha)
});
let merge_result = store::merge_branch(
&disk_path,
@@ -232,38 +238,15 @@ pub async fn merge_pr(
&pr.title,
);
// Always release the advisory lock, even on error; upload to Tigris only on
// success. A false return means the durable upload failed/timed out: the merge
// commit applied to local disk but never reached object storage, so the next
// acquire_write reverts it from the stale archive. Report 5xx so the client
// retries rather than marking the PR merged in the DB while the merged tree
// was silently lost. Fail BEFORE merge_pr / touch_repo / webhooks. (P1
// data-loss guard, same as receive-pack.) On that failure, roll the local
// merge back to the captured tip BEFORE the lock releases.
let upload_ok = guard
.release_with_failure_cleanup(merge_result.is_ok(), |path| {
if let Some(oid) = &prior_target {
if let Err(e) = store::set_ref(path, &target_ref, oid) {
tracing::warn!(repo = %record.name, err = %e,
"failed to roll back local merge commit after failed durable upload; \
a local-fast-path read may serve the un-uploaded merged tree");
}
}
})
.await;
// Always release the advisory lock — even on error; upload to Tigris only on success.
guard.release(merge_result.is_ok()).await;
let merge_sha = merge_result.map_err(|e| AppError::Git(e.to_string()))?;
if !upload_ok {
tracing::error!(repo = %record.name,
"durable upload failed after merge; reporting failure so the client retries");
return Err(AppError::Git(format!(
"durable storage upload failed for {}",
record.name
)));
}
state.db.merge_pr(&pr.id, &merger_did).await?;
state
.db
.merge_pr(&pr.id, &merger_did, merged_source_head.as_deref())
.await?;
let _ = state.db.touch_repo(&record.id).await;
webhooks::fire_event(
@@ -461,166 +444,156 @@ pub async fn list_comments(
#[cfg(test)]
mod tests {
// super::* brings in the handler plus the axum Extension/Path/State
// extractors and the Utc/Uuid/PullRequest types the seed uses.
use super::*;
use std::path::Path as StdPath;
use std::time::Duration;
use axum::body::Body;
use axum::http::{Method, StatusCode};
use axum::routing::post;
use axum::Router;
use std::process::Command;
use tower::ServiceExt;
// Object store whose upload() parks forever, so release()'s bounded timeout
// is the only thing that completes it, modeling a stalled durable PUT.
// exists() is false so acquire_write never downloads over the local bare repo.
struct StallStore;
#[async_trait::async_trait]
impl crate::git::tigris::ObjectStore for StallStore {
async fn exists(&self, _o: &str, _r: &str) -> anyhow::Result<bool> {
Ok(false)
}
async fn upload(&self, _o: &str, _r: &str, _p: &StdPath) -> anyhow::Result<()> {
std::future::pending::<()>().await;
Ok(())
}
async fn download(&self, _o: &str, _r: &str, _p: &StdPath) -> anyhow::Result<()> {
Ok(())
}
async fn delete(&self, _o: &str, _r: &str) -> anyhow::Result<()> {
Ok(())
}
}
const OWNER_DID: &str = "did:key:z6MkpTHR8VNsBxYAAWHut2Geadd9jSwuBV8xRoAnwWsdvktH";
const STALE_SHA: &str = "9999999999999999999999999999999999999999";
// The failed-upload rollback on the merge path. merge_pr writes the merge
// commit to LOCAL disk, then release() uploads to durable storage. When
// that upload times out the handler must 5xx (before merge_pr/webhooks)
// AND roll the target branch back to its pre-merge tip while the advisory
// lock is held, so a local-fast-path read never serves the un-uploaded
// merged tree. RED with the cleanup closure at ~pulls.rs:244 reverted to
// `|_| {}`: main stays at the merge commit.
#[sqlx::test]
async fn merge_failed_upload_rolls_back_target_ref(pool: sqlx::PgPool) {
use axum::response::IntoResponse;
use std::process::Command;
let mut state = crate::test_support::test_state(pool.clone()).await;
let repos_dir = tempfile::TempDir::new().unwrap();
state.repo_store = crate::git::repo_store::RepoStore::new(
repos_dir.path().to_path_buf(),
Some(std::sync::Arc::new(StallStore)),
pool.clone(),
)
.with_release_upload_timeout(Duration::from_millis(200));
let owner = "z6mergerollbackowner";
let name = "mergerollback";
let owner_slug = owner.replace([':', '/'], "_");
let bare = repos_dir
.path()
.join(&owner_slug)
.join(format!("{name}.git"));
state
.db
.upsert_mirror_repo(owner, name, &bare.to_string_lossy(), None, false)
.await
.unwrap();
fn git(args: &[&str], dir: &StdPath) -> String {
let out = Command::new("git")
.args(args)
.current_dir(dir)
.output()
.unwrap();
assert!(
out.status.success(),
"git {args:?} failed: {}",
String::from_utf8_lossy(&out.stderr)
);
String::from_utf8_lossy(&out.stdout).trim().to_string()
}
// Scratch repo: c1 on main, c2 on feature; the bare clone carries both
// branches, so the merge of feature into main has real work to do.
let work = tempfile::TempDir::new().unwrap();
git(&["init", "-q", "-b", "main", "."], work.path());
git(&["config", "user.email", "t@t"], work.path());
git(&["config", "user.name", "t"], work.path());
std::fs::write(work.path().join("f.txt"), "one").unwrap();
git(&["add", "f.txt"], work.path());
git(&["commit", "-q", "-m", "c1"], work.path());
git(&["checkout", "-q", "-b", "feature"], work.path());
std::fs::write(work.path().join("g.txt"), "two").unwrap();
git(&["add", "g.txt"], work.path());
git(&["commit", "-q", "-m", "c2"], work.path());
std::fs::create_dir_all(bare.parent().unwrap()).unwrap();
fn git(dir: &std::path::Path, args: &[&str]) {
let out = Command::new("git")
.args([
"clone",
"--bare",
"-q",
&work.path().to_string_lossy(),
&bare.to_string_lossy(),
])
.args(args)
.current_dir(dir)
.env("GIT_AUTHOR_NAME", "t")
.env("GIT_AUTHOR_EMAIL", "[email protected]")
.env("GIT_COMMITTER_NAME", "t")
.env("GIT_COMMITTER_EMAIL", "[email protected]")
.output()
.unwrap();
.expect("git runs");
assert!(
out.status.success(),
"clone --bare failed: {}",
"git {args:?} failed: {}",
String::from_utf8_lossy(&out.stderr)
);
}
let pre_merge_tip = store::ref_oid(&bare, "refs/heads/main")
.unwrap()
.expect("main exists before the merge");
/// A bare repo with `main` and a `feature` branch one commit ahead.
/// Returns the bare path and the real `feature` tip.
fn bare_repo_with_feature(
root: &std::path::Path,
owner_slug: &str,
) -> (std::path::PathBuf, String) {
let work = root.join("work");
std::fs::create_dir_all(&work).unwrap();
git(&work, &["init", "-q", "-b", "main", "."]);
std::fs::write(work.join("a.txt"), "one").unwrap();
git(&work, &["add", "."]);
git(&work, &["commit", "-qm", "one"]);
git(&work, &["checkout", "-qb", "feature"]);
std::fs::write(work.join("b.txt"), "two").unwrap();
git(&work, &["add", "."]);
git(&work, &["commit", "-qm", "two"]);
git(&work, &["checkout", "-q", "main"]);
// Seed the open PR row the handler looks up.
let record = state.db.get_repo(owner, name).await.unwrap().unwrap();
let now = Utc::now().to_rfc3339();
let bare_dir = root.join(owner_slug);
std::fs::create_dir_all(&bare_dir).unwrap();
let bare = bare_dir.join("demo.git");
git(
root,
&[
"clone",
"-q",
"--bare",
work.to_str().unwrap(),
bare.to_str().unwrap(),
],
);
let tip = Command::new("git")
.args(["rev-parse", "refs/heads/feature"])
.current_dir(&bare)
.output()
.unwrap();
let tip = String::from_utf8_lossy(&tip.stdout).trim().to_string();
assert_eq!(tip.len(), 40, "feature tip must resolve");
(bare, tip)
}
/// Merging freezes the stored head at the source commit the merge actually
/// consumed. The PR row is loaded before the write lock is taken, so a push
/// that lands in between leaves the row's own `head_commit` stale; seeding a
/// value that matches nothing on disk stands in for that race, and the
/// merged PR must still come out pointing at the real `feature` tip.
#[sqlx::test]
async fn merge_stamps_the_source_head_it_merged_over_a_racing_value(pool: sqlx::PgPool) {
let tmp = tempfile::tempdir().unwrap();
let owner_slug = OWNER_DID.replace([':', '/'], "_");
let (bare, feature_tip) = bare_repo_with_feature(tmp.path(), &owner_slug);
let mut state = crate::test_support::test_state(pool.clone()).await;
state.repo_store =
crate::git::repo_store::RepoStore::for_testing(tmp.path().to_path_buf(), pool);
let now = Utc::now();
let repo = crate::db::RepoRecord {
id: uuid::Uuid::new_v4().to_string(),
name: "demo".into(),
owner_did: OWNER_DID.into(),
description: None,
is_public: true,
default_branch: "main".into(),
created_at: now,
updated_at: now,
disk_path: bare.to_string_lossy().into_owned(),
forked_from: None,
machine_id: None,
};
state.db.create_repo(&repo).await.unwrap();
let pr = PullRequest {
id: Uuid::new_v4().to_string(),
repo_id: repo.id.clone(),
number: 1,
title: "add b".into(),
body: None,
author_did: OWNER_DID.into(),
source_branch: "feature".into(),
target_branch: "main".into(),
status: "open".into(),
merged_by_did: None,
merged_at: None,
head_commit: None,
created_at: now.to_rfc3339(),
updated_at: now.to_rfc3339(),
};
state.db.create_pr(&pr).await.unwrap();
// The stale value the racing push scenario leaves on the row.
state
.db
.create_pr(&PullRequest {
id: Uuid::new_v4().to_string(),
repo_id: record.id.clone(),
number: 1,
title: "t".into(),
body: None,
author_did: owner.to_string(),
source_branch: "feature".into(),
target_branch: "main".into(),
status: "open".to_string(),
merged_by_did: None,
merged_at: None,
created_at: now.clone(),
updated_at: now,
})
.set_open_pr_heads(&repo.id, "feature", STALE_SHA)
.await
.unwrap();
// Merge as the owner: the merge commit applies locally, the durable
// upload stalls and times out, and the handler must report 5xx.
let resp = super::merge_pr(
State(state.clone()),
Extension(AuthenticatedDid(owner.to_string())),
Path((owner.to_string(), name.to_string(), 1)),
)
.await;
let status = match resp {
Ok(_) => StatusCode::OK,
Err(e) => e.into_response().status(),
};
assert!(
status.is_server_error(),
"a stalled durable upload on merge must 5xx, got {status}"
);
let db = state.db.clone();
let app = Router::new()
.route(
"/api/v1/repos/{owner}/{repo}/pulls/{number}/merge",
post(merge_pr),
)
.with_state(state);
// The target branch must still point at the pre-merge tip: the failed
// upload rolled the merge commit back while the lock was held.
let tip = store::ref_oid(&bare, "refs/heads/main")
.unwrap()
.expect("main still exists after the rollback");
let response = app
.oneshot(crate::test_support::signed_request_as(
OWNER_DID,
Method::POST,
"/api/v1/repos/z6MkpTHR8VNsBxYAAWHut2Geadd9jSwuBV8xRoAnwWsdvktH/demo/pulls/1/merge",
Body::empty(),
))
.await
.unwrap();
assert_eq!(response.status(), StatusCode::OK);
let merged = db.get_pr(&repo.id, 1).await.unwrap().unwrap();
assert_eq!(merged.status, "merged");
assert_eq!(
tip, pre_merge_tip,
"a failed durable upload must roll the target branch back to its \
pre-merge tip; a local-fast-path read served the un-uploaded merge"
merged.head_commit,
Some(feature_tip),
"the merge must stamp the source head it consumed, not the racing value"
);
}
}
File diff suppressed because it is too large. Load diff
+657
View File
@@ -0,0 +1,657 @@
//! Commit status claims: the write path and the combined read.
//!
//! POST /api/v1/repos/:owner/:repo/statuses/:sha — append one claim (owner only)
//! GET /api/v1/repos/:owner/:repo/commits/:sha/status — the combined projection
//!
//! Claims are append-only: a producer reporting twice for the same context
//! leaves both rows and the visible status is a projection over the history.
//! Reporting twice means two REQUESTS, though. An exact repeat of one already
//! accepted is answered with 200 and the row it wrote, not a second row: the
//! projection elects the highest `seq`, so a replayed claim would not duplicate
//! a verdict but overturn the one that superseded it.
use axum::extract::{Extension, Path, State};
use axum::http::StatusCode;
use axum::Json;
use chrono::Utc;
use serde::{Deserialize, Serialize};
use uuid::Uuid;
use crate::auth::AuthenticatedDid;
use crate::db::StatusClaim;
use crate::error::{AppError, Result};
use crate::state::AppState;
/// Claims for one (repo, commit, producer, context). Generous for a real
/// producer; a producer past it is misbehaving.
const MAX_CLAIMS_PER_TUPLE: i64 = 100;
/// Distinct contexts for one (repo, commit). The context string is caller-chosen
/// and free-form, so the tuple cap alone bounds nothing.
const MAX_CONTEXTS_PER_COMMIT: i64 = 50;
/// Claim rows for one repo within the trailing window the db layer counts over.
/// The commit SHA is caller-chosen and never existence-checked, so a writer at
/// the two caps above can still fan out over fresh 40-hex strings without this
/// one. It bounds the RATE of that fan-out rather than a lifetime total, because
/// nothing prunes the table: a lifetime bound would close the surface on a repo
/// permanently while answering with a status every client retries.
const MAX_CLAIMS_PER_REPO_PER_WINDOW: i64 = 10_000;
#[derive(Deserialize)]
pub struct CreateStatusRequest {
pub state: String,
pub context: String,
pub target_url: Option<String>,
pub description: Option<String>,
}
/// The wire state set (KTD-1): GitHub's four commit-status states, so absence
/// stays distinguishable without a fifth value no client understands.
const CLAIM_STATES: [&str; 4] = ["error", "failure", "pending", "success"];
/// Bounds on the signature material this write path persists verbatim. Every one
/// of the four is caller-influenced: the headers come off the request, the
/// signing string grows with the component list the caller's Signature-Input
/// chose, and the body is the caller's. Without a bound the claim log's row size
/// is set by whoever writes to it, so these are explicit and generous rather than
/// implicit and absent. A conforming `sign_request` produces roughly 100, 200 and
/// 400 bytes for the first three.
const MAX_SIGNATURE_CHARS: usize = 512;
const MAX_SIGNATURE_INPUT_CHARS: usize = 1024;
const MAX_SIGNING_STRING_CHARS: usize = 4096;
/// The body is a `CreateStatusRequest`, whose own fields are already capped at
/// roughly 3.3 KB in total by the three limits below; this leaves room for JSON
/// framing and nothing more.
///
/// Enforced twice, at two different moments, because the handler's check alone
/// is not the bound it advertises: `require_signature` buffers the whole body
/// before any handler runs, so by the time [`create_status`] measures it the
/// allocation has already happened. `build_router` applies the same number as a
/// `RequestBodyLimitLayer` at the transport, outside the auth layers, which is
/// what makes it a bound on what is ALLOCATED and not only on what is STORED.
/// The handler keeps its own check regardless — it is the one that holds for a
/// handler reached by any other route table, and it is what the material size
/// tests drive.
pub(crate) const MAX_REQUEST_BODY_BYTES: usize = 8192;
const MAX_CONTEXT_CHARS: usize = 255;
const MAX_TARGET_URL_CHARS: usize = 2048;
const MAX_DESCRIPTION_CHARS: usize = 1024;
/// The 201 body for an accepted claim: what the client needs to identify the row
/// it just wrote, and nothing else.
///
/// Deliberately not the stored [`StatusClaim`]. That struct carries the signature
/// material, and serializing it echoed the signature, the signing string and the
/// whole request body back on every write — the body as a JSON array of integers.
/// Write-time provenance belongs in the row, not on the wire, which is the same
/// line [`StatusEntry`] draws on the read side.
#[derive(Serialize)]
pub struct CreatedStatus {
pub id: String,
/// The database-assigned ordering key, so the client can name its own row.
pub seq: i64,
pub repo_id: String,
pub commit_sha: String,
pub state: String,
pub context: String,
pub target_url: Option<String>,
pub description: Option<String>,
pub producer_did: String,
pub created_at: String,
}
impl CreatedStatus {
fn from_claim(claim: StatusClaim, seq: i64) -> Self {
Self {
id: claim.id,
seq,
repo_id: claim.repo_id,
commit_sha: claim.commit_sha,
state: claim.state,
context: claim.context,
target_url: claim.target_url,
description: claim.description,
producer_did: claim.producer_did,
created_at: claim.created_at,
}
}
}
/// POST /api/v1/repos/:owner/:repo/statuses/:sha
pub async fn create_status(
State(state): State<AppState>,
Extension(auth): Extension<AuthenticatedDid>,
Path((owner, name, sha)): Path<(String, String, String)>,
material: Option<Extension<crate::auth::SignatureMaterial>>,
Json(req): Json<CreateStatusRequest>,
) -> Result<(StatusCode, Json<CreatedStatus>)> {
// Read-visibility first, then owner, and the order is the security property.
// authorize_repo_read denies a quarantined repo before the visibility gate and
// answers with the repo's own not-found, byte-identical to a missing repo, so
// a caller who cannot read the repo cannot learn it exists. Loading the repo
// and comparing the owner would answer 403 there, turning this endpoint into
// an existence oracle. require_repo_owner then 403s a non-owner of a repo the
// caller can read, where existence is not secret.
let (record, _rules) =
crate::api::authorize_repo_read(&state, &owner, &name, Some(&auth.0), "/").await?;
crate::api::require_repo_owner(&record, &auth.0)?;
let commit_sha = normalize_sha(&sha)?;
let claim_state = validate_state(&req.state)?;
let context = validate_context(&req.context)?;
let target_url = validate_target_url(req.target_url.as_deref())?;
let description = validate_description(req.description.as_deref())?;
// KTD-5: the signature material is captured here because the request carries
// the only copy, and `signing_string` is the exact byte sequence the
// middleware verified. Its absence means this handler was reached without
// `require_signature` — a server misconfiguration (a route group that lost
// its auth layer, a handler mounted somewhere new), not anything the client
// did, so it is a 500 and no row is written. Storing an empty payload instead
// would leave every claim from that moment on unverifiable with nothing going
// red, which is exactly the absence-renders-as-success shape.
let Some(Extension(material)) = material else {
return Err(AppError::Internal(anyhow::anyhow!(
"status claim write reached without verified signature material — \
the route is not behind require_signature"
)));
};
// The body is carried only behind the `PersistsSignedBody` marker, which
// this route's group applies outside the auth layers. Its absence means the
// marker did not reach the middleware, because the layer was dropped or
// reordered. That is the same class of misconfiguration as the branch above
// and gets the same answer. Storing an empty body instead would write claims
// that carry a signature over a digest of bytes nobody kept, which is
// exactly the case `stored_claim_re_verifies_and_a_tampered_row_does_not`
// exists to make impossible.
let Some(body) = material.body.clone() else {
return Err(AppError::Internal(anyhow::anyhow!(
"status claim write reached with signature material carrying no request body: \
the route lost its persist-body marker or applied it inside the auth layers"
)));
};
// Bounded before anything is written. The material is verified, which says
// the caller holds the key, not that what they signed is a reasonable size.
bound("signature", material.signature.len(), MAX_SIGNATURE_CHARS)?;
bound(
"signature_input",
material.signature_input.len(),
MAX_SIGNATURE_INPUT_CHARS,
)?;
bound(
"signing_string",
material.signing_string.len(),
MAX_SIGNING_STRING_CHARS,
)?;
bound("request body", body.len(), MAX_REQUEST_BODY_BYTES)?;
// After the bounds, so the scan below walks a signing string of known size.
require_body_bound_signature(&material, &body)?;
let digest = request_digest(&material, &body);
let (signature, signature_input, signing_string, request_body) = (
material.signature,
material.signature_input,
material.signing_string,
body.to_vec(),
);
let claim = StatusClaim {
id: Uuid::new_v4().to_string(),
// Ignored on insert: the database assigns the ordering key (KTD-3).
seq: 0,
repo_id: record.id.clone(),
commit_sha,
state: claim_state,
context,
target_url,
description,
// Both are the owner identity today (KTD-5); they split when delegated
// capabilities land.
producer_did: auth.0.clone(),
authorizing_did: auth.0,
signature,
signature_input,
signing_string,
request_body,
request_digest: digest,
created_at: Utc::now().to_rfc3339(),
};
let caps = crate::db::ClaimCaps {
per_tuple: MAX_CLAIMS_PER_TUPLE,
contexts_per_commit: MAX_CONTEXTS_PER_COMMIT,
per_repo_window: MAX_CLAIMS_PER_REPO_PER_WINDOW,
};
match state.db.insert_status_claim_capped(&claim, &caps).await? {
crate::db::ClaimInsert::Inserted(seq) => Ok((
StatusCode::CREATED,
Json(CreatedStatus::from_claim(claim, seq)),
)),
// Already recorded, so 200 and the original row rather than 201 and a
// second one. The claim the caller gets back is the stored one, id and
// seq included, which is what makes a retry safe to treat as success.
crate::db::ClaimInsert::AlreadyRecorded(existing) => {
let seq = existing.seq;
Ok((
StatusCode::OK,
Json(CreatedStatus::from_claim(*existing, seq)),
))
}
crate::db::ClaimInsert::CapExceeded(which) => Err(AppError::TooManyRequests(format!(
"claim limit reached for {which}"
))),
}
}
/// Refuse a claim whose signature does not cover the body about to be stored.
///
/// A claim IS provenance, so the signature stored beside it has to bind the
/// bytes stored beside it. It does not do that directly: `signing_string`
/// reaches the body only through a `content-digest` line. And a caller can sign
/// a content-digest the request then never carries. `require_signature` rebuilds
/// the covered values from the request, so an ABSENT `Content-Digest` header
/// makes the covered digest the empty string on the signing side and the
/// verifying side alike; the Ed25519 check passes over method, path and an empty
/// digest, and any body at all rides along. The row that write produces asserts
/// a provenance that cannot be re-verified: the signing string covers a digest
/// of bytes that are not the ones in `request_body`.
///
/// The predicate here is the write-time form of the one `re_verify` runs against
/// a stored row in the tests — recompute the digest of the bytes about to be
/// persisted and require the signing string to carry exactly that line — so what
/// this route accepts and what a third party can later re-verify are the same
/// question, asked at both ends.
///
/// This is a route property, checked where the claim becomes history, and it is
/// deliberately not the only line of defence: requiring the header in
/// `require_signature` closes the same hole one layer earlier and for every
/// signed route, and that change belongs there rather than here because no other
/// route's correctness should depend on the status handler. The two agree and
/// neither substitutes for the other — without the middleware's requirement the
/// empty digest is still accepted everywhere else, and without this one a later
/// relaxation of the middleware would silently resume writing unverifiable
/// history.
///
/// A 400, not a 500: the material is exactly what the client signed, and what
/// they signed does not cover what they sent.
fn require_body_bound_signature(
material: &crate::auth::SignatureMaterial,
body: &[u8],
) -> Result<()> {
let covered = format!(
"\"content-digest\": {}",
gitlawb_core::http_sig::compute_content_digest(body)
);
// Line-exact rather than a substring search: `build_signing_string` emits one
// component per line, and a `contains` would also be satisfied by the digest
// appearing INSIDE some other covered component's value, which proves
// nothing about what the signature covered.
if material.signing_string.lines().any(|line| line == covered) {
return Ok(());
}
Err(AppError::BadRequest(
"the signature does not cover this request body: send a Content-Digest header \
computed over the body and name \"content-digest\" among the signature's covered \
components"
.into(),
))
}
/// The identity of one signed write, as a hex sha-256.
///
/// All four inputs together, because a replay is the same bytes arriving twice:
/// the signature, the input that names what it covers, the canonical string it
/// was verified over, and the body that string covers through a content-digest.
/// The signature alone would nearly do (it covers the other three transitively),
/// but hashing what is actually stored means the digest describes the row rather
/// than a claim about it.
///
/// Each field is length-prefixed so no two different requests can concatenate to
/// the same bytes: without it, a signature ending in some prefix of the next
/// field would collide with the shorter signature that absorbed it.
///
/// This is not a nonce, and deliberately so. A nonce table needs pruning and a
/// pruning window is a second replay window; the claim row IS the record of what
/// was accepted, so uniqueness on it is self-maintaining.
fn request_digest(material: &crate::auth::SignatureMaterial, body: &[u8]) -> String {
use sha2::{Digest, Sha256};
let mut h = Sha256::new();
for part in [
material.signature.as_bytes(),
material.signature_input.as_bytes(),
material.signing_string.as_bytes(),
body,
] {
h.update((part.len() as u64).to_be_bytes());
h.update(part);
}
format!("{:x}", h.finalize())
}
/// One reported context in the combined response. Signature material is
/// deliberately absent: it is write-time provenance, not read-surface data.
#[derive(Serialize)]
pub struct StatusEntry {
pub state: String,
pub context: String,
pub target_url: Option<String>,
pub description: Option<String>,
pub producer_did: String,
pub created_at: String,
}
/// The combined commit status. `state` never leaves the four-value set (KTD-1),
/// so absence is carried by `total_count` 0 with the pending state rather than a
/// fifth value, and `reported_only` (R19) says out loud that the state covers the
/// contexts that reported, not every check a caller expected.
#[derive(Serialize)]
pub struct CombinedStatus {
pub state: String,
pub sha: String,
pub total_count: usize,
pub statuses: Vec<StatusEntry>,
pub reported_only: bool,
}
/// GET /api/v1/repos/:owner/:repo/commits/:sha/status
///
/// The auth extension is optional and MUST be last in the extractor list: the
/// route group carries `optional_signature`, so an unsigned caller reaches here
/// with no `AuthenticatedDid` at all and a public repo still answers.
pub async fn commit_status(
State(state): State<AppState>,
Path((owner, name, sha)): Path<(String, String, String)>,
auth: Option<Extension<AuthenticatedDid>>,
) -> Result<Json<CombinedStatus>> {
let caller = auth.as_ref().map(|Extension(a)| a.0.as_str());
// The gate runs first and on the requested path, before any claim data is
// touched. Its deny is the repo's own not-found, byte-identical to a missing
// repo, so the status surface cannot answer "does this repo exist".
let (record, _rules) =
crate::api::authorize_repo_read(&state, &owner, &name, caller, "/").await?;
let commit_sha = normalize_sha(&sha)?;
let statuses = project_claims(&state, &record, &commit_sha).await?;
Ok(Json(CombinedStatus {
state: combined_state(&statuses).to_string(),
sha: commit_sha,
total_count: statuses.len(),
statuses,
reported_only: true,
}))
}
/// The projection, in one place. Both read surfaces call it, so R3's "the rollup
/// is derived by the same projection as the commit read" holds by construction
/// rather than by two implementations happening to agree.
///
/// KTD-5: current authorization, not write-time authorization. An ownership
/// transfer drops the prior owner's claims from the projection while the
/// append-only history keeps them.
async fn project_claims(
state: &AppState,
record: &crate::db::RepoRecord,
commit_sha: &str,
) -> Result<Vec<StatusEntry>> {
let claims = state
.db
.latest_status_claims(&record.id, commit_sha, &record.owner_did)
.await?;
Ok(claims
.into_iter()
.map(|c| StatusEntry {
state: c.state,
context: c.context,
target_url: c.target_url,
description: c.description,
producer_did: c.producer_did,
created_at: c.created_at,
})
.collect())
}
/// The pull request head rollup (R11). `state` stays inside the four wire values
/// whatever happened to the head (KTD-1): an unresolvable head is carried by
/// `head_resolved`, alongside the pull request's own state, so a client can tell
/// "the head could not be resolved" from "the head resolved and nothing reported"
/// without a fifth state value no client understands.
#[derive(Serialize)]
pub struct PullRequestStatus {
pub number: i64,
/// The pull request's own state: open, closed, or merged.
pub pull_request_state: String,
pub head_resolved: bool,
/// The target commit, absent exactly when `head_resolved` is false.
pub sha: Option<String>,
pub state: String,
pub total_count: usize,
pub statuses: Vec<StatusEntry>,
pub reported_only: bool,
}
/// GET /api/v1/repos/:owner/:repo/pulls/:number/status
///
/// Same optional-auth shape as the commit read, and the auth extension MUST stay
/// last in the extractor list.
pub async fn pull_request_status(
State(state): State<AppState>,
Path((owner, name, number)): Path<(String, String, i64)>,
auth: Option<Extension<AuthenticatedDid>>,
) -> Result<Json<PullRequestStatus>> {
let caller = auth.as_ref().map(|Extension(a)| a.0.as_str());
// The gate runs first, on the requested path, before the pull request row is
// loaded. Its deny is the repo's own not-found, byte-identical to a missing
// repo. Once it passes, a missing pull request number is the plain not-found:
// existence is no longer secret.
let (record, _rules) =
crate::api::authorize_repo_read(&state, &owner, &name, caller, "/").await?;
let pr = state
.db
.get_pr(&record.id, number)
.await?
.ok_or_else(|| AppError::NotFound(format!("PR #{number} not found")))?;
let head = rollup_head(&state, &record, &pr).await?;
let statuses = match &head {
Some(sha) => project_claims(&state, &record, sha).await?,
None => Vec::new(),
};
Ok(Json(PullRequestStatus {
number: pr.number,
pull_request_state: pr.status,
head_resolved: head.is_some(),
sha: head,
state: combined_state(&statuses).to_string(),
total_count: statuses.len(),
statuses,
reported_only: true,
}))
}
/// Branch-head lookups performed by the rollup fallback, counted so a test can
/// assert WORK DONE rather than only the answer returned. A fallback that
/// re-resolved on every read and then discarded the answer would be invisible in
/// the response and visible here.
#[cfg(test)]
pub(crate) static BRANCH_RESOLVES: std::sync::atomic::AtomicUsize =
std::sync::atomic::AtomicUsize::new(0);
/// The rollup's target commit (KTD-4).
///
/// The stored head wins whenever it is set. It is frozen at close or merge, so a
/// closed or merged pull request keeps the commit the decision was made against,
/// and only an OPEN pull request with no stored head falls back to the branch.
///
/// The fallback is one database read: the latest push this node recorded for the
/// pull request's source branch, from `repo_push_events`. It deliberately does
/// NOT go through `git::store` ref listing, which needs an acquired repository
/// path — on a cold node that downloads the whole repository from object storage,
/// and this read group carries no rate limiter, so an anonymous caller could
/// drive repeated downloads with a URL.
///
/// `repo_push_events` is the source rather than `branch_cids` because
/// `record_push_events` writes it for every ref update unconditionally, whereas
/// the sole writer of `branch_cids` sits behind a pin CID. A node with no object
/// pinning configured never writes that table, so a resolve keyed on it could
/// never answer there and every open pull request without a stored head reported
/// `head_resolved: false` forever. The residual limit is that only pushes taken
/// after this shipped have rows, so a branch last pushed before then still does
/// not resolve.
///
/// The resolved head is persisted, which makes an unauthenticated GET write. That
/// is only acceptable because the write is self-limiting: it fires exactly when
/// `head_commit` is absent and it fills it, and the fill is conditioned on that
/// same absence in SQL, so the PERSIST happens at most once per pull request.
/// The resolve attempt itself is not once-only — a pull request whose branch does
/// not resolve stores nothing, so there is nothing to short-circuit on and the
/// lookup runs again on every read until it succeeds.
async fn rollup_head(
state: &AppState,
record: &crate::db::RepoRecord,
pr: &crate::db::PullRequest,
) -> Result<Option<String>> {
if let Some(stored) = pr.head_commit.clone() {
return Ok(Some(stored));
}
if pr.status != "open" {
return Ok(None);
}
// The push path records the full ref while a pull request stores the bare
// branch name, the same mismatch `crate::api::repos::branch_from_ref`
// reconciles on the write side; this is that mapping run backwards. The empty
// branch is excluded for the same reason it is there: `refs/heads/` is not a
// branch and nothing can have written it.
if pr.source_branch.is_empty() {
return Ok(None);
}
let ref_name = format!("refs/heads/{}", pr.source_branch);
#[cfg(test)]
BRANCH_RESOLVES.fetch_add(1, std::sync::atomic::Ordering::SeqCst);
let resolved = state
.db
.latest_push_sha_for_ref(&record.id, &ref_name)
.await?
// A recorded head that is not a commit SHA is no target at all, and
// storing it would leave the rollup pointing at nothing forever.
.and_then(|sha| normalize_sha(&sha).ok());
let Some(sha) = resolved else {
return Ok(None);
};
// Best effort, and deliberately not `?`. The head is already resolved and the
// response does not depend on this write landing, so a transient database
// failure here must not turn a served read into a 500; the next read resolves
// again and re-attempts the fill. Same catch-and-log the sibling writes on the
// push path use (`update_open_pr_heads`, `record_push_events`).
if let Err(e) = state.db.set_pr_head_if_absent(&pr.id, &sha).await {
tracing::warn!(
err = %e,
pr_id = %pr.id,
"failed to persist a resolved pull request head; serving the resolved head anyway"
);
}
Ok(Some(sha))
}
/// KTD-1's fold: any error or failure yields failure, else any pending yields
/// pending, else success. An empty set is pending, never success — total absence
/// of a verdict is its own state and must not read as a pass (R10).
fn combined_state(statuses: &[StatusEntry]) -> &'static str {
if statuses.is_empty() {
return "pending";
}
if statuses
.iter()
.any(|s| s.state == "error" || s.state == "failure")
{
"failure"
} else if statuses.iter().any(|s| s.state == "pending") {
"pending"
} else {
"success"
}
}
/// 40 hex characters, lowercased. The SHA is never existence-checked (KTD-7):
/// a claim describes a commit object, and the object may not have arrived yet.
fn normalize_sha(sha: &str) -> Result<String> {
if sha.len() == 40 && sha.bytes().all(|b| b.is_ascii_hexdigit()) {
Ok(sha.to_ascii_lowercase())
} else {
Err(AppError::BadRequest(
"commit sha must be exactly 40 hexadecimal characters".into(),
))
}
}
/// One size bound on persisted signature material, measured in bytes.
fn bound(what: &str, len: usize, max: usize) -> Result<()> {
if len > max {
return Err(AppError::BadRequest(format!(
"{what} must be at most {max} bytes"
)));
}
Ok(())
}
fn validate_state(state: &str) -> Result<String> {
if CLAIM_STATES.contains(&state) {
Ok(state.to_string())
} else {
Err(AppError::BadRequest(format!(
"state must be one of {}",
CLAIM_STATES.join(", ")
)))
}
}
/// The context is the projection key, so it is trimmed, bounded, and required to
/// be free of control characters.
fn validate_context(context: &str) -> Result<String> {
let trimmed = context.trim();
if trimmed.is_empty() || trimmed.chars().count() > MAX_CONTEXT_CHARS {
return Err(AppError::BadRequest(format!(
"context must be 1 to {MAX_CONTEXT_CHARS} characters"
)));
}
if trimmed.chars().any(char::is_control) {
return Err(AppError::BadRequest(
"context must not contain control characters".into(),
));
}
Ok(trimmed.to_string())
}
fn validate_target_url(url: Option<&str>) -> Result<Option<String>> {
let Some(url) = url else { return Ok(None) };
if url.chars().count() > MAX_TARGET_URL_CHARS {
return Err(AppError::BadRequest(format!(
"target_url must be at most {MAX_TARGET_URL_CHARS} characters"
)));
}
let lower = url.to_ascii_lowercase();
if lower.starts_with("http://") || lower.starts_with("https://") {
Ok(Some(url.to_string()))
} else {
Err(AppError::BadRequest(
"target_url must be an http or https URL".into(),
))
}
}
fn validate_description(description: Option<&str>) -> Result<Option<String>> {
match description {
Some(d) if d.chars().count() > MAX_DESCRIPTION_CHARS => Err(AppError::BadRequest(format!(
"description must be at most {MAX_DESCRIPTION_CHARS} characters"
))),
other => Ok(other.map(str::to_string)),
}
}
#[cfg(test)]
mod tests;
File diff suppressed because it is too large. Load diff
+159 -1
View File
@@ -17,6 +17,53 @@ use crate::state::AppState;
#[derive(Clone, Debug)]
pub struct AuthenticatedDid(pub String);
/// The RFC 9421 material this request was actually verified against, injected
/// alongside [`AuthenticatedDid`] by `require_signature`.
///
/// A handler that records a signed claim as history has to store it: the header
/// values and the canonical signing string are gone once the request ends, and a
/// stored claim nobody can re-verify is not history. `signing_string` is the exact
/// byte sequence the Ed25519 verification succeeded over, not a rebuild — a
/// handler reconstructing it from headers would be verifying a different string
/// than the middleware did.
#[derive(Clone, Debug)]
pub struct SignatureMaterial {
/// The `Signature` header value.
pub signature: String,
/// The `Signature-Input` header value.
pub signature_input: String,
/// The canonical bytes the signature covered.
pub signing_string: String,
/// The buffered request body, on the routes that persist it. The signing
/// string covers it only through the content-digest, so without these bytes
/// a stored claim can be shown to carry *a* valid signature over *some*
/// digest and nothing more. Carried here because the body is consumed
/// downstream by the extractor and cannot be recovered afterwards.
///
/// `Option`, and populated only behind [`PersistsSignedBody`], because a
/// route that never stores the body has no use for a second handle to it.
/// The unlike-sized field is this one: the three above are header-derived
/// and bounded by the server's header limit, while a receive-pack POST
/// carries up to GITLAWB_MAX_PACK_BYTES (2 GB by default). `Bytes`, not
/// `Vec<u8>`, so the handle the marked routes do take is a refcount bump
/// over the buffer the middleware already holds rather than a copy.
pub body: Option<axum::body::Bytes>,
}
/// Marks a route group whose handler persists the signed request body.
///
/// Applied as an extension layer OUTSIDE the auth layers, so it is on the
/// request before [`require_signature`] decides whether to carry the body. A
/// marker applied inside them is never seen, and the failure is silent: the
/// handler still compiles, tests that inject material by hand still pass, and
/// production stores nothing. The status write handler refuses an absent body
/// for that reason.
///
/// A marker rather than a path check because `require_signature` must not learn
/// which URL persists claims; that knowledge belongs at the route table.
#[derive(Clone, Copy, Debug)]
pub struct PersistsSignedBody;
/// Whether `caller` is authorized to push to `record`.
///
/// Phase 1 (`GITLAWB_ENFORCE_OWNER_PUSH`): owner-only, via the canonical
@@ -238,10 +285,28 @@ pub async fn require_signature(request: Request, next: Next) -> Response {
tracing::info!(did = %sig.key_id, "✓ authenticated request");
// The body travels only where it is stored. On every other signed route the
// middleware's handle would keep the buffer alive for the rest of the layer
// chain, until the handler's own extractor consumes the request.
let body = parts
.extensions
.get::<PersistsSignedBody>()
.map(|_| body_bytes.clone());
let material = SignatureMaterial {
signature: sig_header,
signature_input: sig_input,
signing_string,
body,
};
let mut request = Request::from_parts(parts, Body::from(body_bytes));
request
.extensions_mut()
.insert(AuthenticatedDid(sig.key_id.to_string()));
// Carry the verified material forward for handlers that persist a signed
// claim; none of it can be recovered once the request is gone.
request.extensions_mut().insert(material);
next.run(request).await
}
@@ -515,7 +580,6 @@ mod tests {
repo_store: crate::git::repo_store::RepoStore::for_testing(PathBuf::from("/tmp"), pool),
rate_limiter: RateLimiter::new(100, Duration::from_secs(60)),
create_ip_rate_limiter: RateLimiter::new(1000, Duration::from_secs(3600)),
write_rate_limiter: RateLimiter::new(1000, Duration::from_secs(3600)),
push_rate_limiter: RateLimiter::new(600, Duration::from_secs(3600)),
push_limiter_trust: crate::rate_limit::TrustedProxy::None,
sync_trigger_rate_limiter: RateLimiter::new(60, Duration::from_secs(3600)),
@@ -524,6 +588,100 @@ mod tests {
}
}
/// Reports what the signature middleware put in `SignatureMaterial.body`,
/// so a test can tell "captured" from "deliberately not captured" apart
/// from "the middleware never ran".
async fn report_body(material: Option<axum::Extension<SignatureMaterial>>) -> String {
match material {
Some(axum::Extension(m)) => match m.body {
Some(bytes) => format!("captured:{}", bytes.len()),
None => "absent".to_string(),
},
None => "no-material".to_string(),
}
}
fn signed_post(path: &str, body: &[u8]) -> Request {
let kp = Keypair::generate();
let signed = gitlawb_core::http_sig::sign_request(&kp, "POST", path, body);
Request::builder()
.method("POST")
.uri(path)
.header("content-digest", signed.content_digest)
.header("signature-input", signed.signature_input)
.header("signature", signed.signature)
.body(axum::body::Body::from(body.to_vec()))
.unwrap()
}
/// The body handle travels only on a route that asked for it.
///
/// Both arms drive a genuinely signed request through `require_signature`;
/// the only difference is the [`PersistsSignedBody`] marker layer. A
/// middleware that captured unconditionally would make both arms read
/// `captured`, and one that never read the marker would make both read
/// `absent`, so the pair pins the decision rather than the mechanism.
#[tokio::test]
async fn the_body_is_carried_only_when_the_route_marks_itself_as_persisting_it() {
let body = br#"{"state":"success"}"#;
// The marker layer is added AFTER the middleware layer, which is what
// makes it the outer of the two and therefore the one that runs first.
// This is the same relative position `build_router` uses on the status
// write group, where the marker sits outside `add_auth_layers`.
let marked = Router::new()
.route("/", axum::routing::post(report_body))
.layer(middleware::from_fn(require_signature))
.layer(axum::Extension(PersistsSignedBody));
let resp = marked.oneshot(signed_post("/", body)).await.unwrap();
assert_eq!(resp.status(), StatusCode::OK);
let seen = axum::body::to_bytes(resp.into_body(), 2048).await.unwrap();
assert_eq!(
String::from_utf8(seen.to_vec()).unwrap(),
format!("captured:{}", body.len()),
"a route carrying the persist marker must receive the buffered body"
);
let unmarked = Router::new()
.route("/", axum::routing::post(report_body))
.layer(middleware::from_fn(require_signature));
let resp = unmarked.oneshot(signed_post("/", body)).await.unwrap();
assert_eq!(resp.status(), StatusCode::OK);
let seen = axum::body::to_bytes(resp.into_body(), 2048).await.unwrap();
assert_eq!(
String::from_utf8(seen.to_vec()).unwrap(),
"absent",
"a signed route that never persists the body must not be handed a handle to it"
);
}
/// The marker is read from the request the middleware was given, and the
/// request it hands downstream still carries it. Both halves matter: the
/// first is the capture decision, the second is that rebuilding the request
/// from its parts does not drop extensions a later layer may want.
#[tokio::test]
async fn the_marker_survives_the_request_rebuild() {
async fn report_marker(marker: Option<axum::Extension<PersistsSignedBody>>) -> String {
match marker {
Some(_) => "marked".to_string(),
None => "unmarked".to_string(),
}
}
let app = Router::new()
.route("/", axum::routing::post(report_marker))
.layer(middleware::from_fn(require_signature))
.layer(axum::Extension(PersistsSignedBody));
let resp = app.oneshot(signed_post("/", b"{}")).await.unwrap();
assert_eq!(resp.status(), StatusCode::OK);
let seen = axum::body::to_bytes(resp.into_body(), 2048).await.unwrap();
assert_eq!(
String::from_utf8(seen.to_vec()).unwrap(),
"marked",
"the marker must reach the handler, so the middleware saw it too"
);
}
#[tokio::test]
async fn require_ucan_chain_no_header_passes_through() {
let state = make_test_state(Keypair::generate().did());
+5 -48
View File
@@ -1,45 +1,18 @@
use clap::{Parser, Subcommand};
use clap::Parser;
use std::path::PathBuf;
/// Optional admin subcommands. When none is given, the binary runs the node
/// daemon as before — the default (no-subcommand) startup path is unchanged.
#[derive(Subcommand, Debug, Clone)]
pub enum Command {
/// Dry-run (default) or, with --execute, delete empty spam-burst repos owned
/// by the known burst DID. Never touches the hard-excluded DIDs, and verifies
/// each repo is empty (zero git refs) per repo before selecting it.
PurgeSpam {
/// Actually delete the candidates. Omit for a dry-run that prints the
/// candidate list and deletes nothing.
#[arg(long, default_value_t = false)]
execute: bool,
},
}
#[derive(Parser, Debug, Clone)]
#[command(name = "gitlawb-node", about = "gitlawb node daemon", version)]
pub struct Config {
/// Admin subcommand to run instead of the node daemon. Absent = run the node.
#[command(subcommand)]
pub command: Option<Command>,
/// Directory where bare git repositories are stored. `global` so it can
/// follow a subcommand (e.g. `gitlawb-node purge-spam --repos-dir …`).
#[arg(
long,
env = "GITLAWB_REPOS_DIR",
default_value = "./data/repos",
global = true
)]
/// Directory where bare git repositories are stored
#[arg(long, env = "GITLAWB_REPOS_DIR", default_value = "./data/repos")]
pub repos_dir: PathBuf,
/// PostgreSQL connection URL (Supabase or any Postgres instance). `global` so
/// admin subcommands accept it in either position.
/// PostgreSQL connection URL (Supabase or any Postgres instance)
#[arg(
long,
env = "DATABASE_URL",
default_value = "postgresql://localhost/gitlawb",
global = true
default_value = "postgresql://localhost/gitlawb"
)]
pub database_url: String,
@@ -221,22 +194,6 @@ pub struct Config {
)]
pub db_max_connections: u32,
/// Maximum connections in the dedicated advisory-lock pool, separate from
/// the main pool above. Every in-flight repo write (git-receive-pack, fork,
/// merge) pins one connection here for its whole lifetime — the write, its
/// metadata/fan-out tail, and the bounded post-write archive upload — so
/// size it to the expected peak number of concurrent distinct-repo writers,
/// NOT small. Keeping it separate from GITLAWB_DB_MAX_CONNECTIONS is what
/// stops a push burst from starving normal request handlers. Keep
/// (main pool + lock pool) within the database server's max_connections.
#[arg(
long,
env = "GITLAWB_DB_LOCK_POOL_MAX_CONNECTIONS",
default_value_t = 32,
value_parser = clap::value_parser!(u32).range(1..)
)]
pub db_lock_pool_max_connections: u32,
/// Maximum time a request waits for a pool connection before failing with
/// 503, in seconds. Bounds queueing when the database is slow or down.
#[arg(
File diff suppressed because it is too large. Load diff
+6 -27
View File
@@ -35,6 +35,9 @@ pub enum AppError {
#[error("invalid request: {0}")]
BadRequest(String),
#[error("too many requests: {0}")]
TooManyRequests(String),
#[error("incomplete: {0}")]
Incomplete(String),
@@ -44,9 +47,6 @@ pub enum AppError {
#[error("git service timed out: {0}")]
Timeout(String),
#[error("service unavailable: {0}")]
Unavailable(String),
#[error("database error: {0}")]
Db(#[from] sqlx::Error),
@@ -132,6 +132,9 @@ impl IntoResponse for AppError {
// IcaptchaProofRequired is handled above (it carries extra headers/fields).
AppError::IcaptchaProofRequired { .. } => unreachable!("handled before this match"),
AppError::BadRequest(msg) => (StatusCode::BAD_REQUEST, "bad_request", msg.clone()),
AppError::TooManyRequests(msg) => {
(StatusCode::TOO_MANY_REQUESTS, "rate_limited", msg.clone())
}
AppError::Incomplete(msg) => {
(StatusCode::UNPROCESSABLE_ENTITY, "incomplete", msg.clone())
}
@@ -139,14 +142,6 @@ impl IntoResponse for AppError {
// 504, distinct from the 500 git_error and from the read-gate's 404 /
// the auth 401, so the client can tell a deadline from a failure.
AppError::Timeout(msg) => (StatusCode::GATEWAY_TIMEOUT, "git_timeout", msg.clone()),
// 503, retryable: a transient resource contention (e.g. the repo lock
// pool is pinned by a write burst or a purge holds the key), distinct
// from the 500 git_error so the client knows to back off and retry.
AppError::Unavailable(msg) => (
StatusCode::SERVICE_UNAVAILABLE,
"service_unavailable",
msg.clone(),
),
AppError::Db(e) if db_unavailable(e) => (
StatusCode::SERVICE_UNAVAILABLE,
DB_UNAVAILABLE_CODE,
@@ -187,20 +182,4 @@ mod tests {
StatusCode::INTERNAL_SERVER_ERROR
);
}
#[test]
fn unavailable_maps_to_503_distinct_from_git_500() {
// The create-repo lock-acquire failure returns Unavailable so a client sees
// a retryable 503, not a terminal 500 git_error.
assert_eq!(
AppError::Unavailable("lock pool pinned".into())
.into_response()
.status(),
StatusCode::SERVICE_UNAVAILABLE
);
assert_eq!(
AppError::Git("x".into()).into_response().status(),
StatusCode::INTERNAL_SERVER_ERROR
);
}
}
-45
View File
@@ -61,51 +61,6 @@ pub fn create_issue(repo_path: &Path, issue_id: &str, json: &str) -> Result<()>
Ok(())
}
/// Delete an issue ref. Used to roll back a locally written issue whose
/// durable upload failed, so a retry does not duplicate it. The dangling
/// blob object is harmless.
pub fn delete_issue_ref(repo_path: &Path, issue_id: &str) -> Result<()> {
let ref_name = format!("refs/gitlawb/issues/{issue_id}");
let output = Command::new("git")
.args(["update-ref", "-d", &ref_name])
.current_dir(repo_path)
.output()
.context("failed to run git update-ref -d")?;
if !output.status.success() {
let stderr = String::from_utf8_lossy(&output.stderr);
anyhow::bail!("git update-ref -d failed: {stderr}");
}
Ok(())
}
/// Read the object id an issue ref currently points to (resolving an 8-char
/// prefix), returning `(full_id, oid)`. Returns None if no such issue exists.
/// Used to snapshot an issue ref's pre-mutation state so a failed durable
/// upload can restore it, mirroring `create_issue`'s rollback.
pub fn issue_ref_oid(repo_path: &Path, issue_id: &str) -> Result<Option<(String, String)>> {
let full_id = match resolve_issue_id(repo_path, issue_id)? {
Some(id) => id,
None => return Ok(None),
};
let ref_name = format!("refs/gitlawb/issues/{full_id}");
let oid = match crate::git::store::ref_oid(repo_path, &ref_name)? {
Some(oid) => oid,
None => return Ok(None),
};
Ok(Some((full_id, oid)))
}
/// Reset an issue ref back to a captured object id — rolls back a local
/// mutation (e.g. a close) whose durable upload failed, so a local-fast-path
/// read does not serve the un-uploaded state. `full_id` must be the resolved
/// UUID (from [`issue_ref_oid`]). The now-dangling object is harmless.
pub fn restore_issue_ref(repo_path: &Path, full_id: &str, oid: &str) -> Result<()> {
let ref_name = format!("refs/gitlawb/issues/{full_id}");
crate::git::store::set_ref(repo_path, &ref_name, oid)
}
/// List all issue refs and return their JSON content.
pub fn list_issues(repo_path: &Path) -> Result<Vec<String>> {
// List all refs under refs/gitlawb/issues/
File diff suppressed because it is too large. Load diff
@@ -1,431 +0,0 @@
//! Read-path download coordination (KTD-3) for [`RepoStore`].
//!
//! The read path's cache-miss handling lives here: the per-repo async mutex
//! that serializes concurrent readers of the same repo, the holder's fetch +
//! extract + under-lock publish, and the RAII temp-dir cleanup. `acquire` and
//! `acquire_fresh` stay in the parent module and call into `download_published`.
//! These methods reach the parent's private fields (`download_locks`) and
//! private methods (`try_lock_repo`) directly, because a child module can see
//! its parent's private items.
use super::*;
/// Outcome of a coordinated read-path download (KTD-3).
pub(super) enum DownloadOutcome {
/// The local dir is present: this reader downloaded and published it under
/// the advisory lock, or a concurrent reader did and this one served it.
Published,
/// The download was discarded without publishing: the advisory lock was
/// contended (a live writer or purge holds it), the lock pool errored, or
/// the archive vanished under the lock (purged mid-download). The caller
/// degrades to its missing-path or serve-local-copy outcome.
Skipped,
}
/// RAII cleanup for a read-path download temp dir (finding 1). Its `Drop`
/// removes the dir (best effort), so a handler future cancelled mid-download
/// (axum drops it on client disconnect, the same hazard `RepoWriteGuard` /
/// `RepoLockGuard` Drop impls exist for) cannot leak a fully-populated repo
/// dir. `disarm` forgets the dir once the publish rename has consumed it.
struct TempDownloadDir {
path: PathBuf,
armed: bool,
}
impl TempDownloadDir {
fn new(path: PathBuf) -> Self {
Self { path, armed: true }
}
fn path(&self) -> &Path {
&self.path
}
/// Disarm: the temp has been consumed (renamed into place), so `Drop` must
/// not try to remove it.
fn disarm(mut self) {
self.armed = false;
}
}
impl Drop for TempDownloadDir {
fn drop(&mut self) {
if self.armed {
let _ = std::fs::remove_dir_all(&self.path);
}
}
}
/// RAII removal of a download-coordination map entry (finding 4). Its `Drop`
/// removes the entry from `download_locks` whenever the map still holds THIS
/// Arc (ptr_eq-guarded so a newer entry inserted after an earlier removal is
/// never clobbered). Removal is guaranteed on every path (a normal return or
/// `?` drops it in the holder; once a download is in flight it travels with
/// the spawned task and drops at settle, covering holder timeout and a
/// handler future cancelled mid cold-read, which axum triggers on client
/// disconnect), which is what keeps the map from growing per arbitrary
/// requested name: the pre-fix explicit removals ran only at the labelled
/// return points, which a cancelled future skipped, leaking the entry. The
/// outer map is a std::sync::Mutex so
/// this Drop can prune it synchronously (Drop cannot be async); it is only ever
/// held briefly for a get/insert/remove, never across an await.
struct DownloadEntryGuard {
locks: Arc<std::sync::Mutex<HashMap<String, Arc<Mutex<()>>>>>,
key: String,
entry: Arc<Mutex<()>>,
}
impl Drop for DownloadEntryGuard {
fn drop(&mut self) {
// A poisoned map is still safe to prune — recover the guard and remove.
let mut map = self.locks.lock().unwrap_or_else(|p| p.into_inner());
if map
.get(&self.key)
.is_some_and(|cur| Arc::ptr_eq(cur, &self.entry))
{
map.remove(&self.key);
}
}
}
impl RepoStore {
/// Coordinated read-path download (KTD-3). Serializes concurrent readers
/// of the same repo on a per-repo async mutex: the first one in becomes
/// the holder and runs [`download_and_publish`](Self::download_and_publish);
/// a contended reader awaits the holder, re-checks the local dir on wake,
/// and serves what the holder published instead of re-downloading. Only
/// callers that already confirmed the archive exists reach this, and the
/// map entry is removed on completion, so the map cannot grow per
/// arbitrary requested name.
pub(super) async fn download_published(
&self,
owner_did: &str,
repo_name: &str,
owner_slug: &str,
local_path: &Path,
store: &Arc<dyn ObjectStore>,
local_existed_at_entry: bool,
) -> Result<DownloadOutcome> {
let map_key = format!("{owner_slug}/{repo_name}");
let entry = {
let mut map = self.download_locks.lock().unwrap();
Arc::clone(
map.entry(map_key.clone())
.or_insert_with(|| Arc::new(Mutex::new(()))),
)
};
let held = match Arc::clone(&entry).try_lock_owned() {
Ok(g) => g,
Err(_) => {
// Bounded coalescing wait: once a holder's download is in
// flight, the spawned task owns this lock (and the map entry)
// until it settles, whatever happens to the holder (timeout or
// cancellation), so an unbounded lock().await here would park
// this reader for as long as a stall lasts. On elapse, report
// the same timeout error the holder would. Do NOT touch the
// map entry on this path: pruning it while the download is in
// flight would let the next arrival start a duplicate download.
let waited = tokio::time::timeout(
self.release_upload_timeout,
Arc::clone(&entry).lock_owned(),
)
.await;
let g = match waited {
Ok(g) => g,
Err(_) => {
if local_path.exists() {
// A concurrent reader published while we waited.
return Ok(DownloadOutcome::Published);
}
return Err(anyhow::anyhow!("read download timed out"));
}
};
if local_path.exists() {
// A concurrent reader published while we waited — serve it.
// The holder's own guard prunes the map entry on its exit.
return Ok(DownloadOutcome::Published);
}
// The holder degraded without publishing; this reader takes over.
// The prior holder removed the map entry on its way out, so
// re-register THIS entry before downloading, so readers arriving
// now coalesce onto it instead of starting a duplicate download
// (finding 3). Narrows the duplicate-download window; publishes
// stay serialized by the advisory lock regardless.
{
let mut map = self.download_locks.lock().unwrap();
map.entry(map_key.clone())
.or_insert_with(|| Arc::clone(&entry));
}
g
}
};
// RAII: remove the map entry when this holder's work is done (finding
// 4). Created only once THIS reader holds the per-repo lock, so a
// parked or timed-out waiter never prunes the entry out from under a
// live holder; its Drop is ptr_eq-guarded, so it never clobbers a
// newer entry. Ownership moves into download_and_publish and from
// there into the spawned download task itself, so on EVERY settle path
// (success, download error, holder timeout, holder cancellation) the
// entry stays registered and the lock stays held until the in-flight
// download settles: waiters queue on the ONE download instead of each
// timeout or cancel wave starting a fresh full download of the same
// object.
let entry_guard = DownloadEntryGuard {
locks: Arc::clone(&self.download_locks),
key: map_key,
entry: Arc::clone(&entry),
};
self.download_and_publish(
owner_did,
repo_name,
owner_slug,
local_path,
store,
local_existed_at_entry,
entry_guard,
held,
)
.await
}
/// The holder's half of the coordinated download: fetch and extract the
/// archive into a temp sibling with NO advisory lock held (the network
/// phase must never pin a lock-pool connection — a cold-read burst across
/// distinct repos must not drain the writer-sized pool), then take the
/// per-repo advisory lock only around the publish: re-check the archive
/// still exists under the lock (a purge deletes it under this same lock,
/// so a post-purge downloader discards rather than resurrecting), swap
/// the temp copy into place, release. Advisory contention or a lock-pool
/// error discards the temp copy and degrades — no blocking lock or pool
/// wait exists anywhere on the read path.
///
/// `local_existed_at_entry` selects the stale-swap guard (finding 2): a cold
/// `acquire` passes `false`, an `acquire_fresh` refresh passes `true`. Under
/// the lock, before the swap, we detect a writer that published a fresher
/// copy during our unlocked download window and serve theirs instead of
/// clobbering it with our now-stale temp.
///
/// `entry_guard` and `held` are this holder's per-repo coordination (the
/// download_locks map entry and the locked per-repo mutex). Both move into
/// the spawned download task, which returns them through its JoinHandle on
/// success; on any other settle path (download error, task panic, holder
/// timeout, holder cancellation) they drop when the task settles, so
/// waiters keep queueing on the SAME in-flight download until it settles
/// rather than each starting a duplicate one.
#[allow(clippy::too_many_arguments)]
async fn download_and_publish(
&self,
owner_did: &str,
repo_name: &str,
owner_slug: &str,
local_path: &Path,
store: &Arc<dyn ObjectStore>,
local_existed_at_entry: bool,
entry_guard: DownloadEntryGuard,
held: tokio::sync::OwnedMutexGuard<()>,
) -> Result<DownloadOutcome> {
let parent = local_path.parent().context("repo path has no parent")?;
std::fs::create_dir_all(parent).context("creating repo parent dir")?;
let file_name = local_path
.file_name()
.context("repo path has no file name")?
.to_string_lossy();
// Best-effort sweep of leftover temp siblings from a prior download whose
// cleanup never ran (a process crash or kill mid-download, where no Drop
// and no owning task survive to remove the temp). Serialized per
// repo by the download mutex, so no concurrent same-repo download owns a
// live temp here; scoped to THIS repo's prefix, so other repos are
// untouched.
let tmp_prefix = format!(".{file_name}.tmp-download.");
if let Ok(entries) = std::fs::read_dir(parent) {
for entry in entries.flatten() {
if entry.file_name().to_string_lossy().starts_with(&tmp_prefix) {
let _ = std::fs::remove_dir_all(entry.path());
}
}
}
// Stale-swap signal (finding 2): capture the local dir's mtime BEFORE the
// unlocked download. Under the lock, a changed mtime (acquire_fresh,
// refreshing over an existing copy) or the dir's mere appearance (cold
// acquire) means a writer published a fresher copy during our download
// window, so we serve theirs rather than swap our stale temp over it.
let entry_mtime = std::fs::metadata(local_path)
.and_then(|m| m.modified())
.ok();
// Unique per-download temp target (same parent as local_path so the
// publish rename stays on one filesystem); mirrors the extract temp
// naming in tigris.rs. Wrapped in an RAII guard so a handler future
// cancelled mid-download (axum drops it on client disconnect) cannot
// leak the populated temp dir (the same hazard the guard Drop impls
// below exist for, finding 1).
let temp = TempDownloadDir::new(parent.join(format!(
".{file_name}.tmp-download.{}",
uuid::Uuid::new_v4()
)));
// Ownership of the download AND the temp-dir guard moves into a spawned
// task, so timeout and handler cancellation share ONE cleanup path
// (finding F3 and its cancellation twin). `store.download` runs its tar
// extraction in `spawn_blocking`, and neither a tokio timeout nor a
// dropped handler future cancels a spawn_blocking task: pre-fix,
// dropping the download future (a timed-out await, or axum dropping the
// whole handler on client disconnect) detached the extraction, whose
// later rename resurrected `temp.path()` AFTER the guard's Drop had
// removed it, leaving an orphan only a later same-repo download sweeps.
// The spawned task is never dropped mid-flight, so the download future
// always runs to settle. On success it returns the still-armed guard
// through the JoinHandle: the caller consumes it and publishes below.
// If the caller timed out or was cancelled, the JoinHandle is gone, so
// the runtime drops the returned guard when the task settles and its
// Drop removes the temp dir; cleanup is tied to the extraction's
// completion, never racing it (this subsumes the previous detached
// janitor). A download error drops the guard inside the task with the
// same at-settle timing.
let store_dl = Arc::clone(store);
let (dl_owner, dl_repo, dl_target) = (
owner_slug.to_string(),
repo_name.to_string(),
temp.path().to_path_buf(),
);
// The per-repo coordination (`held`, `entry_guard`) moves into the task
// alongside the temp guard: whatever happens to THIS caller (timeout
// below, or a handler future cancelled by axum on client disconnect),
// the lock stays held and the map entry stays registered until the
// in-flight download settles, so waiters and new arrivals queue on the
// ONE download instead of starting duplicates. On success the tuple
// returns through the JoinHandle: the caller publishes under `held` as
// before. If the caller timed out or was cancelled, the runtime drops
// the returned tuple when the task settles, in field order: `temp`
// (dir removed), then `held` (lock released), then `entry_guard` (map
// entry pruned), the same unlock-before-prune order as a normal
// return.
let dl_task = tokio::spawn(async move {
// The guards travel as one `(held, entry_guard)` tuple so that
// wherever it drops (here on error, in the runtime at settle after
// a caller timeout or cancellation, or in the caller after a
// normal publish), its field order gives unlock-before-prune.
match store_dl.download(&dl_owner, &dl_repo, &dl_target).await {
Ok(()) => Ok((temp, (held, entry_guard))),
// The download settled, so cleanup is safe. Explicit drops pin
// the order (async-block captures have no guaranteed one):
// temp removed, lock released, map entry pruned last.
Err(e) => {
drop(temp);
drop(held);
drop(entry_guard);
Err(e)
}
}
});
// Bound the wait, not the work: a stalled GET would park this reader
// indefinitely. On timeout the caller gets the same error as before,
// but the coordination does NOT unwind: it lives in the task (see
// above) and is released only at settle. Waiters therefore stay queued
// on the SAME in-flight download (each bounded by its own timeout in
// download_published) instead of each timeout wave starting a fresh
// full download of the same object; the task keeps running and cleans
// up its temp when the download settles.
// `_coordination` is `(held, entry_guard)`, kept alive to the end of
// this function exactly as the separate guards were before: the
// publish below still runs with the per-repo lock held, and the map
// entry is pruned only on exit (unlock first, prune second, per the
// tuple's field order).
let (temp, _coordination) =
match tokio::time::timeout(self.release_upload_timeout, dl_task).await {
Ok(Ok(Ok(returned))) => returned,
Ok(Ok(Err(e))) => return Err(e),
// The task panicked; its unwind dropped the guards, removing
// the dir, releasing the lock, and pruning the map entry.
Ok(Err(join_err)) => {
return Err(anyhow::Error::from(join_err).context("read download task failed"))
}
Err(_) => {
warn!(repo = %repo_name, timeout_secs = self.release_upload_timeout.as_secs(),
"read download timed out — discarding");
// Dropping the JoinHandle detaches the task; the runtime
// frees temp dir, lock, and map entry at settle (see the
// spawn comment above).
return Err(anyhow::anyhow!("read download timed out"));
}
};
let guard = match self.try_lock_repo(owner_did, repo_name).await {
Ok(Some(g)) => g,
Ok(None) => {
debug!(repo = %repo_name,
"read download discarded — repo lock held by a live writer or purge");
return Ok(DownloadOutcome::Skipped);
}
Err(e) => {
warn!(repo = %repo_name, err = %e,
"read download discarded — could not acquire repo lock");
return Ok(DownloadOutcome::Skipped);
}
};
// Re-check under the lock: the archive gone here means a purge won the
// key mid-download — discard, never publish (the read-side twin of the
// upload paths' under-lock dir re-check). Bounded so a stalled HEAD
// cannot pin the advisory lock + its lock-pool connection. Both a
// timeout and an exists() error collapse to "not present" and take the
// discard arm: fail closed rather than publish unconfirmed state.
let archive_present = matches!(
tokio::time::timeout(
self.release_upload_timeout,
store.exists(owner_slug, repo_name),
)
.await,
Ok(Ok(true))
);
if !archive_present {
warn!(repo = %repo_name,
"read download discarded — archive gone under lock (purged?)");
guard.release().await;
return Ok(DownloadOutcome::Skipped);
}
// Stale-swap guard (finding 2): a writer that locked, published a fresher
// copy, and released during our unlocked download window must not have
// its work clobbered by our now-stale temp. The only concurrent mutator
// of local_path is a writer or a purge (readers of the same repo
// serialize on the download mutex), so the signals below unambiguously
// mean "a writer published here."
let writer_published = match (local_existed_at_entry, entry_mtime) {
// acquire_fresh over an existing copy: presence is always true, so it
// cannot signal a writer; a CHANGED mtime can (the writer replaced
// the dir via rename, minting a fresh mtime).
(true, Some(entry_m)) => std::fs::metadata(local_path)
.and_then(|m| m.modified())
.map(|now| now != entry_m)
.unwrap_or(false),
// Cold acquire, or acquire_fresh whose dir was absent at entry: the
// dir being present now means a writer published it during the window.
_ => local_path.exists(),
};
if writer_published {
warn!(repo = %repo_name,
"read download discarded: a writer published a fresher copy during the download window");
guard.release().await;
// temp guard's Drop removes the stale temp; serve the fresher local copy.
return Ok(DownloadOutcome::Published);
}
let swapped = (|| -> std::io::Result<()> {
if local_path.exists() {
std::fs::remove_dir_all(local_path)?;
}
std::fs::rename(temp.path(), local_path)
})();
guard.release().await;
match swapped {
Ok(()) => {
temp.disarm(); // the rename consumed the temp; nothing to clean up
Ok(DownloadOutcome::Published)
}
// temp guard's Drop removes the temp if the rename left it behind.
Err(e) => Err(e).context("publishing downloaded repo into place"),
}
}
}
+121
View File
@@ -86,6 +86,127 @@ pub async fn receive_pack(
.body(Body::from(output))?)
}
// ── receive-pack report-status ────────────────────────────────────────────
//
// PORTED VERBATIM from PR #384 (`fix/issue-26-split-1-durable-post-receive`),
// which introduced these two functions for the same reason: `receive_pack`
// forwards git's protocol response untouched, and a zero exit says nothing
// about whether any individual ref was taken. Kept byte-for-byte identical so
// the two branches converge on one parser rather than growing two that disagree
// about what "accepted" means; whichever lands second should drop its copy.
/// Parse the git-receive-pack report-status output to determine per-ref
/// success/failure. Returns `(unpack_ok, per_ref_results)` where
/// `per_ref_results` is a list of `(ref_name, is_ok)`.
///
/// The report-status format (after the sideband framing) is:
/// ```text
/// unpack ok\n (or: unpack fail\n)
/// ok <refname>\n (per successful ref)
/// ng <refname> <reason>\n (per rejected ref)
/// \n (empty line terminates)
/// ```
///
/// Returns `None` if the output cannot be parsed (e.g. the client did
/// not request report-status, or the output is truncated). In that
/// case the caller should treat all refs as uncertain.
pub fn parse_report_status(output: &[u8]) -> Option<(bool, Vec<(String, bool)>)> {
let text = std::str::from_utf8(output).ok()?;
// Strip sideband framing: each line starts with a pkt-line length
// prefix and a channel byte (1 = stdout, 2 = stderr). The actual
// data starts after the first `0000` flush packet or after we
// strip sideband bytes.
let stripped = strip_sideband(text)?;
// DIVERGES FROM #384 by these three lines, and deliberately.
//
// When the client asked for `side-band-64k` — which `git push` over smart
// HTTP does — receive-pack multiplexes the report onto band 1, and what
// travels on that band is itself a pkt-line stream. One pass of
// `strip_sideband` peels the outer framing and leaves `000eunpack ok\n...`,
// whose first line is not `unpack ok`, so the parse below returns None and
// every ref falls back to inconclusive: the parser would answer "cannot
// tell" for exactly the pushes it exists to classify. A second pass peels
// the inner layer. It is a no-op on the single-wrapped shape, because
// plain report text does not begin with four hex digits and
// `strip_sideband` declines it.
let stripped = strip_sideband(&stripped).unwrap_or(stripped);
let lines: Vec<&str> = stripped.lines().collect();
if lines.is_empty() {
return None;
}
// First non-empty line is "unpack ok" or "unpack fail".
let unpack_line = lines.iter().find(|l| !l.is_empty())?;
let unpack_ok = if unpack_line.starts_with("unpack ok") {
true
} else if unpack_line.starts_with("unpack fail") {
false
} else {
return None;
};
let mut results = Vec::new();
for line in &lines[1..] {
let line = line.trim();
if line.is_empty() {
break;
}
if let Some(rest) = line.strip_prefix("ok ") {
results.push((rest.to_string(), true));
} else if let Some(rest) = line.strip_prefix("ng ") {
// "ng <refname> <reason>" — skip the reason
let ref_name = rest.split_whitespace().next()?.to_string();
results.push((ref_name, false));
}
}
Some((unpack_ok, results))
}
/// Strip git sideband framing from a pkt-line encoded output.
/// Sideband-encoded lines start with a 4-hex-digit length, then a
/// channel byte (0x01=stdout, 0x02=stderr), then payload. Returns
/// the decoded payload lines concatenated, or `None` if the framing
/// is malformed.
fn strip_sideband(text: &str) -> Option<String> {
let mut output = String::new();
let mut pos = 0;
let bytes = text.as_bytes();
loop {
if pos + 4 > bytes.len() {
break;
}
let len_str = std::str::from_utf8(&bytes[pos..pos + 4]).ok()?;
let len = usize::from_str_radix(len_str, 16).ok()?;
if len == 0 {
// Flush packet — end of sideband stream
break;
}
if len < 4 || pos + len > bytes.len() {
break;
}
let pkt_data = std::str::from_utf8(&bytes[pos + 4..pos + len]).ok()?;
pos += len;
// Sideband: first byte is channel (1=stdout, 2=stderr)
if let Some(payload) = pkt_data.strip_prefix('\x01') {
output.push_str(payload);
} else if pkt_data.starts_with('\x02') {
// stderr — skip (git error messages)
} else {
// Not sideband encoded — pass through
output.push_str(pkt_data);
}
}
if output.is_empty() {
None
} else {
Some(output)
}
}
/// Sends SIGTERM to a child's whole process group on drop, unless disarmed first.
///
/// A served `git upload-pack`/`receive-pack` forks helpers such as `pack-objects`.
+1 -266
View File
@@ -63,105 +63,6 @@ pub fn list_refs(repo_path: &Path) -> Result<Vec<(String, String)>> {
Ok(refs)
}
/// Read the object id a single ref currently points to. Returns None if the
/// ref does not exist. Used to snapshot a ref's pre-write state so a failed
/// durable upload can roll the local write back.
pub fn ref_oid(repo_path: &Path, ref_name: &str) -> Result<Option<String>> {
let output = Command::new("git")
.args(["rev-parse", "--verify", ref_name])
.current_dir(repo_path)
.output()
.context("failed to run git rev-parse")?;
if !output.status.success() {
return Ok(None);
}
let oid = String::from_utf8_lossy(&output.stdout).trim().to_string();
if oid.is_empty() {
return Ok(None);
}
Ok(Some(oid))
}
/// Force a single ref back to a captured object id — rolls back a local write
/// (e.g. a merge commit) whose durable upload failed, so a local-fast-path
/// read does not serve the un-uploaded state. The now-dangling objects are
/// harmless.
pub fn set_ref(repo_path: &Path, ref_name: &str, oid: &str) -> Result<()> {
let output = Command::new("git")
.args(["update-ref", ref_name, oid])
.current_dir(repo_path)
.output()
.context("failed to run git update-ref")?;
if !output.status.success() {
let stderr = String::from_utf8_lossy(&output.stderr);
bail!("git update-ref failed: {stderr}");
}
Ok(())
}
/// Restore a repo's refs to a previously captured snapshot: reset every
/// snapshot ref to its captured object id, and delete any ref that exists now
/// but was absent from the snapshot (created by the intervening write). Used
/// to roll back a receive-pack whose durable upload failed, so the next
/// local-fast-path read does not serve refs that never reached object storage.
/// Best-effort per ref: an individual failure is collected and reported, but
/// the remaining refs are still attempted.
pub fn restore_refs(repo_path: &Path, snapshot: &[(String, String)]) -> Result<()> {
use std::collections::HashSet;
let mut errors: Vec<String> = Vec::new();
// Reset every snapshot ref to its captured oid (recreates refs the write
// deleted, and rewinds refs the write advanced).
for (ref_name, oid) in snapshot {
if let Err(e) = set_ref(repo_path, ref_name, oid) {
errors.push(format!("{ref_name}: {e}"));
}
}
// Delete refs that exist now but were not in the snapshot (created by the
// write being rolled back). A failure to LIST the current refs is collected
// like any per-ref failure, never returned early: the snapshot-reset half
// above already ran, and bailing here would discard its collected errors.
let snapshot_names: HashSet<&str> = snapshot.iter().map(|(r, _)| r.as_str()).collect();
match list_refs(repo_path) {
Ok(current) => {
for (ref_name, _) in &current {
if !snapshot_names.contains(ref_name.as_str()) {
match Command::new("git")
.args(["update-ref", "-d", ref_name])
.current_dir(repo_path)
.output()
{
Ok(output) if output.status.success() => {}
Ok(output) => {
let stderr = String::from_utf8_lossy(&output.stderr);
errors.push(format!("{ref_name} (delete): {stderr}"));
}
Err(e) => {
errors.push(format!(
"{ref_name} (delete): failed to run git update-ref -d: {e}"
));
}
}
}
}
}
Err(e) => {
errors.push(format!("(extra-ref sweep) listing current refs: {e}"));
}
}
if !errors.is_empty() {
bail!(
"failed to restore {} ref(s): {}",
errors.len(),
errors.join("; ")
);
}
Ok(())
}
/// Read the current HEAD commit hash of a repository.
/// Returns None if the repo is empty (no commits yet).
pub fn head_commit(repo_path: &Path) -> Result<Option<String>> {
@@ -554,176 +455,10 @@ pub fn repo_disk_path(repos_dir: &Path, owner_did: &str, repo_name: &str) -> Pat
#[cfg(test)]
mod tests {
use super::{branch_diff_names, list_refs, restore_refs};
use super::branch_diff_names;
use std::path::Path;
use std::process::Command;
fn run_git(dir: &Path, args: &[&str]) -> String {
let out = Command::new("git")
.args(args)
.current_dir(dir)
.output()
.unwrap();
assert!(
out.status.success(),
"git {args:?} failed: {}",
String::from_utf8_lossy(&out.stderr)
);
String::from_utf8_lossy(&out.stdout).trim().to_string()
}
/// Bare repo with main at c2 and keeper at c2, whose "pre-push" state was
/// main and keeper both at c1. Returns (workdir, baredir, bare_path, c1, c2).
fn advanced_bare_repo() -> (
tempfile::TempDir,
tempfile::TempDir,
std::path::PathBuf,
String,
String,
) {
let work = tempfile::TempDir::new().unwrap();
run_git(work.path(), &["init", "-q", "-b", "main", "."]);
run_git(work.path(), &["config", "user.email", "t@t"]);
run_git(work.path(), &["config", "user.name", "t"]);
std::fs::write(work.path().join("f.txt"), "one").unwrap();
run_git(work.path(), &["add", "f.txt"]);
run_git(work.path(), &["commit", "-qm", "c1"]);
let c1 = run_git(work.path(), &["rev-parse", "HEAD"]);
std::fs::write(work.path().join("f.txt"), "two").unwrap();
run_git(work.path(), &["add", "f.txt"]);
run_git(work.path(), &["commit", "-qm", "c2"]);
let c2 = run_git(work.path(), &["rev-parse", "HEAD"]);
let dir = tempfile::TempDir::new().unwrap();
let bare = dir.path().join("repo.git");
let out = Command::new("git")
.args([
"clone",
"--bare",
"-q",
&work.path().to_string_lossy(),
&bare.to_string_lossy(),
])
.output()
.unwrap();
assert!(out.status.success(), "clone --bare failed");
// "The push" advanced main to c2 (already there from the clone), moved
// keeper c1 -> c2, and created feature at c2.
run_git(&bare, &["update-ref", "refs/heads/keeper", &c2]);
run_git(&bare, &["update-ref", "refs/heads/feature", &c2]);
(work, dir, bare, c1, c2)
}
fn sorted_refs(bare: &Path) -> Vec<(String, String)> {
let mut refs = list_refs(bare).unwrap();
refs.sort();
refs
}
/// A single unrestorable snapshot entry (invalid oid, `set_ref` fails) must
/// not abort the restore: every OTHER snapshot ref is still reset and the
/// push-created extra ref is still deleted, and the fn returns Err carrying
/// the failure. RED with the reset loop reverted to `set_ref(..)?`: the
/// broken entry is first in the snapshot, so an early return leaves keeper
/// at c2 and feature alive.
#[test]
fn restore_refs_partial_failure_still_resets_rest_and_deletes_extras() {
let (_work, _dir, bare, c1, c2) = advanced_bare_repo();
let snapshot = vec![
(
"refs/heads/broken".to_string(),
"not-a-valid-oid".to_string(),
),
("refs/heads/main".to_string(), c1.clone()),
("refs/heads/keeper".to_string(), c1.clone()),
];
let err = restore_refs(&bare, &snapshot).unwrap_err();
let msg = format!("{err:#}");
assert!(
msg.contains("refs/heads/broken"),
"the aggregated error names the failed ref: {msg}"
);
let refs = sorted_refs(&bare);
assert_eq!(
refs,
vec![
("refs/heads/keeper".to_string(), c1.clone()),
("refs/heads/main".to_string(), c1.clone()),
],
"keeper and main are reset to c1 and feature is deleted despite the \
broken entry (c2 was {c2})"
);
}
/// Happy path unchanged by the hardening: a fully valid snapshot restores
/// exactly (refs rewound, created ref deleted) and returns Ok.
#[test]
fn restore_refs_happy_path_restores_snapshot_exactly() {
let (_work, _dir, bare, c1, _c2) = advanced_bare_repo();
let snapshot = vec![
("refs/heads/main".to_string(), c1.clone()),
("refs/heads/keeper".to_string(), c1.clone()),
];
restore_refs(&bare, &snapshot).unwrap();
let refs = sorted_refs(&bare);
assert_eq!(
refs,
vec![
("refs/heads/keeper".to_string(), c1.clone()),
("refs/heads/main".to_string(), c1),
],
"restore must reproduce the snapshot exactly"
);
}
/// When the internal `list_refs` (extra-ref sweep) fails, the snapshot-reset
/// half must still have run and its collected failures must surface in the
/// aggregated error, not be discarded by an early `?` return. The repo is a
/// real bare repo declaring repositoryformatversion=999, so both `set_ref`
/// and `list_refs` fail deterministically. RED with the sweep reverted to
/// `let current = list_refs(repo_path)?;`: the raw for-each-ref error
/// propagates without the "failed to restore" aggregate or the ref name.
#[test]
fn restore_refs_aggregates_when_internal_list_refs_fails() {
let dir = tempfile::TempDir::new().unwrap();
let bare = dir.path().join("repo.git");
let out = Command::new("git")
.args(["init", "--bare", "-q", &bare.to_string_lossy()])
.output()
.unwrap();
assert!(out.status.success(), "git init --bare failed");
std::fs::write(
bare.join("config"),
"[core]\n\trepositoryformatversion = 999\n\tbare = true\n",
)
.unwrap();
assert!(
list_refs(&bare).is_err(),
"precondition: list_refs must fail on the corrupted repo"
);
let snapshot = vec![("refs/heads/main".to_string(), "a".repeat(40))];
let err = restore_refs(&bare, &snapshot).unwrap_err();
let msg = format!("{err:#}");
assert!(
msg.contains("failed to restore"),
"must return the aggregated error, not the raw list_refs error: {msg}"
);
assert!(
msg.contains("refs/heads/main"),
"the set_ref failure from the reset half must survive the sweep \
failure: {msg}"
);
assert!(
msg.contains("listing current refs"),
"the sweep failure itself is also collected: {msg}"
);
}
#[test]
fn branch_diff_names_lists_changed_paths() {
let td = tempfile::TempDir::new().unwrap();
+10 -22
View File
@@ -11,21 +11,6 @@ use anyhow::{Context, Result};
use aws_sdk_s3::Client as S3Client;
use tracing::{debug, info};
/// Object storage for git bare-repo archives. The seam that lets `RepoStore`
/// (and tests) depend on storage behavior without a live bucket. `TigrisClient`
/// is the production implementation.
#[async_trait::async_trait]
pub trait ObjectStore: Send + Sync {
/// Whether an archive exists for this repo.
async fn exists(&self, owner_slug: &str, repo_name: &str) -> Result<bool>;
/// Upload a local bare repo directory as an archive.
async fn upload(&self, owner_slug: &str, repo_name: &str, local_path: &Path) -> Result<()>;
/// Download and extract a repo archive to local disk.
async fn download(&self, owner_slug: &str, repo_name: &str, local_path: &Path) -> Result<()>;
/// Delete a repo archive.
async fn delete(&self, owner_slug: &str, repo_name: &str) -> Result<()>;
}
/// Wrapper around the S3 client with the configured bucket.
#[derive(Clone)]
pub struct TigrisClient {
@@ -50,12 +35,9 @@ impl TigrisClient {
fn repo_key(owner_slug: &str, repo_name: &str) -> String {
format!("repos/v1/{owner_slug}/{repo_name}.tar.zst")
}
}
#[async_trait::async_trait]
impl ObjectStore for TigrisClient {
/// Check if a repo archive exists in Tigris.
async fn exists(&self, owner_slug: &str, repo_name: &str) -> Result<bool> {
pub async fn exists(&self, owner_slug: &str, repo_name: &str) -> Result<bool> {
let key = Self::repo_key(owner_slug, repo_name);
match self
.s3
@@ -77,7 +59,7 @@ impl ObjectStore for TigrisClient {
}
/// Upload a local bare repo directory to Tigris as a tar.zst archive.
async fn upload(&self, owner_slug: &str, repo_name: &str, local_path: &Path) -> Result<()> {
pub async fn upload(&self, owner_slug: &str, repo_name: &str, local_path: &Path) -> Result<()> {
let key = Self::repo_key(owner_slug, repo_name);
debug!(key = %key, path = %local_path.display(), "uploading repo to tigris");
@@ -107,7 +89,12 @@ impl ObjectStore for TigrisClient {
}
/// Download a repo archive from Tigris and extract to local disk.
async fn download(&self, owner_slug: &str, repo_name: &str, local_path: &Path) -> Result<()> {
pub async fn download(
&self,
owner_slug: &str,
repo_name: &str,
local_path: &Path,
) -> Result<()> {
let key = Self::repo_key(owner_slug, repo_name);
debug!(key = %key, path = %local_path.display(), "downloading repo from tigris");
@@ -141,7 +128,8 @@ impl ObjectStore for TigrisClient {
}
/// Delete a repo archive from Tigris.
async fn delete(&self, owner_slug: &str, repo_name: &str) -> Result<()> {
#[allow(dead_code)]
pub async fn delete(&self, owner_slug: &str, repo_name: &str) -> Result<()> {
let key = Self::repo_key(owner_slug, repo_name);
self.s3
.delete_object()
+36 -179
View File
@@ -1,4 +1,3 @@
mod admin;
mod api;
mod arweave;
mod auth;
@@ -71,12 +70,6 @@ async fn main() -> Result<()> {
let mut config = Config::parse();
// Admin subcommands run out-of-band and exit, never starting the node. The
// no-subcommand path below is the unchanged daemon startup.
if let Some(command) = config.command.clone() {
return run_admin_command(command, &config).await;
}
// Merge the embedded seed list of public network nodes into the runtime
// bootstrap peers. Operators can opt out via GITLAWB_BOOTSTRAP_DISABLE_SEEDS.
bootstrap::merge_seeds(&mut config);
@@ -286,22 +279,8 @@ async fn main() -> Result<()> {
None
};
let object_store =
tigris.map(|t| std::sync::Arc::new(t) as std::sync::Arc<dyn git::tigris::ObjectStore>);
// Advisory-lock guards get their own pool, separate from the app pool
// (db.pool()) that serves normal handlers, so a concurrent-push burst pins
// lock connections here instead of starving request handlers. Sized to peak
// writers via GITLAWB_DB_LOCK_POOL_MAX_CONNECTIONS.
let lock_pool_acquire_timeout = std::time::Duration::from_secs(config.db_acquire_timeout_secs);
let lock_pool = Db::connect_lock_pool(
&config.database_url,
config.db_lock_pool_max_connections,
lock_pool_acquire_timeout,
)
.await
.context("connecting advisory-lock pool")?;
let repo_store =
git::repo_store::RepoStore::new(config.repos_dir.clone(), object_store, lock_pool);
git::repo_store::RepoStore::new(config.repos_dir.clone(), tigris, db.pool().clone());
// Per-DID limiter for the creation endpoints. Keyed on the authenticated
// DID (attacker-varied), so bound its key set to cap memory.
@@ -315,24 +294,36 @@ async fn main() -> Result<()> {
// capped regardless of how many identities it mints. Sized well above any
// legitimate per-IP creation rate; GITLAWB_CREATE_RATE_LIMIT overrides, 0
// disables. Bounded key set — the key is a client-influenced IP.
let create_ip_rate_limiter = build_ip_limiter("GITLAWB_CREATE_RATE_LIMIT", 120, "creation");
// Per-IP brake for the authenticated non-creation write surface (issue/PR
// comments, labels, stars, merges, protect, replicas, visibility, tasks,
// bounties, profile, GraphQL mutations). Its own bucket, separate from the
// creation brake, so a write flood cannot drain the creation budget and vice
// versa (same rationale as the sync_trigger / peer_write split). Sized above
// any legitimate per-IP write rate — real agent automation makes many small
// writes per hour. GITLAWB_WRITE_RATE_LIMIT overrides; 0 disables. Bounded
// key set — the key is a client-influenced IP.
let write_rate_limiter = build_ip_limiter("GITLAWB_WRITE_RATE_LIMIT", 600, "write");
let create_limit = std::env::var("GITLAWB_CREATE_RATE_LIMIT")
.ok()
.and_then(|v| v.trim().parse::<usize>().ok())
.unwrap_or(120);
let create_ip_rate_limiter = rate_limit::RateLimiter::new_bounded(
create_limit,
std::time::Duration::from_secs(3600),
200_000,
);
if create_limit == 0 {
tracing::warn!("GITLAWB_CREATE_RATE_LIMIT=0 — per-IP creation rate limiting disabled");
}
// Push-path flood brake: max git-receive-pack requests per client IP per
// hour (counts both the info/refs advertisement and the push POST). Sized
// for heavy agent automation while still stopping flood traffic (the June
// 2026 attack pushed several times per second per IP). GITLAWB_PUSH_RATE_LIMIT
// overrides; 0 disables. Bounded key set — the key is a client-influenced IP.
let push_rate_limiter = build_ip_limiter("GITLAWB_PUSH_RATE_LIMIT", 600, "push");
let push_limit = std::env::var("GITLAWB_PUSH_RATE_LIMIT")
.ok()
.and_then(|v| v.trim().parse::<usize>().ok())
.unwrap_or(600);
let push_rate_limiter = rate_limit::RateLimiter::new_bounded(
push_limit,
std::time::Duration::from_secs(3600),
200_000,
);
if push_limit == 0 {
tracing::warn!("GITLAWB_PUSH_RATE_LIMIT=0 — per-IP push rate limiting disabled");
}
// Which forwarded header the edge is trusted to set. Default None (trust
// nothing, key on the socket peer). Fly nodes set GITLAWB_TRUSTED_PROXY=fly;
@@ -340,7 +331,7 @@ async fn main() -> Result<()> {
let push_limiter_trust = rate_limit::TrustedProxy::from_env_value(
&std::env::var("GITLAWB_TRUSTED_PROXY").unwrap_or_default(),
);
tracing::info!(trust = ?push_limiter_trust, "push rate limiter trust configured");
tracing::info!(trust = ?push_limiter_trust, push_limit, "push rate limiter configured");
// Peer-sync flood brakes, keyed on the resolved client IP (per-DID is useless
// here — a did:key farm self-registers). Two buckets so an unsigned notify
@@ -382,7 +373,6 @@ async fn main() -> Result<()> {
repo_store,
rate_limiter,
create_ip_rate_limiter,
write_rate_limiter,
push_rate_limiter,
push_limiter_trust,
sync_trigger_rate_limiter,
@@ -418,17 +408,22 @@ async fn main() -> Result<()> {
// Periodic cleanup of expired rate limit entries + consumed-proof ledger
{
// Clone the whole state so the sweep uses AppState::cleanup_rate_limiters
// (the single source of truth that reaps EVERY limiter, including
// write_rate_limiter) rather than a hand-maintained list that can omit one.
let state = state.clone();
let rl = state.rate_limiter.clone();
let create_ip_rl = state.create_ip_rate_limiter.clone();
let push_rl = state.push_rate_limiter.clone();
let sync_trigger_rl = state.sync_trigger_rate_limiter.clone();
let peer_write_rl = state.peer_write_rate_limiter.clone();
let db = state.db.clone();
let mut shutdown_rx = state.subscribe_shutdown();
tokio::spawn(async move {
loop {
tokio::select! {
_ = tokio::time::sleep(std::time::Duration::from_secs(300)) => {
state.cleanup_rate_limiters().await;
rl.cleanup().await;
create_ip_rl.cleanup().await;
push_rl.cleanup().await;
sync_trigger_rl.cleanup().await;
peer_write_rl.cleanup().await;
let now = std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.map(|d| d.as_secs() as i64)
@@ -569,110 +564,6 @@ async fn main() -> Result<()> {
Ok(())
}
/// Pure resolver for a usize rate-limit env value. Returns `(resolved, invalid)`:
/// an absent or empty value resolves to `default` and is NOT invalid; a non-empty
/// value that fails to parse resolves to `default` and IS invalid (so the caller
/// warns rather than silently disabling the intended, usually stricter, brake).
fn resolve_rate_limit(raw: Option<&str>, default: usize) -> (usize, bool) {
match raw.map(str::trim) {
None | Some("") => (default, false),
Some(t) => match t.parse::<usize>() {
Ok(n) => (n, false),
Err(_) => (default, true),
},
}
}
/// Read a rate-limit env var, warning on a present-but-unparseable value. A typo
/// like `GITLAWB_WRITE_RATE_LIMIT=5O` must not silently revert to the default and
/// leave the operator believing a stricter cap is in force.
fn rate_limit_from_env(var: &str, default: usize) -> usize {
let raw = std::env::var(var).ok();
let (value, invalid) = resolve_rate_limit(raw.as_deref(), default);
if invalid {
tracing::warn!(
var,
value = raw.as_deref().unwrap_or(""),
default,
"invalid rate-limit value — using default; the intended brake is NOT applied"
);
}
value
}
/// Build a bounded per-client-IP rate limiter from an env var: resolve the limit
/// (honoring the present-but-unparseable warning in `rate_limit_from_env`), build
/// the limiter with the shared 1-hour window and 200k-key cap, and warn when the
/// limit is 0 (disabled). Collapses the identical create/write/push setup.
fn build_ip_limiter(var: &str, default: usize, label: &str) -> rate_limit::RateLimiter {
let limit = rate_limit_from_env(var, default);
let limiter =
rate_limit::RateLimiter::new_bounded(limit, std::time::Duration::from_secs(3600), 200_000);
if limit == 0 {
tracing::warn!("{var}=0 — per-IP {label} rate limiting disabled");
} else {
tracing::info!(%var, limit, "per-IP {label} rate limiter configured");
}
limiter
}
/// Dispatch an admin subcommand. Connects the database directly (no server, no
/// p2p, no metrics) and runs the requested tool to completion, then exits.
async fn run_admin_command(command: config::Command, config: &Config) -> Result<()> {
match command {
config::Command::PurgeSpam { execute } => {
let acquire_timeout = std::time::Duration::from_secs(config.db_acquire_timeout_secs);
let db = Db::connect(
&config.database_url,
config.db_max_connections,
acquire_timeout,
)
.await
.context("connecting to database for purge-spam")?;
// Build the object store so purge is authoritative against remote
// state: the emptiness recheck consults the archive (not just
// possibly-stale local disk) and a successful purge deletes the
// archive too. When no bucket is configured this stays None and purge
// is local-only, exactly as before.
let object_store: Option<std::sync::Arc<dyn git::tigris::ObjectStore>> = if config
.tigris_bucket
.is_empty()
{
None
} else {
match git::tigris::TigrisClient::new(&config.tigris_bucket).await {
Ok(client) => Some(std::sync::Arc::new(client)),
Err(e) => {
// Fail closed: without the configured store we cannot
// do the authoritative recheck, and a local-only purge
// on a Tigris deployment is the stale-view delete this
// guards against. Refuse to run rather than fall back.
anyhow::bail!(
"purge-spam: GITLAWB_TIGRIS_BUCKET is set but the object-store client failed to initialize ({e}); refusing to run a local-only purge on a Tigris deployment"
);
}
}
};
// Give the purge-spam store a dedicated advisory-lock pool, separate
// from `db`'s app pool. The guard then holds a lock-pool connection
// while `delete_repo_by_id` runs on the app pool, so a purge at
// GITLAWB_DB_MAX_CONNECTIONS=1 does not self-deadlock (KTD3).
let lock_pool = Db::connect_lock_pool(
&config.database_url,
config.db_lock_pool_max_connections,
acquire_timeout,
)
.await
.context("connecting advisory-lock pool for purge-spam")?;
let repo_store =
git::repo_store::RepoStore::new(config.repos_dir.clone(), object_store, lock_pool);
admin::run_purge_spam(&db, &repo_store, &config.repos_dir, execute)
.await
.map(|_| ())
}
}
}
fn spawn_shutdown_signal(tx: watch::Sender<bool>) {
tokio::spawn(async move {
#[cfg(unix)]
@@ -1133,40 +1024,6 @@ fn load_or_create_keypair(config: &Config) -> Result<Keypair> {
}
}
#[cfg(test)]
mod rate_limit_env_tests {
use super::resolve_rate_limit;
#[test]
fn absent_uses_default_without_warning() {
assert_eq!(resolve_rate_limit(None, 600), (600, false));
}
#[test]
fn empty_or_whitespace_uses_default_without_warning() {
assert_eq!(resolve_rate_limit(Some(""), 600), (600, false));
assert_eq!(resolve_rate_limit(Some(" "), 600), (600, false));
}
#[test]
fn valid_value_parses() {
assert_eq!(resolve_rate_limit(Some("50"), 600), (50, false));
assert_eq!(resolve_rate_limit(Some(" 50 "), 600), (50, false));
// 0 is a VALID value (disables the brake); the ==0 warning is handled
// separately at the call site. It must NOT be flagged invalid here.
assert_eq!(resolve_rate_limit(Some("0"), 600), (0, false));
}
#[test]
fn present_but_unparseable_defaults_and_flags_invalid() {
// The L8 case: a fat-fingered stricter cap ("5O", letter O) must default
// AND be flagged so the caller warns, not silently disable the brake.
assert_eq!(resolve_rate_limit(Some("5O"), 600), (600, true));
assert_eq!(resolve_rate_limit(Some("abc"), 600), (600, true));
assert_eq!(resolve_rate_limit(Some("-1"), 600), (600, true));
}
}
#[cfg(test)]
mod gossip_ssrf_tests {
use super::ping_peer_health;
+5 -1
View File
@@ -235,7 +235,11 @@ pub fn record_auth_failure(route: &str, reason: &str) {
}
}
/// Record one sync_queue item outcome. `status` ∈ {done, failed, skipped}.
/// Record one sync_queue item outcome. `status` ∈ {done, failed, skipped,
/// rejected, deferred}. `rejected` is terminal and caused by a guard (a
/// malformed slug, or a path resolving outside repos_dir); `deferred` leaves
/// the row pending for a later pass, so a queue stalled on an operator
/// condition is visible rather than looking idle.
pub fn record_sync_processed(status: &str) {
if let Some(c) = SYNC_PROCESSED.get() {
c.with_label_values(&[status]).inc();
+19 -273
View File
@@ -70,23 +70,10 @@ impl RateLimiter {
self.window.min(Duration::from_secs(1))
}
/// Returns `true` if the request is allowed. Thin wrapper over
/// [`check_retry`](Self::check_retry) for callers that only need the
/// allow/deny decision, not the rejection's Retry-After delay. Test-only now:
/// production 429 sites use `check_retry` so they can advertise Retry-After.
#[cfg(test)]
pub(crate) async fn check(&self, key: &str) -> bool {
self.check_retry(key).await.is_none()
}
/// Record a request against `key`. Returns `None` when it is allowed, or
/// `Some(retry_after)` when it is rejected — the delay a client should wait
/// before retrying, so a 429 can advertise a value consistent with the
/// actual window instead of a constant.
pub(crate) async fn check_retry(&self, key: &str) -> Option<Duration> {
// max_requests == 0 means the limiter is disabled, not "block all".
if self.max_requests == 0 {
return None;
return true;
}
let now = Instant::now();
let mut state = self.state.lock().await;
@@ -98,25 +85,10 @@ impl RateLimiter {
.timestamps
.retain(|t| now.duration_since(*t) < self.window);
if window.timestamps.len() >= self.max_requests {
// Insertion order is preserved, so after the retain the first
// element is the oldest live request. It ages out (freeing a
// slot) one window after it landed, so that remaining time is
// the delay to advertise. Clamp to >= 1 so a 429 never tells a
// client to retry immediately.
let retry = window
.timestamps
.first()
.map(|oldest| {
self.window
.as_secs()
.saturating_sub(now.duration_since(*oldest).as_secs())
})
.unwrap_or(0)
.max(1);
return Some(Duration::from_secs(retry));
return false;
}
window.timestamps.push(now);
return None;
return true;
}
// New key. Enforce the cap BEFORE inserting so a flood of distinct keys
@@ -137,11 +109,7 @@ impl RateLimiter {
}
drop(last_sweep);
if state.len() >= self.max_keys {
// The key is untracked (never inserted), so there is no oldest
// entry to derive a delay from. Advertise the full window as a
// conservative bound: capacity frees as other keys expire, at
// the latest one window from now.
return Some(self.window.max(Duration::from_secs(1)));
return false;
}
}
state.insert(
@@ -150,7 +118,7 @@ impl RateLimiter {
timestamps: vec![now],
},
);
None
true
}
pub async fn cleanup(&self) {
@@ -162,41 +130,6 @@ impl RateLimiter {
!w.timestamps.is_empty()
});
}
/// Test-only: insert a fully-expired entry (no live timestamps) that the next
/// `cleanup()` must reclaim. Models a key whose window has aged out, without
/// depending on wall-clock sleeps or a short window. Used cross-module by the
/// AppState cleanup guard test (H2).
#[cfg(test)]
pub(crate) async fn insert_reclaimable_for_test(&self, key: &str) {
self.state
.lock()
.await
.insert(key.to_string(), Window { timestamps: vec![] });
}
/// Test-only: number of distinct keys currently tracked.
#[cfg(test)]
pub(crate) async fn tracked_keys(&self) -> usize {
self.state.lock().await.len()
}
/// Test-only: seed a key's bucket with entries of the given ages (how long
/// ago each request landed), so a test can drive the Retry-After computation
/// off a controlled oldest-entry age without wall-clock sleeps. Ages that
/// would underflow the monotonic clock collapse to `now`.
#[cfg(test)]
pub(crate) async fn seed_aged_entry_for_test(&self, key: &str, ages: &[Duration]) {
let now = Instant::now();
let timestamps = ages
.iter()
.map(|a| now.checked_sub(*a).unwrap_or(now))
.collect();
self.state
.lock()
.await
.insert(key.to_string(), Window { timestamps });
}
}
pub async fn rate_limit_by_did(request: Request, next: Next) -> Response {
@@ -208,8 +141,13 @@ pub async fn rate_limit_by_did(request: Request, next: Next) -> Response {
.map(|a| a.0.clone());
if let (Some(limiter), Some(did)) = (limiter, did) {
if let Some(retry_after) = limiter.check_retry(&did).await {
return too_many_requests(retry_after);
if !limiter.check(&did).await {
return (
StatusCode::TOO_MANY_REQUESTS,
[("retry-after", "60")],
"rate limit exceeded — try again later",
)
.into_response();
}
}
@@ -312,13 +250,13 @@ impl<S: Send + Sync> axum::extract::FromRequestParts<S> for PeerAddr {
}
}
/// The shared 429 response for the rate-limit middlewares (per-DID and per-IP
/// alike). Route-agnostic: the message stays generic (the offending path, when
/// logged, is recorded at the call site).
pub fn too_many_requests(retry_after: Duration) -> Response {
/// The shared 429 response for the per-IP flood brakes. Route-agnostic: this
/// middleware now serves the push path AND the peer-sync routes, so the message
/// stays generic (the offending path is recorded in the warn log below).
pub fn too_many_requests() -> Response {
(
StatusCode::TOO_MANY_REQUESTS,
[("retry-after", retry_after.as_secs().to_string())],
[("retry-after", "60")],
"rate limit exceeded — try again later",
)
.into_response()
@@ -336,9 +274,9 @@ pub async fn rate_limit_by_ip(request: Request, next: Next) -> Response {
if let Some(limiter) = limiter {
if let Some(key) = client_key(request.headers(), peer, limiter.trust) {
if let Some(retry_after) = limiter.limiter.check_retry(&key).await {
if !limiter.limiter.check(&key).await {
tracing::warn!(key = %key, path = %request.uri().path(), "per-IP rate limit exceeded");
return too_many_requests(retry_after);
return too_many_requests();
}
}
}
@@ -667,196 +605,4 @@ mod tests {
"an exhausted IP must be braked with 429 before auth runs, not leak to 401"
);
}
// ── Adversarial TrustedProxy verification through the middleware (U5) ──
// These complement the client_key unit tests above by driving hostile
// headers through rate_limit_by_ip on a real router: the property under test
// is that the *bucket key* cannot be rotated by a spoofer.
async fn post_with(
router: &axum::Router,
peer: SocketAddr,
hdrs: &[(&str, &str)],
) -> StatusCode {
use tower::ServiceExt;
let mut b = axum::http::Request::builder()
.method(axum::http::Method::POST)
.uri("/api/v1/repos");
for (k, v) in hdrs {
b = b.header(*k, *v);
}
let mut req = b.body(axum::body::Body::empty()).unwrap();
req.extensions_mut().insert(ConnectInfo(peer));
router.clone().oneshot(req).await.unwrap().status()
}
#[tokio::test]
async fn xff_spoofed_leftmost_hop_cannot_rotate_bucket() {
// XForwardedFor mode, budget 1. A spoofer varies the client-controlled
// leftmost hop every request but the trusted proxy always appends the
// same rightmost hop. All requests must key to the rightmost hop, so the
// second is braked despite a fresh leftmost value.
let router = ip_limited_over_auth_router(IpRateLimiter {
limiter: RateLimiter::new(1, Duration::from_secs(60)),
trust: TrustedProxy::XForwardedFor,
});
let p1: SocketAddr = "10.0.0.1:1".parse().unwrap();
let p2: SocketAddr = "10.0.0.2:1".parse().unwrap();
// First request passes the brake, reaches (failing) auth.
assert_eq!(
post_with(&router, p1, &[("x-forwarded-for", "9.9.9.1, 203.0.113.5")]).await,
StatusCode::UNAUTHORIZED
);
// Different leftmost + different socket peer, SAME rightmost trusted hop:
// braked, because the key is the rightmost hop, not the spoofed value.
assert_eq!(
post_with(&router, p2, &[("x-forwarded-for", "9.9.9.2, 203.0.113.5")]).await,
StatusCode::TOO_MANY_REQUESTS,
"a spoofer varying the leftmost XFF hop must not rotate its bucket key"
);
}
#[tokio::test]
async fn xff_distinct_real_clients_get_distinct_buckets() {
// Distinct trusted (rightmost) hops are distinct clients: neither throttles
// the other, so a busy legitimate client cannot starve a different one.
let router = ip_limited_over_auth_router(IpRateLimiter {
limiter: RateLimiter::new(1, Duration::from_secs(60)),
trust: TrustedProxy::XForwardedFor,
});
let peer: SocketAddr = "10.0.0.9:1".parse().unwrap();
assert_eq!(
post_with(
&router,
peer,
&[("x-forwarded-for", "1.1.1.1, 203.0.113.5")]
)
.await,
StatusCode::UNAUTHORIZED
);
assert_eq!(
post_with(
&router,
peer,
&[("x-forwarded-for", "1.1.1.1, 203.0.113.6")]
)
.await,
StatusCode::UNAUTHORIZED,
"a different trusted client hop must get its own bucket"
);
}
#[tokio::test]
async fn absent_trusted_header_collapses_to_socket_peer() {
// The documented fallback: in a trusted-proxy mode, a request with no
// forwarded header keys on the socket peer. Behind a real edge every
// request shares the proxy's socket, so they collapse onto one bucket —
// asserted here by name so any change to this behavior is deliberate.
let router = ip_limited_over_auth_router(IpRateLimiter {
limiter: RateLimiter::new(1, Duration::from_secs(60)),
trust: TrustedProxy::XForwardedFor,
});
let proxy: SocketAddr = "172.16.0.1:1".parse().unwrap();
assert_eq!(
post_with(&router, proxy, &[]).await,
StatusCode::UNAUTHORIZED
);
assert_eq!(
post_with(&router, proxy, &[]).await,
StatusCode::TOO_MANY_REQUESTS,
"with no trusted header, requests fall back to the socket peer key (collapse)"
);
}
// ── Retry-After derived from the window, not a constant 60 (U5) ───────
// A 429 must advertise the delay until the oldest live request ages out of
// the limiter's window, so the value tracks the real window instead of the
// hardcoded 60 that contradicted every limiter's 3600s window.
async fn resp_from(router: &axum::Router, peer: SocketAddr) -> Response {
use tower::ServiceExt;
let mut req = axum::http::Request::builder()
.method(axum::http::Method::POST)
.uri("/o/r/git-receive-pack")
.body(axum::body::Body::empty())
.unwrap();
req.extensions_mut().insert(ConnectInfo(peer));
router.clone().oneshot(req).await.unwrap()
}
fn retry_after_secs(resp: &Response) -> u64 {
resp.headers()
.get("retry-after")
.expect("429 must carry a retry-after header")
.to_str()
.unwrap()
.parse()
.expect("retry-after must be an integer number of seconds")
}
#[tokio::test]
async fn retry_after_reflects_full_window_for_freshly_filled_bucket() {
// A just-filled bucket's oldest entry is ~now, so the advertised delay
// must be close to the whole window (100s), NOT the old constant 60.
let router = ip_limited_router(IpRateLimiter {
limiter: RateLimiter::new(1, Duration::from_secs(100)),
trust: TrustedProxy::None,
});
let peer: SocketAddr = "203.0.113.50:6000".parse().unwrap();
assert_eq!(resp_from(&router, peer).await.status(), StatusCode::OK);
let resp = resp_from(&router, peer).await;
assert_eq!(resp.status(), StatusCode::TOO_MANY_REQUESTS);
let retry = retry_after_secs(&resp);
assert!(
(95..=100).contains(&retry),
"freshly-filled bucket must advertise ~window (95..=100s), got {retry} (constant-60 bug if 60)"
);
}
#[tokio::test]
async fn retry_after_shrinks_as_oldest_entry_ages() {
// Seed the sole slot with an entry already 95s into a 100s window: the
// next request is rejected and must advertise only the ~5s remaining,
// not 60.
let limiter = RateLimiter::new(1, Duration::from_secs(100));
let peer: SocketAddr = "203.0.113.51:6000".parse().unwrap();
limiter
.seed_aged_entry_for_test(&peer.ip().to_string(), &[Duration::from_secs(95)])
.await;
let router = ip_limited_router(IpRateLimiter {
limiter,
trust: TrustedProxy::None,
});
let resp = resp_from(&router, peer).await;
assert_eq!(resp.status(), StatusCode::TOO_MANY_REQUESTS);
let retry = retry_after_secs(&resp);
assert!(
(1..=10).contains(&retry),
"an aged bucket must advertise the small remaining delay (1..=10s), got {retry}"
);
}
#[tokio::test]
async fn retry_after_clamps_to_at_least_one_second() {
// Oldest entry is 99s into a 100s window: the raw remainder rounds toward
// 1s and must never be advertised as 0 (a 0/blank retry lets a rejected
// client hammer immediately).
let limiter = RateLimiter::new(1, Duration::from_secs(100));
let peer: SocketAddr = "203.0.113.52:6000".parse().unwrap();
limiter
.seed_aged_entry_for_test(&peer.ip().to_string(), &[Duration::from_secs(99)])
.await;
let router = ip_limited_router(IpRateLimiter {
limiter,
trust: TrustedProxy::None,
});
let resp = resp_from(&router, peer).await;
assert_eq!(resp.status(), StatusCode::TOO_MANY_REQUESTS);
let retry = retry_after_secs(&resp);
assert!(retry >= 1, "retry-after must clamp to >= 1, got {retry}");
assert!(
retry <= 2,
"with 99s of a 100s window elapsed, the remaining delay is ~1s, got {retry}"
);
}
}
+73 -52
View File
@@ -14,7 +14,7 @@ use tracing::Level;
use crate::api::{
agents, arweave, bounties, certs, changelog, events, ipfs, issues, labels, peers, profiles,
protect, pulls, register, replicas, repos, resolve, stars, tasks, visibility, webhooks,
protect, pulls, register, replicas, repos, resolve, stars, status, tasks, visibility, webhooks,
};
use crate::auth;
use crate::rate_limit;
@@ -55,50 +55,18 @@ fn add_auth_layers(router: Router<AppState>, state: AppState) -> Router<AppState
.layer(middleware::from_fn(auth::require_signature))
}
/// Attach the shared per-IP write brake to an authenticated write-route group,
/// outermost so a flood is rejected before signature verification. Every
/// authenticated non-creation write group MUST call this or it silently ships
/// unbraked with no compile error. Deliberately a separate call, NOT folded into
/// `add_auth_layers`: the creation routes use a different bucket
/// (`create_ip_rate_limiter`) and the graphql group is not built through
/// `add_auth_layers`, so a helper keyed on auth would attach the wrong limiter.
trait WriteBraked {
fn write_braked(self, limiter: &rate_limit::IpRateLimiter) -> Self;
}
impl WriteBraked for Router<AppState> {
fn write_braked(self, limiter: &rate_limit::IpRateLimiter) -> Self {
self.layer(middleware::from_fn(rate_limit::rate_limit_by_ip))
.layer(axum::Extension(limiter.clone()))
}
}
pub fn build_router(state: AppState) -> Router {
// ── GraphQL routes ─────────────────────────────────────────────────────
let schema = state.graphql_schema.as_ref().clone();
// Per-IP write brake, shared across every authenticated non-creation write
// group (see `AppState::write_rate_limiter`). Built once here so each group
// clones the same bucket; attached outermost on each group so a flood is
// rejected before auth runs. Separate bucket from the creation brake.
let write_ip_limiter = rate_limit::IpRateLimiter {
limiter: state.write_rate_limiter.clone(),
trust: state.push_limiter_trust,
};
let graphql_routes = Router::new()
.route("/graphql", get(graphql_playground).post(graphql_handler))
// Attach the verified DID to /graphql when a signature is present. The
// layer covers only routes added before it, so /graphql/ws (added after,
// read-only subscriptions) stays open. The per-IP write brake covers the
// same /graphql (KTD-5: an HTTP-layer brake cannot tell a query POST from
// a mutation POST, so it counts every /graphql request — acceptable, both
// cost work); /graphql/ws subscriptions stay unbraked.
// read-only subscriptions) stays open.
.layer(middleware::from_fn(auth::optional_signature))
.write_braked(&write_ip_limiter)
.route_service("/graphql/ws", GraphQLSubscription::new(schema));
// ── Task routes (write — require HTTP Signature + per-IP write brake) ──
// ── Task routes (write — require HTTP Signature) ───────────────────────
let task_write_routes = add_auth_layers(
Router::new()
.route("/api/v1/tasks", post(tasks::create_task))
@@ -106,8 +74,7 @@ pub fn build_router(state: AppState) -> Router {
.route("/api/v1/tasks/{id}/complete", post(tasks::complete_task))
.route("/api/v1/tasks/{id}/fail", post(tasks::fail_task)),
state.clone(),
)
.write_braked(&write_ip_limiter);
);
// ── Task routes (read — open) ──────────────────────────────────────────
let task_read_routes = Router::new()
@@ -144,13 +111,7 @@ pub fn build_router(state: AppState) -> Router {
.layer(middleware::from_fn(rate_limit::rate_limit_by_ip))
.layer(axum::Extension(create_ip_limiter));
// ── Write routes — require HTTP Signature + per-IP write brake ────────
// Same per-IP flood defense as the creation routes, on its own `write`
// bucket (see `AppState::write_rate_limiter`). Per-DID is deliberately not
// paired (a DID farm never trips it; busy legitimate agents would false-
// positive). The bundled visibility GET is a read that also draws from the
// write bucket — harmless, since the bucket is sized far above any legit
// per-IP read rate.
// ── Write routes — require HTTP Signature (no rate limit) ─────────────
let write_routes = add_auth_layers(
Router::new()
.route(
@@ -220,8 +181,65 @@ pub fn build_router(state: AppState) -> Router {
axum::routing::delete(agents::deregister_agent),
),
state.clone(),
);
// ── Status claim writes — signature, plus the per-DID throttle AND the
// per-IP flood brake `creation_routes` carries. Not in `write_routes`, which
// has neither: every call here appends a row, so an unthrottled writer grows
// the claim log on demand and the three per-repo caps would be the only bound.
let status_write_routes = add_auth_layers(
Router::new()
.route(
"/api/v1/repos/{owner}/{repo}/statuses/{sha}",
post(status::create_status),
)
.layer(middleware::from_fn(rate_limit::rate_limit_by_did))
.layer(axum::Extension(state.rate_limiter.clone())),
state.clone(),
)
.write_braked(&write_ip_limiter);
// The route's advertised body bound, applied at the transport. It sits
// OUTSIDE the auth layers because `require_signature` collects and hashes
// the whole body before any handler runs: the handler's own check measures
// an allocation that has already happened, so without this layer the 8 KiB
// figure bounded what gets stored and not what gets buffered, and
// concurrent requests multiplied exactly the cost it was written to
// prevent. It sits INSIDE the rate limiters so an oversized request is
// still counted by the per-IP brake before it is refused.
//
// A declared Content-Length over the limit is answered 413 without reading
// a byte; a body that declares nothing (or understates) is truncated at the
// limit and fails where it is read, which `require_signature` reports as an
// unreadable body.
.layer(RequestBodyLimitLayer::new(status::MAX_REQUEST_BODY_BYTES))
// The one route that persists the signed request body, so the one route
// whose signature material carries it. Applied OUTSIDE `add_auth_layers`
// (outermost = runs first) because `require_signature` reads this marker to
// decide whether to carry the body; a layer added inside the auth pair runs
// after the middleware and is never seen. `create_status` refuses an absent
// body rather than storing an empty column, so getting this order wrong
// fails loudly instead of quietly writing unverifiable claims.
.layer(axum::Extension(auth::PersistsSignedBody))
.layer(middleware::from_fn(rate_limit::rate_limit_by_ip))
.layer(axum::Extension(rate_limit::IpRateLimiter {
limiter: state.create_ip_rate_limiter.clone(),
trust: state.push_limiter_trust,
}));
// ── Status claim reads — open, behind `optional_signature` so the handler
// sees the caller when one signs and still answers a public repo for an
// anonymous reader. Without the layer the handler's optional auth extension
// is None for EVERY caller, including the owner, and every private repo
// reads as not-found.
let status_read_routes = Router::new()
.route(
"/api/v1/repos/{owner}/{repo}/commits/{sha}/status",
get(status::commit_status),
)
.route(
"/api/v1/repos/{owner}/{repo}/pulls/{number}/status",
get(status::pull_request_status),
)
.layer(middleware::from_fn(auth::optional_signature));
// Body limit is raised to GITLAWB_MAX_PACK_BYTES (default 2 GB) for git
// routes only — all other API routes keep axum's default 2 MB cap.
@@ -287,8 +305,7 @@ pub fn build_router(state: AppState) -> Router {
post(bounties::dispute_bounty),
),
state.clone(),
)
.write_braked(&write_ip_limiter);
);
// ── Bounty routes (read — open) ──────────────────────────────────────
let bounty_read_routes = Router::new()
@@ -309,10 +326,9 @@ pub fn build_router(state: AppState) -> Router {
let profile_write_routes = add_auth_layers(
Router::new().route("/api/v1/profile", axum::routing::put(profiles::set_profile)),
state.clone(),
)
.write_braked(&write_ip_limiter);
);
// ── Issue routes (write — require HTTP Signature + per-IP write brake) ─
// ── Issue routes (write — require HTTP Signature, no rate limit) ─────
let issue_write_routes = add_auth_layers(
Router::new()
.route(
@@ -324,8 +340,7 @@ pub fn build_router(state: AppState) -> Router {
post(issues::create_issue_comment),
),
state.clone(),
)
.write_braked(&write_ip_limiter);
);
// ── Peer discovery routes ─────────────────────────────────────────────
// Peer writes accept signatures when present and can require them after a
@@ -416,6 +431,10 @@ pub fn build_router(state: AppState) -> Router {
"/api/v1/repos/{owner}/{repo}/events",
get(events::list_repo_events),
)
.route(
"/api/v1/repos/{owner}/{repo}/push-events",
get(events::list_repo_push_events),
)
.route("/api/v1/agents", get(agents::list_agents))
.route("/api/v1/agents/{did}", get(agents::show_agent))
.route("/api/v1/agents/{did}/trust", get(agents::get_trust))
@@ -512,6 +531,8 @@ pub fn build_router(state: AppState) -> Router {
.merge(profile_write_routes)
.merge(creation_routes)
.merge(write_routes)
.merge(status_write_routes)
.merge(status_read_routes)
.merge(git_write_routes)
.merge(git_read_routes)
.merge(issue_write_routes)
-54
View File
@@ -62,16 +62,6 @@ pub struct AppState {
/// resolved client IP is what actually stops a single-source flood (same
/// rationale as `push_rate_limiter`). Keyed by `push_limiter_trust`.
pub create_ip_rate_limiter: RateLimiter,
/// Per-client-IP rate limiter for the authenticated write surface that is
/// not repo/agent creation: issue/PR comments, labels, stars, merges,
/// protect/unprotect, replicas, visibility, tasks, bounties, profile, and
/// the GraphQL `MutationRoot`. Separate bucket from `create_ip_rate_limiter`
/// (its own budget, per the `sync_trigger`/`peer_write` precedent) so a
/// write flood cannot drain the creation budget and vice versa. Per-DID is
/// deliberately NOT paired here: a DID farm never trips it, and busy
/// legitimate agents would false-positive. `GITLAWB_WRITE_RATE_LIMIT`
/// overrides the default, 0 disables. Keyed by `push_limiter_trust`.
pub write_rate_limiter: RateLimiter,
/// Per-client-IP rate limiter for git-receive-pack. Per-DID limits cannot
/// brake a push flood from a DID farm (one throwaway DID per repo), so the
/// push path throttles on the resolved client IP instead.
@@ -128,48 +118,4 @@ impl AppState {
pub fn is_shutting_down(&self) -> bool {
*self.shutdown_tx.borrow()
}
/// Reclaim expired entries from EVERY rate limiter. Single source of truth for
/// the periodic cleanup loop (main.rs), co-located with the limiter fields so a
/// newly-added limiter is cleaned here rather than silently omitted from a
/// hand-maintained list in main() (H2: `write_rate_limiter` was the omitted
/// one). Every limiter field above MUST be swept here; the guard test
/// `cleanup_rate_limiters_reaps_the_write_limiter` fails if it is not.
pub(crate) async fn cleanup_rate_limiters(&self) {
self.rate_limiter.cleanup().await;
self.create_ip_rate_limiter.cleanup().await;
self.write_rate_limiter.cleanup().await;
self.push_rate_limiter.cleanup().await;
self.sync_trigger_rate_limiter.cleanup().await;
self.peer_write_rate_limiter.cleanup().await;
}
}
#[cfg(test)]
mod tests {
/// H2 guard: the write-surface limiter must be reaped by the shared cleanup
/// routine. Seeds a reclaimable entry into `write_rate_limiter` and asserts
/// `cleanup_rate_limiters` removes it. Goes RED if `write_rate_limiter` is
/// dropped from `cleanup_rate_limiters` — the exact H2 omission, now guarded.
#[tokio::test]
async fn cleanup_rate_limiters_reaps_the_write_limiter() {
let state = crate::test_support::test_state_lazy();
state
.write_rate_limiter
.insert_reclaimable_for_test("some-ip")
.await;
assert_eq!(
state.write_rate_limiter.tracked_keys().await,
1,
"precondition: the reclaimable entry is tracked"
);
state.cleanup_rate_limiters().await;
assert_eq!(
state.write_rate_limiter.tracked_keys().await,
0,
"write_rate_limiter must be reaped by cleanup_rate_limiters (H2)"
);
}
}
File diff suppressed because it is too large. Load diff
+147 -1
View File
@@ -77,7 +77,6 @@ fn build_state(db: Arc<crate::db::Db>, pool: PgPool) -> AppState {
repo_store: crate::git::repo_store::RepoStore::for_testing(PathBuf::from("/tmp"), pool),
rate_limiter: RateLimiter::new(100, Duration::from_secs(60)),
create_ip_rate_limiter: RateLimiter::new(1000, Duration::from_secs(3600)),
write_rate_limiter: RateLimiter::new(1000, Duration::from_secs(3600)),
push_rate_limiter: RateLimiter::new(600, Duration::from_secs(3600)),
push_limiter_trust: crate::rate_limit::TrustedProxy::None,
sync_trigger_rate_limiter: RateLimiter::new(60, Duration::from_secs(3600)),
@@ -110,6 +109,153 @@ pub(crate) fn signed_request_as(did: &str, method: Method, uri: &str, body: Body
.expect("request builder")
}
/// Slice a region out of a module's own source (obtained with `include_str!`)
/// and drop full-line `//` comments from it.
///
/// Several guards in `api/` assert that a piece of PRODUCTION code does or does
/// not contain something: that the receive-pack handler really calls the
/// push-event recorder, that the status module never acquires the repository.
/// Each guard carried its own copy of this parse, and three copies of a parser
/// is three places for the slicing to drift apart while every guard still reads
/// green.
///
/// `start` and `end` are literal anchors. The region runs from the first
/// occurrence of `start` (inclusive; the top of the file when `None`) to the
/// next occurrence of `end` (exclusive; end of file when `None`).
///
/// Returns `None` when an anchor is missing, rather than silently falling back
/// to a larger or empty region. Each caller supplies its own message for that
/// case and keeps its own assertion that the region really covers what it claims
/// to, which is what stops a bug in this one helper from turning every guard
/// that uses it into a vacuous pass.
///
/// Comment lines are stripped so that a doc comment naming the very call a guard
/// searches for cannot stand in for the call itself.
pub(crate) fn scrape_source_region(
src: &str,
start: Option<&str>,
end: Option<&str>,
) -> Option<String> {
let from = match start {
Some(anchor) => src.find(anchor)?,
None => 0,
};
let rest = &src[from..];
let to = match end {
// Search past the first CHARACTER, so an `end` anchor that also matches
// at the very start of the region cannot collapse it to nothing. A fixed
// one-byte skip does the same job on ASCII and lands mid-character on a
// multibyte first character, where the slice fails and the helper answers
// `None`, which is indistinguishable, to every caller, from a missing
// anchor.
Some(anchor) => {
let skip = rest.chars().next().map_or(0, char::len_utf8);
rest.get(skip..)?.find(anchor)? + skip
}
None => rest.len(),
};
Some(
rest[..to]
.lines()
.filter(|l| !l.trim_start().starts_with("//"))
.collect::<Vec<_>>()
.join("\n"),
)
}
#[cfg(test)]
mod scrape_source_region_tests {
use super::scrape_source_region;
const SRC: &str = "prelude line\nfn target() {\n // calls_the_thing()\n calls_the_thing();\n}\nfn later() {\n calls_the_thing();\n}\n";
#[test]
fn stops_at_the_end_anchor_so_a_later_function_cannot_satisfy_a_guard() {
let body = scrape_source_region(SRC, Some("fn target()"), Some("\n}")).unwrap();
assert!(body.contains("calls_the_thing();"));
assert!(
!body.contains("fn later()"),
"the region must stop at the first column-zero brace, got: {body}"
);
}
#[test]
fn strips_full_line_comments_so_a_doc_comment_cannot_stand_in_for_code() {
let commented = "fn target() {\n // calls_the_thing();\n}\n";
let body = scrape_source_region(commented, Some("fn target()"), Some("\n}")).unwrap();
assert!(
!body.contains("calls_the_thing"),
"a commented-out call must not survive the scrape, got: {body}"
);
}
#[test]
fn no_start_anchor_reads_from_the_top_of_the_file() {
let body = scrape_source_region(SRC, None, Some("fn later()")).unwrap();
assert!(body.starts_with("prelude line"));
assert!(!body.contains("fn later()"));
}
#[test]
fn no_end_anchor_reads_to_the_end_of_the_file() {
let body = scrape_source_region(SRC, Some("fn later()"), None).unwrap();
assert!(body.trim_end().ends_with('}'));
}
/// A missing anchor is None, never a silently wider or empty region: that is
/// what lets each caller keep its own "the thing I meant to scan is gone"
/// message instead of asserting against whatever the fallback produced.
#[test]
fn a_missing_anchor_is_none_rather_than_a_fallback_region() {
assert!(scrape_source_region(SRC, Some("fn absent()"), Some("\n}")).is_none());
assert!(scrape_source_region(SRC, Some("fn target()"), Some("~absent~")).is_none());
}
/// A region whose first character is multibyte is scraped, not reported
/// missing.
///
/// The end-anchor search skips the region's first CHARACTER. Skipping a
/// fixed byte instead lands mid-character here, the slice fails, and the
/// helper answers `None`, which every caller reads as "the anchor is gone"
/// and reports as a missing region. A guard that says "not found"
/// for input that is present is worse than one that fails loudly, and the
/// five callers sharing this helper all inherit it.
#[test]
fn a_multibyte_first_character_does_not_read_as_a_missing_anchor() {
let src = "é prelude\nfn target() {\n body();\n}\nfn later() {}\n";
let body = scrape_source_region(src, None, Some("\n}"))
.expect("a region starting on a multibyte character must be scraped");
assert!(
body.starts_with("é prelude") && body.contains("body();"),
"the region must cover the multibyte start through the end anchor, got: {body}"
);
assert!(
!body.contains("fn later()"),
"the region must still stop at the end anchor, got: {body}"
);
// The same through a multibyte START anchor, so the skip is measured
// from the region rather than from the top of the file.
let body = scrape_source_region(src, Some("é"), Some("\n}"))
.expect("a multibyte start anchor must be scraped");
assert!(body.starts_with('é'), "got: {body}");
}
/// The end anchor is searched past the region's first character, so a
/// region whose own start also matches the end anchor is not collapsed to
/// nothing.
#[test]
fn an_end_anchor_matching_at_the_start_does_not_collapse_the_region() {
let src = "\n}\nfn target() {\n body();\n}\n";
let body = scrape_source_region(src, None, Some("\n}")).unwrap();
assert!(
body.contains("fn target()") && body.contains("body();"),
"got: {body}"
);
}
}
#[cfg(test)]
mod tests {
use super::*;
+1 -1
View File
@@ -1,7 +1,7 @@
[package]
name = "gl"
description = "The gitlawb CLI — identity management, node control, MCP server"
version = "0.6.0" # x-release-please-version
version = "0.7.1" # x-release-please-version
edition.workspace = true
rust-version.workspace = true
license.workspace = true
+170
View File
@@ -550,4 +550,174 @@ mod tests {
);
assert!(key_trusted, "key stays trusted with operator origin");
}
#[test]
fn new_withholds_key_from_untrusted_origin_but_attaches_for_operator() {
let guard = ICAPTCHA_ENV_LOCK.lock().unwrap();
let prev_key = std::env::var_os("GITLAWB_ICAPTCHA_API_KEY");
let prev_op = std::env::var_os("GITLAWB_ICAPTCHA_URL");
std::env::set_var("GITLAWB_ICAPTCHA_API_KEY", "secret-bearer");
// Arm 1: No operator configured, node advertises an attacker origin.
// The key has no trusted destination; the advert is rejected.
std::env::remove_var("GITLAWB_ICAPTCHA_URL");
let untrusted =
IcaptchaCfg::new("did:key:zTEST", Some("https://evil.example".into()), None);
// Arm 1b: No operator configured, no URL advertised. The fallback goes
// to the default public origin, and with no operator to trust, the key
// must not ride.
let no_operator_no_advert = IcaptchaCfg::new("did:key:zTEST", None, None);
// Arm 2: Operator origin configured and advertised — key attached.
std::env::set_var("GITLAWB_ICAPTCHA_URL", "https://icap.mynode.example");
let operator_match = IcaptchaCfg::new(
"did:key:zTEST",
Some("https://icap.mynode.example/v1".into()),
None,
);
// Arm 3: Operator configured, default host advertised. The default host
// is allowlisted and selected, but its origin differs from the
// operator's, so key must not ride.
let default_advert = IcaptchaCfg::new(
"did:key:zTEST",
Some("https://icaptcha.gitlawb.com/v2".into()),
None,
);
// Arm 4: Operator configured, no URL advertised (the node sent only
// x-icaptcha-level, not x-icaptcha-url). resolve_solver_url falls back
// to the operator origin and marks the key trusted. This is the common
// API-key deployment path — a regression that additionally required
// url.is_some() for key attachment would leave all current tests green
// but make these fallback challenge/answer requests omit Authorization
// and fail against the protected operator service.
let operator_fallback = IcaptchaCfg::new("did:key:zTEST", None, Some(3));
// Arm 5: Operator configured, node advertises an attacker origin.
// The advert is rejected and the solver falls back to the operator's
// own origin, which keeps the key trusted — this is the exfiltration
// scenario the test name describes (withhold key from *untrusted*
// origin, not from the operator's fallback).
let operator_hostile_advert =
IcaptchaCfg::new("did:key:zTEST", Some("https://evil.example".into()), None);
// Arm 6: Operator configured, same host but different port.
// origin_key includes the port, so :8443 must not match the
// operator origin at the default :443 — the advert is rejected
// and the solver falls back to the operator origin.
let operator_diff_port = IcaptchaCfg::new(
"did:key:zTEST",
Some("https://icap.mynode.example:8443/evil".into()),
None,
);
// Restore env and release the lock BEFORE asserting, so a failing
// assertion can't poison the shared lock or cascade into a sibling.
match prev_key {
Some(v) => std::env::set_var("GITLAWB_ICAPTCHA_API_KEY", v),
None => std::env::remove_var("GITLAWB_ICAPTCHA_API_KEY"),
}
match prev_op {
Some(v) => std::env::set_var("GITLAWB_ICAPTCHA_URL", v),
None => std::env::remove_var("GITLAWB_ICAPTCHA_URL"),
}
drop(guard);
let mut failures: Vec<String> = Vec::new();
if untrusted.api_key.is_some() {
failures.push(format!(
"arm 1 (no operator, attacker advert): expected api_key=None, got {:?}",
untrusted.api_key
));
}
if untrusted.url != DEFAULT_URL {
failures.push(format!(
"arm 1 (no operator, attacker advert): expected url={DEFAULT_URL}, got {}",
untrusted.url
));
}
if no_operator_no_advert.api_key.is_some() {
failures.push(format!(
"arm 1b (no operator, no advert): expected api_key=None, got {:?}",
no_operator_no_advert.api_key
));
}
if no_operator_no_advert.url != DEFAULT_URL {
failures.push(format!(
"arm 1b (no operator, no advert): expected url={DEFAULT_URL}, got {}",
no_operator_no_advert.url
));
}
if operator_match.api_key.as_deref() != Some("secret-bearer") {
failures.push(format!(
"arm 2 (operator match): expected api_key=Some(\"secret-bearer\"), got {:?}",
operator_match.api_key
));
}
if operator_match.url != "https://icap.mynode.example/v1" {
failures.push(format!(
"arm 2 (operator match): expected url=https://icap.mynode.example/v1, got {}",
operator_match.url
));
}
if default_advert.api_key.is_some() {
failures.push(format!(
"arm 3 (operator + default advert): expected api_key=None, got {:?}",
default_advert.api_key
));
}
if default_advert.url != "https://icaptcha.gitlawb.com/v2" {
failures.push(format!(
"arm 3 (operator + default advert): expected url=https://icaptcha.gitlawb.com/v2, got {}",
default_advert.url
));
}
if operator_fallback.api_key.as_deref() != Some("secret-bearer") {
failures.push(format!(
"arm 4 (operator fallback, no advert): expected api_key=Some(\"secret-bearer\"), got {:?}",
operator_fallback.api_key
));
}
if operator_fallback.url != "https://icap.mynode.example" {
failures.push(format!(
"arm 4 (operator fallback, no advert): expected url=https://icap.mynode.example, got {}",
operator_fallback.url
));
}
if operator_hostile_advert.api_key.as_deref() != Some("secret-bearer") {
failures.push(format!(
"arm 5 (operator + hostile advert): expected api_key=Some(\"secret-bearer\"), got {:?}",
operator_hostile_advert.api_key
));
}
if operator_hostile_advert.url != "https://icap.mynode.example" {
failures.push(format!(
"arm 5 (operator + hostile advert): expected url=https://icap.mynode.example, got {}",
operator_hostile_advert.url
));
}
if operator_diff_port.api_key.as_deref() != Some("secret-bearer") {
failures.push(format!(
"arm 6 (operator + different port advert): expected api_key=Some(\"secret-bearer\"), got {:?}",
operator_diff_port.api_key
));
}
if operator_diff_port.url != "https://icap.mynode.example" {
failures.push(format!(
"arm 6 (operator + different port advert): expected url=https://icap.mynode.example, got {}",
operator_diff_port.url
));
}
assert!(failures.is_empty(), "\n{}", failures.join("\n"));
}
}
+65
View File
@@ -0,0 +1,65 @@
# fly.toml app configuration file generated for gitlawb-node-2 on 2026-07-22T09:47:18+08:00
#
# See https://fly.io/docs/reference/configuration/ for information about how to use this file.
#
app = 'gitlawb-node-2'
primary_region = 'sjc'
[build]
dockerfile = '../../Dockerfile'
[env]
GITLAWB_AUTO_SYNC = 'true'
GITLAWB_BOOTSTRAP_PEERS = 'https://node.gitlawb.com,https://node3.gitlawb.com'
GITLAWB_HOST = '0.0.0.0'
GITLAWB_KEY = '/data/keys/identity.pem'
GITLAWB_MAX_PACK_BYTES = '524288000'
GITLAWB_P2P_PORT = '7546'
GITLAWB_PORT = '7545'
GITLAWB_PUBLIC_URL = 'https://node2.gitlawb.com'
GITLAWB_REPOS_DIR = '/data/repos'
ICAPTCHA_MODE = 'enforce'
ICAPTCHA_PUBKEY = 'xjyPNqIbvc9U-kwXW6u9mDqRJ7E2UUMOaJdUWhpEXq8'
ICAPTCHA_REQUIRED_LEVEL = '3'
[[mounts]]
source = 'gitlawb_data'
destination = '/data'
[http_service]
internal_port = 7545
force_https = true
auto_stop_machines = 'off'
auto_start_machines = true
min_machines_running = 1
[http_service.concurrency]
type = 'connections'
hard_limit = 500
soft_limit = 400
[http_service.http_options]
idle_timeout = 120
[[http_service.checks]]
interval = '15s'
timeout = '5s'
grace_period = '30s'
method = 'GET'
path = '/ready'
[[services]]
protocol = 'udp'
internal_port = 7546
[[services.ports]]
port = 7546
[[vm]]
size = 'shared-cpu-1x'
memory = '1024'
[[metrics]]
port = 9091
path = '/metrics'
+65
View File
@@ -0,0 +1,65 @@
# fly.toml app configuration file generated for gitlawb-node-3 on 2026-07-22T09:47:19+08:00
#
# See https://fly.io/docs/reference/configuration/ for information about how to use this file.
#
app = 'gitlawb-node-3'
primary_region = 'nrt'
[build]
dockerfile = '../../Dockerfile'
[env]
GITLAWB_AUTO_SYNC = 'true'
GITLAWB_BOOTSTRAP_PEERS = 'https://node.gitlawb.com,https://node2.gitlawb.com'
GITLAWB_HOST = '0.0.0.0'
GITLAWB_KEY = '/data/keys/identity.pem'
GITLAWB_MAX_PACK_BYTES = '524288000'
GITLAWB_P2P_PORT = '7546'
GITLAWB_PORT = '7545'
GITLAWB_PUBLIC_URL = 'https://node3.gitlawb.com'
GITLAWB_REPOS_DIR = '/data/repos'
ICAPTCHA_MODE = 'enforce'
ICAPTCHA_PUBKEY = 'xjyPNqIbvc9U-kwXW6u9mDqRJ7E2UUMOaJdUWhpEXq8'
ICAPTCHA_REQUIRED_LEVEL = '3'
[[mounts]]
source = 'gitlawb_data'
destination = '/data'
[http_service]
internal_port = 7545
force_https = true
auto_stop_machines = 'off'
auto_start_machines = true
min_machines_running = 1
[http_service.concurrency]
type = 'connections'
hard_limit = 500
soft_limit = 400
[http_service.http_options]
idle_timeout = 120
[[http_service.checks]]
interval = '15s'
timeout = '5s'
grace_period = '30s'
method = 'GET'
path = '/ready'
[[services]]
protocol = 'udp'
internal_port = 7546
[[services.ports]]
port = 7546
[[vm]]
size = 'shared-cpu-1x'
memory = '1024'
[[metrics]]
port = 9091
path = '/metrics'
+1 -1
View File
@@ -46,7 +46,7 @@ build_darwin_arm64() {
echo "==> Building darwin-arm64 (native cargo)..."
cd "$ROOT"
export PATH="$HOME/.cargo/bin:$PATH"
cargo build --release -p gl -p git-remote-gitlawb
cargo build --release --locked -p gl -p git-remote-gitlawb
cp target/release/gl "$OUT/gl-darwin-arm64"
cp target/release/git-remote-gitlawb "$OUT/git-remote-gitlawb-darwin-arm64"
chmod +x "$OUT/gl-darwin-arm64" "$OUT/git-remote-gitlawb-darwin-arm64"
+75
View File
@@ -0,0 +1,75 @@
#!/usr/bin/env bash
#
# Dependency-purity gate for the `gitlawb-core` crate.
#
# gitlawb-core is embedded by every consumer (the gl CLI, git-remote-gitlawb,
# the node daemon), so it must stay lean. This script recomputes gitlawb-core's
# NORMAL (non-dev, non-build) transitive dependency set and fails if it contains
# any crate not present in ci/gitlawb-core-allowed-deps.txt.
#
# Hard-fail direction: a crate present now but NOT allowlisted. That is the case
# the gate exists to catch (core silently gaining a heavy dependency).
# Informational only: an allowlisted crate no longer present (stale entry) — a
# legitimate dependency removal should not red CI, so it is reported, not failed.
#
# Runnable from anywhere in the repo.
set -euo pipefail
ROOT="$(git rev-parse --show-toplevel)"
ALLOW="$ROOT/ci/gitlawb-core-allowed-deps.txt"
if [ ! -f "$ALLOW" ]; then
echo "ERROR: allowlist not found at $ALLOW" >&2
exit 1
fi
# `cargo tree` resolves WITHOUT --locked on purpose, unlike the blocking cargo
# jobs in pr-checks. This gate answers one question, whether gitlawb-core's
# dependency closure gained something unallowlisted, and a stale Cargo.lock
# would red it for a reason that has nothing to do with that closure. Lock
# staleness is already caught by the --locked cargo jobs, so it does not need a
# second gate here.
#
# Resolving can refresh Cargo.lock as a side effect, so snapshot and restore
# it: this check must never leave the working tree dirty when run locally.
lock_backup="$(mktemp)"
cp "$ROOT/Cargo.lock" "$lock_backup"
restore_lock() { cp "$lock_backup" "$ROOT/Cargo.lock"; rm -f "$lock_backup"; }
trap restore_lock EXIT
# Current normal-dependency closure of gitlawb-core, one crate name per line.
# Must match the regen command documented in the allowlist header exactly.
current="$(
cargo tree -p gitlawb-core --edges normal --prefix none --manifest-path "$ROOT/Cargo.toml" \
| sed -E 's/ v[0-9].*$//' \
| grep -v '^gitlawb-core$' \
| sort -u
)"
# Allowlist with comments and blank lines stripped.
allowed="$(grep -vE '^[[:space:]]*(#|$)' "$ALLOW" | sort -u)"
# comm needs sorted input; both sides are sorted above.
offenders="$(comm -23 <(printf '%s\n' "$current") <(printf '%s\n' "$allowed"))"
stale="$(comm -13 <(printf '%s\n' "$current") <(printf '%s\n' "$allowed"))"
if [ -n "$stale" ]; then
echo "NOTE: allowlisted crates no longer in gitlawb-core's dependency tree"
echo " (safe to prune from ci/gitlawb-core-allowed-deps.txt):"
printf ' %s\n' $stale
echo
fi
if [ -n "$offenders" ]; then
{
echo "ERROR: gitlawb-core gained dependencies not on the allowlist:"
printf ' %s\n' $offenders
echo
echo "gitlawb-core must stay embeddable and lean. If a new dependency is"
echo "intentional, add it to ci/gitlawb-core-allowed-deps.txt (regen command"
echo "is in that file's header). Otherwise, drop the dependency."
} >&2
exit 1
fi
echo "gitlawb-core dependency purity: OK ($(printf '%s\n' "$current" | wc -l | tr -d ' ') normal deps, all allowlisted)."