`cargo audit` started failing on every branch today: h2 0.4.13 is vulnerable to
RUSTSEC-2026-0258, "h2 unbounded empty DATA frames", fixed in 0.4.16. The
advisory was published after main's last green run, so the existing green ticks
on open PRs are stale results rather than evidence — anything that re-runs CI
goes red until this lands.
h2 arrives transitively through hyper, so no manifest changes: the lockfile is
the whole fix.
Edited by hand rather than with `cargo update -p h2`, deliberately. Running the
update on this machine also rewrote nine unrelated entries — `syn` 2.0.117 to
1.0.109 for data-encoding-macro-internal, `windows-sys` 0.61.2 to 0.52.0 for
errno, `socket2` 0.6.3 to 0.5.10 in three places — all DOWNGRADES, and all
still present with `--precise 0.4.16`. That is local index drift against
whatever generated main's lockfile, not something this fix needs, and it does
not belong in a security bump. Only h2's `version` and `checksum` lines change
here; `cargo check --locked` accepting the result is what confirms 0.4.16 needs
no dependency changes of its own.
Verified with `cargo check --locked --workspace` (exit 0) and, on a branch that
carries the Windows test-gating fix, the full suite: gitlawb-core 104, gl 328,
git-remote-gitlawb 58. `--all-targets` cannot be checked from main on Windows
because the unix-only test code does not compile there, which is what #330's
first commit fixes; Linux CI covers it.
satisfies_threshold counted raw signature entries against the maintainer
list, so a certificate carrying the same maintainer's signature duplicated
across two entries satisfied a 2-of-N threshold with only one real
signer. Dedupe by collecting valid signers into a set before comparing
against the threshold, so each maintainer counts once regardless of how
many times their signature appears in the certificate.
The domain separation owner_slug + ':' + repo_name has no length prefix, so
it is injective only while owner_slug cannot contain a colon. State that on
the pub(crate) function and guard it with a debug_assert, since a raw DID
would collide (('did:key:abc','x') and ('did','key:abc:x') hash the same).
Spelling the escape character as `ESCAPE '\'` puts a backslash inside an
ordinary SQL string literal. Under the legacy `standard_conforming_strings=off`
setting that backslash escapes the closing quote, so the literal runs on and the
statement fails to parse. The failure happens before the bound parameters are
considered, so every prefix lookup breaks, including prefixes that contain no
metacharacter at all, and short certificate IDs stop resolving entirely.
Connection settings come from outside the process, so that mode can arrive from
a database- or role-level setting.
Use `!` as the LIKE escape character instead, escaping `%`, `_`, and `!` in the
bound pattern. The statement then holds no backslash and parses the same either
way, without needing a second layer of backslash escaping. A backslash in the
prefix becomes an ordinary character, which is the literal match the search
wants anyway.
Cover it with a test that runs the production query through a second pool that
requests the legacy mode at connection startup, so a future return to a
parser-mode-dependent literal is caught rather than only wildcard handling under
the default configuration.
Refs #319
The deadline test hand-copied the advisory-lock key derivation, and that
copy drifted once the key moved to SHA-256, silently breaking the lock
collision the test is built around. Import the production function
instead so the key cannot diverge again.
Short-ID resolution passes a caller-provided prefix into a LIKE pattern. Any '%', '_' or '\' in that prefix is interpreted as a LIKE wildcard rather than a literal character, so lookups can match certificates outside the requested prefix.
Escape the three metacharacters in the prefix and add an explicit ESCAPE '\' clause so prefix matching is always literal.
Add a regression test covering literal '%', '_' and '\' prefixes.
Co-Authored-By: cairn-code <[email protected]>
Cross-model review round, findings corroborated by the in-process reliability
reviewer.
Both channel waits were unbounded. If the reader thread never reached its read,
or never returned after the peer close, the test wedged the binary instead of
failing, and cargo test has no per-test timeout to rescue it. Both are now
recv_timeout with bounds far above any real scheduling delay, so hitting one is
a genuine defect report rather than a flake, and the failure names which wait
stalled.
The handshake paragraph also overstated what the handshake buys. It listed "the
reader thread not yet scheduled to reach the read at all" among the states the
handshake removes, which reads as though the whole scheduling window is closed.
It is not: the handshake removes thread spawn from the window, and a reader
descheduled between its own Started send and its read is still a false-green
path. That path is now stated plainly instead of being left to the following
paragraph to correct.
Re-verified after the change: probe RED 50/50 under a deleted
set_nonblocking(false) at full-suite parallelism, RED on the named message under
a deleted set_read_timeout, 50/50 green unmutated, fmt and clippy clean.
Two findings from review of the probe test.
The helper has two postconditions and only one was guarded. Deleting
set_read_timeout left every test green while a stalled peer would park the
shim's only accept-loop thread forever, since handle_conn runs inline in that
loop. std exposes read_timeout() as a getter, so this half is checkable exactly
with no timing component. Confirmed load-bearing: deleting the line reds the
probe on the new message.
The probe also normalized one socket and then read from a try_clone of it, so
it only proved anything if blocking mode is shared across the two. On Linux it
is, because try_clone is dup(2) and O_NONBLOCK lives on the shared open file
description. On Windows try_clone goes through WSADuplicateSocketW and WSASocketW
(std os/windows/io/socket.rs), which mints a new socket descriptor, and whether
FIONBIO follows it is a question this test would have depended on without being
able to answer. It would have failed silently and green, on the one platform
where the underlying defect is real. The clone was unnecessary, so the reader
now owns the socket the helper normalized and the question does not arise.
Also correct two overstatements in the probe's own doc comment. It called
itself deterministic while its verdict is a 250ms window, and it described the
residual false-green path as "the few instructions between the signal and the
call" when the window is a full cross-thread round trip: signal, read, return,
channel send, and this thread waking. Both now say what the code does.
Re-verified after the change: probe RED 50/50 under a deleted
set_nonblocking(false) at full-suite parallelism, RED on the named message under
a deleted set_read_timeout, and 50/50 green unmutated. The end-to-end test came
in at 39/50 on this run against the first mutation, a third sample alongside 43
and 47, so its doc comment now reports the spread instead of two numbers.
The regression test added alongside the shim fix could pass with the fix
removed. It synchronized on a channel send that happens after accept() but
before handle_conn runs, so the request bytes could already be buffered by
the time the shim read them, and the read then succeeded on a non-blocking
socket. Measured at 43 and 47 of 50 full-suite runs catching a deleted
set_nonblocking(false), so it went false-green roughly one run in eight.
Extract the normalization into normalize_accepted_stream and assert its
postcondition directly, with no client and no request-arrival race. The new
probe puts a connected socket into non-blocking mode, calls the helper, and
checks that a reader parks instead of returning WouldBlock. Blocking mode has
no getter to query: std exposes none and Winsock has no FIONBIO getter at all,
so the property has to be observed through behavior.
The reader signals Started immediately before its read and the main thread
waits for that signal unbounded before opening the 250ms window. Without the
handshake the window would also be satisfied by a thread that has not been
scheduled yet, which would reintroduce the same silent hole one level down.
Peer close comes after the assertion, never before; closing first unblocks the
read and fails the test even when the helper is correct.
Also correct a platform claim this PR had in five places. The bug is not
Windows-only. An accepted socket inherits the listener's flag on BSD-derived
stacks, so macOS and the BSDs are exposed too; Linux is the exception because
std uses accept4 there, which does not inherit file status flags
(rust-lang/rust#67027). A macOS contributor could hit this today while the
comments said it was impossible. The macOS half is documented rather than
executed, and the comments now say so.
Verification, full-suite runs at default parallelism:
helper body mutated (set_nonblocking(false) deleted): probe RED 50/50
call site mutated (helper call deleted): probe GREEN 0/50,
integration test 45
and 46 of 50
unmutated: 50/50 green
The call-site row is the probe's blind spot, measured rather than assumed: it
calls the helper directly, so only the integration test notices a deleted
call, and it does so at a sampled rate rather than reliably. That residual is
recorded in the integration test's doc comment.
The test reproduces acquire_write's advisory-lock key derivation locally so
a held session lock can stall the handler. repo_store's advisory_lock_key
now uses SHA-256, so the DefaultHasher copy no longer collided and the
stall never happened. Update the local copy to the SHA-256 derivation.
Resolve conflicts: keep the SHA-256 advisory-lock key change alongside
upstream's connection-affine RepoWriteGuard rework and its F1/F3b/F4/U8
test suite; RUN-A-NODE.md keeps both the updated Public URL row and the
rolling-upgrade advisory-lock row.
start_shim sets the listener non-blocking so its accept loop can poll the
stop flag. On unix accept() hands back a blocking socket, so that never
showed; on Windows the accepted socket inherits the listener's FIONBIO
state and std does not reset it, so every connection arrived non-blocking.
handle_conn reads with read_line(..).unwrap_or(0), which turns the
resulting WouldBlock into "peer closed" and returns without writing a
response, dropping the connection and aborting the client mid-request. The
shim accepts as soon as the handshake completes, strictly before the
request bytes arrive, so the window was open on every connection and
closed only because the 5ms accept poll usually lagged the client. That is
what made real_git_withheld_shaped_first_post fail on windows-latest with
os error 10053 and zero POSTs recorded.
Normalizing at handle_conn entry also restores the 30s read timeout on the
next line, which a non-blocking socket silently made meaningless.
The regression test forces the Windows socket shape explicitly, so it is
load-bearing on every platform rather than only the one that exhibits the
bug, and it establishes the accept-before-request interleaving with a
handshake rather than a sleep: a timing-based version cannot flake red but
can flake green under load, which would make the guard vacuous.
Rebased onto current main. Keep budget-timeout wrappers on get_by_cid;
convert sqlx failures with Into<AppError> (not AppError::Internal) so
pool closed/timeouts become 503 db_unavailable. Same for list_pins and
arweave anchors, with closed-pool regressions.
Add a regression test for verify_strict: a signature forged under a weak (small-order) public key satisfies the verification equation but must be rejected. The non-strict verify path accepts it.
Co-Authored-By: cairn-code <[email protected]>
Rebased onto current main. Keep Overloaded/UnresolvableDid arms; make
Internal and non-unavailable Db responses opaque with server-side logs,
and pin exact JSON in unit + anonymous route tests.
Resolves jatmn's #192 P2. run_bounded's clean-exit teardown only terminated
the job when one had been established; when assign_job returned None it did
nothing, so a git-remote-gitlawb helper descendant that outlived the `git
fetch` leader while holding the inherited stdout/stderr pipe write-ends stalled
the reader joins on the helper's ~300s HTTP timeout, past the bound run_bounded
promises. The timeout path's reap_tree (taskkill /T) fallback cannot rescue the
clean path: once the leader has cleanly exited it no longer roots the
descendant in the parent-PID tree, so taskkill /T cannot reach the orphan.
The Job Object is the only reliable Windows teardown, so make it mandatory:
assign_job now returns JobHandle and panics per-step if the job cannot be
established, and both teardown paths unconditionally terminate it. Nested
unnamed jobs with KILL_ON_JOB_CLOSE are standard on the shipped
x86_64-pc-windows-msvc / CI runners, so this costs no coverage; it converts a
silent ~300s hang into a loud, actionable failure and strengthens the bound
(KILL_ON_JOB_CLOSE kills the descendant at latest when JobHandle drops).
reap_tree, whose only Windows use was the removed None arm, is narrowed to
cfg(all(not(unix), not(windows))).
The change is entirely cfg(windows). Verified by cross-compiling the tests for
x86_64-pc-windows-gnu (compiles clean, clippy -D warnings clean, no dead_code
from the reap_tree narrowing) and by the unchanged Linux git-remote-gitlawb
suite (green). The Windows runtime teardown is reasoned-not-run here; CI's
Windows job is the runtime gate.
run_bounded's clean-exit descendant reap was Unix-only: on Windows a
git fetch leader that exits cleanly leaves the git-remote-gitlawb helper
blocked in its ~300s HTTP request holding the inherited pipes, and
taskkill /T can't reach a descendant once the leader is gone, so the
reader joins blew past the promised bound. The child is now assigned to
a Job Object at spawn (the analog of the Unix process_group(0)), and the
job is terminated on both the timeout and clean-exit paths, which kills
the whole tree regardless of leader liveness. KILL_ON_JOB_CLOSE is the
safety net. The clean-exit regression is now cross-platform so it
exercises this on a future Windows CI lane (#228).
Unix tests unchanged and green; the Windows path is compile-verified for
x86_64-pc-windows-msvc (runtime coverage lands with #228's CI lane).
The test accepted any nonzero git exit (with posts>=1) as the recorded #191
withheld rejection, so a truncated response or a later HTTP failure after the
first POST could pass green. Assert the specific `expected ACK/NAK` signature so
only that break passes, and pin LC_ALL=C so the diagnostic stays the untranslated
English string on a git build with l10n installed. (#192 F6)
The gitlawb_fetches_total gate keyed on whether the request carried `done`,
which both undercounts a no-done completion (a client can finish a flush-
terminated have round with `ACK <oid> ready` + pack and no `done` -> counted 0)
and, on the filtered path, counts a rejected response per POST (the pre-#191
two-POST negotiation the client refuses was counted twice).
Derive completion from the response instead. upload_pack / upload_pack_excluding
now return whether they served a pack; response_served_pack detects the PACK
magic in both side-band-64k and raw framing. The plain path counts when the
response delivered a pack (catches no-done, skips negotiation-only rounds so an
N-round fetch counts once); the filtered path stays gated on the finalizing
`done` round until #191 makes that negotiation valid. Also tighten
request_finalizes_fetch to reject 0003 framing and trailing bytes after the
`done` pkt-line. (#192 F1/F2 + CodeRabbit repos.rs:761)
init() did an unsynchronized REGISTRY.get() check then OnceLock::set(...).
expect(...) for each metric, so two concurrent callers both passed the check
and the loser panicked at 'set INFO once'. Two tests call init(), so the
parallel test runner could hit this. Guard the body with std::sync::Once so it
runs exactly once. Verified: a 32-thread barrier stress-test panics before the
fix and passes after (run in isolation; not committed since the process-global
registry makes the race non-deterministic in the shared-binary suite). (#192)
A non-UTF-8 or non-hex 4-byte length prefix was collapsed to 0000 via
unwrap_or, so a garbled header was handled exactly like a flush -- masking
corrupted/desynchronized protocol input and letting accumulated have lines
go out as a synthesized flush round. Propagate the parse error instead;
a real 0000 still parses as a valid flush. (#192)
run_bounded only tore down the process group on the deadline path. When the git
leader exits on its own but a descendant (the git-remote-gitlawb helper mid-HTTP
request) outlives it holding the inherited stdout/stderr write-ends, the reader
joins blocked on the helper's ~300s timeout, past the deadline run_bounded
promises. Close the group on the clean-exit path too, guarded by a kill(-pgid, 0)
liveness probe so a recycled pid is never signalled (the process-group ID stays
reserved while any member is alive). Adds a leader-exits-first fixture and a
regression test.
Verified by execution: with the clean-path reap removed the new test blocks ~10s
on the descendant (RED); with the reap it returns promptly.
git_upload_pack recorded a fetch and observed a pack size on every successful
POST. In HTTP stateless-RPC the plain serve path streams a pack only on the
finalizing `done` round, so an N-round incremental fetch was reported as N
completed fetches. The filtered path (upload_pack_excluding) instead serves a
self-contained full pack on the single POST that reaches it, with no `done`, so
gating purely on `done` would drop those private (path-scoped) fetches to zero.
Count once on the POST that actually completes a fetch: the `done` round on the
plain path (request_finalizes_fetch parses pkt-lines and fails closed on a
malformed body), or the one filtered-serve POST (served_filtered_pack).
should_count_fetch captures the decision; a completed fetch is exactly one such
POST, so a multi-round negotiation is never double-counted.
The observe_pack_size call still measures the request body, not the served pack;
that mislabel predates this change and is left as a follow-up.
Verified by execution: a three-round plain fetch and a no-`done` filtered serve
each record exactly one completion; reverting the gate to count per POST makes
the plain fetch record 3, and breaking the decision to AND drops both to 0.
The large-output and descendants tests re-invoke the test binary as their
fixtures instead of spawning sh/seq, so they run on x86_64-pc-windows-msvc;
the non-unix timeout path tears down the whole fetch tree with taskkill /T
before joining the drainers instead of killing only the leader. Also folds
the receive-pack POST block into post_pack_round (review nitpick), removing
the duplicated status-check/forward logic.
Address jatmn's two P2 review findings on real_git_fetch.rs.
F1 (INV-22): run_bounded killed only the top-level `git fetch` on the
timeout path, so the git-remote-gitlawb helper descendant kept the
inherited stdout/stderr pipe write-ends open and the reader joins blocked
on the helper's ~300s HTTP timeout instead of the 30s deadline. Spawn the
child with process_group(0) and, on timeout, tear the whole group down
(SIGTERM, ESRCH poll, SIGKILL escalation, bounded cap, then reap) before
joining, mirroring gitlawb-node/src/git/smart_http.rs. libc is added as a
cfg(unix) dev-dep; non-unix keeps the child.kill() fallback so the
always-on test still compiles. New regression run_bounded_reaps_descendants_holding_the_pipe:
a backgrounded sleep inherits the pipe and outlives a 1s-deadline leader;
reverting to a leader-only kill turns it RED (elapsed ~10s).
F2 (INV-21): the non-success branch of real_git_withheld_shaped_first_post
accepted any `git fetch` failure and only printed the POST count, so a
failure before the first POST (broken advertisement, helper lookup,
connection) passed as the expected withheld rejection. Assert posts >= 1
before accepting the nonzero exit. Forcing a pre-POST failure (0 POSTs)
turns the assertion RED.
F1: a nonempty have-batch terminated by EOF means git aborted mid-negotiation, but
the fetch loop folded RoundEnd::Eof into the Done arm, appended a synthetic
0009done, and POSTed it — so a cancelled private fetch still issued an authorized,
signed upload-pack request and made the node generate and stream a full pack to a
client that had gone away. Split the arm: only a real done pkt finalizes; a
nonempty EOF terminates without a POST. The nonempty_eof_batch_posts_as_final_round
test that locked in the wrong behavior is flipped to assert no POST.
F2: fetch_with_helper piped stdout+stderr but only polled try_wait, so a fetch
emitting more than an OS pipe buffer (~64 KiB) blocked git before it exited and the
harness reported a false 30s deadlock. Drain both streams on reader threads while
enforcing the deadline, so the deadline measures a real protocol hang only.
F3: the integration repos were removed only on the success path (predictable
pid/counter names), so a timeout/panic leaked real git repos and could poison a
later run. Root them in a RAII tempfile::TempDir that drops on unwind.
RED->GREEN: nonempty_eof_batch_aborts_without_posting (pre-fix POSTs a synthetic
done → the aborted-negotiation POST fails against the single-response server; fixed
makes one request, no POST). repos_are_cleaned_up_on_unwind (temp dir gone after a
panic unwinds past the guard). Full git-remote-gitlawb suite 42 unit + 3 integration
green, fmt + clippy clean.
Adds executed coverage for the negotiation branches the first pass left
unrun: an invalid pkt-line length (rejected, not underflowed), a non-EOF
read error (propagated, not swallowed as end-of-stream), an immediately
empty upload-pack request (skips the POST), a nonempty have-batch
terminated by EOF (POSTed as a final done round), and an empty
receive-pack body (skips the POST). Each was mutation-checked to fail
without its guard, so the coverage is load-bearing rather than vacuous.
A code-review pass found the negotiation loop would POST once per bare `0000`
flush that carried no haves, so a malformed `wants + N*0000 + EOF` stream
amplified into N signed POSTs. Real git never sends a content-free mid-negotiation
flush and the peer is upstream of the local git process, so this was not
reachable in practice, but it left the loop unbounded per input. Skip a round
that carries no new haves (an empty flush) and only POST rounds with have content
or a terminator, which also makes the loop provably bounded by git's finite have
set. The fresh-clone done-with-no-haves round still POSTs.
A committed integration test drives a real `git fetch` through the built helper
against a real `git upload-pack --stateless-rpc`, so the stateful-to-stateless
ACK bridging is proven by execution rather than reasoned. The main.rs mock tests
cannot falsify it: a pre-scripted Cursor never reacts to the server's ACKs. An
in-test shim replicates the node's v0 serving; the fetch is forced to at least
two negotiation rounds (a fixture that resolved in one round fails the test) and
the resulting object graph is verified.
The second scenario drives the withheld-blob shape (a full pack on the first
POST, as upload_pack_excluding does) through real git and records the outcome:
real git rejects a pack where it expected an ACK continuation ("expected
ACK/NAK, got ..."), so a multi-round fetch of a withheld repo does not complete.
The helper forwards and terminates cleanly rather than hanging, so this is a
node-side concern, not a helper defect, left as a follow-up per the plan's
Withheld-Path Decision. The test guards the helper's non-hang behavior and
surfaces the break if the assumption ever changes.
git-remote-gitlawb advertised `connect`, so git spoke the stateful native
protocol, but handle_connect collapsed the exchange into one GET plus one POST.
A multi-round fetch (more than ~32 overlapping commits) deadlocked: git sent a
flush-terminated have-batch and blocked for an ACK/NAK the helper never sent,
while the helper blocked reading for a `done` git never sent.
Turn Phase 2 into a per-round stateless-RPC client loop. read_upload_pack_round
returns one round at a time (at a flush, done, or EOF) instead of buffering past
flushes waiting for `done`; negotiate_upload_pack captures the wants once and
POSTs a self-contained request per flush-terminated batch: wants, one flush,
every have accumulated so far, and exactly one terminator. Each ACK/NAK is
streamed back to git so it advances, until the done round returns the pack. The
node's `git upload-pack --stateless-rpc` keeps no state between POSTs, so wants
and all prior haves are re-sent every round and no intermediate flush survives
into a body (which would truncate the negotiation server-side).
Signing is preserved per round: every POST carries the Phase-1 decision (signed
after the 404 escalation for a private repo, anonymous for a public one), and a
mid-negotiation denial surfaces through the sanitized error path rather than
reading as an empty or successful fetch. The receive-pack path is unchanged.
Covers the deadlock repro, multi-round accumulation on the wire, per-round
signing for private and public fetches, mid-negotiation denial surfacing, the
withheld-shaped forwarding path, and single-round non-regression.
The git_read_semaphore doc claimed it bounds both info/refs advertisements,
but the receive-pack advertisement draws from git_push_advert_semaphore
(api/repos.rs), so an operator sizing the read pool for both would undercount
their read capacity.
README's settings table also omitted the served-git concurrency env vars that
.env.example already documents: MAX_CONCURRENT_GIT_OPS, MAX_CONCURRENT_GIT_PUSHES,
MAX_CONCURRENT_READS_PER_CALLER, and MAX_CONCURRENT_PIN_TASKS.
GITLAWB_MAX_CONCURRENT_PIN_TASKS caps how many pin loops run at once, not how
much object-id list memory is retained. On the local IPFS path the caller
materializes the list before pin_new_objects_gated acquires, so a parked loop
still holds its full list and the parked count is capped only per repo by
EncryptInflight. config.rs and the pin_new_objects_gated docstring already say
this; the env template still said the knob bounds the memory.
The drain barrier sampled target = count + 1 before connecting the flush and
waited for any next increment. A leak accepted after that sample and before
the flush was counted satisfied the wait; the test then subtracted the flush
as if it were that leak, so leak_count could be 0 with a real leak present
(false green).
Tag the flush connection instead: the accept loop records each accepted
connection's source port, and drain waits until the flush's own source port
is recorded (the flush is never counted). Because accept() is FIFO, when the
flush is recorded every connection that completed its handshake before it is
recorded too, so the post-drain count still covers all egress in flight.
Verified with a probe that reproduces the reported scenario (leak accepted
before the flush is now counted) before removing it.
DefaultHasher is not frozen by the Rust standard and has shifted across
toolchain versions, so the same (owner_slug, repo_name) produced a
different i64 advisory-lock key on different builds and broke
cross-machine write-exclusion for the same repo.
Hash with SHA-256 instead and pin the result with a golden-value test,
plus a one-axis-at-a-time differential test so a regression that drops
either parameter is caught. Document the SHA-256 re-keying caveat on
acquire_write and in the RUN-A-NODE.md operator checklist (drain writes
or cut over single-node during a rolling upgrade).
The tripwire bound a hard-coded 127.0.0.2 alias, which macOS does not route
by default (only 127.0.0.1), so the test could not run on apple-darwin
targets. Switch the fixture to [::1] (IPv6 loopback), which is present by
default on Linux, macOS, and Windows and is NOT covered by NO_PROXY
(127.0.0.1, localhost).
The negative control still holds: hyper-util's proxy matcher has no implicit
loopback bypass — a host is only proxied-around when literally listed in
NO_PROXY (verified in the pinned hyper-util source). Both the positive
control (direct reach of [::1]) and negative control (proxy intercept) still
pass.
- Tripwire: fail loudly (expect) when 127.0.0.2 cannot be bound, instead of
printing SKIP and returning green with zero coverage of the negative
control. Linux routes 127/8 as loopback; macOS needs an explicit alias,
and the message says so.
- Observer: restore a bounded drain/ack barrier (Mutex + Condvar). The
detached accept thread signals after each accept; the test opens a flush
connection and blocks until it is acknowledged. accept() is FIFO, so every
leak connection that completed its handshake before the flush is counted
before the flush. Leaks that start their handshake after the drain remain
the irreducible window, now documented as such.
- support: document that arm_blackhole / disarm_proxy_env mutate process-wide
env vars and are safe only because each consuming binary has exactly one
test; a future second test must be serialized against them.
The stabilise() polling loop cannot prove no more connections are coming,
only that none arrived in the last 100ms. A carefully timed background
leak could schedule its connection after stabilise stabilises and the test
would pass despite a live egress. Replace with a simple counter read
before and after obtain_proof. Connections accepted during the synchronous
call are counted; a leak whose TCP handshake completes after the counter
is read is outside the assertion window — this is now documented as a
known limitation of the single-threaded test harness.
Also narrow the fire-and-forget claim: the observer catches leaks that a
closed-port blackhole would miss (because it counts connections regardless
of error propagation), but the timing race inherent in a detached accept
thread means the guarantee is best-effort for post-return handshakes.
Replace the closed-port proxy with a ProxyObserver (TCP listener with
acknowledged accept-loop flushing) so fire-and-forget leaks whose error
is discarded are caught and the count is synchronized before assertion.
Add a blackhole tripwire asserting obtain_proof fails against 127.0.0.2
with the proxy armed, narrowed to a connect-level error to prevent
vacuous passes. Share arm_blackhole across both test files via a
tests/support module.
Document the observer's coverage scope: it only witnesses leaks on the
single exercised happy path; error/retry branches and non-proxy
transports are outside its guarantee.
Two review findings from the last round.
The object-store readability probe swallowed EACCES on objects/pack exactly
like the benign NotFound of a loose-only store, and never checked the loose
fan-out at all. A present object behind an unreadable directory therefore
looked absent: git cat-file printed a clean missing, the probe said the store
was fine, and /ipfs/{cid} answered 404 for content that exists. The same bool
drives classify_store_fault, so an unreadable store also came back as a
terminal 500 rather than a retryable 503. Only NotFound stays benign now, and
the probe additionally opens the object's own fan-out path. That check is
scoped to the oid (validated as 40 or 64 hex before any path component is
derived) rather than sweeping all 256 directories, so an unrelated broken
directory cannot veto every lookup in the repo, and the caller pays one open
instead of a directory drain it does not need. An unreadable directory now
fails closed: absence cannot be certified through something we cannot read, so
a missing object there sheds a retryable 503 instead of a 404.
Both pin loops held the shared pin permit across a synchronous, unbounded
git read, so a wedged filesystem pinned a slot in a pool that defers rather
than sheds, and blocked a runtime worker while it did. The read now runs
through the bounded, process-group-reaped helper under spawn_blocking, against
the batch deadline, and the Pinata loop gets the batch budget it never had.
The budget gate is shared between the two loops so they cannot drift, and it
declines to start a read it cannot pay for rather than spawning a child only
to reap it.
A fault stops the batch only when the object store is genuinely unreadable,
where every remaining object would fail the same way. An object-scoped fault
skips that object and continues, because one corrupt object should not cost a
healthy batch, and the recovery path re-offers the same list and would break at
the same place forever.
Jatmn's #209 round: known_ungated is empty on main and the three listed
handlers are gated, so the guide must not teach them as exceptions. Restore
the signature/certificate section with a trusted positive case and a forged
rejection case instead of blessing a forged "valid" drive.
Scope the denial-test guidance to each route's actual authorization
rule instead of a blanket non-owner assertion, since several
documented routes (star_repo, replica actions, bounty actions,
author-based closing) intentionally authorize non-owners.
Name the known_ungated exceptions (list_webhooks, list_replicas,
list_protected_branches) instead of implying full repo-scoped read
coverage.
Describe the listed jobs as workflow checks that run on every PR
rather than merge-blocking CI, matching the live main branch ruleset
(one required review, no required status checks).
Encode the contributor-facing gate rules (the three owner-gate forms in
use and the by-design non-owner exceptions, path-scoped reads and the
listings contract, denial semantics, deny-test expectations, migration
versioning, client denial surfacing, MSRV and the exact CI gate set) in
a root AGENTS.md that coding agents load automatically, and point
CONTRIBUTING.md at it. Enforcement labels state only what is on main
today.
Whitelist eight safe AppError variants; opaque Git and anything else.
Add a source-scrape guard for map_err sites, a real sqlx context chain
test, and a tasks(limit) ceiling pin at 200.
Drop the head-file fetch and Rust lexer. Match test attributes on
added lines only, with horizontal-whitespace anchoring, covering bare
#[test], #[cfg(test)], namespaced forms (#[tokio::test(...)]), optional
leading ::, and raw-identifier spellings. needs-tests is advisory, so
the simpler matcher is enough.
Addresses jatmn's review: an exhausted 15-file scan or an oversized/
unreadable blob no longer asserts "no test found" (needs-tests was
being applied on unverified evidence), blobs over 5MB are skipped
before decoding, the "\ No newline at end of file" patch marker no
longer miscounts as an added context line, and #[r#test] /
#[tokio::r#test] raw-identifier attributes are now recognized.
Use function-pointer map_err to clear redundant_closure. Opaque only
when a sqlx::Error is in the anyhow chain so claim/finish business
messages survive. Clamp GraphQL tasks(limit); add mutation coverage.
Addresses jatmn's second round of review on pr-triage's needs-tests check:
hitting the per-PR fetch cap now logs and stops scanning instead of
throwing (which was skipping label reconciliation entirely), file content
comes from the Git blob API instead of the Contents API to avoid the
encoding: none gap for 1-100MB files, the added-line check now requires a
non-whitespace attribute character (not just added whitespace) on the
added line, stripped literals leave a placeholder so they can't glue
adjacent tokens into a false match, and the namespaced-test regex now
accepts a leading :: for absolute-path attributes.
Route query and mutation DB failures through a shared helper that
logs the cause chain server-side and returns a generic client
message, so unauthenticated POST /graphql cannot leak sqlx text.
A diff hunk carries no lexical context from lines outside it, so scanning
patch text alone could misclassify an added line that sits inside a comment
or raw string opened earlier in the file (or outside the hunk's context
window). Fetch the head-version file content via the contents API and lex it
in file order, checking only the lines the diff actually added.
The needs-tests detection regex only matched a literal #[test] or #[cfg(test)
immediately after #[, so it missed namespaced test attributes like
#[tokio::test] or #[async_std::test]. Any PR whose only new tests were async
(the common pattern in this codebase, e.g. crates/gl/src/agent.rs) got falsely
labeled needs-tests even when tests were added in the same commit.
Reproduced against PR #198: it added two #[tokio::test]-only tests in the same
commit as the source change, and the triage bot still commented needs-tests.
Widens the regex to also match any attribute ending in ::test], verified
against representative patches including PR #198's actual diff.
Closes#201
The `test (beta)` lane failed on 17f9eda reading `leader.pid`, NotFound, while
`test (stable)` passed the same commit. The test armed a 100ms watchdog deadline
and the fake git records its pid as its first statement, so a runner slow enough
to delay `/bin/sh` past 100ms gets the whole process group torn down before the
pid is ever written.
Reproduced by delaying the fake git's first statement by 300ms, which yields the
identical NotFound panic at the same line. With the deadline at 2s the same
injection passes at 0.3s, 1.0s and 1.8s of start delay, so the race window moves
from 100ms to 2s.
The property under test is unchanged: a pre-fix build stands the watchdog down on
drain and blocks on `child.wait()` for the child's full `sleep 30`, which still
trips the 10s recv budget. Confirmed by injecting that exact defect, which goes
red on the budget assertion rather than on some other failure.
local_add_refusal had six unit tests and the wiring had none, so
replacing the whole match at the call site with an unconditional "Added
to local peer list." left every test green. That is the shape the store
already names: a helper test plus a status-mapping test does not prove
the call site.
Extract the selection into local_add_report, following the same
print-selection-helper precedent used for the certificate DID hint, and
assert both directions: a refusal never yields the success line, and a
200 never yields the warning. Collapsing the match to always return the
success line now turns that red. The call site is left as a two-line
dispatch carrying no logic.
The scan matched a verb-plus-table needle per line, so it was whitespace
and case sensitive. The identical statement written across lines walked
straight past it, and that form is how the longer queries in this file
are already written, so it is the shape a new writer most likely takes.
Normalize the whole file before matching, mapping offsets back to line
numbers so each statement is still attributed to the function issuing
it. The scan is split into a helper so the property can be asserted
against a synthetic source: the tree contains no multi-line peers write
today, so a test that only ran against the real tree would prove nothing.
Fixtures derive from the needles at runtime for the reason the needles
themselves do, which is that a literal here is found by the scan of this
very file.
The check ran before anything looked at whether the row existed, so it
refused a pure liveness refresh of a row already in the table. Nothing is
protected by that: the row is already there, and an identical-URL refresh
changes no authority. What it did instead was freeze every peer a
deployed database already holds from before this gate, since upsert_peer
validated nothing then and the handler accepted any parseable DID. Those
rows take a 400 on every unsigned announce and cannot be corrected by a
signed one either, because no key resolves.
Now the DID is judged only when no row exists, which is the only place
refusing it prevents anything. The read-then-write race is benign: if a
row appears in between, the write becomes an update and the existing
repoint guard still refuses any http_url change.
The capped read means a peer answering with a body past the cap leaves
truncated JSON behind, and parsing that with ? before the status check
turned a 503 into "invalid JSON response", losing the status the user
needs. A non-JSON error page from a caller-chosen peer URL does the same.
The failure check now runs first and falls back to the body text when
there is no message field. Bound by a test that drives the command
against a mock peer, since reordering the two statements compiles and
passes every unit test in the file.
The peer-add refusal warning added earlier in this branch copied the
node's message straight into stderr, and both announce bodies were read
unbounded, so a hostile --node endpoint, a compromised local node, or a
MITM on plain http could put ANSI or OSC escapes into the user's
terminal. The accepted-announce block had the same hole on the success
path, where the DID and URL come from a peer whose URL the caller chose.
sanitize_node_msg and read_body_capped move from sync.rs to http.rs,
which already owns the wire boundary, and both paths in peer.rs now use
them. Only the display is sanitized: the values forwarded to the local
node stay verbatim, since that is data rather than terminal output.
The split reported the cause in the message but left the code as
bad_request, so a client telling this apart from the other validation
failures on the announce route had to substring-match prose. The auth
middleware already answers unresolvable_did for the same failure on a
request's keyid; the peer-write path now answers the same way. Status is
unchanged at 400, and an unsupported DID method keeps bad_request.
The unproven insert path folded every to_verifying_key failure into
UnsupportedDidMethod, so an oversize, malformed, or non-ed25519 did:key
was told its DID method is unsupported, which is false and leaves the
caller nothing to act on. Split the arm by cause: a parse or resolution
failure now returns UnresolvableDid carrying the underlying reason, in
the same wording the signed path already uses, while a foreign method
keeps methodNotSupported. Every input stays refused in the 400 class.
The refusal test fed local_add_refusal an invented message, so it could
never go red when the node's wording changed. Drive it with the Display
of PeerWriteDenied::UnprovenRepoint and assert the same substring the
node-side pin asserts against a real router response.
The announce route now answers 403 when an unproven caller tries to
repoint an existing peer. gl peer add discarded the local node's reply
with let _ and printed the success line unconditionally, so a refused add
reported as a completed one. The refusal is split into a small function so
the path is assertable, and it is tested for a 403 with a message, a
refusal with no body, and the accepted case. The local add stays best
effort: a transport failure warns rather than failing the command, which
is what the surrounding code already assumed.
The README described the staged rollout as accepting unsigned peers on
these routes, which is no longer the whole rule. With signed peer writes
off, an unsigned announce may still register an unseen peer and refresh a
row whose URL is unchanged, but changing an existing peer's http_url needs
a signature from that peer's own DID, and only a did:key can be registered
unsigned.
The gate restricting unproven inserts to did:key checked the method label
only. Did::from_str validates that the method is one of key, web, or
gitlawb and never inspects the key material, so did:key:notarealkey passed
and created exactly the row nobody can ever correct that the gate exists
to prevent. It now resolves the key the way the signature path does.
Six test fixtures using invented did:key strings stopped inserting once
the gate became real. That failure is the evidence it now reads key
material rather than a prefix, and it means those fixtures were exercising
a path no real DID could take. They carry derivable values now.
Resolving the key put a quadratic base58 decode on a route that accepts
anonymous callers by default. Measured through the router, a 64,000
character did:key cost 8.05s against 4.6ms for the previous label check,
on a tokio worker with no spawn_blocking, and a 1.5 MB body reaches the
handler. The bound sits in to_verifying_key so it also covers the
HTTP-signature keyid, which reads an untrusted string through the same
function. An ed25519 method-id is a fixed 48 characters, so 64 is slack.
Its test asserts the error names the length rather than merely being an
error, because an oversized id fails either way and a check for is_err()
would pass with the guard removed, having already paid the cost.
Three last_ping_ok assertions were vacuous: the seed helper never granted
reachability, so they compared false against false and would have held
against code that never cleared the flag. The helper grants it now and
asserts the seed took. Nothing was hiding behind them.
Drives the must-not set through the production router with real RFC 9421
signatures rather than injected extensions, so the signed path is exercised
as a caller would reach it, in both require_signed_peer_writes modes.
Covers the allowed cases and every rejection: unsigned insert of an unseen
DID, unsigned repoint refused with the row byte-identical after, a
signature from the row's own DID accepted, a signature from a different
DID refused, an identical-URL re-announce refreshing only last_seen, and a
did:web announce refused with no row created.
The signature-from-a-different-DID case is the RUSTSEC-2022-0009 shape,
where libp2p-core accepted a valid signature without checking it derived
the claimed identity. It asserts the key is bound to the row rather than
to anything the request carries.
The announce handler's keyid-mismatch branch had no test at all and
returned 400. It now returns 403 and is tested: a caller denied for want
of proof of control over the targeted row gets the same response class
whether or not it presented a signature.
The completeness ledger derives its set from every writer of the peers
table rather than from upsert_peer's callers, because a caller scan cannot
see a future writer that issues its own SQL. A second table records the
call-site authority for main.rs's bootstrap announce-back, which issues no
SQL and so can never appear in the writer ledger; its authority choice has
no runtime test in this change and is recorded as such.
The type system carries the guard: the authority parameter has no default,
so a new caller cannot omit it. The scan is the backstop for a raw
statement that bypasses upsert_peer entirely, and stays minimal rather
than copying the handler-directory apparatus, which does not apply to a
scan over SQL literals.
upsert_peer ended in an unconditional ON CONFLICT DO UPDATE SET http_url,
so any unauthenticated caller could repoint any peer. A repointed row
steers the sync worker's git remote, the post-receive notify fan-out,
trigger_sync, and the public GET /api/v1/resolve/{did}, none of which
consult the reachability flag that #270's stopgap resets.
An unsigned write may now insert an unseen DID but may never change an
existing row's http_url; changing one requires a verified signature from
that row's own DID. The gate is at the db boundary rather than in the
announce handler because there is a second writer: the bootstrap
announce-back in main.rs reads a peer's own JSON response body with no
proof of anything, and now declares itself unproven, which makes it
insert-only by construction.
The proven variant carries the DID it proves and upsert_peer compares it
against the row being written. A bare proven/unproven flag would say that
something was proven without saying which DID, leaving the real check in
the handler and reproducing RUSTSEC-2022-0009, where a valid signature was
accepted without checking it derived the claimed identity.
An unproven insert is restricted to did:key. Only did:key can ever
authenticate here, so a did:web or did:gitlawb row created unsigned would
become unwritable by anyone once this rule lands.
The unproven path derives its refusal from rows_affected, because an
ON CONFLICT DO UPDATE with a guarding WHERE reports zero rows rather than
raising, and a silent no-op is the fail-open this gate exists to prevent.
The refusal is a typed error the handler maps to 403, downcast out of the
anyhow chain the way sqlx errors already are, so it does not render as a
500.
Observed at c83cbc5 before the fix: an unsigned announce naming an
existing peer returned 200 OK and left the attacker's URL stored. That
test is the RED this change turns green.
Two of #270's reachability tests now drive the proven path. Under this
rule an unproven repoint is refused, so they can no longer express a URL
change through it; they assert the same behavior as before.
The coverage gate matched `# x-release-please-version` anywhere in a manifest,
so an annotated `version` under any other table satisfied it. A crate could
leave the version cargo actually reads unannotated, pass the gate, and still
freeze at release time, with release-please rewriting the decoy line instead:
the exact failure this gate exists to catch.
Scope the match to the [package] table.
IcaptchaCfg::new normalizes an empty GITLAWB_ICAPTCHA_API_KEY to None
via .filter(|s| !s.is_empty()) before storing api_key. No test drove
the empty-string value through new(), so a regression would attach an
empty bearer token to the challenge and answer requests and ship green.
The new test drives an empty key through new() on a trusted operator
origin and asserts it normalizes to None, with a contrast arm showing
a real key still attaches. Mutation check: removing the filter fails
this test and no other.
Add a lone non-key DID fixture check to get_profile_does_not_match_non_key_did to verify that a bare short query returns None when only a non-key profile exists. Add profile_did_case_sql_matches_normalize_owner_key to pin PROFILE_DID_CASE_SQL against Rust normalize_owner_key across the full owner-key boundary matrix. Add cfg(unix) target gates to Unix-specific symlink and permission test helpers.
Refs #298
get_profile equates bare short ids with did:key forms. upsert_profile
must UPDATE by existing.did so a did🔑 alias against a bare-stored
profile actually persists. set_profile_cid uses the same CASE match.
Add a regression test covering both write paths.
Replace the broad LIKE suffix match in profile resolution with the same
normalize_owner_key / CASE pattern already used by get_repo, so non-key
DID methods that share a short-id suffix cannot resolve the wrong profile.
Add a sqlx test locking in bare, did:key, and non-key DID behavior.
Both doc comments describing GIT_SERVICE_TIMEOUT_SECS_MAX said the 100-year
ceiling sits "an order of magnitude" under the ~584-year limit of a
u64-nanosecond Instant. The real margin is 3,153,600,000 seconds against
18,446,744,074, a factor of about 5.85. An order of magnitude means ten.
Worth noting how it spread, since the fix is trivial and the pattern is not.
The phrase was written once on the constant, then copied into the
ipfs_request_budget_secs doc by the commit that was at that moment correcting
a different false claim in the same file. Nobody re-ran the division at either
site, because quoting an existing sentence feels like citation rather than
assertion.
The direction of the error is the part that matters: a vague "well under"
would have been true and would have invited a reader to check. The
precise-sounding "order of magnitude" was false and discouraged checking.
Comments only; the diff carries no executable line, checked by filtering it
for non-comment changes. fmt and clippy clean under the CI form.
Review round on the three preceding commits.
The knob's doc said a value past the ceiling "aborts every /ipfs/{cid}
request in a release build". That is wrong, and three reviewers plus an
outside model landed on it independently: the ceiling is 100 years, the
Instant addition does not actually overflow until roughly 584 years on a
u64-nanosecond Instant, so values in between are representable and are
rejected as a deliberate safety margin, not because they panic. Reworded
to say that, since a confidently wrong comment is worse than the vague
one it replaced.
Also on the knob:
- GIT_SERVICE_TIMEOUT_SECS_MAX has two consumers now, so its own doc says
so, and notes this knob derives only the Instant addition and not the
lease-steal multiply.
- The "clears every disable setting" phrasing overclaimed: u32::MAX sits
above the ceiling. Now names the documented sentinels instead.
Operator-facing docs: the sibling knob's ceiling commit updated both
README.md and .env.example in the same change, and this one had not. Both
now carry the accepted range, matching that precedent.
House style: five comment lines added by the preceding commits used em
dashes, which this project does not allow. Four were carried over from the
text being edited and one was copied from the sibling test. All replaced.
Checked mechanically over the added lines rather than by eye.
Two comments restated an argument in full immediately after pointing at
the canonical site for it. Trimmed to the pointer plus the local delta.
Scoping correction worth recording: the two preceding commits claimed "no
behavior change". That holds for repos.rs, state.rs, main.rs and
test_support.rs, but a config.rs field doc IS clap --help output, so the
pin-knob doc edit changes operator-visible help text. No runtime behavior
changes either way.
Verification: full suite 733 passed, fmt clean, clippy clean under the CI
form. The range guard was separately proven load-bearing in both
directions by mutation: reverting to range(1..) goes RED on the rejection
assertions, and tightening the ceiling to 500_000_000 goes RED on the
disable-value assertions.
Known residual, not fixed here: the bound is parse-side only, so a Config
built by direct construction rather than through clap can still hold an
unrepresentable value.
The three handler peeks said they shed "before any DB/disk work", which
reads as an admission bound on the DB window. It is not one. The peek
holds no permit, and permits are only held from the authoritative acquire
below, after the visibility and rate gates, so a burst arriving while
permits are free all proceeds into the DB and none of it sheds. What the
peek actually gives is a cheap 503 for one request once the pool has
ALREADY filled, sparing that request's DB work.
The gate ordering that makes it a peek is deliberate and stays: a denied
or rate-limited request must consume no slot, and one source must not
hold global slots through the DB/visibility window. Reordering to bound
the window would undo that. So this corrects the claims and changes no
behavior; bounding the DB window would need an admission mechanism the
peek is not.
test_support.rs's receive-pack docstring additionally called the peek
"the permit" and "the first statement". Both were wrong at this head: it
is a non-holding peek, and the authoritative write-pool acquire sits
after the per-repo lease. Corrected along with the shed claim. The
"remove the block and this goes red" sentences stay, since those tests
drive an already-saturated pool, which is the case the peek does handle.
Sweep over "before any DB", "before DB", "pre-DB". Left alone with
reason: api/ipfs.rs:142 describes a HELD walk-admission acquire, so its
claim is true; graphql/mutation.rs:201 and test_support.rs:528 are about
auth ordering, unrelated; the repos.rs test docstrings at 4194/4213/4556/
4618/4682 describe test mechanics against a saturated pool, which is the
case the peek delivers.
No behavior change: the diff is comments and docstrings only, checked by
filtering the diff for non-comment lines.
The pin pool's docs claimed it bounded "how many MB-scale lists are held
at once across all repos". It does not. On the local IPFS path the caller
materializes the object list before pin_new_objects_gated acquires, so a
task parked waiting for a permit is still holding its full list, and how
many tasks park is capped only per repo by EncryptInflight. Across
distinct repos that retained memory is unbounded, which is the gap a
reviewer raised against this claim.
Correct the claim at each site rather than implying the pool closes an
exposure it cannot see. The pool does bound concurrent pin loops, and
therefore lists held while being pinned; that part was always true and is
kept. The residual is now named where each claim lives, including on the
config knob, so an operator does not read the knob as a memory cap it
cannot deliver.
Bounding the retention is a real change to the capture shape (the Pinata
twin already avoids it by acquiring before it derives) and is not
attempted here.
Sites corrected: pin_new_objects_gated's docstring and its F2b empty-list
comment, the empty-list test's docstring, AppState::pin_semaphore and
AppState::encrypt_inflight field docs, the max_concurrent_pin_tasks knob
doc, and the construction comment in main.
Sweep over pin_semaphore, F6, and "held at once". Left alone with reason:
ipfs_pin.rs 16/31/174-200/518 describe the loop that genuinely holds the
permit, so their claims are about hold duration and are true; repos.rs
1029-1036 and 2276-2294 describe the Pinata path, which acquires before
materializing, so its "at most pin_semaphore permits' worth exist at once"
holds; repos.rs 5898 claims concurrent loops are bounded, which is the
true concurrency claim; the F6 hits in api/ipfs.rs are that file's own
review numbering for walk-permit ordering, unrelated to this pool;
constructions and test fixtures carry no claim.
No behavior change: the diff is comments and docstrings only, checked by
filtering the diff for non-comment lines.
ipfs_request_budget_secs parsed range(1..), so every positive u64 was
accepted, but get_by_cid derives the request deadline as
Instant::now() + Duration::from_secs(that). That addition is an explicit
overflow check rather than a debug-only one, so an oversized value aborts
every /ipfs/{cid} request in a release build instead of setting a very
long budget. The route is anon-reachable, so an operator typo is felt
publicly.
Reuse GIT_SERVICE_TIMEOUT_SECS_MAX, the ceiling the sibling knob already
carries: the bound is representability, not a policy view of a sane
budget, so the large "set it very large to disable" values operators were
told to use still parse. Reject rather than clamp, matching the other
bounds in this config.
Execution evidence:
- RED before the change: the new test failed with "3153600001 is past the
representable ceiling and must be rejected at parse time" while its
disable-value assertions passed, so the test is not vacuous in either
direction.
- GREEN after: config::tests 13 passed, including the sibling
git_service_timeout regression test.
Sweep for the same defect class, re-run with a widened shape because the
original grep missed a site. Passes: Instant::now() sites; from_secs()
fed by config fields; config fields reaching a Duration through an
arithmetic expression; config fields reaching a tokio timer constructor.
The arithmetic and timer passes each return exactly one hit,
heartbeat_interval_hours (config.rs 150, main.rs 916, operator.rs 161),
which has no range at all and multiplies by 3600 before the Duration.
That knob is a separate defect on a path this branch does not touch, so
it is recorded for a follow-up rather than widened into here. Every other
Instant addition derives from the already-bounded git_service_timeout_secs
or from a constant.
A second-model pass on the previous two commits caught an upgrade footgun in the
first and stale prose in the second.
The 365-day ceiling was tighter than the defect required. The help text has
always told operators to set this very large to disable the bound, and clap
accepted every positive u64, so values like 999999999 (~31 years) are working
production "off" settings that neither overflow `* 2 + 60` nor exceed an Instant
deadline. A node running one of those would have failed to start after this
upgrade, over a value that was never the defect. The ceiling moves to 100 years,
which clears every such setting and stays an order of magnitude under the
~584-year limit of a u64-nanosecond Instant, the tightest representation on any
platform we build for. Every value that worked before still parses; only the
ones that would have panicked are rejected. The test now pins the pre-existing
disable values alongside the rejections so the distinction cannot quietly
regress.
Clamp-and-warn was the suggested remedy and is not what landed: it would keep
accepting a value the node cannot honor and silently substitute a different one,
where the range says what is representable and fails fast, matching how every
other bound in this file behaves.
The lease-bound comment and its test still described the pre-ceiling world. Both
now say what is true, and the test deliberately keeps u64::MAX rather than moving
to the ceiling as suggested: Config is reachable by direct construction, and at
the ceiling the arithmetic does not overflow, so a test pinned there would pass
with the saturation removed and prove nothing. Re-confirmed load-bearing after
the rewording.
Tracing the lease-bound overflow to its other consumers turned up a worse
instance of the same defect. This PR routed git_service_timeout_secs into
build_filtered_pack and blob_paths, and both build a deadline as
`Instant::now() + Duration::from_secs(..)`. That addition panics on overflow in
RELEASE as well as debug, so a large-but-accepted value crashes the serve path
rather than merely disabling a timeout.
Hardening each arithmetic site as it is found would leave the next one open, so
the bound goes where the value enters: clap now rejects anything above 365 days
at parse time, which keeps every derived duration in range at once and fails
fast at boot instead of at the first path-scoped clone. A year is many orders of
magnitude above any real clone or push, so it remains the practical way to
disable the bound.
The test pins both derivations at the maximum and rejects the value one past it
and u64::MAX. RED when the upper bound is dropped from the value_parser.
GITLAWB_GIT_SERVICE_TIMEOUT_SECS accepts every positive u64, so deriving the
per-repo write-lease steal bound as `* 2 + 60` panics every push in a debug
build across the top of the accepted range, and in release wraps to a short
Duration that would let a waiter steal a live push's lease.
Saturating keeps the knob's documented meaning: a timeout that large means the
service bound is effectively off, so the steal bound should be effectively
infinite rather than tiny.
The regression drives git_receive_pack rather than the arithmetic, so the
call-site wiring is what is under test. RED before at repos.rs:1733 with
"attempt to multiply with overflow", GREEN after; re-confirmed load-bearing by
reinstating the unchecked expression.
cargo audit fails on main, so every open PR inherits a red check for a
problem that is not in its diff.
ruint 1.17.2 carries RUSTSEC-2026-0220: Uint shift operations report
incorrect overflow flags and truncate shift amounts. It reaches us
transitively through alloy-primitives, which alloy pulls into both
gitlawb-node and gl for the on-chain operator client. Our own U256 use in
operator.rs is pow, mul, div, and comparison rather than shifts, so the
direct exposure looks limited, but alloy's internals are not audited here
and the fix is a semver-compatible bump.
Lockfile only; no manifest change. The bump adds six arkworks crates
(ark-ff, ark-ff-asm, ark-ff-macros, ark-serialize, ark-serialize-derive,
ark-std), taking the tree from 715 to 721 dependencies. That surface
increase is the cost of the fix and is worth noting rather than passing
over: it arrives through ruint's own feature set, not through anything we
asked for.
None of it reaches gitlawb-core, so the dependency purity check still
passes at 83 allowlisted normal deps.
Verified: cargo audit exits 0, cargo build --locked --release, 564 node
tests and 83 core tests pass.
Review finding from jatmn, round nine. RepoWriteGuard::release closes the
connection when pg_advisory_unlock returns Err (F3b): the await resolved, so
the session is alive and still holds the lock, and returning that connection
to the pool hands the next caller a lock nobody tracks. The Drop backstop
carried the same hazard and only logged.
Both arms are fixed.
The spawned detached unlock now closes the connection through
close_conn_bounded on the error path, instead of letting the async block end
and return it to the pool.
The off-runtime arm was worse than the finding described. It dropped an
already-taken connection with no unlock attempted at all, and this turned out
to panic rather than leak: PoolConnection::drop calls rt::spawn for its
return-to-pool task, whose no-runtime fallback panics with "this
functionality requires a Tokio context". So any off-runtime guard drop
panicked in a destructor today. It now detaches, which gives up the pool slot
and yields a plain PgConnection whose drop closes the socket, ending the
session that holds the lock. Drop cannot await, so detach is the whole
disposal available here. Confirmed against the vendored sqlx 0.8.6 source
rather than assumed, and min_connections is 0 everywhere in this repo.
Three tests, modeled on the release F3b cases, on a pool with the idle reaper
disabled so a reaped session cannot pass the assertion for us. Observed RED
first:
write_guard_dropped_off_runtime_disposes_the_connection ... FAILED
panicked at sqlx-core-0.8.6/src/pool/connection.rs:208:
this functionality requires a Tokio context
then: dropping a write guard off a Tokio runtime must not panic
write_guard_drop_with_failing_unlock_does_not_return_the_connection ... FAILED
timed out waiting for the connection whose detached unlock errored to be
closed rather than returned to the pool still holding the session lock
write_guard_drop_with_successful_unlock_keeps_the_connection ... ok
The success case is green on both sides on purpose: it is the must-not case,
so it only ever catches a widening of disposal onto the healthy path.
GREEN after, and all 12 guard tests pass, so the release coverage did not
regress. Both new guards mutation-proved load-bearing, each red on its own
message, 2/2.
Full suite 729 passed, 0 failed. fmt and clippy clean.
Both new comments said saturating the pool takes ~8 distinct source IPs,
each also rate-limited. Neither half holds for an IPv6 caller with a routed
/64: the sub-cap and the rate limiter both key on rate_limit::client_key,
which with TrustedProxy::None falls through to the full peer address with no
prefix folding, so one caller has 2^64 keys. max_keys does not compensate,
since keys are freed on permit drop and live keys never approach the 100,000
default.
The comments now say what the cap actually bounds, concurrent slots per
resolved client key, and name the residual.
The keying fix stays deferred as a design call, and that deferral is
defensible: full-IP keying predates this PR, so these caps help IPv4 and
single-address callers without regressing anything. Landing a comment that
overstates the guarantee is not defensible, in a round that corrects a
weaker overclaim one screen away.
The per_source_push_cap docstring added earlier in this round repeated the
same claim, so it is corrected here rather than shipping fresh.
Comments only, no behaviour change. check, fmt clean.
The pinata_object_list_for_refs docstring said a visibility tightening that
lands after tail-start is covered by the reconciliation sweep. No such sweep
exists; state.rs says so directly, that a dropped job would be lost forever
precisely because there is none.
The claim is dropped and nothing replaces it. An earlier draft wanted to
substitute a next-push claim, which no site establishes either, and swapping
one unbacked claim for another is the same defect.
Four pre-existing sites make the same claim (repos.rs:30, push_delta.rs at 8,
189 and 281 with warns at 349 and 358). They predate this PR and are left for
a follow-up rather than widening a nine-round diff.
advert_per_caller_cap_sizing_is_never_zero opened with a local closure
re-implementing (pushes / 8).max(1), so it bound nothing. Production derives
that expression twice, and the second site was added after the test was
written. per_source_push_cap now lives in rate_limit.rs next to
PerCallerConcurrency, the type the value feeds, and both main.rs sites and
the test call it.
The finding was confirmed by execution before the fix. Deleting .max(1) from
both production sites with the closure still in place:
injected (2 sites), re-running ... still green
VACUOUS the guard's own defect was injected and nothing failed
0/1 load-bearing
And after, deleting .max(1) from the helper:
injected (1 site), re-running ... red, on the expected message
LOAD-BEARING matched 'advert cap must be >= 1 for pushes=1'
1/1 load-bearing
The first attempt at that second proof came back RED-WRONG-REASON against
'minimum write pool must derive cap 1, not 0': the loop's assertion for
pushes=1 fires first and aborts before that line is reached. Same property,
earlier assertion.
No mutation residue. Test green, fmt and clippy clean.
separator_prevents_the_owner_name_boundary_collision asserted that
("a","b_c") and ("a_b","c") produce different keys. Those differ under any
join at all, bare included (ab_c against a_bc), so deleting the separator
from repo_identity_key left the test green. It could not fail for the reason
it named.
The inputs are now ("a","bc") against ("ab","c"), both abc under a bare
join. The test docstring said the separator was \0; the code joins with /.
Corrected. The docstring on repo_identity_key itself carried the same wrong
collision example, so it is corrected too.
Proven by mutation rather than argued, joining with no separator:
old inputs, injected, re-running ... still green
VACUOUS the guard's own defect was injected and nothing failed
0/1 load-bearing
new inputs, injected, re-running ... red, on the expected message
LOAD-BEARING matched 'owner/name boundary must not be ambiguous'
1/1 load-bearing
Separator restored, no mutation residue. All 6 repo_identity_key_tests pass.
The walk's deadline clamp was computed on the async side, before
spawn_blocking was called, then used inside the closure. Time spent queued
for a blocking thread went uncharged, so the walk ran with a budget larger
than the request's true remainder. The clamp is now derived inside the
closure at task start.
This is the sibling of the upload-pack site fixed in 28a6ca4, and unlike
that one it sits on the anonymously reachable route.
A queue delay that eats the whole remainder saturates the clamp to zero and
the bounded walk fails closed through the existing taint arm, no verdict,
which is the safe direction.
Testing gap, recorded at the site rather than papered over: the queue-delay
path is reasoned, not executed. Observing it needs a runtime with the
blocking pool pinned and parked, and the sqlx::test harness gives no seam
for that. 28a6ca4 shipped with the same gap for the same reason, verified
against its diff and message rather than assumed. The tests in this file
saturate the admission semaphores, never the blocking pool, so there was
nothing to reuse. An assertion on the arithmetic would only restate the
implementation.
cargo test ipfs::: 24 passed, 0 failed.
pin_new_objects runs under a pin_semaphore permit and that pool defers rather
than sheds. Both sinks in ipfs_pin.rs built a bare reqwest::Client::new(),
which carries no timeout, so a silent endpoint parked the pool indefinitely.
Both now use the shared no-redirect client from build_http_client() through a
module-local OnceLock, rather than the hand-rolled twin its docstring forbids.
The client alone does not bound the hold: without a ceiling the loop is O(N)
with N chosen by the pusher. A 120s batch budget is taken once at loop start
and gated at the top of every iteration, so no object's work begins with zero
budget left. The remainder is also passed to each add as a per-request
timeout, which is what lets one large healthy upload run past the client's 10s
default without letting the batch run forever. RequestBuilder::timeout
replaces the client value for that request only and leaks nothing to other
callers, verified by execution against the pinned reqwest.
An earlier version of this commit classified a dead endpoint by
reqwest::Error::is_timeout() || is_connect() and broke the batch. That was
wrong in both directions and is gone. It missed the ordinary dead-peer shapes,
since a peer closing mid-request yields IncompleteMessage and a peer RST
mid-upload yields ConnectionReset, both observed with is_timeout and is_connect
false, so a load-balancer idle timeout evaded it entirely. And it fired on a
slow but alive endpoint, because .timeout() is a total deadline covering body
upload, so a healthy multi-megabyte add was read as a dead endpoint and
abandoned the rest of the push. A wall-clock bound needs neither judgement.
The docstring now states what is bounded and what is not, rather than implying
the hold is closed. Not bounded: the git read, since store::read_object shells
two bare Command::output() calls with no timeout or process-group reaping and
this loop does not run it under spawn_blocking; the DB round-trips; and the
pool itself, since the Pinata replication task holds the same semaphore across
a full git re-derivation with no deadline. Truncation is stated plainly: a
batch stopped at the deadline leaves the rest unpinned and nothing sweeps them
up, so recovery is opportunistic through a later full-scan push.
Tests written first, both RED by assertion rather than by a harness timeout:
pin_new_objects_stops_the_batch_at_its_deadline ... FAILED
the batch must stop partway, not pin all five and not stall on the
first: pinned 5
pin_new_objects_does_not_abandon_the_batch_on_a_slow_but_alive_endpoint ... FAILED
a slow but progressing endpoint must pin both objects: an upload past
the client's 10s default is not a dead endpoint
left: 1, right: 2
Both guards mutation-proved load-bearing against present-but-wrong mutants,
each red on its own message: the loop-top break widened to continue, and the
per-request timeout replaced with None.
Also bounds encrypted_pin.rs's pin_git_object call, where a hang wedged a
repo's EncryptInflight key. Its Drop backstop covers unwind, not hang.
Full suite 730 passed, 0 failed. fmt and clippy clean.
icaptcha-client became a workspace member in #138, after #130 last wrote
release-please-config.json, and nothing updated extra-files. release-please
rewrites a version line only when the line carries the annotation AND the
manifest is listed, so the crate was skipped every release and sat at 0.4.0
while the other five walked to 0.7.1.
Nothing broke: every intra-workspace path dep is bare, so no version
requirement could go unsatisfied; no crate is published to a registry; and
the crate reads no CARGO_PKG_VERSION of its own. The cost was the workspace
Cargo.toml comment asserting a lockstep rule the config did not keep, which
is what the next person adding a crate would read and copy.
Add the missing entry, and gate the rule so it cannot silently lapse again.
The gate derives its required set from `cargo metadata`, so a new crate reds
CI whether or not its author knows the gate exists, and it covers all three
release-silent shapes: a member absent from extra-files, a member whose
version line lost the annotation, and an extra-files path that is no longer
a member.
Next release moves icaptcha-client 0.4.0 to the current workspace version in
one step. Nothing depends on that version, so the jump is cosmetic.
The absent-CID path disambiguates a clean `missing` by re-running the probe,
but the re-probe took the same deadline with no check that any budget was left.
A first probe that nearly exhausted the budget still spawned a second child
that could only be reaped, and the watchdog's SIGTERM grace plus SIGKILL settle
carried the whole call to roughly twice the budget. Measured on
object_type_bounded with a 1s budget and a probe that burns 0.9s of it: two
spawns and 2021ms elapsed.
That is the same ~2x occupancy shape this PR exists to remove, on a different
handler, and it is worse here than on the served-git path: /ipfs/{cid} carries
only optional_signature, and an absent CID drives this branch once per repo, so
an unauthenticated caller spraying well-formed absent CIDs pays the overshoot
per repo. It also degraded an honest absence into a retryable 503.
Re-probe only when the remaining budget covers what the first probe actually
took, since the re-probe runs the same command. Below that, taint to Transient
rather than spawn: an inconclusive disambiguation is not an absence verdict, so
it must not become a false 404 either. The same 1s case is now one spawn and
finishes inside the budget.
Both introduced by this PR (d39fc8d, the F5 disambiguation); neither
object_type_bounded nor the re-probe exists at the merge base, so this is not a
pre-existing condition. The companion test pins that an ample budget still
re-probes, so the check cannot silently disable the disambiguation it guards.
The shared deadline computed the walk's budget on the async side, before the
task was queued on the blocking pool. withheld_blob_oids_bounded starts its own
clock when it actually runs, so a budget measured at queue time handed the walk
a full budget from whenever the pool got to it: the read permit was held for
queue delay PLUS the budget, and only the serve phase saw the outer deadline
already spent.
Computing the remainder inside the closure charges that delay against the shared
deadline, which is what makes the ~1x bound in the docs true rather than
approximate. Not a regression from the previous commit (the old code passed a
constant git_timeout, so the delay went uncharged there too, and the combined
hold was queue delay plus two full budgets), but the claim this PR now prints in
README and .env.example should be exact.
The two shared-deadline tests still pass and still fail on revert; they cover
the budget arithmetic, not the queue-saturation path. Reproducing that needs a
runtime with the blocking pool pinned and parked, which the sqlx::test harness
does not give a seam for, so the queue-delay path is reasoned, not executed.
Two table rows needed correcting for the changes in this PR: the
GITLAWB_GIT_SERVICE_TIMEOUT_SECS row now says the classification walk and the
pack serve share one deadline on the path-scoped upload-pack path, so the value
bounds them combined; and the GITLAWB_IPFS_REQUEST_BUDGET_SECS row no longer
claims the probe and content-read subprocesses are unbounded, while keeping the
object-store-readability sweep named as a live residual.
The other thirteen changed lines are punctuation only, no wording changes: em
dashes replaced with colons, commas, or sentence breaks, and one pair of curly
quotes straightened. The repo's prose-style hook validates a whole file rather
than a diff, so these pre-existing characters were rejecting every edit to
README.md, including the two rows above.
Four prose corrections on this PR's surfaces, no behavior change.
The receive-pack advertisement no longer draws from the write pool, it draws
from a dedicated advert pool, but five comments still described the old
shape: the info/refs per-source block, the cap construction in main.rs, the
State field doc, and a test doc plus its assertion message. The state.rs and
main.rs ones mattered most because they contradicted the operator text being
written in this same commit.
The /ipfs probe and content-read subprocesses each run under their own
deadline now (the lesser of git_service_timeout_secs and the remaining
budget, reaped by process-group teardown), but four surfaces still claimed
they had no duration bound past the pre-start budget check. The correction
stays scoped rather than declaring the route safe: the probe's
object_store_readable check is a synchronous filesystem sweep with nothing to
reap, so a wedged filesystem can still hold the walk slot past the deadline,
and that residual is now stated where the old warning used to be. A grep for
the full phrase misses two of these surfaces, since one wraps across doc-comment
lines and one says "duration clamp" instead, so the sweep uses the fragment.
Both push-side per-source concurrency caps are derived as
max_concurrent_git_pushes / 8 with a floor of 1 and have no environment
variable, so neither was documented anywhere an operator reads. The write-side
one is load-bearing for push availability: it is acquired before the global
write permit, and owner enforcement defaults off, so without it one host
minting disposable did:key identities could monopolize the write pool. They
are described in .env.example beside the read cap rather than in README's
table, which is keyed by variable name.
The hung-walk assertion in visibility_pack printed {err}, showing only the top
anyhow context and dropping the underlying io error. A beta-lane failure
surfaced as "failed to spawn git for-each-ref" with no errno, which made it
undiagnosable. {err:#} prints the chain. The underlying flake is still
undiagnosed: it did not reproduce in 123 local runs and the stable lane passed
on the same commit, so this only improves the next occurrence.
The path-scoped git-upload-pack branch spent git_service_timeout_secs twice:
once on the withheld-blob classification walk, then a second full budget on
the serve. One clone could hold a read-pool permit for ~2x the configured
budget, which is the occupancy hole this PR otherwise exists to close.
Thread one Instant deadline through both phases, mirroring
fail_closed_full_scan_objects and build_filtered_pack: the walk runs against
the remainder and the serve gets what the walk left, computed after the
blocking task's await so it cannot pick up a fresh budget. A walk that
consumes the whole budget saturates the serve to zero, which surfaces as
GitServiceTimeout -> 504 rather than running unbounded. Under-serving one
slow clone is the safe direction versus over-holding the pool.
Both serve arms are covered because a fix that reached only the plain arm
would leave the ~2x hold reachable by any clone of a repo that actually
withholds a blob. Each test was observed RED before the change (status 200,
the serve completing on a fresh budget) and GREEN after, and reverting the
filtered arm alone flips only its own test, so neither guard is a proxy for
the other.
The knob's meaning changed for this path, so .env.example now says the value
bounds the two phases combined rather than each separately.
Review of the previous two commits found the SQL correct and the comment
wrong in two places, plus two implementations of the intent that all four
tests accepted.
Comment fixes:
The claim that four http_url consumers "never read the flag at all" named
the public resolve route, which does read it: api/resolve.rs republishes it
as `reachable`, as does the peer listing. The accurate split is that only
the federated repo fan-out in api/repos.rs GATES on the flag, four consumers
act on a repointed URL regardless of it, and two surfaces republish it.
Three reviewers caught this independently.
The atomicity sentence implied the conflict row lock protects the flag. It
orders announces against each other and nothing else: mark_peer_ping writes
by DID with no http_url predicate, so a probe of the previous URL that lands
after a reset can still re-grant the flag until the next round. Said so, and
qualified the self-healing claim on cosmetic differences, which holds only
if no further announce lands first.
Two tests added, each proven load-bearing against a shape the original four
accept:
cosmetic_url_difference_counts_as_a_change
Pins the exact-comparison decision. Normalizing the comparison
(rtrim(peers.http_url, '/') vs rtrim($2, '/')) passes the other five and
fails only this one. The original four only ever compare identical or
wholly different hosts, so none of them constrain the comparison.
same_url_reannounce_still_advances_last_seen
The reset must ride the existing UPDATE, not gate it. Hoisting the
condition to a statement-level WHERE passes the other five while
skipping the update entirely on a same-URL re-announce, so liveness
stops advancing and the peer ages out on last_seen. Observed with both
timestamps identical.
Gates: cargo test -p gitlawb-node 516 passed 0 failed; clippy --all-targets
-D warnings clean workspace-wide; fmt --check clean.
Refs #270
The ON CONFLICT branch rewrote http_url and left last_ping_ok alone, so an
unsigned announce could repoint an existing peer's URL and inherit the
reachable=true that the previous host earned. api/repos.rs filters the
federated fan-out on that flag and then stamps each entry from the peer's
response body with the peer's DID, so the inheritance is what made the
injection instant rather than probe-delayed.
Clear the flag when the URL actually changes, and leave it alone when it does
not. In the DO UPDATE branch peers.http_url is the existing pre-update row and
the comparison runs under the conflict row lock, so this needs no second round
trip and does not race a read-then-write. Both writers funnel through
upsert_peer, so the announce handler and the bootstrap announce-back are both
covered.
Observed transition on the tests from the previous commit:
before url_change_clears_reachability ... FAILED (last_ping_ok assertion)
after url_change_clears_reachability ... ok
the other three ... ok throughout
Both guards were confirmed load-bearing by reverting, not by reading a green
suite. Removing the CASE clause entirely turns url_change_clears_reachability
RED again. Flattening it to last_ping_ok = (peers.http_url IS NOT DISTINCT FROM
$2) still passes three of the four cases and is caught only by
same_url_reannounce_does_not_grant_reachability, which is why that case exists.
This bounds the automatic inheritance; it does not close the rewrite. Four
other http_url consumers never read this flag, and until #248 lands the
unauthenticated ping route can write it back from the attacker's own host.
#273 is the closure.
Closes#270
upsert_peer's ON CONFLICT branch rewrites http_url and says nothing about
last_ping_ok, so a repointed peer keeps the reachability the previous host
earned. These four cases state the behavior the fix has to produce, and they
land before it so the failure is observed rather than assumed.
Pre-fix results:
url_change_clears_reachability ... FAILED
same_url_reannounce_keeps_reachability ... ok
fresh_peer_inserts_unreachable ... ok
same_url_reannounce_does_not_grant_reachability ... ok
panicked at crates/gitlawb-node/src/db/mod.rs:5737:
a repointed peer must re-earn reachability, not inherit it
The failure is on the last_ping_ok assertion; the http_url assertion above it
passes, so the row is being rewritten and only the gate is wrong.
The three that already pass are regression pins, not filler.
same_url_reannounce_keeps_reachability guards against clearing the flag on
every conflict rather than only on a change.
same_url_reannounce_does_not_grant_reachability guards the other direction and
is the only case that fails if the conditional is flattened to
last_ping_ok = (peers.http_url IS NOT DISTINCT FROM $2), which would let any
unsigned same-URL re-announce set the flag TRUE with no probe.
fresh_peer_inserts_unreachable guards the INSERT branch.
Refs #270
Spawning the replication tail above guard.release() also moves it above the
Tigris upload, so a ref can be announced while the shared durable copy is still
the old one, and a disconnect there cancels the upload while the detached tail
still pins and announces.
Accepted rather than fixed. Upload-then-announce was never guaranteed: release
tolerates a failed upload by design, warning and continuing to the unlock, so an
announce over a stale Tigris copy was already reachable before this reorder, and
it self-heals through acquire_fresh's local fallback plus the next push. The
alternative, detaching release and the tail together, would return 200 before
the durable copy lands, which changes the client contract more than the window
it closes.
A handler-level "the next acquire_write succeeds" check does not discriminate:
it still passes with the RepoWriteGuard::Drop backstop disabled. Measured with
the backstop off, the lock is held at the drop and reads free about two seconds
later from a session outside the pool. #[sqlx::test] builds its pool with
idle_timeout(1s), so the guard's connection is reaped once the handler future
drops it, the session ends, and postgres frees the lock with no help from the
code under test. acquire_write retries for far longer than that window, so it
waits the release out and reports success either way.
write_guard_release_cancelled_mid_unlock_frees_the_lock is the real proof: it
probes from a connection held out of the pool, 400ms after the drop, inside that
window rather than past it.
v12 was main's current_max + 1, which INV-7a says is necessary and not
sufficient: the runner keys the applied set on the integer alone, so a
version another in-flight branch also claims is skipped in full on
whichever side merges second. No error, no warning, schema_migrations
still reading healthy, and the column simply absent, after which
dequeue_pending_syncs fails on every poll and sync ingestion stalls.
Two open branches already claim into that range. #135/#173 holds
through 14 (15 once it rebases past main's v11) and #253 took 16. 17
clears both, and the reservation is recorded in a comment so the next
author can see it. Gaps are harmless; the runner iterates the array and
never requires contiguity.
Not doing the other half here. INV-7a's load-bearing rule is the name
check that makes the silent skip impossible rather than merely
unlikely, and that lands with #253 on test/replay-guards-253; adding it
on this branch would collide with that work.
Three review fixes on the two commits above.
The docstring claimed more than the code delivers. UPDATE ... RETURNING
does not order its output, so the batch is the right set in no
particular order (nothing in process_batch depends on the order); and
nothing is claimed, since the rows stay pending with no lock held past
the statement, so two workers against one database can still be handed
the same batch. Single-worker is the existing assumption and
FOR UPDATE SKIP LOCKED is what would change it. It also implied the
caller handles a failed stamp robustly; the caller logs and skips the
poll, which is worth stating plainly now that this writes and can fail
for reasons a plain SELECT could not.
The outer UPDATE now repeats status = 'pending'. Between the subquery
and the update a concurrent worker can settle a row, and without the
predicate the statement would stamp and return a row that had already
left the pending set.
Both starvation tests now assert the healthy row is still pending
before the poll that frees it. Without that they would keep passing if
the batch size ever grew past the stuck set, having silently stopped
exercising head-of-line yield, and the larger test now also asserts the
stuck rows rotated rather than settled.
The post-receive tail was spawned after `guard.release(...)`, which on a
successful push awaits the Tigris upload and then the advisory unlock while
the handler future is still tied to the client connection. The pack has
already landed on disk by then, so a disconnect in that window left a durable
push whose pins, recovery copy and announce were never spawned: the dropped
tail F2 closed, one step earlier in the handler.
Spawn it above `release`, gated on an explicit `receive_result.is_ok()`. The
`?` below the spawn can no longer be what keeps a rejected push from pinning
and announcing a half-applied repo, so the success check is now a named flag
that `release` consumes too. The tail is read-only on the repo directory and
holds neither the lease nor the advisory lock, so overlapping it with the
upload waits on nothing the handler still owns.
Tests: a handler-level disconnect driven through a parked `release` (new
store-level test seam), plus the must-not direction, a failed receive-pack
that spawns no tail. The U5 source gate now binds the new boundary: spawn
inside `if push_succeeded`, above `guard.release(push_succeeded)`.
A row the worker cannot make progress on stays pending, and while its
ordering key never moved it stayed among the oldest rows forever,
holding the fixed-size window against every healthy repo behind it.
Classifying more error kinds as terminal does not close this: an owner
directory that exists but denies traversal lets create_dir_all succeed
and defers at path_within_root instead, per-repo rather than as a
whole-repos_dir outage.
dequeue_pending_syncs now selects and stamps in one statement, ordering
by COALESCE(attempted_at, enqueued_at), so the key means least recently
handed out. Doing it in the dequeue rather than at each deferral branch
makes it hold by construction: no call site can forget it, a batch that
dies mid-loop still leaves its rows stamped, and a failed stamp is a
failed dequeue the caller already handles. enqueued_at keeps meaning
enqueue time so backlog age stays measurable.
The column lands as migration v12, not a v1 append: v1 never re-runs on
an existing install, so appending there would leave every deployed node
dequeueing against a column that does not exist. The upgrade-path test
simulates a v11 node and is the only test that catches that placement,
since every #[sqlx::test] otherwise starts from a fresh database.
N stuck rows still cost N slots per rotation, delaying a healthy row by
up to ceil(N/10) polls, so the attempts column and a retry cap remain
worth doing; this removes the starvation, not the residual cost.
create_dir_all reports AlreadyExists when a regular file, dangling
symlink, or link loop occupies repos_dir/<owner>, and none of those
clear on a retry. Classifying them as transient left the row pending,
and since dequeue_pending_syncs is oldest-first over a fixed batch of
10, a handful of them held the window and starved every healthy repo
behind them. This restores the pre-#272 behavior, where the same state
failed the clone and hit the terminal Err arm.
A concurrent mkdir resolves to Ok rather than AlreadyExists, because
create_dir_all falls back to is_dir() on EEXIST, so the race is not
caught by this classification.
The key is logged as the `repo` field on the lease waiter-cap shed and the
steal-bound warning (state.rs), and it was joined with a NUL. Those are the two
messages an operator reads to find a contended repo, and a NUL-hostile log sink
truncates the field at the separator, rendering two different repos' warnings
identically. Before this PR that field was a UUID or a mirror id, both printable,
so the NUL was a regression this change introduced.
Join with '/' instead. It cannot occur in the owner slug by construction - the
slug is owner_did.replace([':','/'], "_") - and it mirrors the shape of the disk
path the key exists to reproduce. A test asserts the key carries no control
characters so this cannot regress silently.
Rotating the U2 lease test's row to a UUID id is not incidental: the mirror path
that seeds it uses 'owner_short/name' as the row id, which now coincides with the
identity key and would leave the test unable to tell the two keys apart. The
rotated id is also the real shape of the bug. Re-verified RED against the reverted
call site after the change.
Two follow-ups from code review.
The pending-on-IoError arm had no operator-visible signal: the only
record_sync_processed calls were the done and failed arms, so a queue
stalled on an unmounted repos_dir looked identical to an idle one. Both
deferring arms now record "deferred" and the guard rejections record
"rejected", which also distinguishes a retryable stall from a terminal
refusal. There is still no attempt cap; bounding retries needs an attempts
column on sync_queue, which is a migration and belongs in its own change.
The two escape tests were asserting that repos_dir stays empty, but the
owner directory is created before containment has a parent to canonicalize,
so that assertion was detecting the create_dir_all side effect rather than
an escape. They now assert no mirror exists anywhere under repos_dir.
Their comments were also wrong about which layer holds. Mutation testing
says the slug rule and containment are each independently sufficient for
that input: removing either alone leaves the tests green, and only removing
both turns them red, with the mirror observed written outside repos_dir.
The comments now say that, and point at the per-layer witnesses instead:
./hello isolates the slug rule because it resolves back inside repos_dir
and containment approves it, and the two symlink tests isolate containment
because no character rule can see a symlink.
Code review caught an availability regression introduced by the guard added
earlier on this branch. validate_owner_did bails only above 256 chars, but
the owner half becomes a single path component and Linux NAME_MAX is 255, so
a 256-char owner passed validation and then failed create_dir_all with
ENAMETOOLONG on every attempt. The worker left such a row pending, and
dequeue_pending_syncs is oldest-first with a fixed batch of ten, so ten
unsigned notify calls held the entire window and starved every healthy repo
behind them. Before this branch the same row reached git clone, failed, and
was retired.
Two changes. Reject the length in the shared validator, so an undeliverable
slug never enters the queue from either boundary. Then split the
create_dir_all arm by whether the error can ever clear: a transient one
still leaves the row pending to be retried, a permanently uncreatable path
is marked failed rather than re-picked forever.
The earlier test missed this by asserting 257, one past the bound. It now
pins both sides: 256 rejected, 255 still accepted.
Also make the two permission-bit tests fail loudly instead of returning
early. One probed with symlink_metadata on a path that was never created,
which is Err for every user, so its root guard could never fire; the other
returned green after an eprintln that cargo swallows on a passing test, so
a root runner would have reported ok while exercising nothing.
The two throwaway probes from the investigation, in attack form, asserting
the closed outcome. The router-level one drives the full router with no auth
extension, because the point it pins is reachability: the attack needs no
signature and no operator config change, only an unsigned announce followed
by an unsigned notify.
The worker-level one seeds the peer row directly, since process_batch
resolves the peer URL before it looks at the slug and an unseeded row dies
at the no-peer arm, which would leave the test green with the guard gone.
It asserts on the parent directory too, because git clone removes the
destination on failure but leaves the parent.
Note for reviewers: reverting the worker slug check alone does not recreate
the absolute-path escape, since containment catches it and leaves the row
pending. That check is independently load-bearing for ./hello, which
canonicalizes back inside repos_dir and so passes containment.
The post-receive replication tail was spawned at the end of git_receive_pack, after
touch_repo, the trust-score update, per-ref certificate issuance and webhook fan-out.
A client or proxy disconnect anywhere in that window dropped the tail, and with it
this push's pins, recovery copy, and announcements - even though the push was already
durable on disk.
Spawn it immediately after the receive_result ? instead. That is the first point where
the push is known to have landed: guard.release() a few lines up runs on failure too,
so anchoring there would spawn a tail for a push git rejected, pinning and announcing
a half-applied repo - the state release() deliberately refuses to upload to Tigris,
and one a hostile pusher can produce on demand by aborting a pack mid-transfer.
RefUpdate gains Clone so the tail takes the parsed updates while the certificate and
webhook loops keep iterating their own copy.
This makes the tail run concurrently with certificate issuance rather than after it,
so a ref can be announced before its signed certificate exists. Accepted, and stated
here rather than left for a reviewer to find: cert issuance already fails open (errors
are logged and skipped) and the gossip event carries cert_id: None regardless, so no
announce consumer reads a certificate out of it.
Evidence is an ordering gate in inv22_gates.rs rather than a cancellation-race test,
matching the F3 gate's instrument: forcing a disconnect to land inside the cert/webhook
window is timing-dependent and would be flaky, while the ordering is precisely what
changed. The gate is mutation-verified in both directions - moving the spawn back below
the webhooks fails the upper bound, hoisting it above the ? fails the lower one.
The character rules on the slug cannot see a symlink. Two cases get through
them: a symlinked owner directory, and a symlinked <name>.git leaf. The leaf
is the one a parent-only check misses, since the parent canonicalizes clean,
exists() follows the link, and the fetch branch then runs git -C through it
with the mirror refspec, force-overwriting refs outside repos_dir.
So the predicate canonicalizes the candidate when symlink_metadata says it
exists, and its parent only when it does not. Canonicalizing the candidate
unconditionally would reject every first clone, which is total loss of
mirroring, and the must-not-break test covers that direction.
Three-valued rather than a bool because the worker has to tell a hostile
path from a filesystem that could not answer. Outside is terminal; an I/O
error leaves the row pending, since dequeue only selects pending and a
transient EACCES would otherwise retire a legitimate repo permanently.
The predicate is pure. create_dir_all lives at the call site so the admin
purge path can ask a containment question without mutating anything.
Two single-site wiring fixes from the same review round.
withheld_recipients_gated held its git_encrypt_semaphore permit in the async frame
and then awaited spawn_blocking, so dropping the future released the permit while the
uncancellable walk kept running. Every other post-receive scan helper moves the permit
into the closure - replication_withheld_set says so in as many words - so this aligns
the last site with the F4 contract. The regression drops the future mid-walk and
asserts the pool stays at zero while the git child is still alive.
git_info_refs passed a literal "git" to smart_http::info_refs while upload-pack and
receive-pack both pass state.git_bin, so no fake-git harness could drive the
advertisement path. The regression points git_bin at a fake emitting a distinctive
advertisement and asserts it reaches the response body; before the fix the body came
back from real system git (agent=git/2.43.0).
The global and per-source walk permits were handler locals, so a client disconnect
released them the moment the future dropped while the uncancellable spawn_blocking
probe/walk/read kept running against a slot the pool now considered free. A disconnect
flood could admit replacements while prior blocking git children still occupied
threads and PIDs, weakening GITLAWB_MAX_CONCURRENT_IPFS_WALKS and
GITLAWB_IPFS_WALK_PER_SOURCE on a surface any unauthenticated caller can reach.
Hold the pair in an Arc and clone it into each of the three blocking closures. The
permits release when the last clone drops, which collapses both failure directions
into one case: a disconnect leaves the abandoned closure holding the slot, and a
panicking closure leaves the handler holding it. No loop arm changes and the
JoinError arms keep today's taint-and-continue.
A move-and-return shuttle was implemented and rejected. It is correct only if every
arm that continues the loop re-binds the returned value, which the compiler cannot
enforce - the absent-object arm Ok(Ok(None)) => continue, taken on nearly every
iteration of a random-CID scan, drops it - and it forces a second decision about the
panic arms whose least-bad answer would let one broken repo shed every other owner's
/ipfs lookups.
The regression strengthens the existing admission test: it now asserts the slot is
still held after the future is dropped and before the child is killed, with a
child-still-alive precondition so it cannot pass vacuously. Only the walk site runs
under state.git_bin and can be pinned that way; the probe and read deliberately shell
to real git, so a source-scan gate in inv22_gates.rs binds all three clone sites
instead. Removing any one clone turns that gate red.
/api/v1/sync/notify accepts unsigned callers by default, and its only gate
was that node_did appear in the peer table, which one unsigned announce of
the caller's own DID satisfies. req.repo then reached enqueue_sync verbatim.
Validate after the peer check so an unknown peer still gets the unknown-peer
message, and before enqueue_sync so nothing is written. The 400 names the
field and carries no path: the validator's text never interpolates the slug,
so a hostile string is not echoed back either.
NOTIFY_BODY used "demo", a slug with no separator, so the three brake tests
would have started asserting against the new 400 instead of the brake.
repo_write_leases and encrypt_inflight keyed on record.id, but RepoStore::local_path
and the pg advisory lock key on the sanitized owner slug plus repo name, and the bare
repo on disk is reused when a repo is deleted and recreated under the same slug. The
row id rotates across that recreate, so both in-process serializers stopped serializing
exactly where the shared objects/ directory still needed them: a disconnected push's
reaper could still be tearing down while a new push took a different key and proceeded.
Add repo_identity_key(owner_did, repo_name) and use it at both sites. Three details are
load-bearing and carry tests: the sanitization is repo_disk_path's replace([':','/'],"_")
rather than normalize_owner_key, which folds a did🔑 prefix and would disagree with
the disk path; the separator is a NUL, without which owner 'a' + name 'b_c' and owner
'a_b' + name 'c' collide; and both call sites pass record.owner_did / record.name rather
than the request path segments, since get_repo normalizes DID aliases and the raw
segments would mint two keys for one directory.
The sanitization is deliberately not injective. Two did:web spellings can fold to one
slug, which would misroute encrypt coalescing if it were reachable, but repos.disk_path
is NOT NULL UNIQUE and holds exactly this derivation, so two such rows cannot coexist.
A test pins that fold in place: making the key injective is what would let two owners
sharing one objects/ directory push concurrently.
process_batch split item.repo on the first '/' and joined both halves onto
repos_dir. Since PathBuf::join does not normalize and an absolute second
component replaces the path, a queued `a//tmp/x` cloned an attacker's
origin to /tmp/x.git with repos_dir left empty.
The worker is the real choke point rather than defence in depth: three
writers reach sync_queue (the notify handler, trigger_sync, and the
gossipsub ref-update handler at p2p/mod.rs), and rows queued before this
change are already in the table.
The tests serve the composed remote URL for every hostile slug, not just
the escaping one. With an unreachable remote those rows end failed anyway
because the clone fails, so the scenarios would have passed before the
guard existed and proven nothing.
The sync queue carries a peer-supplied `repo` string that the worker turns
into a filesystem path, and `PathBuf::join` does not normalize, so an
absolute second component replaces the accumulated path. `a//tmp/x`
resolved to `/tmp/x.git`, outside repos_dir entirely.
Validate the slug once, in the module that already owns the owner and name
rules, and return the two halves so callers cannot re-split and drift. The
only rule added here is the leading `.`/`-` check on the owner half, which
`validate_owner_did` lacks because it also serves full DIDs.
Each guard is covered by a case that isolates it: the earlier rules shadow
the obvious candidates, so `../hello` does not actually exercise the
owner delegation and `a//tmp/x` does not exercise the separator count.
resolve_drain_object_list already re-fetches the repo row by owner/name and
deliberately reads visibility rules under record.id, because a delete+recreate
under the same slug gives the row a new id while the bare repo on disk is
reused. The encrypt write a few lines later still took ctx.repo_id, frozen when
the task was spawned, so recovery-pin metadata for a coalesced drain landed
against the deleted row and was unreachable from the live one.
Return the re-fetched id from the resolver and pass it explicitly to
pin_and_encrypt_objects. The parameter is explicit rather than read from ctx so
the two callers stay honest about which id they mean: the snapshot iteration
passes its own spawn-time id, the drain passes the fresh one.
The Arweave anchor is unaffected - EncryptedManifest carries no repo id and
builds its slug from owner and name, both stable across the rotation.
The Dockerfile cache layer primed from four crate manifests while the
workspace has six members, and crates/gl takes a path dependency on
icaptcha-client. Cargo loads the whole workspace before resolving, so the
layer aborted on the missing manifest and || true swallowed it. Every
image build has been a full cold build since icaptcha-client landed,
including the docker-build smoke test on every PR. Copy all six
manifests, and drop the || true: the layer is expected to succeed, and
tolerating its failure is what let this sit dead unnoticed.
Dockerfile.bins pinned rust:1.85-bookworm against a workspace
rust-version of 1.91. An unmet MSRV is a hard cargo error, so both Linux
musl targets of scripts/build-bins.sh have been unbuildable. Nothing in
CI builds that file, which is why nothing reported it.
Closes#265Closes#266
sync-release-lock ran cargo update --workspace on whatever Rust the
ubuntu-latest image shipped, so the lock it wrote for the release PR came
off an undeclared input. Install the same stable toolchain the other
cargo jobs use. This does not make resolution identical (stable moves,
and the MSRV gate checks the same lock on 1.91), it just removes the
runner image from the equation.
The release-binaries build, both Dockerfiles and the darwin path of
build-bins.sh also resolved without --locked, so a lockfile disagreeing
with the manifests was caught on the PR gates but not where artifacts
are actually produced. Add --locked to all five.
Two of those five cannot reach resolution today and are hardened for
when they can: the Dockerfile dep-cache layer at line 24 copies four of
the workspace's six member manifests, so cargo dies at workspace load and
|| true swallows it, and Dockerfile.bins pins rust:1.85 against a 1.91
MSRV. Both are pre-existing and left alone here. The live coverage comes
from release.yml, Dockerfile:37 and build-bins.sh.
The gitlawb-core dependency-purity gate still resolves without --locked
on purpose, but its comment justified that by saying the build and test
jobs do the same, which stopped being true when those jobs gained
--locked. Restate the real reason: a stale lock would red that gate for
something unrelated to the dependency closure it checks.
The 300s cleanup task swept the DID, create-IP, push, sync-trigger and
peer-write limiters but not ipfs_rate_limiter, so its expired keys sat
until the map hit max_keys and the inline capacity sweep reclaimed them.
Drive the sweep off AppState in a named sweep_rate_limiters so a limiter
added to the state and forgotten here fails a test instead of leaking
keys, and cover all six with a regression that goes red when any one is
dropped from the sweep.
GITLAWB_REPO_LEASE_MAX_WAITERS was also missing from .env.example and the
README settings table, leaving operators without a way to discover or
tune the cap behind the "repo is busy with another push" 503.
Three findings from the review round.
Moving the Pinata gate to the rules-only predicate was necessary, because a
coalesced push has no walk of its own to consult. But it also meant a push whose
OWN walk failed now acquired a global pin permit and ran a second full
re-derivation that failed the same way. Measured on a repo whose rev-list always
fails: twelve git children and three walk attempts, against four and one before.
With eight pin slots that defer rather than shed, eight such pushes stall pins for
every repo on the node. The gate now excludes a non-coalesced push whose own walk
failed, leaving the coalesced path on the rules-only predicate it needs, and the
pin helper returns before taking a permit for an empty object list.
The connection close on the failing-unlock path had no deadline, and release is
awaited inline while the write lease and both admission permits are still held.
The condition that produces that branch is a connection whose last statement
errored, and a blackholed socket to Postgres is one way to get there, so the await
could pin three admission resources for a TCP timeout. It now runs under a five
second bound; on expiry the future is dropped, which closes the socket and ends
the session anyway, which is what frees the lock. What is tested is the deadline.
That sqlx's close is the thing that stalls in production is reasoned from its
source, not reproduced, and the test says so.
The two tests for that path waited a fixed four hundred milliseconds before
asserting. Replacing the sleep with polling was not enough on its own: the test
pool sets a one second idle timeout, so a connection returned to the pool is
reaped shortly after and the session ends without any help from the code under
test. Both tests passed under mutation once the sleep was long enough to reach
that reaper, which means a straightforward flake fix would have quietly gutted
them. They now run against a pool with no idle reaper, so release is the only
thing that can end the session, and both mutations go red again.
Adds tokio's test-util as a dev-dependency for paused-time assertions. Production
dependencies are unchanged and the build stays --locked clean.
The previous attempt moved the per-source write permit above the lease park to
bound buffered pack bodies. Two reviewers demonstrated that it denies far more
than the source doing the parking. A source parked on one repo's lease was shed
on repos with no contention at all, and because the trusted-proxy setting
defaults to trusting nothing and keying on the socket peer, every pusher behind a
proxy, NAT, or CI runner pool resolves to one key: four parked pushes shed every
push from every pusher to every repo for up to twenty-one minutes. That is the
node-wide shed the permit was moved below the lease to prevent in the first
place, reintroduced at an eighth of the threshold.
The permit goes back below the lease. What is actually unbounded is the waiter
set, so that is what is bounded now, per repo, with a shed past the cap.
The cap counts live waiters, never the entry refcount. The refcount includes the
holder, so capping it would let a lease whose owner died without unwinding pin a
slot forever, which is the permanent wedge the steal timeout exists to survive.
The count is claimed in the same critical section as the refcount and held by a
guard scoped to the cancellable wait, so a cancelled or shed waiter gives it back.
An uncontended acquire takes the fast path and spends no waiter budget; that path
does not barge a queued waiter, which was checked over two thousand rounds rather
than assumed.
Two tests now guard the defect that was just removed: a source parked on one repo
is still served on another, and a parked push does not shed an unrelated pusher
sharing its resolved key. Restoring the old ordering turns both red. Capping on
the refcount instead of live waiters turns the steal-backstop test red. Holding
the waiter slot past the wait turns the fast-path test red.
What this bounds is the parked count, not the parked bytes. At a two gigabyte
pack limit the byte figure stays large, and the total across repos, whose
dominant term is the buffering the signature middleware does before any admission
point at all, is tracked in #263. Past the cap, same-repo concurrency sheds
instead of queueing; that is the trade the knob buys.
The two F1 tests asserted "the capped source shed rather than parked" by racing a
ten second timeout, and drove the holding push in ten millisecond slices. Under
`cargo test --workspace` that is a race against the scheduler rather than a test:
a reviewer saw two of five runs go red, and under load the pair stretched from
1.8s to 19.2s while the fake git holding the lease only lived for 10s, past which
every assertion in both tests is meaningless.
Shed versus park is a question of which path ran, so both are now read as state.
A shed is `Err(Overloaded)`. A park is a second reference on the lease entry,
exposed by a test-only `refs_for`; `acquire` takes that reference synchronously
before the cancellable wait, so a waiter is visible the instant it reaches the
lease, and it stays visible because the steal bound is 1260s. The tests spawn
their pushes and poll until one of the two mutually exclusive states appears, so
a starved machine iterates more rather than failing.
Every wall-clock assertion is gone. The worst of them was a sleep followed by
`!is_finished()`: a runtime that had not yet polled that task satisfied it
vacuously, which means the constant-key mutation could have gone green under
load, and the discriminator is the whole reason that test exists.
All three mutations still go red, now in seconds rather than by timeout expiry:
the per-caller acquire below the lease, the global permit above it, and the
per-caller key replaced with a constant.
Also fixes a third flake found on the way: the acquire-deadline test's follow-up
push inherited the deliberately short two second acquire timeout, which has to
cover a real advisory-lock round trip, so its `Overloaded` was indistinguishable
from the drained-pool `Overloaded` the assertion was actually testing for.
The sqlx `databases_pkey` collisions seen alongside this are machine residue, not
a leak from these tests: sqlx derives the test database name from a hash of the
test path and does cleanup-then-insert without a lock, so any two concurrent test
binaries running the same path collide. The orphan count here held at 53 across
every run.
The post-push tail resolved the withheld set and materialized the object list
before it ever consulted the per-repo coalescer, so rapid pushes to one repo each
parked on the encrypt semaphore and repeated the scan before discovering they
should have coalesced. The gate now runs first, keyed on the cheap
listable-at-root predicate that `replication_withheld_set` already computes from
rules alone before it touches the semaphore.
Three things made this less mechanical than it looks, all of them ways a naive
move would have shipped a regression.
The withheld set had a second consumer. It also gated the Pinata pin and the
branch-to-CID upsert, so a coalesced push that skipped the walk would have pinned
nothing and recorded no CID, while a test asserting merely that the announce
spawn ran would have stayed green. That gate moves to the announce predicate too.
Coalescing must not return from the tail. The announce spawn sits below the
coalescer in the same task, and returning early would drop gossip, the GraphQL
broadcast, Arweave anchoring and peer notify, which are per-push and
non-idempotent. A flag skips the encrypt section and falls through.
The publish paths are gated on announce and iterate the ref updates, not the
object list, so handing the tail the cheap predicate would have made a repo whose
walk is failing start gossiping and anchoring, losing the fail-closed property.
`pinata_object_list_for_refs` already recomputes the real set inside the pin
permit, so it now returns that verdict alongside the list and the tail uses it.
No additional walk is introduced.
Moving the gate earlier also opens a window where a push can coalesce while the
admitted one is still walking. If that walk then fails, the admitted task now
still spawns the drain with an empty snapshot so those tips are consumed, rather
than dropping the guard and discarding them. Note a released guard and a dropped
guard are indistinguishable by the in-flight map, since Drop clears the key
either way, so that case is asserted on drained work rather than on the map being
empty.
Each of the four is pinned by a mutation: walking anyway, returning early,
leaving the Pinata gate on the withheld set, and dropping instead of draining
each turn exactly one test red.
`release` discarded the unlock's result and set `released` regardless, so an
unlock that errored while its session stayed alive (a statement timeout, an admin
cancel) returned a connection to the pool still holding the repo's advisory lock,
with the Drop backstop already disarmed by that flag.
The error is now captured. On failure the connection is taken and closed, which
ends the session and makes Postgres drop the lock server-side, and only then is
the guard marked released, because the lock went with the session rather than
with the unlock. `close()` rather than `detach()` since we are already in an async
fn and can wait for the teardown deterministically.
Disposal is the only mechanism on this path. It cannot coexist with an armed Drop
backstop: disposal needs the connection taken, and Drop returns early when there
is none, while leaving the connection in place so Drop can use it just puts the
locked connection back in the pool when that spawned task ends. The Drop doc now
says so rather than claiming to cover a case it cannot.
The cancellation path is untouched. It was already correct, since `released` is
set after the await, and its test still passes unmodified.
The new tests assert against `pg_locks` from a connection held out of the pool
before the acquire, never by calling `acquire_write` a second time. That shortcut
would pass whether or not the fix is present, because the pool can hand back the
same connection and session advisory locks are re-entrant. The unlock error is
injected by leaving the pinned connection in an aborted transaction, which errors
the next statement while keeping the session and its lock alive, and the test
asserts that precondition before releasing.
Not covered: that the failure path still skips the Tigris upload. The test store
builds with no Tigris client, so the upload site is unobservable here without a
new seam, and that block is untouched by this change.
A push buffers its whole pack before the handler runs, then parks on the per-repo
write lease for up to git_service_timeout_secs * 2 + 60, which is 1260s at
defaults. Both admission permits were taken after that park, so nothing accounted
for waiters: any number of requests could sit on one contended repo, each holding
a fully buffered body, invisible to both semaphores.
The per-source sub-cap now runs before the park. It already exists, is sized at a
quarter of the push pool, and is keyed on the resolved source IP, so parked bodies
are bounded at four per source without a new knob or a new cap to tune.
The global permit deliberately stays below the lease. Moving it up is what
f02c5e1 fixed: a lease-blocked waiter holding a global slot sheds pushes to every
other repo node-wide. A per-source key does not have that shape, since exhausting
it denies only the source doing the exhausting, so the two permits belong on
opposite sides of the park and the comment there now says why.
The load-bearing test is not that a capped source sheds, which any cap satisfies,
including the global one this rejects. It is that a different source still parks
and is served while the first is at its cap. Keying the permit on a constant
leaves the shed test green and turns only that one red. Moving the global permit
above the lease turns the f02c5e1 guard red.
Bounded per source, not globally. A distributed set of sources can still each hold
their four, and the unbounded-total term is the pre-auth buffering in #263 rather
than the lease.
Two review findings on the sync-release-lock job.
Fail on every dispatch failure. The step tolerated the 422 a release branch
cut before pr-checks gained its workflow_dispatch trigger returns, on the
theory that release-please would rewrite the branch and heal it. A rewrite
only happens when a later releasable commit lands, so an open release PR
nobody touches is never rewritten, and the tolerated path left the lock-sync
commit this job had just pushed on the release head with no --locked
validation. That is the exact state the job exists to prevent. No release PR
is open today, and after this lands every release branch is cut from a main
that already carries the trigger, so the tolerance protects nothing.
Scope the fallback discovery. `gh pr list` includes fork PRs and a fork's
headRefName is bare, so matching on the `release-please--` prefix alone let
any fork nominate the branch this job checks out, commits to, and dispatches
on: actions/checkout then cannot resolve that fork-only ref here, failing the
job and leaving the real release PR unsynced and unchecked. Discovery now
requires the head repo to be this repository and the author to be the
release-please app, and selects the highest PR number instead of relying on
gh's undocumented default ordering.
Verified the new filter against the hostile inputs, using the jq program as
extracted from the parsed YAML: a fork lookalike, a fork PR claiming the
app author, and a same-repo human branch named `release-please--*` are all
rejected; the real release PR is still selected when listed after a fork and
when a second component release PR is present; an empty listing yields empty.
The previous expression selected the fork's branch on all three hostile cases.
The branch forked before v0.7.0 merged, so the rebase left main's 0.7.0
manifests against workspace lock records still at 0.6.0. That state fails
the gate this PR adds: `cargo metadata --locked` exits 101 with "cannot
update the lock file because --locked was passed".
Regenerated with `cargo update --workspace`, the same command the new
sync-release-lock job runs. Only the 5 workspace crates move 0.6.0 ->
0.7.0; all 252 external dependencies are unchanged, so this is a version
resync and not a dependency bump. `cargo metadata --locked` now exits 0.
Proc-macro helper for curve25519-dalek (already allowlisted). Newer cargo
versions list it under `cargo tree --edges normal` while older ones do not;
the allowlist is a superset so it passes on both. Caught by core-deps-purity
on the first CI run of this branch.
gitlawb-core is embedded by gl, git-remote-gitlawb, and the node daemon,
so it must stay lean. Add a CI gate that fails if its normal (non-dev,
non-build) transitive dependency tree gains any crate not on an explicit
allowlist.
The allowlist is exhaustive-by-construction: a new heavy dependency reds
CI whether or not anyone thought to ban it, which a denylist cannot do.
The checker resolves like the build/test jobs (no --locked, since the
committed lockfile can lag the manifests) and snapshots/restores
Cargo.lock so a local run leaves no working-tree side effects.
- ci/gitlawb-core-allowed-deps.txt: the 83 current normal deps, plus the
regeneration command and rationale in the header
- scripts/check-gitlawb-core-deps.sh: recompute and diff; hard-fail on a
non-allowlisted crate, note-only on a stale (removed) allowlist entry
- pr-checks.yml: new hard-fail core-deps-purity job, matching the pinned
action SHAs and style of the sibling jobs
Settles the review's design question: a valid signature alone proves
internal consistency — the payload was signed by whatever key the
certificate itself names — so a hostile source could self-sign and
pass. --verify now also requires the issuing node_did to match a
trusted anchor: --expect-node when given, else the DID of the node
being queried; unanchorable (node info unreachable, no --expect-node)
fails closed. The displayed verdict wording says what it proves.
Also from review:
- A frozen-literal test pins gl's payload serialization to the node's
canonical byte form, so serialization drift (new field, a
preserve_order feature unifying across the workspace) fails a test
instead of silently rendering every real certificate INVALID.
- The printed unalias remediation uses the idempotent-safe
'unalias gl 2>/dev/null' form everywhere (doctor, install.sh, brew
caveats), matching what doctor already suggested.
- Acknowledged, deliberately left: bash-side alias detection and
127/8-range loopback matching (noted as out of scope for a
warning-level heuristic).
- cert show: the node-DID comparison is contextual; a node-info fetch
error now prints a note instead of failing the command after the
certificate and verdict were already shown.
- doctor: loopback detection parses the URL host exactly — substring
matching misread localhost.example and localhost-in-path URLs.
- init: a detached HEAD gets its own hint (switch to a branch first)
instead of a guessed 'git push gitlawb main'.
- status: every fetch AND push URL of each remote is inspected — a
gitlawb:// push URL behind an https fetch URL now counts.
- doctor/install.sh: unalias detection is command-aware — comments,
'unalias global', and gl-only-in-a-trailing-comment no longer
suppress the warning (BSD-grep-compatible pattern; ordering vs
oh-my-zsh load documented as accepted slack). Eight new test cases.
We ship `gl` and `git-remote-gitlawb` to Windows users via release.yml but run
no Windows tests: pr-checks.yml is ubuntu-only, and the single windows-latest in
the repo is the release build step, which never runs `cargo test`. So changes to
the shipped Windows CLIs (e.g. the multi-round fetch fix in #192) reach Windows
users with zero test runs on Windows.
Add a `test-windows` job that runs `cargo test -p gl -p git-remote-gitlawb` on
windows-latest with `continue-on-error: true`, mirroring the informational beta
lane: a Windows-only failure is visible on the PR but does not block merge. The
two crates are the client CLIs and depend on no database, so the job needs no
Postgres service (service containers do not run on windows-latest regardless);
git is preinstalled on the runner for the real_git_fetch harness.
This makes the #[cfg(windows)] regressions from #192 load-bearing instead of
never-run. Un-gating the currently #[cfg(unix)] harness tests is the deferred
follow-on in #228.
install.sh detects the common shadowing setup (omz git plugin, no
unalias in ~/.zshrc) and prints the fix right after installing; the
generated Homebrew formula gains a caveats block saying the same, so
brew install/upgrade surfaces it too. Catching this at install time
beats letting the first gl invocation print git's error.
oh-my-zsh's default git plugin ships gl='git pull', silently shadowing
gl in every interactive zsh — the symptom is the baffling 'fatal: not a
git repository'. Aliases beat PATH, so no installer can fix it; doctor
now detects the setup (explicit alias gl= in ~/.zshrc, or the omz git
plugin without an unalias) and prescribes the one-line fix. Best-effort
heuristic, warning-level, covered by four unit tests.
A localhost GITLAWB_NODE red-flagged every self-hosted and dev-harness
setup even when the node was healthy; the connectivity check already
fails loudly when the configured node is unreachable. The env check now
passes with an explanatory note.
show printed the signing payload and told the user to verify offline
themselves. It now rebuilds the node's canonical payload (field order
mirrors gitlawb-node cert issuance), derives the Ed25519 key from the
certificate's node DID, verifies, and prints a VALID/INVALID verdict;
--verify makes the exit code reflect it for scripting. Round-trip +
tamper + malformed-signature unit tests included.
- git init -b main (plain-init fallback for git <2.28) so a fresh repo
can't land on master while every hint says main.
- The closing hint checks HEAD: with no commits it prints the
add/commit step first — a bare 'git push gitlawb main' at that point
fails with 'src refspec main does not match any', the exact trap the
acceptance test walked into. The current branch name is used in the
hint either way.
gl init names its remote 'gitlawb', but detect_gitlawb_remote() only
ever read origin — so gl status reported 'not in a gitlawb repo' in the
very repo gl init had just set up. Scan all remotes, preferring origin
(clones) then gitlawb, then any other gitlawb:// URL. (#231 acceptance
test finding)
Address review: the alerter reflected CI events rather than main's
current state, which could both cry wolf and stay silent.
- Freshness gate: a run only opens or closes anything when its commit is
still main's tip — re-running an old red run after a fix has landed no
longer opens a false stop-ship issue (and a stale green can't close a
current one).
- Trigger covers timed_out and startup_failure — both are red mains that
previously alerted nothing. cancelled stays excluded as a human abort.
- Recovery: a green run on the tip comments and closes open ci-main-red
issues, turning the alert into a current-state signal.
- queue: max — the single-slot pending queue let a later completion
displace a pending red alert under burst merges.
- Issue lookup filters out pull requests carrying the label.
- github-script converged on v8.0.0 (SHA verified against the tag) in
pr-triage too.
Two near-simultaneous red-main events could both pass the open-issue
lookup and each create a ci-main-red issue (or 422 racing createLabel).
A workflow concurrency group with cancel-in-progress: false serializes
them; the default single-slot pending queue is enough since a superseded
alert's state is re-read by the newest run anyway.
Two individually-green PRs can break main in combination when the second
merges on checks that ran against a stale base (#113 + #145 compiled fine
apart, but #145 added a field to ReceivedRefUpdate that #113's new test
initializer never set — E0063 on merged main, fixed by #229).
- pr-checks: add merge_group trigger so the suite can gate a GitHub merge
queue, which re-validates every PR against the true merged result.
- main-red-alert: new workflow that opens/updates a labeled issue whenever
a push-to-main PR Checks run fails, so a red main is loud instead of
silent (release-please PRs inherit main breakage, as #182 showed).
buildx's --output name= does no lowercasing (unlike metadata-action),
so ghcr.io/Gitlawb/node is an invalid reference and the per-arch digest
push would fail outright. Compute the lowercase image once in the
resolve step and reuse it, matching the manifest job.
The v0.6.0 release run's multi-arch Docker build wedged for 3+ hours
emulating the arm64 Rust release build under QEMU, and — because the
workflow concurrency group is cancel-in-progress:false — every queued
release run sat blocked behind it.
- docker is now a matrix: amd64 on ubuntu-latest, arm64 on
ubuntu-24.04-arm — both native, mirroring release-binaries. Each leg
smoke-tests the image (--version must report the released version)
and pushes by digest only.
- docker-manifest stitches the digests into the :X.Y.Z, :X.Y and
:latest tags with buildx imagetools, so a half-finished matrix can
never publish a partially-tagged image.
- workflow_dispatch grows docker_backfill_tag (and npm_backfill_tag
becomes optional): each channel backfills independently for an
already-cut release — needed to publish the missing v0.6.0 image.
A backfill run must be a narrow npm-only operation. Without an event
guard, workflow_dispatch also ran release-please with contents+PR write
permissions and could create or advance a release PR as a side effect.
npm-publish is unaffected: its !cancelled() condition tolerates the
skipped need, and all other jobs already gate on release_created.
The npm-publish job silently skipped every release since 0.4.x because
NPM_TOKEN was never configured — the guard made the job green while
@gitlawb/gl sat at 0.3.7 on the registry. Replace token auth entirely:
- npm Trusted Publishing: the job's GitHub OIDC identity (id-token: write)
is exchanged with the registry; no long-lived secret to configure, leak,
or rotate. Provenance is attested automatically. All five @gitlawb
packages exist on the registry now, so OIDC no longer has the
first-publish bootstrap problem that forced token auth in #127.
- Fail loud: the secret guard is gone. If a package's trusted publisher
isn't configured on npmjs.com, the publish errors visibly instead of
green-skipping.
- setup-node 24 + npm >=11.5.1 (OIDC exchange support; older npm falls
back to absent token auth with a confusing error).
- workflow_dispatch backfill: publish an already-cut GitHub release
(e.g. v0.6.0) to npm by tag — binaries come from the existing release
assets, so only the npm-publish job runs.
PRs #113 and #145 raced: #145 added owner_did to ReceivedRefUpdate and
updated its own call sites, but the seed initializer in
list_repo_events_fails_closed_when_repo_lookup_errors (added by #113)
was never updated, so main fails to compile test targets (E0063).
This breaks clippy --all-targets, both test jobs, and MSRV on CI,
including the release PR #182.
owner_did stays None here on purpose: the test exercises the legacy
None-branch slug fallback.
Co-authored-by: Kevin Codex <[email protected]>
* fix(gl): point doctor's version check at Gitlawb/node, not the frozen Gitlawb/releases repo
check_version queried https://api.github.com/repos/gitlawb/releases/releases/latest.
Gitlawb/releases is a separate repo that stopped receiving tags after v0.3.8
(2026-03-30). Releases have shipped directly on Gitlawb/node since v0.4.0, via
release-please, and install.sh already treats Gitlawb/node as the canonical
release source. The old query target meant gl doctor's version check could
never again detect a real update.
Points the check at Gitlawb/node instead, matching install.sh's own default,
and makes the GitHub API base injectable so the fix is covered by a test
against a mocked server rather than the real GitHub API.
Closes#197
* fix(gl): check HTTP status before parsing the release response; strengthen up-to-date test
Addresses CodeRabbit review on #198:
- check_version now checks resp.status().is_success() before parsing the body,
so a GitHub rate-limit or other API error surfaces as its actual HTTP status
instead of a generic "could not parse latest tag".
- test_check_version_up_to_date now also asserts the detail message, so the
test can't silently pass on an unmatched mock request (mockito's fallback for
an unhandled request is a 501, which check_version also maps to Ok).
* test(gl): add unit test for non-2xx check_version response
* fix(gl): remove trailing blank line at EOF in doctor.rs
cargo fmt --all -- --check fails on the extra terminal blank line, and the
fmt + clippy workflow runs that check as a required gate.
* fix(core): honor wildcard capabilities in Ucan::can, matching is_attenuated_by
Ucan::can did strict equality on both resource and action, so a validly
delegated wildcard capability (with: "*", or can: "*" / "repo/admin") could
never match a concrete runtime request even though is_attenuated_by treats
those same values as wildcards during delegation-chain verification.
Not a security issue: the mismatch only produces false negatives (denying a
capability the delegation chain actually grants), never false positives.
Dormant today since no production code path calls can() yet — auth's
caller_authorized_to_push is pure did_matches(), with the UCAN-capability
check noted as future (Phase 2) work.
Makes can() mirror is_attenuated_by's semantics: the stored capability plays
the "parent" role, so with == "*" matches any resource and can == "*" or
"repo/admin" matches any action.
Closes#199
* test: add can_honors_action_wildcard unit test for Ucan::can
* feat(core): add shared is_bidi_format predicate for terminal sanitizers (#183)
Terminal-bound error sanitizers must strip Cf bidi/format controls as well as Cc control bytes (INV-6), or a right-to-left override in attacker-influenceable bytes can visually reorder the displayed line (Trojan Source, CWE-451). gl already did this with a local is_bidi_format; two sibling sanitizers shipped with only the Cc half (#183). Centralize the reordering-subset predicate here so all consumers reuse one audited definition instead of re-deriving the set. Deliberately excludes legitimate Cf chars (ZWJ, soft hyphen).
* refactor(gl): consume shared is_bidi_format from gitlawb-core (#183)
Replace gl's local is_bidi_format copy with the centralized gitlawb-core predicate so exactly one definition exists in the workspace and the three terminal sanitizers cannot drift apart. Behavior-preserving; the existing strip test is unchanged and a must-not-over-strip guard (ASCII + Arabic U+0627 + ZWJ U+200D survive) pins the reordering-only scope.
* fix(git-remote): strip Cf bidi controls from terminal-bound error bodies (#183)
safe_error_body_excerpt filtered only Cc control bytes, so a Cf bidi override in a hostile or MITM'd node's HTTP error body reached the operator's terminal and could visually reorder the line (Trojan Source, CWE-451). Drop the reordering Cf subset via the shared gitlawb-core::sanitize::is_bidi_format, without setting pending_space so stripped chars leave no phantom gap. Covered per bidi class plus a transport-path proof through handle_connect, RED observed before the fix.
* fix(icaptcha): strip Cf bidi controls from terminal-bound error excerpts (#183)
sanitize_excerpt filtered only Cc, so a Cf bidi override in a node-advertised iCaptcha service's reason/body reached the terminal on the bail! path (Trojan Source, CWE-451). Drop the reordering Cf subset via the shared gitlawb-core predicate, without pending_space. The existing sanitize_strips_control_chars_and_escapes test asserted only is_control() and so was vacuous for this class (why the gap shipped); a dedicated per-class bidi test now covers it, RED observed before the fix.
* style: apply rustfmt to #183 sanitizer additions
* test(#183): pin a leading-position bidi char in the strip tests
The per-class strip tests placed bidi chars only mid-string. A bidi char at index 0 exercises the empty-excerpt/out path (where the pending_space guard behaves differently), confirming a leading override still yields no phantom leading space. Traced correct; now asserted.
* test(review): pin that bidi chars don't consume the length cap (#183)
Correctness, adversarial, and testing reviewers all flagged that no test locks in the drop-before-count ordering: a dropped bidi char must not consume MAX_ERR_CHARS budget. Behavior verified correct by execution during review; this test guards it against regression (interleaved bidi across the cap still yields exactly MAX_ERR_CHARS visible chars, no phantom spaces).
---------
Co-authored-by: t <t@t>
* test(icaptcha): assert obtain_proof consumes the mocked service
Add an integration test that runs obtain_proof against a mockito server and
asserts both /v1/challenge and /v1/answer were consumed, so the flow is proven
to call the service rather than short-circuit or make a live call. Adds mockito
as a dev-dependency.
* test(icaptcha): block non-mock egress so the no-live-call guard is real (#184)
The mock-consumed assertions only prove the two mock endpoints were hit; they
cannot see a request that also leaks the DID, answer, or API key to DEFAULT_URL
or any other origin, so the test's stated "no live call" guarantee was untested.
Blackhole every non-loopback destination for the flow: NO_PROXY lets the
loopback mock through while ALL_PROXY routes any other host to a closed port, so
the client contacting anything but the injected mock fails the request. Verified
load-bearing: pointing cfg.url at DEFAULT_URL makes obtain_proof fail with a
proxy tunnel error (RED), and the loopback-mock path still passes (GREEN).
* test(icaptcha): blackhole scheme-specific proxy vars so the no-live-call guard covers https
The mock-consumed guard blackholed non-loopback egress with ALL_PROXY only, but
reqwest honors HTTPS_PROXY/https_proxy (and the http equivalents) OVER ALL_PROXY
for https URLs. On a runner whose environment has a working HTTPS_PROXY, a
regression could still leak the DID/answer/API key to the https DEFAULT_URL while
both loopback mock calls are consumed and the test stayed green. Override the
scheme-specific proxy vars (both cases) to the closed port too; NO_PROXY keeps the
loopback mock direct. Single #[test] per process, so the override needs no restore.
Verified: a throwaway repro showed reqwest routing an https request to HTTPS_PROXY
past the ALL_PROXY blackhole (the gap); with the fix the override sends that egress
to the closed port even when a working HTTPS_PROXY is pre-set; the guard still goes
RED when cfg.url points off-loopback.
---------
Co-authored-by: t <t@t>
* fix(node): carry full owner DID on ref-update wire event (#144)
* test(node): add ref-update owner_did round-trip, DB, and API tests (#144)
* fix(node): move owner_did migration to v10, add upgrade-path test
* fix(node): add owner_did to GraphQL, broadcast, and REST feed surfaces (#144)
* fix(node): address review feedback — drop ipfs regression, fix doc, improve migration test, use local owner_did
* fix(node): fix migration v10 test to actually start from v9 schema; complete owner_did API assertion
* fix(node): show local owner on global feed, assert nullable in migration test, add owner_did echo assertions
* fix(node): prefer stored wire owner_did over lossy slug-local resolution; mirror projection in GraphQL
* fix: address review findings — restore iCaptcha PoW/origin matching, fix feed owner_did projections
* fix: match icaptcha-client Cargo.toml ordering with upstream/main
sha2 was at line 16 in our branch but at line 18 in upstream/main.
The 3-way merge in CI's pull/145/merge checkout kept both lines,
causing a 'duplicate key' error. Moving sha2 after tracing matches
upstream/main exactly, so the merge commit will have a single entry.
* fix(node): bind wire owner_did to local repo before surfacing as trusted
The peer-supplied owner_did (HTTP notify + gossipsub receive) and the
repo slug arrive over untrusted paths, yet REST and GraphQL projected the
wire value as trusted feed ownership.
- P1: add Db::resolve_ref_update_owner_did so a wire owner_did is echoed
only when it matches the canonical owner of the actual local repo at that
slug; otherwise it is dropped (returns null). The canonical DID is
returned, never the raw wire bytes.
- P3: legacy owner_did = None rows are now attributed via an exact,
normalized, unique local match (get_repo), not the prefix-tolerant drop
gate, so a cross-method slug collision cannot synthesize the wrong owner.
- Unify REST (global + per-repo gossip) and GraphQL on the shared resolver
so the two surfaces cannot drift. Add tests for forged-wire drop and
exact-legacy attribution.
* style(node): apply cargo fmt to owner_did resolver
* fix: batch owner_did resolution, propagate errors, preserve full DID for mirrors
P2 batch: replace per-row join_all with resolve_ref_update_owner_dids
that issues at most one get_repo query per distinct local repo in the
page rather than one per event row. Applies to both REST and GraphQL
surfaces.
P2 error: REST handler now propagates DB errors (rather than silently
coercing them to null) by calling the batch method with ?.
P2 mirror: when the validated wire DID carries a full method prefix
(e.g. did:key:z...) and the matching repo is a mirror storing only
the bare key, return the full wire DID instead of the bare key to
preserve the DID-method contract.
P1/P3 already addressed in fb553e6.
* fix: rebase onto current main and use single batch SQL for owner DID resolution
Rebased from stale 3ec8cbf base onto current upstream/main (ad7c2b2),
dropping the already-merged iCaptcha work from the diff. The PR diff
is now only the 11 owner-DID files.
The batch owner-DID resolver now issues a single set-based SQL query
with one OR condition per distinct slug, replacing the loop of per-key
get_repo calls. A 200-row page of 200 distinct remote slugs now costs
1 SQL round-trip instead of 200.
P1/P3 already addressed in c64c082. P2 mirror, error propagation,
and the earlier batch refactor preserved in the rebased commits.
---------
Co-authored-by: Gravirei <[email protected]>
* fix(node): owner-gate list_webhooks to stop leaking webhook URLs (#94)
list_webhooks sat on the optional_signature read group with no auth or
ownership check, so any unauthenticated caller who knew a repo's owner/name
could enumerate its webhooks (target URL, created_by_did, events; only the
secret was redacted).
Gate it read-visibility-then-owner, mirroring create_webhook/delete_webhook:
headerless -> 401, non-reader of a private repo -> 404 (existence hidden,
uniform with the sibling read surfaces), non-owner of a public repo -> 403.
Add a list_webhooks row to the authz_guard drift table so the gate can't
silently regress.
* fix(node): read-visibility-gate list_replicas for private repos (#94)
list_replicas sat on optional_signature with no visibility check, so a
private repo's replica URLs were enumerable by anyone who knew owner/name.
Replica lists are a documented public mirror-discovery surface, so gate on
read visibility (authorize_repo_read) rather than owner-only: a public repo
stays anonymously listable, a private repo returns 404 to anyone who cannot
read it.
register_replica registers non-owner DIDs, and a replica operator is not a
visibility reader, so a non-owner replica operator of a private repo gets
404 — the intended contract, pinned by the new test. Add the authz_guard
row.
* fix(node): read-visibility-gate list_protected_branches (#94)
list_protected_branches sat on optional_signature with no visibility check,
leaking a private repo's branch names to any unauthenticated caller. Gate on
read visibility (authorize_repo_read at the root, INV-2 root-listing
exception): public repos stay anonymously listable, private repos 404 to a
non-reader. Add the authz_guard row.
* fix(node): gate list_repo_events for local private repos, keep gossip-only (#94)
list_repo_events served a private repo's ref certificates and gossip ref
metadata (ref names, SHAs) to any unauthenticated caller. The handler also
tolerates a repo it knows only via gossip (no local row) and legitimately
serves its events, so a blanket get_repo->authorize_repo_read swap would have
404'd that federation path.
Gate only the locally-hosted branch: when get_repo returns a record, run
visibility_check on it at the root and 404 on deny before fetching either the
cert or gossip events. A gossip-only repo (no local row) keeps its existing
public federation-feed behavior; the global /api/v1/events/ref-updates feed
remains a separately tracked deferral. Add the authz_guard row (visibility_check
marker, since the gate is direct to avoid a second get_repo).
* fix(gl): sign the webhook list request now that GET /hooks is owner-gated (#94)
`gl webhook list` built its NodeClient with no keypair, so the GET went out
anonymously. With the node now owner-gating GET /hooks, that request returns
401. Thread a --dir through the List subcommand and load the keypair, matching
`gl webhook create`/`delete`, so the listing is signed as the owner.
(Replica and protected-branch listings stay anonymous for public repos under
read-visibility gating, so their gl clients need no change.)
* fix(review): sign the webhook list GET (get_signed), not plain get (#94)
Code review caught that NodeClient::get() never attaches the RFC 9421
signature even when a keypair is loaded — only get_signed/post/put/delete
sign. The earlier gl webhook-list fix loaded the keypair but still called
get(), so `gl webhook list` (and the MCP webhook_list tool) 401'd against
the now-owner-gated endpoint. Switch both to get_signed.
Harden the two cmd_list tests to require a Signature/Signature-Input header on
the mock (verified: they fail against plain get(), pass with get_signed) — the
missing assertion that let the bug through. Add the untested owner-of-private-
repo webhooks path (200 + redaction). Fix the stale module doc and the
authz_guard bucket comment for list_webhooks.
* fix(gl): sign read-visibility GETs so private-repo owners can read their own repos (#94)
The #94 server change read-visibility-gates GET /replicas and
/branches/protected: public repos stay anonymous, private repos require the
owner/reader to authenticate. Four gl read commands sent unsigned GETs and so
broke (404) or silently degraded (empty output) for a private repo's owner:
gl protect list, gl repo owner (silent empty protected list), gl repo info
(silent missing replica count), gl repo replicas (and it had no --dir flag).
Add NodeClient::get_maybe_signed — signs when an identity is present, stays
anonymous otherwise (mirrors the conditional signing of post/put/delete) —
and route the four call sites through it. Add --dir to `gl repo replicas`.
Harden the protect-list tests to assert the request is signed (verified: they
fail on plain get()). Public-repo anonymous reads are unchanged.
* fix(review): sign the primary repo fetch in gl repo info/owner too (#94)
Round-2 review caught that cmd_info and cmd_owner loaded a keypair and signed
their replica/protected-branch sub-fetch but left the PRIMARY GET
/api/v1/repos/{owner}/{name} on plain get(). get_repo is read-visibility-gated,
so for a private repo that primary fetch 404'd first and the command bailed
before reaching the signed sub-fetch — the half-fix never actually let an owner
inspect their own private repo. Route both primary fetches through
get_maybe_signed.
Add the missing signing guards: harden test_cmd_owner_is_owner to require a
signature on both mocks, and add tests for cmd_replicas and cmd_info (both were
untested) asserting the request is signed when an identity is present.
Out of scope (pre-existing, gated before #94, untouched here): gl repo commits,
gl pr list/view/diff, and the MCP repo_get/repo_commits/pr_* tools call other
already-gated read endpoints unsigned and are broken for private repos
independently of this change.
* test(review): execute the previously-reasoned-only paths (#94)
Close the vetting gaps from the review:
- node: a listed reader (non-owner) passes the read gate but is still refused
the webhook list (403), and headerless on an existing-private vs an absent
repo both 401 — proving the 401 fires before any lookup, so it is not an
existence oracle.
- node: the read-visibility surfaces (replicas, protected branches) admit a
listed reader who is not the owner (the allow-list branch of visibility_check),
while a non-reader stranger still 404s.
- gl: get_maybe_signed's anonymous fallback (no identity -> unsigned GET) is now
exercised, RED-proven by flipping it to get_signed (which signs and fails the
no-signature-header assertion).
* test(review): execute the MCP dispatch and gl->node signature seams (#94)
Close the last two reasoned-only paths by execution:
- MCP webhook_list: a call_tool("webhook_list", ...) dispatch test against a
mock node asserts the /hooks request carries a signature (get_signed).
RED-proven: flipping the arm back to get() fails the header assertion.
- gl->node end-to-end: a real RFC-9421 signature produced exactly as the gl
client's get_signed does (gitlawb_core::http_sig::sign_request over GET +
empty body) is run through the node's actual optional_signature middleware,
which verifies it and authorizes the owner (200 + webhook body). RED-proven:
signing over POST while sending GET makes the node reject it. This stitches
the gl signing side and the node verifying side in one test rather than
mocking one end.
* fix(review): fail closed when the repo lookup errors in list_repo_events (#94)
`get_repo(...).await.ok().flatten()` collapsed a lookup Err into None, so a
transient DB failure skipped the read-visibility gate and the handler served a
private repo's gossip ref-update events. Propagate with `?` instead: an error
now maps to AppError::Internal (500) and fails closed, while a genuine Ok(None)
(repo not hosted locally) stays the intentional ungated pass-through.
Add a regression test that forces a get_repo error (drop the column its SELECT
reads) and asserts 500 with no secret in the body. The injection is
self-verifying: it asserts the lookup succeeds pre-drop and errors post-drop, so
the test can't pass vacuously if get_repo's projection changes later.
* fix(node,gl): close residual gaps in repo-read gating (#113)
Address jatmn's review feedback on the read-surface gate:
- gl webhook: resolve owner from the signing identity, not the node DID.
cmd_create/list/delete all routed through the local resolve_owner helper,
which only ever returned the node root DID, so for a repo owned by the
local identity the owner-gated /hooks route returned 404/403. Drop that
helper and reuse repo::resolve_owner_repo_pair (accepts owner/repo, else
derives from --dir/keypair). Fixed across all three subcommands so
create-writes and list/delete-reads stay on the same owner.
- gl webhook list: check the HTTP status before reading `webhooks`. A
401/403/404 body has no webhooks field and previously printed
"No webhooks" and exited 0, hiding the rejection; now bails with the
server message like cmd_create.
- visibility::is_owner: delegate to api::did_matches so the read gate and
require_repo_owner share one normalization. Fixes the under-match (bare
owner stored, full did:key caller, as mirror rows) that denied the owner
with the opaque 404, and the cross-method over-match.
- authz_guard: pin both halves of list_webhooks (authorize_repo_read and
require_repo_owner) so removing the existence-hiding gate fails the drift
test.
cargo fmt/clippy clean; 644 workspace tests pass.
* test(node)+fix(gl): close events reader-coverage gap, align label-list signing
Self-review follow-ups on the read-gate work:
- list_repo_events: add list_repo_events_admits_listed_reader, pinning the
allow-list branch of the events gate (a listed non-owner reader gets 200
with the cert; a non-reader stranger 404s with no leak). Mirrors the
read_visibility_admits_listed_reader coverage the sibling replica and
protected-branch surfaces already had; the events handler was the only
gated read surface missing it.
- gl repo: cmd_label_list now loads an identity and uses get_maybe_signed
instead of a bare anonymous get(), matching cmd_replicas/cmd_info. Public
repos stay anonymously listable; a private-repo owner can read their own
labels once /labels is read-visibility-gated, with no silent client drift.
fmt/clippy clean; 645 workspace tests pass.
* fix(review): pin the label-list signing and the events 404 sha-no-leak
Final-review hardening of the prior two follow-ups:
- gl repo label-list tests: require the signature/signature-input headers on
the signed-branch mocks, and add an anonymous (no-identity) test asserting
the header is absent. Without the header assertion the tests passed even
with a bare get(), so the signing change was not actually pinned
(RED-verified: reverting to get() now fails both signed-branch tests).
- list_repo_events_admits_listed_reader: also assert the cert sha (not just
the ref name) is absent from the stranger's 404 body, matching the
list_repo_events_is_read_visibility_gated leak-check convention.
fmt/clippy clean; 646 workspace tests pass.
* fix(gl): error on 2xx webhook-list body missing the webhooks array (#113)
A 2xx /hooks response without a `webhooks` array could hide a node/API
contract regression. cmd_list now errors with context instead of mapping
it to "No webhooks". An empty-but-present array stays a valid no-webhooks
result. Adds a test for the missing-array case.
* fix(gl,node): scope MCP webhook_list owner to keypair, gate list_labels on read visibility (#113)
- MCP webhook_list: default owner to the signing keypair's short DID (explicit
owner arg still overrides) instead of the node root DID, so the signed,
owner-gated /hooks request resolves to the caller's own repo. Mirrors the
gl webhook list fix. resolve_owner and webhook_create/delete left untouched.
- list_labels: gate on authorize_repo_read so a private repo's label names 404
for non-readers (public repos stay anonymously listable), matching the sibling
read surfaces and making the existing client comment true. Added the
authz_guard marker and a read-visibility integration test.
- gl label list: check HTTP status before reading the body so the now-gated
404 surfaces as an error instead of printing "No labels" and exiting 0,
matching the sibling list clients (replicas, protected branches).
* fix(gl): surface denied repo info reads instead of fabricating a repo (#113)
cmd_info parsed the GET /api/v1/repos/{owner}/{name} body without checking
the HTTP status, so a private-repo JSON 404 was rendered as a fake repo
summary (owner / ? / public / main) and exited 0. Capture the status and
bail on non-2xx, matching the sibling list commands.
Also assert the anonymous 404 label-list body does not leak the label name,
closing the last unread-body branch in the read-visibility gate test.
* test(node): drop #113-gated handlers from KNOWN_UNGATED; sync lockfile to 0.4.0
The repo-scoped egress guard's allowlist (added in #122, merged after this
branch was opened) still listed the read surfaces this branch gates. Now that
list_labels, list_repo_events, list_webhooks, list_replicas, and
list_protected_branches are read-visibility gated, the staleness assert
requires removing them from the allowlist. The #120 surfaces (certs, issues,
bounties, stars) stay listed; git_info_refs stays until #119 lands. Lockfile
versions follow the 0.4.0 release the rebase landed on.
* fix(gl): fail the MCP webhook_list tool on non-2xx node responses
The MCP webhook_list path deserialized the response body and returned it as a
successful tool result without checking the HTTP status, so a 401/403/404 JSON
error (missing identity, wrong owner, private or deleted repo) was reported as
success with the error body as the result. Check status().is_success() before
deserializing and surface the node's message as a tool error, matching the
`gl webhook list` path. Covered by a non-2xx test (RED without the check).
* fix(gl): read the protected_branches key in repo owner
gl repo owner parsed the protected-branch response as a bare array or a
"branches" key, but the node returns {"protected_branches":[...],"count":N},
so it always printed "Protected: (none)" even when branches were protected.
Extract the parse into parse_protected_branches matching the gl protect list
shape, and update the owner tests to mock the real schema instead of bare
arrays (which is why the bug went uncaught).
* test(gl): cover the deny/negative arms reconciled by the merge
The merge changed three gl branches that no existing test executed (they
were only reasoned): resolve_owner_did with no local identity, cmd_info on
an authenticated-but-denied 403, and get_maybe_signed with no keypair. Add
a test per branch asserting the must-not case (error/unsigned, never a
fabricated repo card). Verified load-bearing by mutation: disabling the
cmd_info status gate turns the 403 test red.
---------
Co-authored-by: t <t@t>
- steal_after: the reclaim is NOT a guarantee that only a leaked lease is
reclaimed. A waiter's timeout starts at acquire(), not at the FIFO head, so a
same-repo backlog whose cumulative wait exceeds steal_after can steal while an
earlier waiter still writes; correctness rests on the retained pg advisory
lock (which serializes the stealer at acquire_write), not on the bound.
- pinata_object_list_for_refs: recomputes from the tail-start rules snapshot,
not a fresh read at pin-worker time, so a rule tightened AFTER tail-start is
not reflected (matches the old retained-list behavior; reconciliation sweep is
the backstop). Removed the overstated 'rule tightened since the push is honored'.
- RepoWriteGuard::Drop: note that on runtime shutdown the detached unlock task
may be dropped before it polls, bounded by pool teardown / connection close.
The deterministic-fault terminal 500 was checked before the transient
truncated_by 503, unconditionally. When one repo is corrupt (deterministic)
while a DIFFERENT repo is transiently skipped in the same scan, the requested
CID may live in the transient repo -- a retryable 503 is correct, but the
handler returned a terminal 500 and a conformant client won't retry, hiding
retrievable content until the unrelated corrupt repo is repaired.
Gate the 500 on truncated_by.is_empty(): it fires only when a deterministic
fault is the SOLE reason nothing served; a co-occurring transient taint falls
through to the retryable 503. A pure deterministic fault still terminally 500s
an absent lookup by design. Adds a co-occurrence regression test.
Code review found a DoS the F3 lease introduced: git_receive_pack acquired the
global and per-source write permits before block-waiting on the per-repo lease,
so a second same-repo push that parked on the lease pinned a scarce global
write-pool slot (1 of 32) for up to steal_after while sending zero bytes. A few
hostile sources stacking same-repo pushes could hold every slot on idle
lease-waiters and shed 503 on every push to every other repo node-wide.
Move both permit acquisitions to after the lease acquire so a lease-blocked
waiter pins no write-pool slot, and keep a cheap non-holding availability peek
before the DB lookup so a push flood on a saturated pool still sheds 503 without
touching Postgres (the authoritative held permit is taken after the lease).
Adds a blocked-waiter-holds-no-permit regression test.
On client disconnect, RepoWriteGuard::Drop released the per-repo advisory
lock immediately (it holds no child/pgid), while KillGroupOnDrop's detached
reaper was still tearing down the git process group for ~4s. A second same-node
push could then acquire the repo and race the first's still-writing objects/
dir and Tigris upload.
Add an in-process per-repo write lease (block-and-wait, keyed like the advisory
lock) that SUPPLEMENTS the retained cluster-wide PG lock. The lease rides into
the disconnect reaper via a clone on the write-path AdmissionGuard (None on all
read paths) and a second clone held across guard.release() for the clean-path
Tigris upload, so it frees only after the group is reaped. A bounded-wait steal
reclaims a leaked lease (block-and-wait has no coalesce degradation). The lease
is always taken before the PG lock and released after, so the two serializers
cannot invert. Deterministic fake-git race test + inv22 gate row.
The detached Pinata task moved each push's full object-id list into the
closure and parked it on pin_semaphore.acquire_owned(); a tokio semaphore
bounds active holders, not the waiter queue, so a slow Pinata backend let N
queued pushes each retain an MB-scale OID list (unbounded memory/tasks).
Capture only the small ref-update tuples and re-derive the object set via the
existing reaped/deadline-bounded git helpers AFTER the pin permit is acquired,
so retained memory is O(ref tuples), not O(object lists). Per-ref effects are
unchanged (still one spawn per push, no coalescing or shedding, so no dropped
announcements). Adds an inv22 gate row forbidding a retained object list across
the acquire, plus re-derivation-equivalence and reaping tests.
list_all_repos and list_visibility_rules_for_repos ran as bare awaits while
holding the scarce IPFS walk permits, so a query blocked in Postgres pinned the
slots past GITLAWB_IPFS_REQUEST_BUDGET_SECS and shed later requests as 503s.
Wrap both in the remaining request-deadline budget; on timeout the RAII permits
drop and the handler sheds a retryable budget 503. Fails closed: a timeout on
the visibility-rules query denies the request rather than serving an unfiltered
listing.
Appends an inv22 gate row asserting both queries stay deadline-wrapped.
Probe existence via `git cat-file --batch-check` (structured `<oid> missing`
on exit 0) instead of matching English `cat-file -t` prose, so a git wording
change cannot flip a miss into an error or vice-versa. A bad-config/corrupt
repo on a readable store is now a terminal, non-retryable 500 (not a false 404,
and not a retryable 503 that would fan out cat-file retries); a transient
unreadable store stays 503. Error bodies are opaque (no raw git stderr).
Layers onto the existing object_store_readable + re-probe disambiguation.
Await pg_advisory_unlock while the pooled connection is still owned by
self; take it and set released only after the await resolves. A cancel
mid-unlock now leaves the Drop backstop armed (conn still Some), so the
session-level advisory lock is no longer leaked onto a pooled connection.
Adds a cancel-mid-unlock regression test (via a test-only pre-unlock gate)
and an append-only inv22 gate row asserting the take/released orderings.
EncryptInflight bounds the outstanding post-push encryption-task set to one per
repo, but its state.rs doc implied a global cap it does not deliver, and each pin
loop (ipfs_pin + pinata pin_new_objects) holds a full per-push object-id list
while walking it — so N distinct authenticated first-pushes could hold N MB-scale
lists at once, unbounded.
Add a global pin_semaphore (GITLAWB_MAX_CONCURRENT_PIN_TASKS, default 8) and gate
BOTH pin loops through it: they DEFER when the pool is full, never drop a pin.
Tighten the state.rs comment to state the bound is PER-REPO, with the cross-repo
residual (throttled by auth + rate limits, memory bounded by the pin permit)
called out honestly.
Also correct .env.example GITLAWB_DB_MAX_CONNECTIONS (20 -> 48): with the #174 F1
boot validation, 20 is below max_concurrent_git_pushes(32)+headroom(8) and would
fail startup — closing the INV-24 doc gap from the F1 change.
Tests: pin_new_objects_gated_defers_when_pin_pool_exhausted (RED without the
permit acquire); max_concurrent_pin_tasks range/default.
object_type_bounded treated any bare `fatal:` cat-file exit as absence. A packed
object whose pack/idx is transiently unreadable (permissions, or a mid-repack
race) emits `could not get object info` — byte-identical to a genuine miss — so a
present object could return Ok(None) and the /ipfs scan a definitive 404 instead
of the intended retryable 503. The stderr cannot separate the two (the CID is
pre-validated, so that string is also the only legitimate absence path), so the
disambiguation must be out of band.
On the collided fatal, probe object_store_readable (objects/ listable, every
pack/idx openable). Unreadable -> Err (taint -> 503). Readable -> re-probe once;
still absent -> Ok(None), now present -> the type. This narrows but does not close
the concurrent-repack TOCTOU (the check samples a different instant), stated
honestly rather than claimed deterministic.
Test: object_type_bounded_unreadable_pack_is_error_not_absence packs a real repo,
chmod 000 the pack, and asserts Err (root-guarded). RED with the readability check
disabled: the present blob returned Ok(None).
fail_closed_full_scan_objects ran phase 1 (replicable_blob_set_bounded) then
phase 2 (all_blob_oids) with a FRESH Instant::now() + timeout for phase 2, under
one held scan permit. A large-but-successful phase 1 plus a full phase 2 held the
permit ~2x the configured budget instead of the one whole-scan budget.
Compute one deadline before phase 1 and share it: phase 2 takes the shared
Instant directly, phase 1 takes the remaining-until-deadline. Total occupancy is
now ~1x. The completeness cost is honest: a repo whose phase 1 nears the budget
under-pins this push (fail closed) rather than over-holding, so size the budget so
both phases normally fit.
Test: full_scan_shares_one_deadline_across_both_phases drives a slow-but-
successful phase 1 that eats the budget; the shared deadline reaps phase 2 and the
scan fails closed (empty). RED before: phase 2's fresh budget completed and kept
the non-blob candidate.
The GET /ipfs/{cid} object-type probe and content read ran bare synchronous
Command::output() on the async worker, budget-checked only before they started.
A hung or corrupt object store therefore pinned a Tokio runtime worker and both
held IPFS admission permits indefinitely, so enough requests exhausted the route
despite GITLAWB_IPFS_REQUEST_BUDGET_SECS. The walk beside them already used
spawn_blocking + the reaped bounded runner; these two stages were left bare.
Extract run_bounded_git_raw (returns ExitStatus/stdout/stderr so callers can
classify exit codes; run_bounded_git stays a thin bail-on-nonzero wrapper for the
walk callers) and add store::object_type_bounded / read_object_content_bounded on
it, preserving object_type's Ok(None)-on-absence vs Err classification so the
serve path's 404-vs-503 semantics are unchanged. Run both /ipfs stages in
spawn_blocking under a deadline clamped to min(git timeout, remaining budget),
with the IPFS permits held across the awaited spawn_blocking.
Test: get_by_cid_hung_probe_is_reaped_and_sheds_503 hangs git on a FIFO alternates
with no feeder; the reaped probe sheds a 503 in bounded time (GREEN), while a
neutralized deadline blocks the handler forever (RED). The existing probe-error /
corrupt-repo tests still assert the absence-vs-error classification at the handler.
The post-receive replication tail (withheld/candidate/full-scan resolution, then
the encrypt+recovery spawn and the announce spawn) ran inline in the request
future and parked on git_encrypt_semaphore before the durable try_begin gate. A
client/proxy disconnect while parked dropped the future and silently lost this
push's pins, recovery copy, and announcements, with no reconciliation sweep to
recover them (state.rs documented this exact residual).
Move the whole tail into an independently owned tokio task and return the git
response immediately, so a disconnect can no longer drop the parked work. Each
push owns its own tail, including its own always-spawned announce, so the
announce is never brought under the per-repo encrypt coalescing (which would
drop a coalesced push's per-ref announcements). Update state.rs to retire the
disconnect-loses-work residual.
Test: receive_pack_landed_push_returns_without_parking_on_scan_pool asserts the
handler returns 200 while the scan pool is held (RED with the tail inline: the
request future parks and the timeout fires; GREEN detached). The burst
serialization test is retimed to poll for scan completion since the withheld
walk is now detached too.
acquire_write took the pg advisory lock, then awaited Tigris, then built the
guard. An outer tokio::time::timeout firing during the Tigris await dropped the
future with the lock held and no guard, so no unlock ran and later pushes to the
repo spun 60s. release() also unlocked on an arbitrary pooled connection, not the
one that locked (session advisory locks are connection-affine), so even the
normal release was unreliable.
Pin a PoolConnection in RepoWriteGuard, build the guard before the lock query,
run the lock loop through it, unlock on that same connection in release(), and
add a Drop backstop that spawns a detached affine unlock (Handle::try_current so
an off-runtime drop logs instead of panicking). Two distinct-session tests
(drop-without-release, affine release) go RED without the fix, GREEN with it.
Pinning a connection per active write is a DB-pool DoS unless the pool clears
the concurrent-write cap: the shipped default (20) was below
max_concurrent_git_pushes (32). Raise the db_max_connections default to 48 and
add Config::validate() to reject db_max_connections < max_concurrent_git_pushes
+ 8 headroom at boot; wire it into startup.
The budget-knob docs claimed the overshoot was bounded by one unclamped
probe; an unclamped subprocess is not a bound, so the README,
.env.example, config help, and module comments now say plainly that a
hung git probe holds the walk slot for the hang's duration. New
regression: an acquire-tainting ghost row ahead of a healthy public copy
must not stop the scan; the later found short-circuit still serves 200.
object_type now distinguishes repo-level git failures (not-a-repository,
error:-prefixed corruption) from genuine object absence, so a corrupt
repo taints the /ipfs scan into the retryable 503 instead of vouching
for a definitive 404; the drain reads visibility rules by the freshly
fetched record id, closing a latent fail-open on id divergence. The four
budget gates collapse into one stage-labelled helper, the three scan
sites share state::acquire_scan_permit (inv22 tripwire repointed and
sever-verified), and the Tigris stall tests use a local accept-and-park
listener instead of a non-routable address.
try_begin now merges a losing push's tip pairs into the in-flight key's
pending slot in the same critical section as the presence check, bounded
at 1024 pairs with a full-scan marker on overflow (force_full_scan
threads to the resolver; empty tips never encode the marker). The task
loop-drains: finish_or_take_pending either hands back the batch with the
key retained, or removes the key and disarms the guard atomically, so a
successor task's key survives the guard drop. Drains re-fetch rules
fresh and replay the tail pipeline under the U4 helper gates only; the
loop holds no task-level pool permit, so a hot repo cannot deadlock the
pool at size 1.
replication_withheld_set's walk arm, resolve_candidates_for_push's git
stages, and fail_closed_full_scan_objects each take a
git_encrypt_semaphore permit before their spawn_blocking and carry it
inside the closure, so a push burst holds at most pool-many concurrent
scans and a started scan finishes with its permit. Deletion-only pushes
and repos without path-scoped rules never park. Contention defers the
landed push's tail (logged with queue_wait_ms), never errors it; the
acquire sites document the accepted disconnect-during-park residual.
GITLAWB_IPFS_REQUEST_BUDGET_SECS (default 600) is captured as one
deadline at handler entry: every acquire and expensive walk is clamped
to the remaining budget, every stage checks remaining before starting,
and exhaustion taints the scan into the retryable 503. A walk in flight
is never aborted at the tokio layer (that would free the walk permit
while the blocking thread still runs); its clamped git deadline plus
group teardown ends it. Docs state the residual overshoot: the in-flight
stage's kill/reap slack plus the unclamped probe subprocesses.
A request-scoped taint list replaces the silent folds: any repo skipped
without a verdict (acquire timeout, probe or read failure, walk failure,
cap or ceiling exhaustion) turns the terminal into a retryable 503 with
Retry-After, and 404 is reserved for a scan in which every candidate
reached a verdict. The walk cap now counts only expensive allowed-set
walks and skip-continues on exhaustion so a plain public copy past the
cap still serves; a new GITLAWB_IPFS_MAX_REPO_VISITS knob (default 1024)
bounds the acquire+probe cost class and stops the scan, documented with
its worst-case object-store fetch count.
drive_git_child hands the AdmissionGuard back on success so
upload_pack_excluding can thread it across the rev-list and pack-objects
stages; on timeout or error it is dropped post-reap, and on future-drop
it rides the detached reaper as before. The upload-pack handler builds
the guard once ahead of the withheld/plain branch, so a client
disconnect mid-filtered-serve no longer frees the read-pool and
per-source permits while the git process group is still being reaped.
P2-1: git_info_refs treated every service other than git-receive-pack as a read op
and did the DB/visibility/Tigris work under a read permit, validating the service
string only downstream in smart_http. So an unauthenticated `?service=anything` to
a public repo consumed a read slot + storage work before rejection. Validate the
service is exactly git-upload-pack or git-receive-pack immediately after parsing,
returning 400 before the pre-DB shed and the permit. Handler-layer regression:
with the read pool exhausted, `?service=git-explode` returns 400 (validated first),
not 503 — removing the validation makes it 503 (RED).
U5/INV-24 docs: document the new knobs from U2/U3 in .env.example and the README
env-var table — GITLAWB_GIT_ACQUIRE_TIMEOUT_SECS, GITLAWB_MAX_CONCURRENT_IPFS_WALKS,
GITLAWB_IPFS_WALK_PER_SOURCE, GITLAWB_IPFS_MAX_REPOS_WALKED, GITLAWB_IPFS_RATE_LIMIT.
Each is already enforced on the path it names (U2/U3 tests), so the docs match the
implemented routing.
After a successful path-scoped push the handler spawns a DETACHED task that runs
pin_new_objects then withheld_recipients_gated, which acquire_owned().await's on
git_encrypt_semaphore — it DEFERS (blocks) when the pool is full. The semaphore
caps active walks but nothing capped how many detached tasks were spawned and
PARKED on that await: N rapid path-scoped pushes spawned N tasks, each holding
cloned object lists/rules/paths/keys — an unbounded parked-waiter set (P2-2).
2a54c15 deliberately chose defer-not-shed because dropping the walk loses the
withheld-blob recovery copy and there is no reconciliation sweep to rebuild it.
So bound the OUTSTANDING-TASK set by per-repo coalescing rather than shedding:
EncryptInflight (a bounded Arc<Mutex<HashSet<repo_id>>>) + try_begin — before the
spawn, if a task for the repo is already in-flight skip the duplicate (the pending
walk covers the newer objects); otherwise spawn and move an RAII
EncryptInflightGuard into the task that removes the key on drop (completion, error,
or panic-unwind). This bounds the set to <=1 pending task per repo and never drops
work; withheld_recipients_gated's defer is unchanged.
The second detached post-push spawn (Pinata replication, also runs pin_new_objects)
is scoped out with rationale: it parks on no semaphore (no unbounded waiter set)
and does per-push per-ref work (branch->CID, gossip, subscription, Arweave,
peer-notify) keyed to this push's ref_updates — coalescing it would DROP a later
push's announcements, a correctness regression.
Regressions (mutation-verified RED->GREEN): the outstanding set is bounded to 1
per repo under saturation (never-coalesce -> 32 tasks RED); a coalesced repo's key
is released when its task ends so it is reprocessed, never permanently skipped
(guard-drop no-op -> repo locked out, recovery copy lost forever, RED — the exact
durability regression the fix prevents); distinct repos each admit; cold-set first
push admits; plus an inv22_gates structural tripwire that fails if the handler gate
is removed.
GET /ipfs/{cid} (get_by_cid, publicly reachable via optional_signature) ran
allowed_blob_set_for_caller_bounded in spawn_blocking inside a per-repo loop with
NO global/per-source admission and NO route rate limit — the surface had zero
concurrency or rate control. A permissionless caller could fan out concurrent
full-history git walks (each up to the walk timeout) exhausting blocking-pool
threads and PIDs outside every served-git pool (P1-3).
Acquire a global git_ipfs_walk permit (try_acquire_owned, shed 503) plus a
per-source sub-cap (with_default_max_keys, reject-before-insert, keyed via
client_key/TrustedProxy so XFF can't spoof it) ONCE after CID validation, held as
handler locals across the whole request — including every spawn_blocking walk, so
the slot reflects real blocking-thread occupancy (a tokio timeout cannot cancel
spawn_blocking). None key -> global pool only. Cap repos walked per request so one
request can't serialize N full-history walks. Attach rate_limit_by_ip +
IpRateLimiter to the /ipfs route (mirrors the git/create/peer routers; the
extension is required or the middleware is a silent no-op). Visibility/authz
semantics unchanged.
New knobs: GITLAWB_MAX_CONCURRENT_IPFS_WALKS (32), GITLAWB_IPFS_WALK_PER_SOURCE
(4), GITLAWB_IPFS_MAX_REPOS_WALKED (64), GITLAWB_IPFS_RATE_LIMIT (600/hr, 0
disables) — all range-validated and enforced on-path.
Handler-layer regressions (all mutation-verified RED->GREEN): shed-at-capacity
503 (delete acquire -> falls through), per-source cap sheds same source / admits
another, None-key arm sheds on the global pool, map self-bounds reject-before-insert,
the walk permit is held while the spawn_blocking walk runs (available_permits()==0;
dropping it before the loop -> RED), repos-walked cap (remove break -> 2 walks),
and the route IP rate limit actually fires 429 (drop the Extension -> RED no-op).
The served-git admission permit is acquired, then RepoStore::acquire*/advisory-lock
runs before the bounded git runner starts — with no local deadline. acquire/acquire_fresh
await Tigris HEAD/GET, and acquire_write's advisory-lock loop awaits a per-iteration
pg_try_advisory_lock that can block indefinitely on a hung Postgres pool. Since
GITLAWB_GIT_SERVICE_TIMEOUT_SECS only starts once git spawns, a stalled backend pins
the permit and drains the pool until every later request 503s (P1-2).
Wrap each git-path acquire (git_info_refs read/advert, git_upload_pack, git_receive_pack)
and the /ipfs per-repo acquire loop in tokio::time::timeout(git_acquire_timeout_secs, ..).
On expiry the git paths return AppError::Overloaded (503 + Retry-After); the permit is a
handler-local at that point (moved into the AdmissionGuard only after acquire) so the early
return frees the slot. The /ipfs per-repo acquire keeps its fail-closed `continue` on
timeout (never serves an un-acquired repo). No repo_store.rs signature change: the outer
timeout cancels a mid-sleep/mid-fetch_one future, so no in-loop deadline is needed.
New knob GITLAWB_GIT_ACQUIRE_TIMEOUT_SECS (default 30, range 1..), modeled on
git_service_timeout_secs, kept separate because acquisition and git execution are distinct
cost centers.
Handler-layer regression receive_pack_acquire_deadline_sheds_and_releases_permit: holds the
same pg advisory lock acquire_write derives on a second connection so the real loop must
retry, drives git-receive-pack with the deadline at 2s and write pool size 1, and asserts
503 at ~2s + the permit recovers + a follow-up admits once the lock frees. Removing the
timeout wrapper turns it RED (blocks ~31s to the test ceiling on the ~59s advisory loop).
On the plain (non-path-scoped) info_refs / upload-pack / receive-pack paths the
global + per-source admission permits were handler-locals that dropped the
instant the handler future dropped on a client disconnect, while KillGroupOnDrop
reaps the git process group in a DETACHED task (SIGTERM -> grace -> SIGKILL ->
reap). So a disconnect-spammer admitted replacements while prior groups were
still alive, defeating the process cap and the per-source cap during teardown.
be0cdd6 already fixed this for the path-scoped upload-pack walk; this closes the
residual plain paths (P1-a).
Introduce AdmissionGuard, a move-only holder of the permits, threaded through
info_refs/upload_pack/receive_pack/run_git_service into drive_git_child's
KillGroupOnDrop. On disconnect the guard moves into the detached reaper and drops
only after the group is ESRCH-confirmed reaped; on success/timeout disarm()
returns it for the earliest provably-free drop. The rev-list/pack-objects
visibility-walk callers hold no admission and pass None.
Thread git_bin through upload_pack/receive_pack (they hardcoded "git") so the
plain POST path is drivable by an injected fake git, matching the existing
injectable seam on info_refs/run_git_service/upload_pack_excluding.
Handler-layer regression upload_pack_plain_permit_held_through_group_reap_after_disconnect:
isolates the global read pool (size 1, per-source + rate limiter permissive), a
SIGTERM-ignoring descendant keeps the group alive across the reap window, and
asserts available_permits()==0 held on disconnect + a cross-source replacement
sheds 503, then frees after ESRCH. Reverting the guard-into-reaper move (release
immediately) turns it RED at the held-through-reap assertion. Plus a None-key arm.
A source-scan tripwire that fails when a new site reintroduces the concurrency-cap
class PR #174 fixed, or when one of the five gates (U1-U5) is removed. INV-22: a
permit held per op recovers only if every path is duration-bounded and reaps the
group before releasing admission, and every detached git task carries admission.
Lives in tests/ (a separate crate) on purpose — a guard scanning the file it lives
in would match its own identifier literals and pass vacuously; scanning src/ from
here keeps every check load-bearing. Checks: run_bounded_git confirms child exit via
child_terminated_without_reaping (U1); KillGroupOnDrop launches the reaper via
Handle::try_current (U2); git_receive_pack acquires state.git_write_per_caller (U4);
the encryption walk runs through withheld_recipients_gated over git_encrypt_semaphore
(U5); and the bounded recipients walk is invoked exactly once — only inside the gated
helper — so a new spawn_blocking bypass (count > 1) fails (P1-e non-bypass).
Proven load-bearing: a synthetic second call site turns the count tripwire RED, and
each named gate maps to a fix whose own regression already goes RED on revert.
After a successful path-scoped push the handler released its write permit and
then ran a DETACHED tokio task whose spawn_blocking(withheld_blob_recipients_bounded)
performed another full-history git walk under no admission. N fast completed
pushes spawned N concurrent full-history walks past GITLAWB_MAX_CONCURRENT_GIT_PUSHES
(which bounds only the in-handler phase), and the per-child bounding from 71389e6
caps each walk's duration but not their count (P1-e).
Add git_encrypt_semaphore, a pool of its own (sized from max_concurrent_git_pushes,
no new knob — Q1) so a long background walk never holds a foreground write slot and
a handler holding a write permit can't self-deadlock. Route the walk through a new
withheld_recipients_gated helper that acquires the pool before the walk and DEFERS
(blocks) when full rather than shedding — dropping the walk would lose the
withheld-blob recovery copy, so durability stays fail-closed.
Regression: encrypt_walk_defers_when_pool_exhausted drives the gated helper with the
pool exhausted and asserts the walk blocks and does not run its rev-list, then runs
once a permit frees. RED before (the walk runs regardless of the pool), GREEN after.
The detached push task calls this exact helper.
The authenticated git-receive-pack POST acquired only the global write semaphore
— no per-source sub-cap, unlike the read and anon-advert pools. Owner enforcement
defaults off, so one host minting disposable did:key identities could open
max_concurrent_git_pushes slow POSTs from a single source IP and 503 every other
source's push; the 600/hour push limiter bounds arrival rate, not in-flight
concurrency (P1-d).
Add git_write_per_caller (a PerCallerConcurrency sized like the advert cap,
max_concurrent_git_pushes/8), and acquire it before the global write permit,
keyed on the resolved source IP via read_caller_key — never the signed DID (a DID
farm defeats a DID key). This required adding the PeerAddr + HeaderMap extractors
to git_receive_pack, which it lacked (without them the key is None and the cap is
inert). Neutralized the shared acquire helper's log wording now that it serves
both read and write paths.
Regression: receive_pack_per_source_write_cap_sheds_capped_source_not_others — a
source at its write sub-cap sheds Overloaded/503 (proving the extractors resolve a
key), while a different source is not shed. RED before (the capped source proceeds
past admission to a git error instead of shedding), GREEN after; the existing
receive-pack advert cap test still passes.
On the path-scoped upload-pack path the per-source and global read permits were
handler locals, so a client disconnect dropped the handler future and released
both permits while the awaited spawn_blocking withheld-blob walk kept running
(spawn_blocking can't be cancelled). A permitless caller could disconnect-spam to
exceed the read cap while real git work continued (P1-b).
Move both permits into the blocking task and return them out: on success they
flow back so the serve phase keeps them; on a dropped future the returned tuple
(with the permits) is discarded only when the blocking task completes, so
admission tracks the walk's real duration.
Regression: upload_pack_permit_held_through_walk_after_disconnect drives the
handler to mid-walk (fake git hangs on rev-list), disconnects, and asserts the
global read slot stays held (available_permits == 0). RED before (the slot frees
to 1 the instant the future drops), GREEN after. The test roots the repo store at
its own TempDir so it is isolated per run.
On a client disconnect KillGroupOnDrop sent a lone SIGTERM with no escalation or
reap, so a group member that ignores SIGTERM (a wedged pack-objects, a malicious
helper) survived while the handler released its concurrency permit — letting
disconnect-spam accumulate live git processes and defeat the load shed (P1-c).
The timeout path already ran the full teardown.
Make the disconnect teardown as strong as the timeout path: the guard now owns
the child and, on drop, launches a detached reaper that runs the same
TERM -> grace -> SIGKILL -> reap-the-group sequence (reap_group_on_timeout).
Owning the tokio Child in the reaper keeps it the sole reaper of the leader, so
it never races tokio's own orphan reaper (a raw waitpid(-pgid) would).
Regression: run_git_service_sigkills_a_sigterm_ignoring_child_on_disconnect, RED
before (the descendant survives the lone SIGTERM), GREEN after; all existing
teardown/timeout/disarm tests still pass (the three hand-built-guard tests were
updated to the owned-child shape).
Residual: capacity (the handler's semaphore permit) still releases at disconnect
rather than strictly after the reaper confirms the group dead, so accumulation is
now bounded to ~one teardown window (<=4s) rather than eliminated; gating the
permit on the reap needs threading it through the serve handlers and is left for
a follow-up (it also touches the receive-pack permit lifetime U4 changes).
A group member (or the leader itself) that closes stdout before the deadline made
the stdout drain return EOF early, which stood the watchdog down via done_tx
before it could ever fire; child.wait() then blocked on the still-live child,
pinning the walk thread and its read/write permit past
GITLAWB_GIT_SERVICE_TIMEOUT_SECS. This is distinct from the descendant case
already covered: there the leader sleeps until the deadline so the watchdog does
time out; here the drain-EOF races ahead of it.
Stand the watchdog down only once the child has actually terminated, detected
without reaping (waitid + WNOWAIT) so the leader's pid stays unreaped and its
pgid un-recycled until the watchdog finishes its teardown and is joined. Past the
deadline the watchdog still owns the full SIGTERM -> grace -> SIGKILL sequence.
Regression: run_bounded_git_reaps_a_leader_that_closes_stdout_then_hangs, RED
before (blocks on child.wait past the recv budget), GREEN after; the three
existing teardown tests still pass (no recycle-hazard regression).
build_filtered_pack_times_out_a_hung_rev_list and _a_hung_pack_objects
spawn a freshly-written fake git directly, bypassing the fake_git_run_with_pids
retry the rest of the module uses. Under cargo test fork-storm load a concurrent
worker forks while the fake's write fd is still open, so execve returns ETXTBSY
and the raw io::Error surfaces where the test asserts GitServiceTimeout, failing
test (stable) intermittently.
Route both tests through build_filtered_pack_or_exec_race_retry, which retries
ONLY errno 26 (ETXTBSY) and keeps the outer 10s watchdog per attempt, so a
genuinely-missing async timeout bound still trips the watchdog and a wrong error
type still fails at the assertion. Production paths unchanged.
The finding-2 wiring test covered only list_all_objects (the full-scan fallback).
Add delta_path_exec_fns_time_out_on_a_hung_git covering the common push path:
peeled_object_type (cat-file), rev_list_delta (rev-list), and list_all_objects_with_type
(cat-file, which all_blob_oids delegates to). Each must return within the watchdog
budget on a hung git; reverting any to a bare Command::output() blocks past the recv
budget (RED-verified).
jatmn P2 (INV-24): the config help, .env.example, and README described routing the
handler no longer does.
- The read pool serves upload-pack and the UPLOAD-PACK info/refs advertisement only;
the anon receive-pack info/refs advertisement draws from its own dedicated pool
(sized like the write pool but disjoint), not the read pool and not the write pool.
- The write pool is the authenticated push POST only.
- The per-caller read cap is keyed on the resolved source IP, never the DID: a
signature does not move a caller off it (the docs claimed a per-DID budget, which
would tell a NATed authenticated client it escapes the shared cap when it does not).
- git_service_timeout_secs now also bounds the push-side pin-candidate discovery
(rev-list / cat-file), reaped via process-group teardown, alongside the info/refs
advertisements and the withheld-blob walk.
Route push_delta's candidate-discovery git children (cat-file / rev-list) through the
bounded run_bounded_git (deadline + process-group SIGKILL teardown) instead of bare
Command::output(). A hung scan no longer pins the git-receive-pack write permit past
the deadline or a client disconnect. Threads git_bin + a shared per-scan deadline
(git_service_timeout_secs) through resolve_candidates_for_push and all_blob_oids.
jatmn/CodeRabbit P1: the walk watchdog stood down as soon as the group LEADER was
reaped. A leader that exits cleanly on SIGTERM while a background member ignores
SIGTERM and closes its inherited pipes let the main drain unblock, the leader be
reaped, and the watchdog skip its SIGKILL escalation — so the handler returned while
the descendant kept running. jatmn's suggested "continue until ESRCH" could not work:
the main thread reaping the leader is what frees the pgid, so continuing to kill(-pgid)
after that races a recycled group.
Restructure the coordination instead: the main thread now defers reaping the leader
until the watchdog thread returns. The watchdog always escalates through an
unconditional group SIGKILL (no leader-reap short-circuit), and because the leader is
still unreaped while every kill(-pgid) fires, the pgid cannot have been recycled. The
grace before SIGKILL is a fixed budget (only paid on the exceptional timeout path).
Regression run_bounded_git_sigkills_a_sigterm_ignoring_descendant_after_leader_exits:
a leader that exits 0 on SIGTERM spawns an sh -c descendant (its own $$, not a ( )
subshell's parent pid) that ignores SIGTERM and closes its pipes; the descendant must
be dead when the runner returns. RED on the old teardown (survives), GREEN now. The
existing leader-ignores-SIGTERM and hung-walk teardown tests still pass.
jatmn P1: run_bounded_git was gated #[cfg(unix)], but its callers
(assert_all_refs_are_commits, blob_paths) and the module are compiled on all
targets, so a non-Unix build (the continue-on-error Windows release target) could
not resolve the function.
Add a #[cfg(not(unix))] twin with the same signature and result semantics. It bounds
a single child (no process-group teardown, which is Unix-only): threads feed stdin
and drain stderr, the main thread drains stdout, and a watchdog thread kills the
child at the deadline. The child is shared with the watchdog behind a mutex the main
thread does not hold while draining, so the watchdog can always acquire it to kill,
and killing closes stdout to unblock the drain. Best-effort (reaps only the direct
child), which is why the hardened group-aware path stays Unix-only.
Verified the twin type-checks by compiling its (portable-std) body on the native
target; the full non-Unix link is CI's Windows job.
jatmn P1: the anon-reachable GET info/refs?service=git-receive-pack drew from the
same git_write_semaphore as the authenticated git-receive-pack POST. The per-source
advert cap reserves no global slots, so ~8 sources (each within the per-IP push rate
limit) could hold all 32 write permits across acquire_fresh + the advertisement, and
the next authenticated push then 503s at admission. That is an anonymous caller
shedding an authenticated push across the auth boundary (INV-10).
Add a dedicated git_push_advert_semaphore, disjoint from the write pool. Both the
pre-DB early-shed peek and the held acquire for the receive-pack advertisement now
draw from it; the write pool is left exclusively for the POST. An advert flood can at
worst exhaust the advert pool (each source still capped by git_push_advert_per_caller
and the per-IP push rate limiter); the authenticated push pool is untouched.
Test receive_pack_advertisement_draws_from_dedicated_advert_pool proves at the handler
layer that the advert sheds when its own pool is saturated AND survives when the write
pool is saturated (the reservation). Both flip under revert, verified independently for
the pre-DB peek and the held acquire.
CodeRabbit review: the held git_permit was acquired at the top of git_info_refs
and git_upload_pack, before the per-source cap, so one source could occupy the
global pool during the DB/visibility window before its sub-cap rejected the
excess. Move the held git_permit to after acquire_read_caller_permit (still
before acquire_fresh/git, so INV-10's bound on the fresh Tigris acquire holds),
and add a cheap pre-DB early shed (available_permits() == 0 -> 503, holds no
permit) so the #62 shed-before-DB property is preserved without holding a permit
across the DB work.
The receive-pack POST is unchanged (owner-only; its top-of-handler git_permit is
deliberate for acquire_write bounding).
Tests: the two #62 shed-before-DB tests now exercise the explicit early check
(load-bearing: disable it and they fall through to the DB); three new
*_per_source_cap_sheds_with_global_capacity tests prove the per-source sub-cap
sheds a capped source even with global capacity free (load-bearing on the caller
cap). Full workspace green.
Follow-up to threading the configured timeout into the walk: the walk now honors
GITLAWB_GIT_SERVICE_TIMEOUT_SECS on both the serve and replication paths, so the
README no longer says a fixed internal deadline.
Close the reasoned-not-run gaps from the code review by making the walk's git
binary and timeout injectable, then driving the missing branches with a real
handler and a fake git instead of reasoning about them.
- Add state.git_bin and *_bounded variants of the walk entry points taking
(git_bin, timeout); the served handlers (upload-pack serve, receive-pack
replication and full-scan and encrypt-pin, and the ipfs gate) now pass the
operator-configured GITLAWB_GIT_SERVICE_TIMEOUT_SECS, so the whole walk is bounded
by the same budget as the other served-git ops rather than a fixed constant. The
git_bin-less wrappers stay for the real-git classification tests.
Newly vetted by execution (not reasoning):
- receive-pack replication path is bounded: replication_withheld_set with an injected
hung git returns within the budget and fails closed, so it cannot pin the write
permit git_receive_pack holds across it.
- a hung withheld-blob walk on the upload-pack POST returns 504 (real handler, real
repo on disk, injected hung git), proving the GitServiceTimeout -> git_service_app_error
wiring end to end.
- the watchdog status-gate: a child that exits successfully is not reported as a
timeout even when the watchdog fired (mutation-checked: drop the guard -> RED).
- SIGKILL escalation: a SIGTERM-ignoring child is still reaped via SIGKILL and the
group is gone; a truly uninterruptible D-state child (unreapable by any signal) is
the documented residual, matching the async teardown.
- the advertisement per-source cap sizing never derives 0.
Full gitlawb-node suite 515 green.
U2 moved the receive-pack info/refs advertisement onto git_write_semaphore to keep
an anonymous read flood from starving the push handshake. But the advertisement is
anon-reachable on public repos and holds its write permit across the slow
acquire_fresh Tigris download, and the only per-source brake on it was the push
RATE limiter, not a concurrency cap. So a multi-source flood of receive-pack
advertisements could hold the write pool's slots across those downloads and shed
authenticated pushes (both the advertisement and the owner-gated git-receive-pack
POST draw from the same pool). U2 thus introduced the first anonymous consumer of
the write pool the state doc promised anon could never reach; the plan's residual
note (no worse than the POST) was wrong, because the POST is owner-gated and the
advertisement is not.
Add git_push_advert_per_caller, a per-source concurrency sub-cap on the receive-pack
advertisement keyed on the resolved source IP (the same PerCallerConcurrency
mechanism U1 uses for reads), sized to an eighth of the write pool so a single
source holds at most that share and saturating the pool takes many distinct source
IPs, each also braked by the per-IP push rate limiter. The upload-pack advertisement
keeps its read-pool per-caller cap; the owner-gated POST is unchanged. Correct the
state doc for git_write_semaphore accordingly.
Handler-layer proof: a source at its receive-pack advertisement cap sheds 503 (RED
before the acquisition, 500-not-503), while a different source and the upload-pack
advertisement are unaffected. Full suite 510 green.
Code review found run_bounded_git's watchdog could return a spurious 504 and
signal a recycled process group. The watchdog runs off a wall clock on its own
thread; done_tx.send() only fires after child.wait() reaps the leader, so a walk
that finished within microseconds of the deadline took the watchdog's Timeout
branch, discarded a fully-captured successful result, and returned GitServiceTimeout
(a 504 for a walk that actually completed). Worse, the Timeout branch SIGTERMed
-pgid unconditionally after the leader was reaped, so a recycled pgid could be
signalled, the exact hazard smart_http guards via disarm-after-wait.
Set a reaped AtomicBool the instant the main thread reaps the child; the watchdog
checks it before every kill and stands down if the leader is already reaped. Gate
the timeout verdict on !status.success(), so a child that exited on its own is never
reported as a timeout even if the watchdog fired late. Add the survived-SIGKILL warn
smart_http's reap already emits, for operator visibility on a wedged (D-state) git.
The hung-walk test stays green (a killed child exits by signal, not success, so it
still surfaces GitServiceTimeout and reaps the group) and the 32 real-git
classification tests stay green (a fast walk is never spuriously killed).
GITLAWB_GIT_SERVICE_TIMEOUT_SECS bounds the info/refs advertisement too:
smart_http::info_refs drives it through drive_git_child under this timeout, with a
passing test proving the 504. The old note claimed it does not. It also claimed the
withheld-blob path is unbounded; after the blob_paths seam bound (this PR) the walk
is bounded and reaped, by a fixed internal deadline rather than this env var, so the
line now states that precisely instead of overclaiming this setting covers it.
The withheld-blob classification walk (blob_paths) fanned out blocking git children
with no deadline and no process-group teardown: git for-each-ref, git cat-file, git
rev-list, a git ls-tree per commit, and an uncounted git rev-parse (via
store::head_commit). A hung or pathologically slow child pinned the caller's
served-git permit for the whole hang, and on client disconnect the spawn_blocking
task and its git children ran on, orphaned. blob_paths is the shared core of five
callers: the upload-pack serve path (holds a read permit) AND, inside
git_receive_pack, the post-push replication and encrypt-then-pin walks (hold the
write permit U2 reserves for pushes). So the same unbounded walk could pin either
pool, and leaving the write-side twin unbounded would have made U2's reservation a
claim that does not match behavior.
Bound every git child at the blob_paths spawn seam on the blocking side: each child
runs in its own process group with a watchdog thread that SIGTERMs (then SIGKILLs)
the group on one shared deadline spanning the whole walk, and retains admission
until the group is reaped. This is the blocking-side counterpart of
smart_http::drive_git_child (spawn_blocking cannot be cancelled by an async
timeout). blob_paths stays sync, so all five callers keep their signatures and the
32 classification tests are unchanged; because every caller funnels through
blob_paths, one seam bounds both the serve and replication paths. The previously
unbounded store::head_commit child becomes a bounded git rev-parse inside the walk.
A walk that hits its deadline carries GitServiceTimeout, which the serve handler now
maps to 504 rather than a generic 500.
Proof: a fake git that hangs on rev-list makes blob_paths return GitServiceTimeout
within the watchdog budget (not block on the child) and the recorded process-group
leader is reaped, not orphaned; neutralizing the watchdog kill makes it hang past
the budget (RED). The 32 real-git classification tests stay green through the
refactor, including detached-HEAD, non-standard-ref, and deleted-in-history cases.
git_info_refs acquired git_read_semaphore for BOTH services, so the push handshake
(GET /info/refs?service=git-receive-pack) competed in the global read pool. An
anonymous clone flood could exhaust that pool and shed a legitimate push with 503
during its required advertisement phase, before it ever reached git_write_semaphore
on the POST. The write pool exists precisely so anonymous reads cannot shed an
authenticated push, but only the POST drew from it.
Select the pool by service: the receive-pack advertisement (phase one of a push)
now draws from git_write_semaphore, like the git-receive-pack POST, so a saturated
read pool cannot starve it. The per-IP push_rate_limiter that already brakes the
advertisement stays as the anti-flood control, and the advertisement stays
reader-visible with no new auth requirement. Because the receive-pack branch is now
a write-path op, it no longer consumes a read per-caller slot.
Handler-layer proofs: with the read pool at zero the receive-pack advertisement
survives while the upload-pack advertisement sheds; with the write pool at zero the
receive-pack advertisement sheds while upload-pack is unaffected; and a receive-pack
advertisement from an IP whose read per-caller budget is full still gets through
(mutation-checked, RED when the skip is neutralized).
read_caller_key returned the authenticated DID when a caller signed, dropping the
source-IP key. Public read routes accept any valid did:key via optional_signature
with no admission step, so one host could mint N disposable DIDs and hold
max_concurrent_reads_per_caller slots under each, multiplying its budget N-fold and
filling the global read pool, while the same host unsigned was capped on its IP.
Key the read sub-cap on the resolved source IP for every caller, signed or not,
mirroring the push path's IpRateLimiter which already throttles on source IP for
this exact DID-farm reason. Drops the now-unused caller_did parameter at both call
sites (git_info_refs, git_upload_pack).
Inverts info_refs_per_caller_cap_keys_on_did_not_ip into
info_refs_per_caller_cap_keys_on_ip_not_did: fill one source IP's slot, then two
requests signed under different DIDs from that same IP both shed 503 (farm
defeated), while a signed request from a different IP keeps its own budget. RED on
the DID-keyed tree, GREEN after.
The read-pool knob was referenced by the push and per-caller entries'
comments but had no example line of its own, so operators couldn't
discover it from the template. Add it with the config default (128).
Review of the served-git concurrency cap found no P0/P1; these are the
verified P2 follow-ups.
config: the max_concurrent_git_ops doc overclaimed that "every capped path
is duration-bounded." The rev-list object enumeration in the withheld-blob
path still runs in an uncancellable spawn_blocking, so a stuck rev-list can
hold its slot until git exits. Scope the guarantee to the streaming stages
and name the residual. Also tighten the fairness claim: the receive-pack
advertisement shares the read pool (a shed advertisement is a cheap retryable
GET); only the push POST is on the isolated write pool.
api/repos: add info_refs_per_caller_cap_keys_on_did_not_ip, the missing
handler proof that a signed caller is keyed by its DID, not its source IP.
Filling the DID slot sheds a request from a free IP; collapsing read_caller_key
to its IP arm turns the assertion green-not-503 (mutation-verified RED).
api/repos: extract acquire_read_caller_permit so both read handlers share one
shed path instead of a duplicated match block.
rate_limit: recover from a poisoned PerCallerConcurrency mutex instead of
panicking. The critical section is pure counter arithmetic and cannot poison
the lock, but a panic there would brick the limiter for every caller.
505 tests pass; clippy -D warnings and fmt clean.
Part of #174.
The max_concurrent_git_ops and git_service_timeout_secs doc-comments (and .env.example) described the info/refs and withheld-blob paths as unbounded follow-up gaps. Both are now closed (#174): the comments reflect the read/write pool split, the per-caller sub-cap, and that every capped path is duration-bounded with process-group teardown. Verified the pattern-doc pre-ship checklist: every git_permit / write-permit / per-caller site holds only a timeout+teardown git path, and all three size knobs are range(1..=1_048_576).
Closes the #174 work. No behavior change.
The filtered-pack path ran the whole rev-list + pack-objects build inside a spawn_blocking, so an outer tokio timeout could not cancel the blocking thread and a client disconnect orphaned the git child while the permit freed (jatmn P1b, the second gap path). Split it: rev-list enumeration stays blocking off the runtime (rev_list_keep), but the streaming pack-objects stage now runs under the shared drive_git_child on the async side, so it is duration-bounded (GitServiceTimeout -> 504) and its process group is reaped on disconnect. build_filtered_pack becomes async and takes a git_bin seam + timeout; upload_pack_excluding threads the git_service_timeout through.
A watchdog-bounded test proves a hung pack-objects times out (rev-list fast, pack-objects hangs). The refactor's happy path is covered by the existing filtered-pack correctness and real-git partial-clone/fetch tests, all still green; disconnect/group-teardown is the shared drive_git_child code proven by the run_git_service tests. 504 tests pass.
Part of #174.
info_refs ran a bare Command::output() with no timeout and no process-group teardown, so a hung git pinned its concurrency slot indefinitely and a client disconnect orphaned the child (jatmn P1b). Extract the timeout + process_group(0) + KillGroupOnDrop core from run_git_service into a shared drive_git_child, and route info_refs through it with an injectable git_bin. A hung advertisement now aborts with GitServiceTimeout (mapped to 504); disconnect reaps the group.
run_git_service's teardown tests all pass through the shared core (proving the group teardown info_refs inherits), the real-git filter tests cover the advertisement happy path, and a new watchdog-bounded test proves a hung advertisement times out. 503 tests pass.
Part of #174.
Adds PerCallerConcurrency, a bounded-keyed in-flight limiter (distinct from the request-rate RateLimiter) so no single caller monopolizes the served-git read pool. Each caller (per-DID when signed via optional_signature, else per-source-IP via client_key) may hold at most max_concurrent_reads_per_caller concurrent reads; over that it sheds 503. The key map is self-bounding (a key is dropped when its in-flight count hits zero) with a reject-before-insert max_keys backstop so a key farm can't grow it (INV-15). Applied in git_upload_pack and both info/refs advertisements, acquired after the visibility gate so a denied request never consumes a slot (KTD7).
Primitive unit-tested (cap + self-bounding + reject-before-insert) and mutation-checked. Handler-layer SC2: same-caller sheds while a different caller passes, proven on BOTH git_info_refs and git_upload_pack with independent mutation probes; plus a None-key bypass test. Per-source-IP keying is trust-config dependent, documented on the config knob. 502 tests pass.
Part of #174.
git-receive-pack now draws from a separate git_write_semaphore (max_concurrent_git_pushes) instead of the shared pool, so a flood of anonymous reads can no longer shed an authenticated push at admission (jatmn P1a). The shared field is renamed git_read_semaphore and continues to gate upload-pack and both info/refs advertisements. The write permit stays above acquire_write so it precedes the Tigris fresh-acquire (INV-10).
Handler-layer tests: write-pool shed (503), and a cross-boundary proof that an exhausted read pool does NOT shed a push; both mutation-checked (routing receive-pack back to the read pool flips each RED). 497 tests pass.
Part of #174.
Add max_concurrent_git_pushes (default 32) and max_concurrent_reads_per_caller (default 16), both clap range(1..=1_048_576) so an oversized value is a clean CLI error, not a Semaphore::new boot panic. The per-caller knob documents that per-source-IP keying is only as granular as GITLAWB_TRUSTED_PROXY. Wiring lands in the following commits; these are the config surface for the #174 concurrency-fairness fix.
Resolves jatmn P1a/P1b groundwork on #174.
PR3 of the #62 served-git hardening stack (timeout #165 and teardown
wiring #150 are merged). A bounded semaphore caps how many upload-pack /
receive-pack / info-refs operations run at once; past the cap a request
is shed with a clean 503 + Retry-After before spawning another git
subprocess, instead of exhausting the PID/thread table. A permit is
acquired at the top of each of the three handlers and held for the whole
op, releasing on return.
The cap is a portable backstop: the compose pids_limit is absent on Fly,
whose 500-connection cap is a different axis. Size --max-concurrent-git-ops
(GITLAWB_MAX_CONCURRENT_GIT_OPS, default 128) below the process budget.
Range 1..=1_048_576 so 0 (shed everything) and an oversized value that
would panic tokio's Semaphore at boot are clean CLI errors.
Known gap, tracked separately: info/refs and the withheld-blob
(upload_pack_excluding) path are not duration-bounded and do not reap
their git child on client disconnect, so a hung git on those two paths
holds its slot until it exits and live git can briefly exceed the cap.
The main pack path (run_git_service) tears its group down on drop.
Tests: Overloaded maps to 503 + Retry-After; the config knob defaults
and rejects out-of-range; git_permit sheds at capacity and releases; and
each of the three endpoints sheds with 503 when the semaphore is
exhausted (load-bearing: drop the permit line and the endpoint test goes
red).
* fix(node): gate GET /ipfs/{cid} on reachable allowed-set, not deny-set (#126)
The IPFS visibility gate used withheld_blob_oids (a deny-set enumerating
only reachable blobs), so a dangling/unreachable blob was absent from the
set and served in cleartext to anonymous callers. Flip to an allowed-set
(allowed_blob_set_for_caller) that enumerates reachable blobs the caller
may read: a dangling blob has no path, is never in the set, and 404s.
* perf(ipfs): check object existence before allowed-blob walk
Move store::read_object before the allowed_blob_set_for_caller
spawn_blocking call so random-CID spray against repos with
path-scoped rules cannot trigger full-history git walks on
repos that don't carry the object.
* refactor(ipfs): improve formatting and readability in get_by_cid function
✓ P3 blocker fixed: cargo fmt applied — the format gate will pass.
✓ P3 cleanup resolved: withheld_blob_oids is still used by replication code in repos.rs, so it stays.
• P2 follow-up: Tree/commit disclosure tracked in #135 — out of scope here.
* refactor(ipfs): streamline object retrieval by separating type and content reading
* docs(ipfs): update get_by_cid comment to reflect split object retrieval
* Run cargo fmt on store.rs
* test(gl): add tests for iCaptcha transparent retry flow (#167)
* fix(clippy): remove redundant borrow in format! arg
* fix(review): use runtime GITLAWB_ICAPTCHA_INSECURE flag instead of cfg(test), add mock assertions
* fix(review): set mock expectations at creation time, tighten insecure scheme check
* fix(test): serialize icaptcha integration tests with global Mutex to prevent env-var race
* fix(icaptcha): compare origins (scheme+host+port) in resolve_solver_url; tighten proof assertion in test
* fix(test): save/restore inherited iCaptcha env vars instead of unconditional remove; share lock in icaptcha-client tests
* fix(test): constrain negative send_once mocks with Matcher::Missing so they actually protect the contract
---------
Co-authored-by: Gravirei <[email protected]>
iCaptcha now requires a proof-of-work with each answer (enforce mode) so that
minting a proof costs CPU — the lever that stops a distributed, multi-IP
proof-farming flood that per-IP rate limits cannot. Teach the sanctioned
client to solve it: parse the `pow` field from the challenge, grind a nonce
whose sha256(challenge:nonce) has >= difficulty leading zero bits, and send it
as `powNonce` on /v1/answer. Bounded search so a hostile difficulty can't
hang the client. Mirrors icaptcha/src/pow.ts. Without this, agents on the
updated client cannot obtain a proof once enforce is on.
New pow module + unit tests; verified end-to-end against the live enforce-mode
service (obtained a level-3 proof incl. PoW).
* fix(node): add per-IP flood brake to repo/agent creation endpoints
The recurring spam-repo floods are not bypassing the iCaptcha gate — they
pass it. The enforced level (3) draws only machine-solvable deterministic
challenges (arithmetic/algebra/sequence), and the /v1/challenge API lets the
caller pin the easy type, so a bot mints a fresh throwaway did:key, solves a
real proof bound to that DID, and creates a repo. The per-DID creation
limiter never trips because each DID makes a single create call.
Add a per-client-IP rate limiter to the creation route group (repos,
register, fork, issues, pulls), reusing the same IpRateLimiter the push path
already uses against the identical DID-farm bypass. It wraps the auth layer
(outermost) so flood traffic is rejected before signature verification.
Default 120/hour/IP, GITLAWB_CREATE_RATE_LIMIT overrides, 0 disables.
* docs(env): document GITLAWB_CREATE_RATE_LIMIT in .env.example
* fix: apply CodeRabbit auto-fixes
Fixed 1 file(s) based on 1 unresolved review comment.
Co-authored-by: CodeRabbit <[email protected]>
* test(node): unit-test the per-IP creation brake runs before auth
The route-level coverage (repo_creation_is_rate_limited_by_ip) is
#[sqlx::test] and needs Postgres, so it is skipped in the plain unit-test
pass. Add a DB-free middleware test that layers the per-IP creation limiter
over an always-401 auth stand-in and asserts an exhausted IP is rejected
with 429 before auth runs — the ordering the DID-farm flood defense relies
on.
---------
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
Co-authored-by: CodeRabbit <[email protected]>
* Keep node alive during database outages
* Fix startup-resilience gaps: reachable 503s, bounded retries, seamless handoff
- Route sqlx errors buried in anyhow chains back to AppError::Db and widen
the unavailable class (Io/Tls/pool), so DB outages return 503 instead of
500 — the previous 503 arm was unreachable dead code
- Bound each connect+migrate attempt (GITLAWB_DB_CONNECT_TIMEOUT_SECS) so a
held migration advisory lock cannot wedge the startup retry loop forever
- Share one bound socket between the degraded and full servers and stop
awaiting the degraded drain: no closed-port window during the handoff and
no slow client can block the full server from starting
- Probe the database from /ready (2s bound) and gate Fly routing/deploys on
it, so mid-life outages drain traffic and a bad DATABASE_URL fails the
deploy visibly instead of going green; classify auth/catalog errors as
likely-permanent (error-level log, straight to max backoff)
- Start the metrics listener before the DB connects so the degraded window
is observable; degraded /health now returns 503 so peer pings and uptime
monitors stop counting a DB-less node as alive
- Move the five GITLAWB_DB_* knobs onto clap Config (validated, in --help),
drop the redundant outer connect timeout, default the pool to 20 conns,
fix the backoff exponent cap that defeated GITLAWB_DB_RETRY_MAX_SECS, and
strip at most one .git suffix in smart-HTTP repo names
* fix: apply CodeRabbit auto-fixes
Fixed 1 file(s) based on 1 unresolved review comment.
Co-authored-by: CodeRabbit <[email protected]>
---------
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
Co-authored-by: CodeRabbit <[email protected]>
* fix(node): gate GET /ipfs/{cid} on reachable allowed-set, not deny-set (#126)
The IPFS visibility gate used withheld_blob_oids (a deny-set enumerating
only reachable blobs), so a dangling/unreachable blob was absent from the
set and served in cleartext to anonymous callers. Flip to an allowed-set
(allowed_blob_set_for_caller) that enumerates reachable blobs the caller
may read: a dangling blob has no path, is never in the set, and 404s.
* perf(ipfs): check object existence before allowed-blob walk
Move store::read_object before the allowed_blob_set_for_caller
spawn_blocking call so random-CID spray against repos with
path-scoped rules cannot trigger full-history git walks on
repos that don't carry the object.
* refactor(ipfs): improve formatting and readability in get_by_cid function
✓ P3 blocker fixed: cargo fmt applied — the format gate will pass.
✓ P3 cleanup resolved: withheld_blob_oids is still used by replication code in repos.rs, so it stays.
• P2 follow-up: Tree/commit disclosure tracked in #135 — out of scope here.
* refactor(ipfs): streamline object retrieval by separating type and content reading
* docs(ipfs): update get_by_cid comment to reflect split object retrieval
* Run cargo fmt on store.rs
* refactor(visibility): route is_owner through the shared did_matches matcher (#153)
* style(profiles): remove redundant borrow in format arg
---------
Co-authored-by: Gravirei <[email protected]>
* feat(node,git): bound a hung served git with a total-duration timeout (#62)
Part 1 of #62. A served git upload-pack/receive-pack that neither finishes
nor disconnects parks forever, pinning a git PID (and, for receive-pack,
holding the write lock); on a public repo an anonymous caller can trigger
it. No tower TimeoutLayer exists and axum imposes no default.
Wrap the child interaction in tokio::time::timeout, bounded by a new
GITLAWB_GIT_SERVICE_TIMEOUT_SECS knob (default 600, must be positive).
Drain stdout/stderr concurrently with the stdin write so a large body
can't deadlock. On expiry run_git_service returns a typed GitServiceTimeout
and reaps the WHOLE process group before returning (SIGTERM so git clears
its .git/*.lock, wait for the leader and its grandchildren like index-pack
to exit, escalate to SIGKILL past a grace, hard-capped), so a caller
releasing the receive-pack write lock can't race a still-live git on the
same repo.
The handlers map GitServiceTimeout to a distinct 504 (not the generic 500
git error, and clear of the read-gate 404 / auth 401 the client keys
retries on) via a pure, unit-testable classifier rather than matching the
anyhow string. Surface git's own non-zero exit (its stderr) before a
stdin-write EPIPE so a malformed body is classified 400, not 500.
Out of scope, deferred in #62: the info/refs advertisement and the
withheld-blob path (spawn_blocking, which tokio timeout cannot cancel);
both remain unbounded (noted in the config knob's docs).
Tests: the timeout fires and tears the group down; it reaps the whole
group before returning (RED if the reap is leader-only); the wrap is
load-bearing (RED via an outer bound, not a hang); the error must be the
typed timeout; the 504 mapping and classifier are unit-tested; config
rejects 0. Full suite green.
* fix(review): harden #62 timeout tests and drop a dead reap flag
- test(node): cover the `protocol error` arm of git_service_app_error
independently (the input has no "bad line length" substring, so it
isolates the second classifier arm and goes RED if that arm is removed)
- test(node): make the reap-before-return test's grandchild IGNORE SIGTERM
(`trap "" TERM`) and outlive the ~4s reap cap, so the SIGKILL escalation
is load-bearing. Previously the grandchild self-exited under the cap, so
neutering the SIGKILL still passed; the escalation was exercised but not
required. Verified: RED with SIGKILL disabled, GREEN with it.
- test(node): add a receive-pack timeout test. Every prior timeout test used
an upload-pack fake; this proves the push path (which also holds the repo
write lock) is bounded too. RED-verified with the internal timeout removed.
- test(node): poll for the pidfile in the reap-before-return timeout test
so a loaded runner can't false-panic before the fake git writes it
- remove the redundant `sigkilled` guard in reap_group_on_timeout; step
== 200 fires exactly once in the 0..400 loop, so no re-entry guard is
needed and the SIGKILL still fires exactly once
* fix(node): log malformed receive-pack as warn, not error (#165)
git_receive_pack's error classifier routed every non-Timeout error,
including client-caused BadRequest (400), through tracing::error!. Mirror
the git_upload_pack arm so a malformed push logs at warn level instead of
as a server error.
* test(node,git): make smart_http timeout tests pass under the parallel runner (#165)
Addresses jatmn's review of 41eca0b.
[P2] The #62 timeout tests failed under `cargo test --workspace`:
- run_git_service_tears_down_group_when_future_dropped re-polled a completed
future ("async fn resumed after completion"): the advance-until-pids loop
ignored the timeout's Ok(_) (future-finished) case and polled the resolved
future again. It now breaks on early completion.
- Under fork-storm load a freshly-written fake `git` transiently fails to exec
(ETXTBSY) or is timed out before recording its pids. Added a
fake_git_run_with_pids retry helper (used by the four pid-reading tests) plus
a matching inline retry on the drop test, with growing backoff for the
correlated bursts and a 12-attempt cap that still fails loudly on a genuine
never-spawns regression. Reverted two tests from a speculative 2000ms bound
back to 300ms/1000ms now that the retry covers the timeout-vs-pidfile race.
[P3] Documented GITLAWB_GIT_SERVICE_TIMEOUT_SECS in .env.example and the README
config table (default 600; bounds upload-pack/receive-pack, not info/refs or the
withheld-blob path).
The tests stay load-bearing: breaking process_group(0), the post-reap disarm,
the timeout-arm reap, or the internal timeout each still turns the corresponding
test red (mutation-verified).
---------
Co-authored-by: t <t@t>
* fix(node): gate GET /ipfs/{cid} on reachable allowed-set, not deny-set (#126)
The IPFS visibility gate used withheld_blob_oids (a deny-set enumerating
only reachable blobs), so a dangling/unreachable blob was absent from the
set and served in cleartext to anonymous callers. Flip to an allowed-set
(allowed_blob_set_for_caller) that enumerates reachable blobs the caller
may read: a dangling blob has no path, is never in the set, and 404s.
* perf(ipfs): check object existence before allowed-blob walk
Move store::read_object before the allowed_blob_set_for_caller
spawn_blocking call so random-CID spray against repos with
path-scoped rules cannot trigger full-history git walks on
repos that don't carry the object.
* refactor(ipfs): improve formatting and readability in get_by_cid function
✓ P3 blocker fixed: cargo fmt applied — the format gate will pass.
✓ P3 cleanup resolved: withheld_blob_oids is still used by replication code in repos.rs, so it stays.
• P2 follow-up: Tree/commit disclosure tracked in #135 — out of scope here.
* refactor(ipfs): streamline object retrieval by separating type and content reading
* docs(ipfs): update get_by_cid comment to reflect split object retrieval
* Run cargo fmt on store.rs
* fix(bounties): add tests for claim_bounty repo-read gate (#160)
* fix(profiles): remove redundant borrow in format arg
---------
Co-authored-by: Gravirei <[email protected]>
* fix(node): gate GET /ipfs/{cid} on reachable allowed-set, not deny-set (#126)
The IPFS visibility gate used withheld_blob_oids (a deny-set enumerating
only reachable blobs), so a dangling/unreachable blob was absent from the
set and served in cleartext to anonymous callers. Flip to an allowed-set
(allowed_blob_set_for_caller) that enumerates reachable blobs the caller
may read: a dangling blob has no path, is never in the set, and 404s.
* perf(ipfs): check object existence before allowed-blob walk
Move store::read_object before the allowed_blob_set_for_caller
spawn_blocking call so random-CID spray against repos with
path-scoped rules cannot trigger full-history git walks on
repos that don't carry the object.
* refactor(ipfs): improve formatting and readability in get_by_cid function
✓ P3 blocker fixed: cargo fmt applied — the format gate will pass.
✓ P3 cleanup resolved: withheld_blob_oids is still used by replication code in repos.rs, so it stays.
• P2 follow-up: Tree/commit disclosure tracked in #135 — out of scope here.
* refactor(ipfs): streamline object retrieval by separating type and content reading
* docs(ipfs): update get_by_cid comment to reflect split object retrieval
* Run cargo fmt on store.rs
* fix(visibility): gate repo-scoped read surfaces on visibility (#120)
* fix(visibility): enhance repo read authorization checks and improve test coverage
* Fix review feedback: clamp limit, memoize authz, identical 404 body, resolved issue id in comments
* cargo update -p crossbeam-epoch: fix RUSTSEC-2026-0204 (invalid pointer dereference)
* Fix review findings: correct authz memoization, page past hidden bounties, gate star/unstar
* Fix findings: sign CLI/MCP reads, keyset pagination, gate claim_bounty
---------
Co-authored-by: Gravirei <[email protected]>
* feat(node): add per-client-IP limiters + knobs for the peer-sync routes (#82)
Plumb two dedicated per-IP RateLimiters into config/state so the peer-sync
enqueue routes can be braked independently: sync_trigger_rate_limiter (tight,
GITLAWB_SYNC_TRIGGER_RATE_LIMIT, default 60/hr) and peer_write_rate_limiter
(generous, GITLAWB_PEER_WRITE_RATE_LIMIT, default 600/hr). Both are bounded and
swept by the periodic cleanup loop, keyed via the node-wide push_limiter_trust.
No route is wired to them yet; that lands next.
* fix(node): require a signature on /sync/trigger and rate-limit the peer-sync routes (#82)
/api/v1/sync/trigger was reachable unauthenticated in the default config and
drove an O(peers) outbound fan-out + per-repo sync_queue enqueue — an anonymous
amplification/DoS. The same enqueue_sink is reachable via /sync/notify, which
also accepts unsigned known-peer requests and had no rate limit.
- Move /sync/trigger into its own always-signed route group (add_auth_layers),
independent of require_signed_peer_writes, and give it a tight per-IP brake.
- Add a separate, generous per-IP brake to the peer-write group
(/peers/announce, /sync/notify); notify's signature behavior is unchanged
(rolling-upgrade compat), it only gains the brake. Separate buckets so an
unsigned notify flood can't drain the signed trigger caller's quota.
- Per-IP (not per-DID): a did:key farm self-registers, so a signature/per-DID
gate does not cap cost (INV-10); the brake keys on the client IP before auth.
Adds an adversarial test matrix (unsigned->401 both modes, IP flood->429,
DID-agnostic brake, forged X-Forwarded-For can't bypass, notify flood->429,
distinct-IP not throttled, cross-route non-exhaustion, 0-disables-but-sig-holds,
announce still accepts unsigned).
* fix(gl): surface node denial on sync trigger and sanitize the error body (#82)
Now that the node gates /api/v1/sync/trigger behind a signature and rate limit,
'gl sync trigger' must not render a 401/429 as a fabricated '✓ sync triggered /
0 peers' success. Check resp.status() before parsing; on a non-2xx, extract the
node's message and bail. Strip C0/C1 control bytes from the node-supplied string
before printing (INV-6) — a hostile node could embed ANSI/OSC escapes in the
body. Adds mockito tests: 401/429 surface as errors (RED without the status
check), control bytes are stripped, and the 200 path is unchanged.
* fix(review): bound gl error-body read, strip bidi controls, genericize 429 message (#82)
Code-review follow-ups on the #82 change:
- gl sync trigger: read the node error body with a capped reader
(read_body_capped) instead of resp.text(), so a hostile/broken node can't
force an unbounded allocation to surface a denial (INV-6, read half).
- sanitize_node_msg: also strip Unicode bidi/format controls (U+202A-202E,
U+2066-2069, LRM/RLM/ALM) — is_control() only covers the Cc category, but a
right-to-left override can reorder the displayed error line.
- too_many_requests(): the shared per-IP 429 middleware now serves the peer-sync
routes too, so drop the push-specific wording ('rate limit exceeded'); the
push-specific 429 in the receive-pack handler is unchanged.
Adds tests for the bidi/length sanitize and the oversized-body path.
* test(#82): close reasoned-not-run gaps with load-bearing tests
Executes the branches that were previously only reasoned:
- read_body_capped byte-cap test (RED against resp.text() — 2MB body -> ≤cap):
vets the bounded-read fix, not just the display cap.
- sync-route 429 body asserts 'rate limit exceeded' and NOT 'push' (RED before
the message was genericized); also finishes genericizing the warn log, which
the response-body change had missed.
- real RFC 9421 signature through the full router reaches the handler (200) in
both config modes — the positive path through the actual gate, not a
direct-mount with an injected DID.
- /peers/announce flood -> 429 (the peer_write brake covers announce, not just
notify).
- trigger_counts extraction unit test (happy + malformed -> 0,0), closing the
count-formatting gap in the 200 path.
No production behavior change beyond the warn-log wording.
* fix(#82): require an identity for sync trigger and fix the rollout docs
Address review feedback on the always-signed /sync/trigger route.
- gl sync trigger now requires a local identity via load_keypair_from_dir and
fails locally with "identity not found" before issuing a request, instead of
.ok() silently sending an unsigned request that can only 401 remotely. Add a
test proving it fails before dialing the node.
- Split /sync/trigger out of the staged unsigned peer-write rollout in the
README and the OSS-readiness audit; only /peers/announce and /sync/notify keep
the rolling-upgrade unsigned behavior.
- Reorder the two peer-sync rate-limit vars in .env.example for dotenv-linter.
- Bump crossbeam-epoch to 0.9.20 (RUSTSEC-2026-0204), matching main, so cargo
audit passes.
---------
Co-authored-by: t <t@t>
RUSTSEC-2026-0204 (invalid pointer dereference in crossbeam-epoch's
fmt::Pointer impl for Atomic/Shared) hard-fails cargo audit on every branch,
since the advisory's addition to the RustSec db. crossbeam-epoch is a fully
transitive dependency (moka -> hickory-resolver -> libp2p-dns), and the
advisory's prescribed fix is >= 0.9.20; moka 0.12.15 accepts >=0.9.18,<0.10.0
so the bump resolves with no other graph changes.
Lockfile-only, single crate. Verified: cargo audit exits 0 and the workspace
builds clean with 0.9.20.
Closes#162.
Co-authored-by: t <t@t>
The aarch64-unknown-linux-musl leg cross-compiled a musl target with the
glibc toolchain (CC=aarch64-linux-gnu-gcc), which aws-lc-sys cannot build
with — the Build step failed on every release and no aarch64-linux asset
has ever shipped. Build it natively on ubuntu-24.04-arm instead (free for
public repos), using the same stock musl-tools recipe as the x86_64 leg,
and drop the smoke-test skip since the binaries now run on their build
runner.
The x86_64-apple-darwin leg targeted macos-13, which GitHub has retired;
the job queues forever and holds the release concurrency group open. Move
it to macos-15-intel.
Both failures also skipped the downstream npm-publish, homebrew-bump, and
web-sync jobs, so v0.4.0 and v0.5.0 were never published there.
Co-authored-by: Claude Fable 5 <[email protected]>
* feat(gl): sanctioned iCaptcha client flow + secure git lifecycle
Builds the client side of the iCaptcha gate so a legitimate agent passes the
challenge transparently, with no manual headers and no GITLAWB_ICAPTCHA_PROOF
env var. Server enforcement is unchanged.
New crate `icaptcha-client` (blocking, shared by gl + the git helper):
- deterministic solvers for the computational challenge types (arithmetic,
algebra, sequence) matching the iCaptcha generators; anagram/logic/LLM fall
back to a solver hook / interactive prompt.
- obtain_proof(): POST /v1/challenge (requesterId = caller DID), solve, POST
/v1/answer, handle escalation, return the signed proof. Optional bearer auth
via GITLAWB_ICAPTCHA_API_KEY.
gl (crates/gl):
- http.rs: unify post/put/delete through one signed sender that, on a 403
iCaptcha challenge (detected via x-icaptcha-url / x-icaptcha-level headers),
solves it and retries the same signed request with the x-icaptcha-proof
header (bounded retry absorbs the ~5 min proof TTL). Removes the env hack.
Emits an actionable hint on 401 not_an_agent (old-CLI / unregistered).
- repo.rs: `repo info` checks status and returns a real 404 instead of a stub
card with `?` fields; `repo info`/`clone` resolve the owner from the local
identity or an explicit owner/name — never the node's own DID.
- doctor.rs: warn on gl-vs-node version drift; add an iCaptcha reachability
check.
git-remote-gitlawb: same shared client — push stays signed (RFC 9421); on a 403
iCaptcha (safety net; push is signed-only, not gated) it solves and retries.
Clone/fetch unsigned; missing repo keeps its clear 404.
gitlawb-node: the 403 icaptcha_proof_required response now advertises the
service url + required level as JSON fields and x-icaptcha-url / x-icaptcha-level
headers (mirroring the human_detected pattern) so clients discover them instead
of scraping the message. The sub == authenticated-DID and expiry/level checks
are unchanged — enforcement is not weakened.
README: documents the full lifecycle (identity -> register -> repo create with
auto-iCaptcha -> push -> clone), the requesterId == DID rule, and proof TTL.
Tested: solver unit tests; full workspace green against Postgres (node still
rejects missing/expired/wrong-subject proofs).
* fix(icaptcha-client,gl): address PR #138 review feedback
Rebased onto main (resolving the git-remote-gitlawb/main.rs conflict from #119)
and addressed the review findings:
- [beardthelion P2 / jatmn P2 / CodeRabbit] Removed the dead iCaptcha retry loop
from the push path. git_receive_pack never calls verify_request (only
create/fork/register are iCaptcha-gated), so a push 403 never carries
x-icaptcha-* headers and the loop could never solve. Taking main's push path
(via #119) deletes it, which also resolves the pack-clone (retries no longer
copy the whole pack) and the interactive-prompt-over-git-stream threads, and
restores the sanitized info/refs error path. git-remote-gitlawb no longer
depends on icaptcha-client.
- [beardthelion P1] Sanitize and length-bound iCaptcha error bodies before they
reach the terminal. The origin is only as trusted as the node that advertised
it, so its non-2xx bodies (and Failed{reason}) are attacker-influenceable; they
went raw into bail! (CWE-150, the #137 class). Now bounded-read + C0/C1-stripped
+ capped, same shape as #137.
- [beardthelion P2] Do not trust or hand credentials to a node-chosen iCaptcha
URL. x-icaptcha-url was used as the base with no scheme/host check and the API
key was sent to it on the first hop (SSRF + key exfiltration). resolve_solver_url
now honors an advertised URL only when https + host-allowlisted (public default
or the operator's GITLAWB_ICAPTCHA_URL), else falls back to the trusted origin;
the API key is attached only to the operator's own configured origin.
- [jatmn P2 / CodeRabbit] gl repo clone with a bare name now derives the short
owner key via resolve_owner_did, matching the other commands' gitlawb://z.../name
URL shape instead of the colon-bearing full DID.
Skipped (with reason): jatmn P3 (doctor GITLAWB_ICAPTCHA_URL) is resolved by the
P2 change — that env var is now read by the write path, so the doctor's probe is
accurate. CodeRabbit User-Agent-version nit lived in the push code reverted to
main. README push wording kept ("signed-only, no per-push challenge") — now
accurate since the retry path is gone; the API-key bullet was updated for the new
trust model.
Tests: added sanitize + resolve_solver_url unit tests; full workspace green.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
---------
Co-authored-by: Claude Opus 4.8 <[email protected]>
* fix(node): close two spam-vector root causes (trust upsert + ungated push)
The June 2026 spam wave kept re-materializing after cleanup because two
paths let an unauthenticated-by-captcha actor keep operating:
1. `update_trust_score` was an upsert. Any authenticated push/issue/PR
from a DID with no agent row silently INSERTed one with a fresh
`registered_at`, bypassing the iCaptcha gate on /api/register — so
deregistered spam DIDs came back the moment they pushed. Make it an
UPDATE-only no-op for unregistered DIDs; registration is the sole way
into the agents table. Regression test added.
2. `git-receive-pack` had no rate limit. The per-DID limiter on the
creation routes can't brake a push flood from a DID farm (one
throwaway identity per repo), so the node absorbed several pushes/sec
per IP, each a Tigris round-trip + peer-notify fan-out. Add a per-
client-IP limiter (Fly-Client-IP / X-Forwarded-For) on the push path,
default 600/h, `GITLAWB_PUSH_RATE_LIMIT` override, 0 disables. Fails
open without a proxy header so self-hosted nodes aren't broken.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
* fix(node): address PR review — advertisement throttle, trusted-proxy IP key, bounded limiter
Resolves the review findings on the push rate limiter:
P1 — advertisement path bypassed the limiter. A push hits
GET info/refs?service=git-receive-pack first, which calls acquire_fresh()
(a Tigris download); it sat in git_read_routes under only optional_signature,
so the flood was reachable via the cheap GET without ever sending the throttled
POST. Now git_info_refs applies the same per-IP brake before the fresh acquire.
P1 — client-supplied forwarded headers were trusted. The key came from
Fly-Client-IP / the first X-Forwarded-For hop, both client-settable, so a
flooder could rotate the header and never fill a bucket. Replaced with a
TrustedProxy policy (GITLAWB_TRUSTED_PROXY = fly | x-forwarded-for | unset):
trust only the operator's edge header, fall back to the real socket peer
(ConnectInfo) otherwise. Fly config trusts Fly-Client-IP; the AWS/Caddy
compose trusts the rightmost X-Forwarded-For hop.
P2 — unbounded limiter key set / insert-before-reject. Added a max_keys cap
(reject-before-insert, inline eviction when full) so a varied key cannot grow
the map, and a rejected request never allocates. Applied to both the per-IP
and per-DID limiters.
P2 — malformed forwarded header disabled the brake. Empty leading XFF hop
returned None and skipped the limiter; empty Fly-Client-IP returned Some("").
client_key now takes the rightmost XFF hop and always falls back to the peer,
so a malformed header can never turn the limiter off.
P2 — missing tests. Added: middleware 429 path, per-peer isolation, key-cap
eviction/rejection, trusted-proxy header resolution, and an integration test
asserting the receive-pack advertisement is throttled before the acquire.
Also serve with into_make_service_with_connect_info so the peer address is
available. New env: GITLAWB_PUSH_RATE_LIMIT, GITLAWB_TRUSTED_PROXY (documented
in .env.example).
Co-Authored-By: Claude Opus 4.8 <[email protected]>
* Update infra/aws/compose.yaml.tftpl
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
* fix(node): amortize the rate limiter's capacity eviction scan
`RateLimiter::check` ran a full O(max_keys) `retain` sweep on every new-key
miss once the map was at capacity — which is exactly the distinct-key-flood
state the cap defends against, so the limiter serialized all traffic behind a
200k-entry scan per request under the mutex.
Gate the inline sweep to run at most once per `sweep_interval` (min of the
window and 1s), tracked by `last_sweep`. Between sweeps a new key is rejected
without scanning; the background `cleanup()` loop still reclaims independently.
The fast path for existing keys is unchanged, the key cap invariant holds
(map never exceeds max_keys, rejected keys never allocate), and post-flood
self-healing stays bounded to one interval. Added a test asserting a burst of
capacity misses does not trigger repeated eviction scans.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
---------
Co-authored-by: Claude Opus 4.8 <[email protected]>
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
create_repo was the one git handler that mapped its error without logging
it (info_refs, receive_pack, acquire all log with repo context). So a
failed init_bare returned an opaque 500 "initializing bare repo" with no
server-side trail to the real cause.
Log the full anyhow chain ({:#}, so the leaf git/filesystem error is kept)
with owner+repo context, matching the other handlers. Response unchanged.
Co-authored-by: mizuki0x <[email protected]>
The PR-blocking audit job ran `cargo install --locked cargo-audit` with no
version, so the scanner floated to the latest release on every run; a
regressed release could red the PR gate repo-wide with no code change. Pin it
to 0.22.2, the version the unpinned install currently resolves to and the one
the audit-gate learnings were validated against. `cargo install --version X`
is an exact pin (not a caret), so this freezes the binary while the advisory
DB is still fetched fresh at scan time, which is what the gate actually keys on.
Keep a forcing function so the pin cannot silently rot: leave the weekly
Scheduled Audit deliberately unpinned as a drift canary. That job is cron-only
(never pull_request/push-triggered), so it never gates a merge; floating there
surfaces a regressed latest as a red weekly run or a changed report, which is
the drift signal, without ever blocking a PR.
Also assert the installed version after the pinned install so an accidental
future unpin fails loud instead of silently running a different scanner,
include the version in the rust-cache key so cache hits stay deterministic,
and document the yank/bump path in the step comment.
Co-authored-by: t <t@t>
The #61 tests build KillGroupOnDrop by hand and never call
run_git_service, so dropping process_group(0), the guard-arming, or the
post-reap disarm left them green. Add two tests that drive the real
run_git_service through an injected fake git (a minimal git_bin seam;
production passes "git") and assert the production spawn path actually
wires the teardown:
- future-drop mid-flight SIGTERMs the whole group, reaping git and its
grandchild (RED without process_group(0) or the guard-arming)
- a completed request disarms the guard, leaving a live grandchild
untouched (RED without the post-reap disarm)
Each assertion was shown RED against its mutation and green when
restored. Parallel-safe: the fake is reached via the seam, not a PATH
mutation.
Co-authored-by: t <t@t>
* fix(gl,ipfs): sign gl ipfs list so it works under the #134 pins auth gate
/api/v1/ipfs/pins now 401s anonymous callers (#134). Load the caller's
identity (--dir, default keystore) and use NodeClient::get_signed; when no
identity exists, propagate load_keypair_from_dir's existing 'gl identity new'
error instead of a raw 401. Tests: signed-headers present, empty, and
no-identity-issues-no-request (mockito).
* fix(gl,node): sign the gl node status pins panel, degrade gracefully
The pins panel signs its /api/v1/ipfs/pins read (#134 gates it behind auth)
via an injectable fetch_pins helper with four states: pins, empty,
unavailable (signed read rejected/errored), and needs-identity (no keypair,
no request issued). cmd_status loads the identity gracefully so a missing
keystore never aborts status; peers/repos/p2p/events panels stay anonymous.
Tests drive fetch_pins directly against a mock node.
* fix(gl): address review — surface pins HTTP errors, correct empty-panel label
- gl ipfs list: check the /ipfs/pins response status before parsing, so a
rejected signed read (e.g. 401) surfaces as an error instead of silently
printing 'No IPFS pins recorded'. Matches the sibling gl ipfs get.
- gl node status: PinsPanel::Empty now renders 'Pinned CIDs: 0' (a reachable
node with zero pins), reserving 'unavailable' for the failure state; the
prior label reused 'IPFS not configured' and misreported the zero-pins case.
- Carry the resolved count in PinsPanel::Pins, removing the duplicated
count-fallback closure between fetch_pins and the render arm.
* test(gl,node): cover fetch_pins error branches and pins-panel rendering
Extract the pins-panel render into a testable pins_status_line helper and
assert all four states' output (closes the untested cmd_status render path,
including the empty->'Pinned CIDs: 0' label). Add fetch_pins tests for the
two error branches that were code-traced but unexecuted: a malformed 200
body and a transport error both degrade to Unavailable without panicking.
* style(gl): rustfmt the ipfs list pins-response parse
* fix(gl,node): honor --dir identity selector in `gl node status` (#146)
`gl node status` always loaded the default keystore via
`load_keypair_from_dir(None)`, so a user who created or selected an identity
with `--dir` saw the status pins panel render "sign in to view" even though
`gl ipfs list --dir` authenticated fine. Add a `--dir` option to
`NodeCmd::Status` and thread it into the signed pins fetch, so both CLI pins
callers select the same identity directory.
An explicit `--dir` that can't be loaded (missing, corrupt, or unreadable) no
longer masquerades as "no identity, run `gl identity new`": the pins identity
is resolved up front into Keyed / Anonymous / DirUnusable, and the unusable
explicit-dir case renders a distinct "no usable identity in <dir>" panel line
that names the path, while a missing default keystore (no `--dir`) still
degrades quietly. A pins failure never aborts the dashboard.
Tests cover both flag branches (Some/None) and the resolver both ways: a good
`--dir` loads Keyed, an unusable one reports DirUnusable/IdentityError without
issuing a request. The new symbols were absent before, so they compile-RED
first.
* test(gl,node): pin the pins-identity resolver to the --dir key
Follow-up test hardening on the #146 --dir wiring after review:
- resolve_pins_auth_loads_keyed_identity_from_explicit_dir now destructures
Keyed and asserts loaded.did() equals the --dir key's DID. A bare Keyed(_)
match passed even when the load fell through to the default keystore; the
DID assertion catches that regression (confirmed red under a simulated
fallthrough, green on the real code).
- add resolve_pins_auth_none_is_never_dir_unusable: an absent --dir must
degrade to Anonymous or Keyed, never DirUnusable. It asserts only the
None to DirUnusable mapping, so it is stable whether or not the test
runner has a default ~/.gitlawb identity.
Test module only; no production behavior change.
---------
Co-authored-by: t <t@t>
* feat(node,visibility): add pure fail-closed feed-visibility filter (#112,#114)
Add ref_update_row_visible: given the deduped local repo set + their
visibility rules, decide whether a received_ref_updates row (keyed by its
peer-supplied slug) is visible to the caller. Matches slug->repo by name
plus prefix-tolerant owner key, and drops any row matching a local repo
the caller cannot read at root; rows matching no local repo (remote/
gossip-only) and malformed slugs pass through. Fail-closed by
construction (no default-keep). Pure/I-O-free so visibility.rs stays
exhaustively unit-tested.
Shared gate for the GraphQL (#112) and REST (#114) ref-updates feeds;
call sites land in the following commits. Includes a DB-layer test
pinning that a private canonical row beats a public mirror row in
list_all_repos_deduped (else the feed gate would over-serve).
* fix(node,graphql): gate refUpdates on read visibility (#112)
The GraphQL ref_updates resolver mapped every received_ref_updates row
straight out with no visibility gate, so an anonymous caller could read
a private repo's ref metadata (refName, SHAs, pusherDid, nodeDid) by
name. Thread the caller DID and retain rows through the shared
ref_update_row_visible filter (deduped local set + rules, same '/'
decision the repos resolver uses); applies to both the repo:Some and
repo:None branches. Remote/gossip-only rows still pass.
Removes the now-live filter's dead_code allow. Tested via the GraphQL
schema: anon leak reproduced before the gate (RED) and closed after
(GREEN); owner still sees their private row; full-DID and truncated-key
slug aliases fail closed; remote rows kept.
* fix(node,events): gate global ref-updates feed on read visibility (#114)
GET /api/v1/events/ref-updates returned every received_ref_updates row
across all repos, including private ones, to any anonymous caller (the
REST analog of #112). Thread the caller DID via optional_signature and
retain rows through the shared ref_update_row_visible filter, deriving
count from the filtered set. Keeps the crate::error::Result return type;
the deduped/rules loads fail closed (500) on a DB error rather than
serving unfiltered. Remote/gossip-only rows still pass.
Tested via the crate's HTTP-API harness: anon leak reproduced before the
gate (RED) and closed after (GREEN); owner still sees their private row;
mixed feed yields only public rows with a filtered count; full-DID and
truncated-key slug aliases fail closed; remote rows kept.
* fix(node,graphql): gate the ref-update subscription broadcast on announce (#112,#114)
Code review surfaced a third reader of the same ref metadata the query
and REST feeds gate: the GraphQL subscription. /graphql/ws is mounted
after the optional_signature layer (no caller to gate against), and the
push handler broadcast every ref update to it unconditionally — outside
the if-announce block that already gates the sibling gossip and Arweave
sends. So an anonymous websocket subscriber received live ref metadata
(repo, refs, SHAs, pusher/node DID) for private repos as they were
pushed — the subscription analog of the #112/#114 leak.
Move the ref_update_tx broadcast inside the existing if-announce guard so
only publicly-readable pushes reach the unauthenticated subscription,
matching gossip and Arweave. Pin the gate decision with a
replication_withheld_set test (announce false for private, true for
public).
Also harden the feed filter's owner-key normalization: normalize the
slug owner to the last ':'-segment (same rule as record_key) instead of
stripping only did🔑, so a multi-segment DID (did:web) and its
full-DID slug match and fail closed rather than over-serving. Regression
test added (RED under the old normalization).
* docs(node,graphql): pin the subscription write-side gate invariant; add did:web keep test
Code review flagged that the unauthenticated /graphql/ws ref_updates
subscription now depends on a single-point invariant: safety rests
entirely on every sender to ref_update_tx being announce-gated, since the
resolver has no caller to gate against. Document that on the resolver so
a future sender can't silently reopen the leak.
Add the symmetric keep-side test for the did:web owner-key normalization
(public multi-segment-DID repo row is kept for anon), so a regression
that over-drops legitimate did:web rows is caught alongside the existing
drop-side test.
* docs(node,visibility): correct ref-update slug normalization rationale (#112,#114)
The comment claimed parity with did_matches/DEDUP_CTE, but the last-segment
normalization deliberately diverges: the emitter (publish_ref_update) broadcasts
the short, method-stripped slug, so a repo's own rows arrive keyed by the
trailing segment and the gate must match on it to stay fail-closed. Reusing the
did:key-aware keep-whole rule would fail open on a did:web repo's short slug.
Add feed_private_didweb_short_slug_dropped_for_anon to pin that load-bearing
canonical case; relabel the full-DID test as the non-canonical hand-forged form.
* fix(node,events): gate ref-update feeds before the limit via paging (#114)
The REST global feed and the GraphQL ref_updates resolver loaded `limit` rows
and dropped private ones in memory, so an anonymous request could get fewer than
`limit` events (or zero) when the newest rows named private local repos, even
though older public ref updates existed.
Add a shared collect_visible_ref_updates helper that applies the fail-closed
gate before the limit: it over-fetches in bounded pages (128 rows, capped at
max(limit, 2048) scanned) until `limit` visible rows are collected or the table
is exhausted. Both feeds route through it so the one gate cannot drift. Back it
with list_ref_updates_page (stable timestamp,id ordering for offset paging) and
drop the now-unused list_ref_updates / list_ref_updates_filtered. The page size
is a parameter on an inner fn so a regression can exercise multi-page paging.
Regressions: newest-private, older-public, small limit still returns the latest
visible rows on both surfaces (RED before, GREEN after); and a page=2 test pins
offset paging across page boundaries (no skip or duplicate).
* docs(node,visibility): correct slug-builder attribution in ref-update gate comment
The comment credited publish_ref_update with building the wire slug, but that
p2p method only broadcasts a pre-built RefUpdateEvent; git_receive_pack assembles
the {last-segment}/{name} slug and hands it over. Fix the attribution.
* fix(node,events): bound feed limit and withhold quarantined mirrors (#114)
Resolve two CodeRabbit findings on the ref-update feeds:
- Clamp the effective limit inside the shared collector (max 200). REST already
capped, but the GraphQL resolver passed its caller-provided limit uncapped,
letting a large request return unbounded rows and scan unbounded DB rows. Both
surfaces are now bounded identically.
- Fold quarantined mirrors into the gate's match universe as unreadable.
list_all_repos_deduped excludes quarantined rows, so a ref-update naming a
quarantined local mirror matched nothing, was misclassified as remote, and was
served to anon, bypassing the withheld-from-every-listing-surface contract. Add
list_quarantined_repos and drop matching rows fail-closed.
Regressions: oversized-limit clamp (201 rows -> 200) and quarantined-mirror
withheld-from-anon, both RED before, GREEN after.
* fix(node,events): gate the repo-scoped events feed on read visibility (#112, #114)
GET /api/v1/repos/{owner}/{repo}/events (list_repo_events) served a repo's ref
certificates and received gossip ref-updates with no caller identity and no
visibility check, so an anonymous caller who guessed a private repo's owner/name
read its ref metadata. It was the last repo-scoped reader in the #112/#114
ref-metadata leak class.
Gate it in two layers, using the caller DID the optional_signature layer already
supplies:
- The cert half is gated by a repo-root authorize_repo_read (keyed by the unique
repo record id): a denied, quarantined, or not-hosted repo returns an opaque 404
(RepoNotFound), never a 403.
- The gossip half is filtered per row through the shared collect_visible_ref_updates.
received_ref_updates rows are keyed by the lossy, non-unique wire slug
{last-segment}/{name}, so two owners (did:web:a:alice, did:web:b:alice) collide on
`alice/name` and a plain slug query would serve a colliding private repo's rows to
anyone allowed to read this one; the per-row gate drops them fail-closed.
Both halves now propagate DB errors as 500 instead of swallowing them into an empty
200. Removes the now-unused list_repo_ref_updates.
Deliberate behavior change: a repo this node does not host returns 404 (it holds no
visibility record for it and fails closed; remote gossip is read via the global
/api/v1/events/ref-updates feed). Drops list_repo_events from the egress-guard
KNOWN_UNGATED allowlist; the source-scrape staleness assert enforces the removal.
Tests cover anon-private, owner-private (both datasets), anon/authed-public,
quarantined, released-from-quarantine, non-owner, not-hosted, did:web (owner and
anon), the slug collision, oversized-limit clamping, and DB-error-fails-closed on
both the gate and the cert fetch.
* fix(node,events): clamp negative ref-update feed limits to zero (#114)
A caller-supplied `?limit=-1` cleared the `.min(MAX_VISIBLE_REF_UPDATES)`
upper cap unchanged and reached `all_events.truncate(limit as usize)` in
list_repo_events, where `-1 as usize` is usize::MAX, so the truncate was a
no-op and the handler returned the full unbounded local ref-cert set.
Floor both feed handlers' limit parse with `.clamp(0, MAX_VISIBLE_REF_UPDATES)`.
Only the repo handler's clamp is load-bearing: it guards the local `truncate`,
which does not pass through the shared collector. The global feed's clamp is a
consistency measure (its limit is re-clamped in the collector, whose want==0
short-circuits before any scan), kept in lockstep so the two per-handler caps
the module comment calls out don't drift.
Tests, both load-bearing by execution:
- repo_events_negative_limit_clamped: seeds local certs (the vulnerable half),
RED (3) before the clamp and green (0) after.
- feed_negative_limit_returns_empty: seeds 5 visible rows; green (0) with the
collector clamp, 5 without it, proving the collector is the global feed's guard.
* fix(node,events): drop quarantined repos before the feed visibility gate (#114)
The ref-update feeds folded quarantined mirrors into the deduped repo set with is_public=false and relied on the row visibility gate to withhold them. That held for anonymous callers but not for one matching the mirror's owner_did: visibility_check short-circuits to Allow for the owner before is_public is read, so an owner-matched quarantined row was served on both the REST global feed and the GraphQL ref_updates resolver.
Drop any row that names a quarantined repo in the shared collector, before the visibility gate runs, so the drop bypasses the owner short-circuit. Quarantine is a status decided at admission and checked separately from the mirror's visibility fields, so it must deny every caller including the owner.
Extract the slug match into one ref_update_row_names_repo predicate reused by both the gate and the quarantine drop, so the two cannot disagree about which rows a repo owns.
Tests cover the owner case at the collector, REST handler, and GraphQL resolver in both bare-key and full-did:key owner forms, plus a must-not case proving the drop withholds only quarantined rows while still serving unrelated visible ones.
* test(node,events): align did:web feed tests with the #141 canonical slug
The rebase onto main adopts #141's normalize_owner_key on the repo-scoped
ref-update read path, which keeps a non-did:key DID intact instead of taking
the last URL segment. Align the two did:web fixtures that still assumed the
old {last-segment}/{name} wire slug:
- repo_events_did_web_owner_reads_own_gossip stored "alice/widget", but the
emit side (api/repos) publishes "did:web:example.com:alice/widget", so the
read query no longer matched and the owner saw only the cert half. Store the
emit-side slug so the gossip KEEP branch is exercised again.
- repo_events_gossip_slug_collision_withheld_from_anon is removed. Its premise
was that two did:web owners collide on the last-segment slug; #141 gives them
distinct full-DID slugs, so the repo-scoped query fetches zero rows and the
per-row drop is never reached (verified by probe: fetched=0). Global-feed
collision coverage stays in feed_full_did_slug_dropped_for_anon and
feed_truncated_key_slug_dropped_for_anon.
* docs(node): correct ref-update slug comments for the #141 canonical form
The rebase onto #141 changed the emit-side wire slug from the last-segment
form to normalize_owner_key (bare short key for a single-segment did:key,
full DID otherwise). Four comments still described the old behavior:
- visibility.rs ref_update_row_names_repo: the emitter builds the slug via
normalize_owner_key, not {last-segment}/{name}. The next_back reduction on
the matcher is what tolerates the untrusted/crafted short form; it does not
mirror an already-stripped emit form.
- events.rs list_repo_events gossip half: the residual collision the per-row
filter still guards is a did:key canonical owner vs its bare short-key
mirror (both normalize to the same slug), not two did:web owners (#141
gives those distinct slugs).
- the two did:web feed-visibility tests had their canonical/crafted framing
backwards: post-#141 the full-DID slug is the emit form and the short slug
is the crafted one.
Comment-only; no behavior change.
* test(node,events): add load-bearing two-repo owner-key collision deny
A public bare-key mirror (z6MkX) and a private did:key canonical repo
(did:key:z6MkX) normalize to the same owner key post-#141, so their gossip
rows share the z6MkX/... slug space on the untrusted wire. Assert an anon
global-feed read sees the public mirror's row but NOT the private canonical
sibling's: the shared gate keys on the full slug (owner + name), so a
readable public repo under an owner key does not unlock that owner's other
private repos' rows. The removed repo-scoped did:web collision test never
covered this pair.
Verified load-bearing: disabling the per-row visibility drop serves
z6MkX/secret to anon (RED); the gate drops it (GREEN).
* fix(node,events): keyset-paginate the ref-update feed collector
collect_visible_ref_updates paged received_ref_updates with LIMIT/OFFSET,
so a concurrent gossip/notify insert (which sorts to the front under
timestamp DESC) shifted every later offset and let one feed response
duplicate or skip a visible row. Switch the collector to a keyset cursor
on (timestamp, id): each page reads rows strictly older than the last row
fetched, so a newer insert lands above the window and cannot shift it.
One collector feeds the REST global, REST per-repo, and GraphQL paths, so
this covers all three; there is no client-facing cursor, so the internal
cursor tracks the last fetched row. Adds tests that OFFSET duplicates a row
under a concurrent insert and that keyset stays stable.
---------
Co-authored-by: t <t@t>
* fix(node,git-remote): gate receive-pack advertisement, sign client fetch/push
The info/refs ref advertisement was gated only for service=git-upload-pack
(the visibility check sat inside `if service == "git-upload-pack"`), so an
anonymous GET /{owner}/{repo}.git/info/refs?service=git-receive-pack returned a
private repo's branch names and commit tips, bypassing the read gate. Run the
visibility "/" check for both services.
Now that the advertisement is gated for both services, git-remote-gitlawb has to
authenticate its own requests, or a private repo's owner can neither fetch nor
push it. Sign the Phase-1 advertisement GET and the Phase-2 pack POST for both
services over path_and_query when an identity keypair is present; public repos
stay anonymous. This also repairs a pre-existing break where an owner could not
fetch their own private repo, because the upload-pack POST was sent unsigned.
Tests: node-side gate (anon denied, owner-signed cleared, both services); a
real-signature seam proving the client's own signature verifies under the node's
verification primitives; real served-content withholding against an on-disk repo
(the owner gets the ref bytes, a denied caller's 404 omits them); and mockito
coverage that the client attaches and omits signatures correctly.
* docs(node,git-remote): correct auth-contract comments on receive-pack gating (#119)
receive-pack POST is signature-required and separately owner-gated, not
visibility_check-gated; push implies read, not read-subset-of-push.
* test(node): assert unsigned receive-pack POST is rejected (#119)
No test covered the signature-required half of the receive-pack auth
contract. Wire the POST route behind require_signature as production does and
assert an unsigned POST returns 401 before reaching the handler; 401 (not the
handler's 404/500) proves require_signature rejected it pre-handler.
* test(node): RAII-clean the served-refs test's temp dirs on panic (#119)
advertisement_serves_real_refs_only_to_authorized_callers created /tmp/<slug>/<name>.git and /tmp/gl-seam-src-<short> and only removed them after the final assertion, leaking those dirs on the runner when an earlier assertion failed. Wrap each known path in a DirGuard whose Drop calls remove_dir_all, bound right after creation, so cleanup runs on success or panic. tempfile::TempDir does not fit here because repo_store::for_testing fixes the path layout.
* test(node): drop git_info_refs from KNOWN_UNGATED now that #119 gates it
The repo-scoped egress guard's allowlist (added in #122, merged to main
after this branch was opened) still listed git_info_refs as ungated. This
branch makes the info/refs visibility gate unconditional for both
git-upload-pack and git-receive-pack, so the handler is now gated and the
staleness assert requires removing it from the allowlist.
* style(node,git-remote): tidy #119 review nits (em dashes, dead binding, test header)
Post-rebase ce-code-review pass on #119. No behavior change:
- replace em dashes in the added comments with commas/colons/parentheses
- drop the dead `let _ = COVERED_COMPONENTS;` binding (and its import); the
`missing_components().is_empty()` assert already covers the intent
- add a section header above the new git-info-refs test block to match the
module's sectioning convention
* test(node): recurse nested api modules in the egress-gate guard
The completeness scan in every_repo_scoped_handler_is_gated used a flat
read_dir over src/api, so a future api/<module>/mod.rs could add a
repo-scoped handler the scrape never inspects. Walk the api tree recursively
via a new collect_rs_files helper (covered by its own unit test), skipping
only the top-level mod.rs so nested module files are checked too.
* fix(git-remote): keep public fetches anonymous, sign only on a 404 retry (#119)
git-remote-gitlawb signed every git-upload-pack request whenever an identity
existed, disclosing the caller's DID to any node it fetched a public repo from,
even though public repos need no auth. Fetch now sends the advertisement GET
anonymously and escalates to a single signed retry only when the node denies it
with 404 (how it withholds a private repo from an unauthenticated reader); if the
retry was needed, the upload-pack POST is signed too. Push (git-receive-pack) is
unchanged and signs from the first request, since a push can never be anonymous.
Verified against the node by execution: an unsigned private fetch returns 404,
not 401, and a signed retry clears the read gate for the owner and for a granted
non-owner reader, while a signed non-reader stays 404.
Tests drive handle_connect end to end (recording mock): public fetch stays
anonymous for GET and POST, private fetch retries signed once and signs the POST,
a keyless fetch does not retry, a still-denied signed retry surfaces 404 once, and
push signs from the first request.
* fix(node): scan nested api sources by relative path in the egress guard (#119)
The egress-gate completeness scan skipped already-covered files by basename
(listed.contains(fname)), so a nested api/<module>/repos.rs would be skipped
because the top-level repos.rs is listed, letting a future nested module add an
ungated repo-scoped handler the scrape never inspects. Extract the skip logic into
unlisted_source_files and match on the path relative to api_root, so only the exact
top-level files already covered are skipped and a nested file with a colliding
basename is still scanned. Completes CodeRabbit's outstanding request.
---------
Co-authored-by: t <t@t>
gossip_task built its own reqwest::Client::new(), whose default redirect
policy follows up to 10 redirects, and used it for the bootstrap announce
and the periodic peer /health ping. A peer host (public at announce time,
per is_public_http_url) could answer /health with 302 Location:
http://169.254.169.254/... and the node would follow it into cloud
metadata or internal addresses. Blind SSRF: only request status feeds
mark_peer_ping.
Reuse the shared state.http_client (Policy::none, already on AppState),
matching sync.rs and the announce fan-out hardened in #78. Extract
ping_peer_health so the redirect-not-followed behavior is covered by a
mockito test (302 to an internal target registered expect(0), asserted
not hit).
* fix(node): prefer canonical repo row over mirror row in get_repo (#124)
get_repo matched both mirror rows (bare short-DID, is_public=true, no
visibility rules) and canonical rows (full did🔑 DID, correct rules),
returning one nondeterministically. When the mirror row won, the
visibility gate saw is_public=true + empty rules, allowing unauthenticated
read of private/withheld content.
Add ORDER BY to prefer canonical rows (UUID id, no slash) over mirror
rows (slash-form id), matching the ranking logic already used by
DEDUP_CTE and dedupe_canonical_repos.
Test seeds mirror FIRST so an unordered fetch_optional returns the
mirror row — proving the test locks in the fix.
* fix(test): date canonical row after mirror to guard CASE term in get_repo fix (#124)
* fix(node): tighten get_repo owner matching to did:key only, not any DID method (#124 P2)
* fix(node): refactor owner DID normalization to a dedicated function in create_repo
* fix(node): align clone URL and slug construction with tightened did:key matching
- Make normalize_owner_key pub(crate) and reuse it in api/repos, api/events, api/peers, and visibility modules.
- Add read-gate authorize_repo_read and to_response clone_url slug tests.
* fix(node): hoist owner-key CASE into shared const + address #124 review feedback
- Hoist 7 verbatim SQL CASE copies into OWNER_KEY_CASE_SQL const
(dedup_cte(), get_repo, count_repos_deduped), keeping one literal
copy in the v7 migration with a cross-reference comment.
- Add 9 unit tests for normalize_owner_key over the full boundary set.
- Add LIMIT 1 to get_repo query for single-row intent.
- Add non_key_owner_bare_short_does_not_match is_owner regression test.
- Update is_owner doc to reflect did:key-only rule.
- Document non-key clone_url disk-path constraint in to_response.
* test(node): add SQL round-trip test for normalize_owner_key vs OWNER_KEY_CASE_SQL
Verifies every boundary value produces identical results from the
Rust function and the SQL CASE expression, preventing silent drift.
---------
Co-authored-by: Gravirei <[email protected]>
* fix(node): gate GET /ipfs/{cid} on reachable allowed-set, not deny-set (#126)
The IPFS visibility gate used withheld_blob_oids (a deny-set enumerating
only reachable blobs), so a dangling/unreachable blob was absent from the
set and served in cleartext to anonymous callers. Flip to an allowed-set
(allowed_blob_set_for_caller) that enumerates reachable blobs the caller
may read: a dangling blob has no path, is never in the set, and 404s.
* perf(ipfs): check object existence before allowed-blob walk
Move store::read_object before the allowed_blob_set_for_caller
spawn_blocking call so random-CID spray against repos with
path-scoped rules cannot trigger full-history git walks on
repos that don't carry the object.
* refactor(ipfs): improve formatting and readability in get_by_cid function
✓ P3 blocker fixed: cargo fmt applied — the format gate will pass.
✓ P3 cleanup resolved: withheld_blob_oids is still used by replication code in repos.rs, so it stays.
• P2 follow-up: Tree/commit disclosure tracked in #135 — out of scope here.
* refactor(ipfs): streamline object retrieval by separating type and content reading
* docs(ipfs): update get_by_cid comment to reflect split object retrieval
* Run cargo fmt on store.rs
---------
Co-authored-by: Gravirei <[email protected]>
The AWS EC2 node (manila) was upgraded to v0.4.0 and enabled iCaptcha
enforce by hand-editing its running /opt/gitlawb/compose.yaml — which would
be lost on instance replacement or a user-data re-run. Codify it so rebuilds
stay enforced:
- variables.tf: add icaptcha_mode (default "enforce"), icaptcha_pubkey
(default the pinned live icaptcha.gitlawb.com Ed25519 key), and
icaptcha_required_level (default 3).
- main.tf: pass the three vars into the compose templatefile render.
- compose.yaml.tftpl: set ICAPTCHA_MODE / ICAPTCHA_PUBKEY /
ICAPTCHA_REQUIRED_LEVEL on the node service environment.
Operators can override per-deployment via terraform.tfvars.
- bootstrap-peers.json: add node2, node3, and manila (manila.gitlawb.com)
to the canonical seed list alongside node.gitlawb.com, and normalize the
Gitlawb operator label to "Gitlawb". DIDs left null — bootstrap merges
peers on http_url and the parser treats did as optional.
- .gitignore: ignore infra/fly/node1.fly.toml. node1's production Fly
config is pulled from Fly (fly config save), not tracked in the repo.
* test(node): guard that every repo-scoped handler is visibility-gated
Mirrors the existing every_in_scope_mutation_has_its_gate source-string CI test on
the egress side: scrapes every repo-scoped handler (any pub/pub(crate) async fn
taking Path<(String,String..)>) and asserts it carries an authz marker, either a
read gate (authorize_repo_read / visibility_check) or a write/owner/self gate
(require_repo_owner / require_owner / did_matches / &auth.0), or is listed in
KNOWN_UNGATED with its tracking issue. Verb-agnostic on purpose: a repo-scoped read
named outside list_/get_ (git_info_refs, replicate_encrypted_blobs, withheld_paths)
is still checked, so a new ungated egress of any name fails CI instead of shipping
as the next leak.
A completeness scan over src/api/ asserts every module declaring a repo-scoped
handler is wired into the guard, so a whole new file cannot add an ungated handler
the scrape never looks at. KNOWN_UNGATED is the live gap list (certs/issues/labels/
bounties/stars per #120, plus events/webhooks/replicas/protected pending PR #113);
the staleness assert forces each entry out the moment its gate lands. A count floor
trips if the scrape silently breaks, and unit tests pin handler_names and
is_repo_scoped.
Verified by execution: removing an allowlist entry trips the missing-gate assert;
listing a gated handler trips the staleness assert; dropping a source file trips the
completeness scan; breaking the parser trips the count floor; and a non-list/get
handler (git_info_refs) is confirmed scraped and gate-checked.
* test(node): don't let a service-conditional gate satisfy the egress guard (#122)
The guard counted a handler as gated on bare substring presence of a gate
marker, so git_info_refs false-passed on visibility_check( even though that
check only runs under `if service == "git-upload-pack"` and leaves the
git-receive-pack advertisement ungated. Add gate_runs_unconditionally, which
ignores markers that sit only inside an `if service ==` block, and track
git_info_refs in KNOWN_UNGATED until #119 makes the gate unconditional. When
that lands, the staleness assert forces the entry's removal.
* fix(review): harden gate_runs_unconditionally against unclosed blocks (#122)
Code review of the egress-guard hardening:
- Clamp the span-advance index so an unclosed `if service ==` block can't
slice past the body end and panic (latent: real source is balanced, but the
scraper should not crash on a pathological future body). Keep span-to-EOF as
the fail-safe direction.
- Document that only `if service ==` is detected, not `match service`, so a
future match-arm gate isn't silently treated as full.
- Add unit cases: gate inside both service blocks (conditional), gate inside
and outside (full), and the unclosed-block no-panic regression.
Phase 2 of the iCaptcha work: extend the proof-of-intelligence gate from
direct writes to federated propagation, so a node that enforces iCaptcha
can defend against spam repos that originate on a node which does not.
The proof now travels with the repo and is re-verified at mirror time:
- Capture & persist: create_repo/fork_repo store the verified proof
(raw token + claims) against the new repo. New repo_icaptcha_proofs table.
- Serve: GET /api/v1/repos/{owner}/{repo}/icaptcha-proof returns the stored
token, gated on whole-repo "/" read like the other replication endpoints.
- Admit at mirror time: the sync worker fetches the origin's proof and
icaptcha::admit_mirror re-verifies it offline — signature + level +
owner-DID binding, but NOT expiry — and consumes the jti in the local
ledger so one solved challenge admits at most one mirror per node.
- Quarantine: in enforce mode a mirror that fails the gate is kept on disk
but hidden from clone/serve/push (authorize_repo_read + the git serve and
receive-pack handlers) and from listings (dedup CTE + federated list), via
a new repos.quarantined column. Shadow logs only; off admits normally.
Fail-safe: off/shadow/no-key always admit.
Hardening from review:
- git-receive-pack also rejects quarantined mirrors (no writes while hidden).
- mirror admission keys "first-time" on DB-row absence, not on-disk presence,
so a clone that crashed before the DB insert is still gated on retry.
- mirror jti is spent against a fixed forward retention window, not the
proof's already-past exp, so the replay record isn't swept early.
Known limitation: an iCaptcha proof binds to the owner DID, not the repo, so
a malicious origin can get one spam repo per harvested public proof admitted
(per-node jti single-use bounds the amplification). Fully binding a proof to
a repo needs a repo-scoped iCaptcha challenge (protocol change, future work).
The operator release HTTP endpoint is deferred (no admin-auth surface yet);
DB methods are in place.
Tested: verify_token expiry-skip + signature/level/DID enforcement on the
propagation path; proof round-trip; quarantine set/release, preservation
across re-sync, exclusion from listings; mirror jti retention survives a
sweep at the proof's exp. Full workspace green against Postgres.
release-type: rust at path . is incompatible with this virtual
workspace root. release-please's Rust strategy unconditionally appends a
CargoToml update for the root Cargo.toml, and that updater throws
"is not a package manifest (might be a cargo workspace)" whenever the
manifest has no [package] section — so every run died on the final step.
Switch to release-type: simple (touches only a version file + CHANGELOG,
never a manifest) and bump each crate version through the generic updater
via extra-files plus an inline x-release-please-version annotation.
Preserves the single vX.Y.Z tag, all action outputs, and the entire
downstream release.yml unchanged.
release-please's Rust workspace strategy versions each member crate in its own
Cargo.toml and cannot rewrite an inherited `version.workspace = true`, so it
failed with 'value at path package.version is not tagged'. Give each of the five
crates an explicit version = "0.3.9" (kept in lockstep by release-please) and
drop the now-unused [workspace.package].version. Inter-crate deps are path-only,
so no version requirements need syncing.
* fix(node): gate GET /ipfs/{cid} on per-caller path-scoped visibility (#110)
get_by_cid served any git object by raw SHA-256 with no auth and no
visibility check. For a public repo with a path-scoped deny rule, an
unauthenticated caller could recover a withheld blob's oid from the
served trees, derive its CID, and fetch the cleartext here, bypassing
the pack-path withholding.
Bind optional caller identity (optional_signature on a dedicated
/ipfs/{cid} sub-router; the pins route stays unsigned, tracked by #121)
and gate each iterated repo row against its OWN rules: visibility_check
at "/" denies the repo, and withheld_blob_oids skips a blob withheld
from the caller when the row carries path-scoped rules. The row is gated
directly rather than via authorize_repo_read, whose fuzzy get_repo
re-resolve could authorize a different physical row than the one read.
The per-repo withheld set is memoized on repo.id within the request.
Adds get_by_cid to the authz_guard drift table (visibility_check marker).
Scope: this closes the direct unauthenticated scan. A stale-public
mirror row still serves withheld content on both this path and the pack
path; that pre-existing leak is tracked separately in #124.
* fix(review): batch visibility-rule lookup and fail-closed walk panic in get_by_cid
Code-review follow-ups for #110:
- Replace the per-repo list_visibility_rules call with a single
list_visibility_rules_for_repos batch query (the existing #97 pattern),
removing the N+1 on an unauthenticated scan and the "one repo's DB error
500s the whole request" semantics.
- Fail closed on a spawn_blocking JoinError (walk task panic) by skipping the
repo, matching the walk-error arm, instead of 500-ing the whole scan.
- Add a regression test for the repo-level "/" deny branch (private repo,
anon denied / owner served) and note the access gate in the module doc.
* test(node): cover the fail-closed walk-error arm in get_by_cid
Induce a withheld_blob_oids walk error (a ref pointing at a non-tree-ish
blob, same technique as visibility_pack::fails_closed_when_a_ref_cannot_be_traversed)
and assert the handler skips the whole repo: the withheld blob 404s with no
leak, and the public blob in the same repo also 404s (proving fail-closed-skip
rather than serve-on-error). Verified load-bearing: mutating the Ok(Err) arm to
treat the error as no-withholding makes it RED (secret served 200).
* ci(release): add Windows target and cross-platform packaging
Add x86_64-pc-windows-msvc to the release matrix. Windows ships a .zip
(7z) while unix keeps .tar.gz; a portable sha256 helper handles
sha256sum/shasum/certutil. Smoke test and packaging now resolve the
.exe suffix on Windows. Release attach uses globs to cover both formats.
* feat(release): vendor npm wrapper and auto-publish to npm
Vendor the @gitlawb/gl npm wrapper (optionalDependencies platform-package
pattern) into npm/, replacing the standalone gl-npm repo. A new npm-publish
job downloads the four unix release tarballs from the GitHub release, lays in
the binaries, stamps every package.json with the release version, and runs
npm publish --provenance. Guarded on NPM_TOKEN so it no-ops until the secret
is configured.
* feat(release): auto-bump Homebrew tap on release
Add a homebrew-bump job that regenerates Formula/gl.rb in Gitlawb/homebrew-tap
from the release tarballs' sha256 checksums (macOS+Linux, arm64+x64) and pushes
the bump. Guarded on HOMEBREW_TAP_PAT so it no-ops until the secret and tap repo
exist. Enables 'brew install gitlawb/tap/gl'.
* feat(release): Windows installer and web auto-sync on release
Add install.ps1 (Windows PowerShell installer, source of truth in node).
Add a web-sync job that opens a PR to Gitlawb/web on each release syncing
install.sh + install.ps1 + public/version.json, plus idempotent drift fixes
(remove orphaned public/bin, fix stale skill.md releases link). Guarded on
WEB_SYNC_PAT. Vercel deploys on merge.
* docs(readme): document npm, Homebrew, and Windows install methods
* ci(release): build only CLI binaries on Windows
The gitlawb-node daemon targets Linux/macOS; building it on Windows risks
a compile failure that would block the whole release (npm/brew/web jobs all
depend on release-binaries). Windows now builds only gl + git-remote-gitlawb;
the smoke-test and package loops follow the per-target binary list.
* fix(release): reconcile release-please manifest with v0.3.9 and bump action to v5
The manifest still said 0.3.8 while the only tag is a manual v0.3.9 and
Cargo.toml is 0.3.9, so release-please had no valid baseline (no v0.3.8 tag)
and never cut a release. Set the manifest to 0.3.9 so release-please diffs
v0.3.9..main and opens a proper release PR for the next version. Also pin
release-please-action to v5.0.0 to match the openclaude pipeline.
* ci(release): set explicit minimal permissions on homebrew-bump and web-sync
CodeQL flagged both jobs for lacking a permissions block, leaving GITHUB_TOKEN
at broad default scopes. Both only need contents: read (cross-repo writes use
dedicated PATs).
* fix(release): harden installers and workflow per review
- install.ps1: fail closed if the .sha256 cannot be fetched; pure ASCII (no BOM)
- npm install.js: fail (exit 1 / throw) when the platform payload or a binary is
missing instead of reporting a successful but broken install
- workflow: top-level permissions {} + persist-credentials: false on read-only
checkouts
- docs: PowerShell-only Windows note; text language tag on npm/README fenced block
* fix(installer): require core CLI binaries on Windows
install.ps1 only failed when zero binaries were present, so a partial archive
missing gl.exe could still report success. Require gl.exe + git-remote-gitlawb.exe
(gitlawb-node.exe stays optional, matching the Windows build) and fail early if a
required binary is absent.
* ci(release): make Windows non-blocking and stop persisting cross-repo PATs
- release-binaries: continue-on-error for the Windows leg so a failure there
no longer fails the job or blocks npm/Homebrew (matches the best-effort intent)
- homebrew-bump/web-sync: persist-credentials: false on the PAT-backed checkouts;
supply the token only on the git push via an x-access-token URL
* fix(release): gate web-sync on binaries and detect native arch in installer
- web-sync now needs release-binaries so the site never advertises a release
whose artifacts failed to build
- install.ps1 reads PROCESSOR_ARCHITEW6432 before PROCESSOR_ARCHITECTURE so a
32-bit PowerShell host on x64 Windows is not wrongly rejected
* ci(release): make npm publish idempotent across reruns
Skip any @gitlawb package version already on the registry so rerunning the
job after a partial publish no longer errors on the first existing package
and leaves the wrapper/platform packages out of sync.
* fix(release): harden web-sync and tap commit per review
- web-sync: verify the Windows ZIP + .sha256 exist on the release before
syncing install.ps1/version.json (Windows is best-effort, so a failed leg
must not make the site advertise a missing installer)
- stage generated files before the no-op check so brand-new untracked files
(formula, version.json) are committed instead of skipped
- gh pr create: only swallow the 'PR already exists' case; surface auth/API errors
- npm README: scope npm/yarn/pnpm/bun install to macOS/Linux/WSL, point native
Windows at the PowerShell installer
* ci(release): guard npm-publish before doing any work
Check NPM_TOKEN in an up-front guard step (like homebrew-bump/web-sync) and
skip checkout, release downloads, extraction, version rewrites, and publish
when it is absent, instead of doing all that work and bailing in Publish.
* fix(release): robust web PR error handling and Windows installer hardening
- web-sync: capture gh pr list output so a real API/auth failure surfaces
instead of being misread as 'PR already exists'
- install.ps1: enforce TLS 1.2 and use -UseBasicParsing so downloads work on
stock Windows PowerShell 5.1; remove binaries no longer shipped on upgrade so
an in-place update can't leave a mixed-version install; handle an empty user
PATH without a leading separator
* feat(node): iCaptcha proof-of-intelligence gate on create_repo + register
Add an optional gate that requires callers to present an iCaptcha proof (an
Ed25519-signed token from icaptcha.gitlawb.com, earned by solving an escalating
challenge) on the two spam-prone endpoints. Proofs are verified OFFLINE with
ed25519-dalek against the published public key, and bound to the authenticated
agent DID so they cannot be shared.
Backward-compatible by default: ICAPTCHA_MODE is `off` unless set, so deploying
changes nothing. `shadow` verifies and logs would-be rejections but always
allows; `enforce` rejects requests lacking a valid, strong-enough proof. If the
public key cannot be loaded at startup the gate stays inert (fail safe) so an
iCaptcha outage can never break repo creation or registration.
- New src/icaptcha.rs: env-driven verifier (OnceLock), /v1/pubkey fetch at
startup or ICAPTCHA_PUBKEY override, proof parse + signature + expiry + level
+ DID-binding checks; mode-aware decide() split out for testing.
- Wired into create_repo and register (X-ICaptcha-Proof header). git push
(receive-pack) is intentionally NOT gated.
- 11 unit tests incl. verification of a real captured proof (JS-signed,
Rust-verified), tamper/expiry/wrong-DID/level/missing-header rejections, and
the off/shadow/enforce/fail-safe mode matrix.
Author: Kevin Codex <[email protected]>
* fix(node): address review on icaptcha gate (validate config, bound key fetch)
Addresses CodeRabbit and @jatmn review feedback on #108:
- parse_mode now returns Option and rejects unrecognized ICAPTCHA_MODE values;
init() warns and falls back to off instead of silently disabling the gate on
a typo like "enforced". [P2]
- ICAPTCHA_REQUIRED_LEVEL: warn on a non-empty unparseable value instead of
silently lowering the threshold to the default. [P2]
- fetch_key uses a reqwest client with a 5s timeout so a hung /v1/pubkey can
never block node startup; still fails inert on timeout/error. [P2]
- .env.example: reorder ICAPTCHA_* keys (MODE, PUBKEY, REQUIRED_LEVEL, URL) to
satisfy dotenv-linter. [P3]
- add parse_mode unit test (rejects junk, accepts documented values).
cargo test -p gitlawb-node icaptcha: 12 passed. clippy + fmt clean.
Author: Kevin Codex <[email protected]>
* fix(node): consume iCaptcha proof jti so a proof can't be replayed
Addresses @jatmn P2 on #108: the verifier checked exp/level/sub but never
consumed the proof's jti, so one solved challenge was a reusable bearer proof
for that DID until expiry, covering every gated create_repo/register in the
window. Now, in enforce mode, each verified proof's jti is recorded once and
replays are rejected.
- db: migration v7 adds icaptcha_consumed_proofs(jti PK, expires_at); new
consume_proof_jti (INSERT ON CONFLICT DO NOTHING -> bool) and
sweep_expired_proofs. DB-backed so it holds across restarts and instances.
- icaptcha: ProofClaims carries jti; verify returns the claims; decide() stays
pure and returns Allow | Reject | Consume{jti,exp}; check() is now async and
spends the jti via the DB, rejecting replays (enforce only; shadow observes
without consuming).
- handlers: create_repo/register await check(&state.db, ...).
- main: periodic sweep of expired proof rows alongside rate-limit cleanup.
- tests: 12 pass incl. decide() yields Consume with the proof's jti.
Author: Kevin Codex <[email protected]>
* fix(node): consume iCaptcha proof only after request passes validation
Addresses CodeRabbit (Major) on #108: check() spent the jti as soon as the proof
verified, but create_repo/register ran it before name/DID/existence checks, so a
rejected request (bad name, already-exists, wrong DID) permanently burned a
valid proof without the action succeeding.
Split verification from consumption:
- icaptcha: replace check() with verify_request() -> ProofGuard (verifies and
rejects invalid/missing proofs early; off/shadow/inert yield a no-op guard)
plus ProofGuard::consume(db) which spends the jti right before the write and
rejects replays. Guard is #[must_use].
- create_repo/register: verify_request() up front, then consume() immediately
before the first write, after cheap validation passes.
Replay protection unchanged (consume-before-write). 12 tests pass; clippy clean.
Author: Kevin Codex <[email protected]>
* fix(node): give iCaptcha rejections a dedicated error code
Addresses CodeRabbit on #108: iCaptcha failure paths returned
AppError::Unauthorized, which maps to the "not_an_agent" API code and conflates
a valid-but-unproven agent with a failed signature. Add a dedicated
AppError::IcaptchaProofRequired variant mapped to the "icaptcha_proof_required"
code (HTTP 401), and use it for both the missing/invalid-proof and replay paths
so clients can distinguish "solve a challenge" from "your signature is invalid".
12 tests pass; clippy + fmt clean.
Author: Kevin Codex <[email protected]>
* fix(node): gate fork_repo with iCaptcha and cover the proof ledger
Address remaining PR #108 review comments:
- fork_repo was the third repo-creation entrypoint and bypassed the
iCaptcha gate. Verify the proof up front (fail fast on missing/invalid)
and consume it only just before the git clone, so cheap rejections
(bad name, conflict, withheld subtree) never burn a valid proof —
matching create_repo's verify/consume split.
- Add #[sqlx::test] coverage for the single-use ledger that the gate's
anti-replay value depends on: consume_proof_jti (first-use vs replay vs
distinct jti) and sweep_expired_proofs (only expired rows removed,
unexpired spent proofs still block replays).
- Return 403 Forbidden (was 401) for IcaptchaProofRequired: the caller is
authenticated but forbidden without a valid proof. The distinct
icaptcha_proof_required code is unchanged.
- Recommend pinning ICAPTCHA_PUBKEY in production in .env.example to
avoid the startup fetch dependency.
The two shadow-mode review comments were already resolved by the earlier
verify/consume refactor (decide/verify are IO-free; check_proof_jti gone).
Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <[email protected]>
* fix(node): NFC-normalize visibility matcher to close NFC/NFD under-withholding (#101)
A deny rule authored in NFC byte-compared unequal to a path committed in NFD
(or vice versa), so blob_paths/withheld_blob_oids failed to classify the blob
and it shipped in cleartext on the replication/pack path. Normalize both the
glob prefix and the candidate path to NFC at the single matcher seam
(glob_matches + specificity); NFC not NFKC so compatibility forms are not
folded into distinct paths. Live serve is unaffected (git resolves byte-exact;
the gate only becomes more restrictive). Adds a matcher unit test and a
replication-path integration test with the variant byte inside the
rule-covered directory name.
* fix(node): fail closed on the full-scan pin path to stop dangling-blob leak (#99)
The withheld set is computed over reachable objects (rev-list --all + HEAD),
but the full-scan pin fallback (cat-file --batch-all-objects) includes dangling
objects the reachable walk never classified. Subtracting the reachable-only
withheld set therefore could not catch a private blob orphaned by a
force-push/amend, which pinned to IPFS/Pinata in cleartext.
Invert the default for blobs on the full-scan path: a candidate replicates only
if it is a non-blob (commits/trees are structural) or a reachable,
visibility-allowed blob; anything else (a dangling blob, or a withheld one) is
dropped. The delta path keeps the cheaper reachable-only subtraction since delta
candidates are always reachable, so no extra walk on the hot path.
Adds list_all_objects_with_type/all_blob_oids (typed enumeration),
replicable_blob_set + replicable_objects_fail_closed, and threads a full_scan
flag through resolve_candidates_for_push. Pin functions now take a pre-filtered
object list. Regression tests: dangling blob excluded end-to-end, fail-closed
unit semantics, and the all-blob universe includes dangling + excludes
non-blobs.
* fix(node): gate repo-listing surfaces on visibility to stop private-repo enumeration (#97)
list_repos, list_federated_repos, and the GraphQL repos query returned every
repo row with no visibility check, so an unauthenticated caller could enumerate
private repos' name, owner, description, and clone URL — and X-Total-Count
leaked the private-repo count even when rows were filtered.
Each surface now keeps a row only when visibility_check at "/" allows the
caller, the same decision the per-repo content endpoints make — not a bare
is_public filter, which would diverge for is_public=false-with-root-reader and
is_public=true-with-root-deny. The "/" decision depends on owner short/full-DID
matching and JSON reader-DID membership, so it is applied in Rust over an
over-fetched set rather than pushed into SQL; the paged path derives
X-Total-Count from the filtered set (dropping the SQL window count) and
paginates after filtering, so neither the page nor the count leaks. Adds a
batch root-rule query (list_visibility_rules_for_repos) to avoid N+1, and
converts list_all_repos_paged into list_all_repos_deduped (mirror dedup
preserved, pagination moved to Rust).
Tests: anonymous hides private repo + count, owner sees own private repo,
authorized root-reader sees an is_public=false repo (proving the visibility_check
path, not is_public), and federated listing hides private from anonymous.
* fix(review): share the listing visibility seam, extract fail-closed pin helper, add listing tests
Code-review follow-up on the VIS-subtree fixes (no behavior change to the three
shipped fixes; all confirmed sound by review):
- DRY the '/' listing gate into one shared visibility::listable_at_root used by
list_repos, list_federated_repos, and the GraphQL repos query, so the three
surfaces enforce one rule instead of three drifting copies
(enforce-shared-rule-on-every-reader-path).
- Extract the full-scan fail-closed pin block into fail_closed_full_scan_objects,
mirroring replication_withheld_set's degraded-path shape; drop the duplicated
comment and the per-site no_rules sentinel (use &[]).
- Add the three listing tests the review flagged as missing: anonymous GraphQL
repos hides a private repo, the paged X-Total-Count path excludes the private
count (the KTD2 exploit shape), and an is_public=true repo under a root deny is
not listed (proves the gate is visibility_check, not a bare is_public filter).
* fix(review): route the last inline root-visibility check through listable_at_root
Round-2 review follow-up. replication_withheld_set still made the announce
decision with an inline visibility_check(.., "/") == Allow — the fourth copy of
the listing gate the seam refactor was meant to collapse. Route it through
crate::visibility::listable_at_root (caller is always None, the anonymous
replication audience), so every reader of the '/' decision now shares one seam.
Also drop an em dash from the fail_closed_full_scan_objects doc comment.
* fix(node): gate the /api/v1/stats repos count through visibility to close the count oracle (#104)
The stats endpoint returned count_repos_deduped(): an unfiltered, unauthenticated
count of every logical repo including is_public=false and mode-A hide-existence
repos. A polling observer could detect that a hidden repo was created (N to N+1),
the exact existence signal mode-A suppresses everywhere else.
Count only the repos an anonymous caller could list: over-fetch the deduped set,
batch-load visibility rules, and keep rows that pass listable_at_root(.., None),
mirroring the listing seam in api/repos.rs. The caller is always None (meta_routes
carries no auth layer). Fail closed: any DB error collapses the count to 0, since
an under-count cannot leak existence.
count_repos_deduped is retained as the tested reference for the DEDUP_CTE dedup
count (its tests pin the CASE the live list paths share); it has no production
caller after this change and is allowed dead outside tests.
Tests: bare-private (no-rule branch), hide-existence mode-A with both repos
is_public=true (the Some(rule) branch), public-under-root-deny, list-parity,
sibling-fields-preserved, and empty-DB.
* test(review): add listing visibility regression guards (#97)
Three gaps the review flagged as uncovered:
- owner-filtered listing (?owner=) still hides a private repo and its count
from anonymous (a distinct SQL bind branch from the unfiltered path).
- offset past the visible set returns an empty page while X-Total-Count stays
the full visible total (guards against deriving the total from the cut page).
- a logical repo present as both a canonical row (with a root-deny rule) and a
gossip mirror row stays hidden: pins that the DEDUP_CTE picks the canonical
survivor so the batch rule lookup finds the deny. If dedup ever picked the
mirror (slash-form id, no rule) the gate would fall back to is_public and leak.
* fix(review): address CodeRabbit feedback on #111
- visibility_pack NFC/NFD test: accept a SHA-1 or SHA-256 object id
(40 | 64) instead of hardcoding 40, matching the fixture guard later in
the file, so the test is not SHA-1-only.
- federated listing test: also assert the response `count` reflects only
the visible repos, guarding against a regression to the pre-filter total.
- add an authenticated-caller GraphQL test: an owner sees their own private
repo via `repos`, covering the resolver's caller-DID path that the
anonymous-only test did not exercise.
- stats: count via the no-stars `list_all_repos_deduped()` (same DEDUP_CTE)
instead of the stars-joined variant, since the count never needs stars.
* fix(review): match owner filter to did_matches for bare-owner mirror rows (#111)
Addresses jatmn's P2. The shared listing CTE filtered owners with
`owner_did = $1 OR owner_did LIKE '%:' || $1`, so a full `did:key:z...`
query did not match a mirror-only row stored with the bare owner `z...`.
#97 routed the no-limit owner-filtered path (used by `gl repo list`)
through this SQL predicate, regressing the full-DID-against-bare-mirror
direction that `crate::api::did_matches` handled in the old Rust path.
Normalize the bound owner to its did:key short form and compare it against
the row's owner key using the same CASE the dedup grouping already applies,
so full and short forms match symmetrically and a non-key DID still matches
only exactly. The no-arg `list_all_repos_deduped()` binds NULL and is
unaffected.
Regression test: a bare-owner mirror-only row is returned when queried by
both the full `did🔑` form and the bare short form. Confirmed RED
(test fails) before the fix.
* fix(node): gate fork_repo on per-caller path-scoped visibility (#98)
fork_repo authorized the caller only at the repo root "/" and then
git clone --mirror'd the full source, discarding the visibility rules.
A non-owner with root read but denied a path-scoped subtree could fork
the full mirror and obtain blobs the filtered git_upload_pack path
withholds.
Refuse the fork when the caller has any withheld glob (per-caller
withheld_globs non-empty), the same decision the read path makes;
owners and authorized subtree readers are unaffected. Fail closed with
a not-found response, before repo_store.acquire, so no withheld object
is materialized. Add unit coverage for the policy and a direct-mount
integration test pinning the gate wiring.
* fix(review): note fork-gate glob-grammar dependency, import withheld_globs
Address code-review findings on the #98 fork gate:
- Document that fork_withheld_blocks's prefix-probe equivalence with the
serve path depends on validate_path_glob keeping "/" the only whole-repo
scope, mirroring has_path_scoped_rule's caveat (latent-coupling guard).
- Import withheld_globs and call it unqualified, consistent with the
sibling visibility_check import in the same module.
* fix(node): anchor the real old_sha and issue a per-ref certificate (#26)
The push handler discarded the real old->new transition before anchoring
and only certified the first ref of a multi-ref push, so the permanent
Arweave record was a weaker, incomplete copy of facts the node had
already parsed.
- Thread the parsed old_sha through the clone moved into the anchor task
instead of hardcoding a 64-zero placeholder. This also fixes the SHA
width mismatch: object ids are 40-hex SHA-1, not 64-hex.
- Issue a signed RefCertificate for every ref the push advanced, each
carrying that ref's real old/new, rather than one cert for the first
ref (which, on an empty push, signed a timestamp string as new_sha).
Lock the anchor-body contract with a test asserting the upload carries
the real old_sha and new_sha, never a placeholder.
Scope is the integrity subset only. Carrying the signed certificate on
Turbo, capturing the pusher signature, chaining certificates, and
gl-side verification remain open under #26.
Refs #26
* fix(node): thread real old_sha through every ref-update fan-out path
PR #72 made the Arweave/DB anchor path per-ref with the real old_sha but
left three sibling fan-out paths in git_receive_pack flattening multi-ref
pushes and/or emitting a placeholder old_sha. Bring them in line:
- gossip publish_ref_update now forwards the real old_sha (was ""), matching
the RefUpdateEvent contract that documents all-zeros for a created ref
- HTTP /sync/notify sends one signed request per ref with its real old_sha;
the receiver is single-ref, so a multi-ref push no longer drops refs 2..N
- GraphQL RefUpdateBroadcast emits one broadcast per ref with the real
old_sha instead of only the first ref + a zero placeholder
No receiver, struct, or wire-schema change; the all-zeros-for-created-ref
convention is preserved on the wire. Addresses jatmn's CHANGES_REQUESTED and
the two CodeRabbit findings.
* test(node): regression-guard the per-ref sync-notify fan-out
Extract the HTTP /sync/notify send into notify_peer_of_ref (one ref) and
notify_peer_of_refs (one peer, looping refs), collapsing the deeply-nested
inline block in git_receive_pack and giving the fan-out a unit-testable seam.
The ref_name is now also logged on every warn arm.
Add mockito regression tests: a two-ref push must emit one request per ref
each carrying its real old_sha (guards against re-flattening to .first() with
a zero placeholder, the bug PR #72 fixed), and a created ref must forward the
all-zeros hash verbatim. Behavior is unchanged; this only adds the seam, the
tests, and ref_name in the logs.
* style(node): match sibling test conventions in repos tests
Prefix the two new sync-notify tests with test_ and use the _mock binding
convention (matching arweave.rs/pinata.rs). No behavior change.
* style(node): fix rustfmt import order and dedupe the zero-sha literal
- Reorder the repos test-module imports so `use crate::...` precedes the
external `gitlawb_core` import (unblocks cargo fmt --check / CI).
- Webhook `created` check reuses the existing ZERO_SHA constant instead of
repeating the 40-hex literal.
- Arweave test fixtures use the canonical 40-hex zero old_sha, matching the
SHA-1 width the per-ref anchoring contract now carries.
* perf(node): run peer notification after local broadcast and anchoring
The post-push background task awaited the HTTP peer-notify fan-out before
the GraphQL subscription broadcast and Arweave anchoring. With per-ref
notification a slow or unreachable peer could delay both local subscribers
and permanent anchoring by the sum of the peer round-trips. Move the
peer-notify loop to run last; peers already learned of the push via the
gossip publish earlier in the task, so this only deprioritizes the
best-effort HTTP fallback.
* style(node): correct notify-ordering comment and reuse ZERO_SHA in test
Review follow-ups on the two prior commits:
- The peer-notify ordering comment claimed peers 'already learned via the
gossip publish', but that publish is gated on a present p2p handle; with
no handle (the very case the HTTP fallback exists for) it does not run.
State the actual rationale: peer-notify is the lowest-priority best-effort
step, ordered last so it cannot delay local broadcast or anchoring.
- The all-zeros notify test still inlined the 40-hex literal; point it at the
ZERO_SHA constant like the production path.
* fix(node): close under-withholding via full ref scope and reachable-only pin set
blob_paths walked only refs/heads/* and refs/tags/* and skipped silently on a
failed git ls-tree, so a blob reachable only through another namespace, or a ref
that failed to traverse, could fall out of the withheld set and ship in cleartext.
Walk every ref and fail closed on traversal error.
The pin enumerators (ipfs_pin, pinata) used git cat-file --batch-all-objects,
which includes unreachable/dangling objects that have no path and cannot be
classified for withholding. Switch them to git rev-list --objects --all so the
pin set matches the reachable graph blob_paths evaluates.
* fix(node): include HEAD in the withheld-blob walk to close pin/serve asymmetry
blob_paths classified objects from for-each-ref only, but the pin set
(git rev-list --objects --all) and git upload-pack both also expose
objects reachable from HEAD. A detached HEAD (or any HEAD pointing
outside the ref-covered set) therefore put a blob into the pin/serve
path that the withholding walk never classified — a latent cleartext
leak of a withheld blob. Walk HEAD too (guarded for unborn HEAD on an
empty repo) so the withheld set is a superset of every exposure path.
Adds a regression test: a blob reachable only via a detached HEAD must
still be withheld (fails without this change).
* fix(node): classify blobs across full history, not just ref tips
blob_paths walked only ref-tip trees (git ls-tree -r <ref>), so a blob
that existed in an older commit and was deleted or rotated at the tip
was never classified, while the pin set (git rev-list --objects --all)
and upload-pack still expose it from history. That under-withheld the
blob into the IPFS/Pinata replication paths in cleartext.
Enumerate all reachable commits (git rev-list --all + HEAD for the
detached case) and walk each commit tree, deduping (oid, path) pairs.
ls-tree per commit is kept over rev-list --objects so every path a blob
lives at is still represented for the per-path visibility check. A ref
that resolves to a non-commit now fails closed explicitly, since
rev-list --all silently skips it where the old per-ref ls-tree errored.
Regression test: a secret blob deleted at the tip but reachable in
history is withheld.
* fix(node): share one history walk and fully peel ref types in withheld classifier
Addresses two review findings on the under-withholding classifier:
- withheld_blob_recipients walked blob_paths twice (once via
withheld_blob_oids, once directly). Extract withheld_from_pairs so a
single history walk feeds both the withheld set and the recipient map.
- The ref-type fail-closed guard used for-each-ref %(*objecttype), which
peels only one tag level and so misclassified a nested annotated tag of
a commit as a non-commit, refusing the whole walk (over-withhold). List
refnames and peel each fully with <ref>^{} through one cat-file
--batch-check; reap the child even if the stdin write fails. Fail closed
on a short read or any non-commit/missing target.
Adds tests: annotated + nested-tag of a commit does not fail closed,
annotated tag of a blob still fails closed, and a blob at both an allowed
and a denied path is not withheld.
* fix(node): drain cat-file stdout concurrently to close the peel deadlock
The ref-type guard peeled every <ref>^{} through git cat-file --batch-check,
writing all queries to stdin before draining stdout. On a repo with enough
output the child's stdout pipe fills, it stops reading stdin, and write_all
blocks forever — hanging every serve/pin that classifies the repo. Batching
the queries does NOT bound this: cat-file echoes the full query on a
'<query> missing' line, so output scales with refname length, and a few
hundred long-named dangling refs already exceed the pipe buffer.
Run the stdin write on a separate thread so the main thread drains stdout
concurrently via wait_with_output, which reaps the child unconditionally
before any error is surfaced; the writer is joined after the drain so it
cannot deadlock, and a vanished stdin pipe becomes a broken-pipe write error.
One pass, no batch bound, deadlock-free regardless of ref count or refname
length.
Extracted as assert_all_refs_are_commits with corrected doc attribution.
Detect the missing-object line by its trailing status word. Adds a
timeout-bounded regression test with 500 long-named dangling refs (>64 KiB of
cat-file output) that fails rather than hangs if the deadlock returns.
* fix(node): parse ls-tree -z raw paths and fail closed on non-UTF-8 to stop under-withholding
Plain `git ls-tree -r` C-quotes any path with non-ASCII or special bytes
(café.txt becomes "secret/caf\303\251.txt"), so the stored path no longer
matched a visibility rule like /secret/** and the blob was served/pinned in
cleartext. Switch blob_paths to `git ls-tree -rz` and split the NUL-delimited
records, which emit paths raw; the TAB field separator survives -z so the
per-record parse is otherwise unchanged.
Parse strictly with str::from_utf8 and fail closed on invalid UTF-8 rather than
from_utf8_lossy: a lossy decode would rewrite an invalid byte in a denied
directory name to U+FFFD, and the mangled path would again miss its deny rule
(the same under-withholding class one layer down).
Tests: a non-ASCII path stays withheld (and the public blob still replicates),
and a non-UTF-8 path fails closed.
* ci(triage): run PR triage on pull_request_target so fork PRs can be labeled
Fork PRs run the triage job with a read-only GITHUB_TOKEN under the
pull_request event regardless of the workflow's declared write
permissions, so addLabels/createComment fail with 403 'Resource not
accessible by integration' for every external contributor (e.g. #103).
Switch the trigger to pull_request_target, which runs in the base-repo
context with the declared write token. The job never checks out or runs
PR head code (it only reads context.payload and the pulls/files API and
manages labels), so it carries none of the untrusted-code-execution risk
that makes pull_request_target dangerous.
* ci(stale): drop needs-issue from auto-close trigger, keep it advisory
needs-issue flags PRs that link no issue, but linking one is a SHOULD in
CONTRIBUTING (required only for protocol-level changes), and many
legitimate small PRs (docs, obvious fixes) have none. Coupling it to the
14/21-day auto-close was overreach: a substantive, well-described PR
could be closed purely for missing an issue reference.
Keep needs-issue as an advisory label and guidance nudge; leave only
needs-description (an empty/thin body, a far stronger low-effort signal)
as the auto-close trigger.
* ci(stale): align stale message with description-only close condition
The auto-close trigger narrowed to needs-description, but the stale
message still told contributors a PR could close for a missing linked
issue. Drop that clause so the warning matches what actually closes a PR.
* style(ci): replace em dashes in triage/stale contributor messages
* docs(contributing): match stale close policy to description-only trigger
Auto-close is driven by needs-description only; a missing issue link stays
advisory. Align the contributor docs with the workflow so the OR wording no
longer implies a missing issue link can close a PR.
* fix(node): fail closed when a recipient DID can't be resolved (#47)
encrypt_and_pin resolved recipient DIDs with filter_map, silently
dropping any that failed to resolve and sealing the envelope to the
remaining subset while still recording the full intended recipient set
as covered. A reader whose DID could not be resolved at seal time (any
did:web/did:gitlawb, or a malformed did:key) was then permanently locked
out: the dedup compares the recorded full set against the desired set,
matches, and never re-seals.
Resolve all recipient DIDs up front via resolve_all_recipients; if any
fail, log and skip the blob rather than seal to a partial set. Nothing
partial is recorded, so the dedup stays honest and the blob re-seals on
a later push once resolution is available. Add a bounded per-blob warn
plus one aggregate coverage warn so a fully-migrated did:gitlawb org
(zero encrypted-pin coverage) is one greppable line, not a scrape, and
give the previously silent read_object and pin_git_object failure arms
warn logs.
* test(node): extract plan_seal so the #47 fail-closed gate is unit-tested
The fail-closed invariant lived only in resolve_all_recipients, tested in
isolation; nothing proved encrypt_and_pin acts on it (a refactor falling
through to a partial seal would pass every test). encrypt_and_pin needs a
live Postgres Db, which CI does not provide, so it can't be tested directly.
Extract the per-blob seal/skip decision into a pure plan_seal() -> SealPlan
(no DB, no IO), behaviour-preserving, and unit-test its branches: seals only
when all recipients resolve, fails closed on any unresolvable DID, skips the
empty set, skips an unchanged tag, re-seals a changed set.
* style(node): rustfmt encrypted_pin (Seal variant + assert_eq wrap)
* feat(node): add has_path_scoped_rule predicate for withheld-walk short-circuit
Shared predicate (rules.iter().any(path_glob != "/")) both serve and
replication paths will use to skip the per-blob withheld walk when no rule
can withhold. Doc records the "/"-gate precondition and the validate_path_glob
dependency. Adds unit tests plus a gate-aware safety invariant test.
* fix(node): skip withheld-walk when no path-scoped rule can withhold (#50)
The serve path (git_upload_pack) ran the full per-ref withheld_blob_oids
walk on every clone/fetch before falling back to a plain upload_pack, even
for repos with no path-scoped rules where the walk can never withhold
anything. Cost grew with #42 (per-ref git ls-tree -r).
Guard both call sites on has_path_scoped_rule: the serve path skips the
spawn_blocking walk entirely and serves upload_pack directly; the
replication path generalizes its is_empty() short-circuit to also cover
root-only rule sets. Safe because the whole-repo "/" gate runs before both
sites, so a denying root rule 404s/withholds upstream and never reaches the
predicate. No change to served-pack contents on the success path.
* fix(node): gate encrypt-then-pin walk on has_path_scoped_rule (#50)
The post-push encrypt-then-pin block in git_receive_pack gated the
withheld_blob_recipients call on `!r.is_empty()`, which is true for a
root-only `/` rule set. withheld_blob_recipients then runs the same
per-ref `git ls-tree` walk (withheld_blob_oids) the short-circuit is
meant to avoid, and returns an empty recipient map anyway. Gate it on
has_path_scoped_rule to match the serve and replication sites; the skip
is behavior-preserving (empty map -> encrypt_and_pin is a no-op).
Also align the two remaining call sites to the visibility_pack import
alias for consistency.
* test(node): cover the serve decision across both has_path_scoped_rule branches
Drives the git_upload_pack serve decision over a real bare repo for the
INV-2 caller (a reader allowed at whole-repo "/" but denied a path-scoped
subtree), exercising both branches the predicate selects: root-only rules
skip the walk and serve the full pack (asserting the walk would withhold
nothing, so the skip is sound), and a /secret/** rule runs the walk and
excludes the secret blob from the served set while keeping the public one.
tar 0.4.45 had a PAX header desynchronization issue (GHSA-3pv8-6f4r-ffg2,
fixed in 0.4.46); rand 0.9.2 was unsound with a custom logger using
rand::rng() (GHSA-cq8v-f236-94qc, fixed in 0.9.3). Both resolve as
lockfile-only bumps with no manifest change. Build and cargo audit pass.
lru 0.12.5 (GHSA-rhfx-m35p-ff5j) stays pinned by aws-sdk-s3 (lru ^0.12.2)
and the hickory advisories remain accepted risk per .cargo/audit.toml (#76).
x25519_secret_from_seed derived the X25519 private scalar from the Ed25519
seed and returned it as a bare [u8; 32]. Because [u8; 32] is Copy with no
Drop (and sha2's digest output likewise has no zeroize), the scalar, the
SHA-512 digest, and the returned temporary were released without being
scrubbed, leaving secret-derived material in freed memory. This is the same
Copy-no-Drop class fixed for the raw seed in #41, on the derived secret.
Return Zeroizing<[u8; 32]>, build the scalar directly into a zeroizing
buffer so no bare secret local persists, and explicitly wipe the SHA-512
digest before it drops. Callers deref the result into crypto_box::SecretKey,
which already scrubs its own copy. No behavior change: the derived scalar and
all decryption output are byte-identical, covered by the existing
ed25519_to_x25519_keypair_agrees and seal_open_round_trip_for_recipients tests.
* fix(node): dedupe mirror and canonical repo rows on list surfaces (#6)
The non-paged GET /api/v1/repos legacy path and list_federated_repos
returned both the short-owner peer mirror row and the canonical did:key
row for the same logical repo, so profiles rendered the repo twice. Only
the paged path collapsed them, in SQL.
Add a dedupe_canonical_repos helper that groups by (normalized owner,
name), keeps the canonical non-mirror row (tie broken by earliest
created_at), and carries the group's latest updated_at onto the
survivor, matching the paged SQL dedup. Apply it at both non-paged
surfaces and cover it with unit tests.
* fix(node): dedupe GraphQL + stats repo surfaces and key mirror detection on the repo id
Addresses jatmn's review on #73 (dedup not applied on every reader path; mirror
detection keyed on a user-settable string).
- GraphQL repos and /api/v1/stats now collapse mirror+canonical rows, via new
Db::list_all_repos_deduped and count_repos_deduped that share the DISTINCT ON
CTE with list_all_repos_paged so the dedup rule cannot drift.
- Mirror detection keys on the structural slash-form id (written only by
upsert_mirror_repo) instead of the description == 'mirrored from peer' string,
in both the SQL paths and dedupe_canonical_repos.
- Deterministic survivor on a full created_at tie (id ASC) in both
implementations.
- Legacy REST list and federated keep their method-scoped did_matches owner
filter in Rust; it does not compose with the method-blind SQL group key, so
those paths intentionally stay on the Rust helper.
- Adds sqlx and unit tests for the new surfaces, the structural marker, and the
tiebreak.
* test(node): cover mirror-only, empty, and count/list-consistency dedup cases
Follow-ups from code review on the dedup change:
- list_all_repos_deduped/count_repos_deduped: mirror-only group survives, empty
table returns empty/0, and count_repos_deduped equals the deduped list length
(guards the two independent SQL queries against grouping-key drift).
- Document list_all_repos as the raw, non-deduped enumeration path (object
lookup only), so it is not mistaken for a listing-surface method.
* style(node): apply cargo fmt to dedup test code
* fix(node): make the repo dedup key did:key-aware so distinct DID methods don't collapse
The dedup grouping key took the last ':' segment of owner_did
(split_part / rsplit), so two repos owned by did:key:X and
did:gitlawb:X with the same name collapsed into one logical repo on
the list, paged, count, stats, and GraphQL surfaces. That is the exact
cross-method collision the codebase already guards against in
did_matches.
Replace it with a did:key-aware key that strips a did🔑 prefix only
when the remainder is a bare id (no ':'), otherwise keeps the full DID,
reproducing did_matches/key_id as an equivalence relation: did:key:X
and a bare mirror X still collapse, while distinct methods never merge.
Applied byte-identically across DEDUP_CTE (DISTINCT ON / PARTITION BY /
ORDER BY), count_repos_deduped, the empty-page count fallback, and the
in-memory dedupe_canonical_repos, so the SQL and Rust paths agree.
The backing index lived in the already-released migration v1, so v1 is
left untouched and a new migration v7 drops idx_repos_owner_short_name
and builds idx_repos_owner_key_name on the matching expression; the
CASE must stay byte-identical to the queries or Postgres stops using it.
Tests cover both the in-memory and SQL paths: distinct methods stay
separate, bare-id and did:key forms collapse, and the residual-colon
guard keeps a malformed did:key:did:gitlawb:X distinct from the bare
method DID. 216 pass.
Webhook creation only scheme-checked the URL, so an authenticated repo
owner could register a hook pointed at loopback, private, or metadata
endpoints and make the node POST to them on repo events. Route the URL
through is_public_http_url, the same validator the peer announce path
already uses.
Wiring it up surfaced a bypass in that shared validator: it folded
IPv4-mapped and compatible IPv6 but not 6to4 or NAT64, so [2002:7f00:1::]
(127.0.0.1) slipped through. Fold 6to4 and the NAT64 well-known prefix,
reject the NAT64 local-use prefix and the rest of 64:ff9b::/32 outright
(the embedded v4 offset varies by prefix length, so a decode would be
error-prone), and reject 0.0.0.0/8. The change only ever rejects more,
so the peer announce handler, the DB-boundary writer, and the boot-time
prune inherit the same hardening.
Closes#81.
* feat(node): add push-delta object enumeration module
Compute the per-push object delta (git rev-list --objects <new> --not <old>)
for the pin path instead of enumerating the whole repo, with a fully-peeled
ref-type guard (cat-file -t '<sha>^{}') that fails closed to a full scan on a
non-commit tip, any git error, or the GITLAWB_PIN_FORCE_FULLSCAN kill-switch.
The whole-repo lister moves here for the fallback and the reconciliation sweep.
Pin-enumeration scope only (N16); the withheld filter is unchanged.
* feat(node): pin the per-push object delta instead of the whole repo
Wire the push-delta enumeration into the IPFS and Pinata pin paths so pin work
scales with push size, not repo size (N16).
- U2: both pin_new_objects twins take a candidate OID set instead of scanning
the whole repo internally; repo_path is retained for reading object bytes.
- U4: extract the anonymous replication withheld gate (caller=None, no caller
param) into a shared helper returning (announce, withheld), so the push path
and the future reconciliation sweep cannot drift. The per-caller read-serve
withheld call in git_upload_pack is deliberately left separate.
- U3: the post-receive handler resolves the push delta once (off-thread),
falls back to a full scan on a non-commit tip / git error / kill-switch, and
passes the candidate set to both pin paths. Logs delta size and fallbacks.
The reconciliation sweep (U5) is deferred: it depends on the history-deep
blob_paths fix (#84) landing on main first.
* fix(review): log degraded pin-candidate paths, extract wiring to push_delta
Code review found the push candidate computation collapsed both a failed
full-scan fallback and a panicked spawn_blocking task to a silent empty set,
diverging from the sibling withheld path that logs and fails closed.
Extract resolve_candidates_for_push into git::push_delta (it owns the
PinCandidates dispatch and full-scan fallback), logging every degraded path
(fallback taken, fallback failed, task panicked) instead of failing silently.
Pinning nothing on a hard failure is still safe (the withheld filter always
runs; the reconciliation sweep backstops durability) but is now observable.
Moving the dispatch out of the HTTP handler also makes it unit-testable.
Add tests: tag-of-tag-of-commit peels to a Delta, and resolve_candidates_for_push
delta + non-commit-tip full-scan-fallback paths.
* fix(node): fail announce closed when withheld walk fails (#90)
A failed or panicked withheld_blob_oids walk left announce=true while
withheld=None, so a push we could not vet would still gossip ref updates,
notify peers, and write a permanent Arweave anchor. Force announce false
in that arm so the gate fails closed on both axes.
validate_path_glob checked `ends_with("/**")` without first stripping it, so any
glob that still ended in `/**` slipped past the wildcard guard even when it held a
`*` elsewhere in the prefix (e.g. `/a*b/**`, `/a/**/**`). The prefix is fed to
glob_prefix/glob_matches as a literal, so such a rule matched no real path and
silently granted nobody, the exact silent-misconfiguration this validator exists
to prevent.
Strip the one permitted trailing `/**` first, then reject any remaining `*`.
While here, close a related hole in the same validator: `glob_prefix` greedily
trims all trailing `**` and `/`, so `//**` collapsed to `/` (whole repo) and
bypassed the explicit `/**` guard, and `//secret/**` became a dead rule whose
prefix never matches a real single-slash path. Reject empty path segments so the
validator and glob_prefix agree on the canonical form.
Adds `/a*b/**`, `/a/**/**`, `/**/**`, `//**`, `//secret/**` to the rejection test.
* feat(node): owner-only push enforcement behind GITLAWB_ENFORCE_OWNER_PUSH (#31)
A valid did:key signature on git-receive-pack is authentication, not
authorization: any party can generate a key, derive its DID, sign, and push to
an unprotected branch. Owner checks only ran on protected branches, so a
non-owner could write to unprotected branches of an arbitrary repo.
Add an opt-in flag GITLAWB_ENFORCE_OWNER_PUSH (default false, mirroring
GITLAWB_REQUIRE_SIGNED_PEER_WRITES) that requires the authenticated pusher to be
the repo owner on every branch. Default-off keeps live nodes unaffected.
Design follows the maintainer's review on #31:
- Authorize on the canonical AuthenticatedDid injected by require_signature, via
Extension<AuthenticatedDid> on the handler — not a re-parse of the headers.
The legacy branch-protection check is left as-is.
- Fail closed: an absent identity or a non-owner caller is rejected. The policy
lives in a pure fn (owner_push_rejection) so it is unit-tested without a DB.
- The gate runs before branch protection, so one rejection yields one error body.
- Factor the duplicated owner-match into auth::is_repo_owner, reused by
receive-pack, protect.rs, and visibility.rs (each keeps its own message).
events.rs / peers.rs are left out — they aren't owner checks.
- Name the push gate by intent (caller_authorized_to_push) so the planned
Phase 2 UCAN git/push capability is a pure addition, not a rewrite.
Tests: owner (full + short DID) allowed, non-owner rejected, missing-DID
rejected, flag-off legacy. Docs: .env.example + docs/RUN-A-NODE.md.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
* fix(node): address review on #68 — 403 status, finish owner-match consolidation
Per @beardthelion's review:
- Return 403 Forbidden (not 400) for authorization denials: the owner-push gate,
the branch-protection denial, and the owner checks in protect.rs / visibility.rs.
400 reads as a malformed request and some git/CI clients retry it.
- Finish the is_repo_owner consolidation: route the branch-protection check and
the repos list filter through the helper. The branch-protection check now uses
the verified AuthenticatedDid (auth.0) instead of re-parsing the headers.
- Drop the second identity source on the push path entirely: the post-push event
recording also uses auth.0, so extract_did_from_auth (and the handler's
HeaderMap param + import) are removed.
- Tests assert the Forbidden variant (so a future status regression fails loudly)
and add direct coverage for is_repo_owner / caller_authorized_to_push.
- Docs: warn that enabling the flag blocks delegated and CI agents until the
Phase 2 UCAN git/push capability is honored.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
---------
Co-authored-by: FrozenRaspberry <[email protected]>
Co-authored-by: Claude Opus 4.8 <[email protected]>
* fix(gl): make /encrypted-blobs recovery parsing schema-strict
Deserialize the /encrypted-blobs 200 body into a typed EncryptedBlobsResponse
instead of poking at a serde_json::Value and defaulting blobs to [] via
unwrap_or_default(). A server schema regression now fails closed (a hard parse
error) rather than silently yielding nothing to recover, mirroring the existing
WithheldPathsResponse convention. The sole caller still does not abort the clone
but now surfaces the error instead of swallowing it into an empty result.
Part of #49.
* fix(gl): paginate Arweave manifest discovery and rank ties by block height
Discovery hard-coded a single first:100 GraphQL query with no pagination, so a
repo with more than 100 encrypted-manifest anchors (one per push) silently lost
recovery coverage for any blob whose most recent seal fell outside that window.
Follow cursor pagination (after + pageInfo) until coverage is complete, deduping
transaction ids and bounding the loop (1000 pages / 10000 ids, degenerate-cursor
guard) so a hostile or buggy gateway cannot grow the downstream per-id fetch loop
without limit. A full page returned without pageInfo, a cap hit, or a failure
after discovery began now warns; a page-1 failure (unconfigured/unreachable
gateway) stays silent as before.
Pagination concatenates pages in gateway order, which made merge_manifests'
equal-timestamp tie-break (previously insertion-order, i.e. accidentally newest
on single-page HEIGHT_DESC) gateway-dependent. Rank candidates by the composite
key (timestamp, Arweave block height, cid): timestamp primary keeps the existing
distinct-timestamp latest-wins; block height breaks equal-timestamp ties by
anchor recency (a not-yet-mined anchor counts as newest), preserving the
pre-pagination behavior; cid is a final deterministic tiebreak for same-block
anchors. Height is parsed defensively so a present-but-unreadable height (e.g. a
gateway stringifying large ints) cannot masquerade as a pending/newest anchor.
Part of #49.
* fix(review): bound edges per Arweave page and cover degenerate-cursor/dedup
Code-review follow-ups on the #49 recovery change:
- Cap edges parsed per GraphQL page at ARWEAVE_PAGE_SIZE so a hostile gateway
returning one oversized page cannot defeat the MAX_TX_IDS budget or flood the
downstream per-id manifest fetch loop (adversarial review).
- Collapse the two identical gateway-failure arms in the discovery loop into a
single warn site, and hoist PAGE_SIZE to a named const tied to the query's
first:100 (maintainability).
- Add tests: per-page edge bound, non-advancing-cursor termination (infinite-loop
guard), and cross-page tx-id dedup fetched exactly once (testing).
Part of #49.
* fix(review): make the degenerate-cursor test assert, and pin the page-size invariant
Round-2 review follow-ups:
- The non-advancing-cursor test bound its mock to a _-prefixed handle with
expect_at_most(3) and never asserted it; mockito defaults assert_on_drop=false,
so the test passed even if the guard were removed. Bind it, tighten to expect(2)
(the guard breaks on the second page, so discovery makes exactly two POSTs), and
assert_async.
- Correct the parse_tx_page docstring: the per-page take() bounds the TxRef vec and
fetch loop, not the response-body deserialization (an unbounded-body concern
shared by every gateway read here).
- Hoist the discovery query to a const and add query_first_matches_page_size so the
query first:N literal and ARWEAVE_PAGE_SIZE are kept equal by a test rather than
a comment.
Part of #49.
Peer sync decided mirror mode from the origin's withheld-paths answer via
classify_mirror, whose _ => Plain arm collapsed both Some(empty) (genuinely
public) and None (lookup 404'd / 5xx / network / parse error) into Plain. For an
existing promisor mirror that meant fetch_repo(.., Plain) unset
remote.origin.promisor + partialclonefilter and ran git fetch --refetch, so a
transient withheld-paths outage downgraded a still-withheld mirror to a full
clone and broke syncs until the endpoint recovered.
Replace classify_mirror with resolve_mirror_mode(withheld, exists, promisor):
only Some(empty) downgrades an existing mirror; None preserves the on-disk
promisor mode. A genuine public transition always returns Some(empty), never
None, so preserving on None cannot mask one; it only suppresses the
transient-error false positive.
Add a three-valued PromisorProbe read from git config --get exit codes so a
transient probe failure (Unknown) is not mistaken for a definitive NotPromisor
and cannot itself trigger the downgrade. One warn marks the preserve path,
derived from the resolved mode so it cannot drift.
Tests cover every resolve_mirror_mode arm (incl. the regression and the
indeterminate-probe case) and the probe's Promisor / NotPromisor / Unknown
outcomes against real git repos.
* fix(core): route seed access through the zeroizing wrapper (#41)
Remove Keypair::seed_bytes(), which returned a bare [u8; 32] copy of the
Ed25519 private seed. Because [u8; 32] is Copy with no Drop, each caller's
copy was released without being scrubbed, leaving secret key material in
freed memory. The zeroizing accessor to_seed() -> Zeroizing<[u8; 32]>
already existed beside it; this makes it the sole seed accessor and migrates
the two in-crate callers (envelope open path and a crypto test) to it via
deref coercion. No behavior change to envelope encryption: the derived
X25519 secret is byte-identical, covered by the existing round-trip tests.
Adds an identity test pinning to_seed() as the only seed accessor.
* test(core): pin to_seed's Zeroizing return type, drop duplicate round-trip
The to_seed_is_sole_seed_accessor test was named for a compile-time
property its body couldn't assert, and its from_seed -> DID check
duplicated from_seed_round_trip. Replace it with a type-annotated
binding that fails to compile if to_seed ever returns a bare [u8; 32]
again, plus the determinism check, which is the only behavior it
uniquely covered.
* fix(node): migrate node_seed pin caller to to_seed()
main grew a third seed_bytes() caller (repos.rs, from #40) after this
branch was cut. Route it through to_seed() too: node_seed is moved into
a long-lived tokio::spawn pin task, so the Zeroizing wrapper scrubs the
copy when that task ends. encrypt_and_pin takes &[u8; 32]; the wrapper
deref-coerces, so the call site is unchanged.
* perf: trim allocations and redundant copies on hot paths
- Cargo.toml: enable thin LTO + symbol stripping for release builds
- git-remote-gitlawb: attach pack body after signing to move (not clone)
potentially large pack bytes, halving peak push memory
- gitlawb-attest: deserialize payload by reference instead of cloning it
- core/http_sig: compare components by &str to avoid per-item String alloc
- node/tigris: drop redundant to_vec() copy of the response body
- node/rate_limit: only allocate the key String on first insert, not per request
- node/sync: resolve the peer table once per batch instead of per item
Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
* perf(sync): skip peer lookup on empty sync batch
Return early from process_batch when dequeue_pending_syncs yields nothing
— the common case on the 30s idle poll — so an empty batch no longer fires
an unnecessary list_peers() DB query (CodeRabbit, PR #80).
Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
* test(sync): cover per-batch peer origin-URL resolution
Extract the peer -> origin-URL lookup from process_batch into a pure
resolve_origin_url() helper and unit-test it (match + trailing-slash trim,
unknown DID, empty peer list). This is the per-batch resolution PR #80
introduced; pulling it out of the DB-bound process_batch lets it be tested
without Postgres. No behavior change. Addresses the needs-tests label.
Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
* fix(tigris): use a unique temp dir per archive extraction
The fixed `.{file_name}.tmp-extract` path let two concurrent extractions of
the same repo share one temp dir, so one's remove_dir_all/unpack could clobber
the other's in-progress work. Suffix the temp dir with a fresh UUID so each
extraction is isolated; this also removes the need to pre-clear a leftover dir
(a unique name can't collide with an interrupted run). The on-failure cleanup
still removes our own dir, and success consumes it via rename (CodeRabbit).
Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
* fix(tigris): serialize repo publish per path to close the swap race
Unique temp dirs let extractions unpack in parallel, but the final
remove_dir_all + rename swap could still interleave for the same local_path,
causing a nondeterministic overwrite or a rename-onto-non-empty failure. Guard
just the swap with a process-wide mutex keyed by local_path so only one
extraction publishes per repo at a time; the expensive unpack stays parallel
(CodeRabbit).
Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
* docs(tigris): restore decompress_repo doc comment ordering
The earlier publish_lock insertion landed between decompress_repo's doc
comment and its fn, gluing the two doc blocks onto publish_lock and leaving
decompress_repo undocumented. Reorder so each fn keeps its own doc. No
behavior change.
Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
* docs(tigris): note publish_lock map is never evicted
Document the known unbounded-growth limitation of the per-path LOCKS map:
bounded by the repo count so negligible for normal use, but high-volume
deployments may want LRU/weak-ref eviction. No behavior change (CodeRabbit).
Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <[email protected]>
* fix(security): reject non-public peer URLs + prune poisoned peers
The unauthenticated POST /api/v1/peers/announce only validated the
http(s) scheme, so anyone could register loopback/private 'peers'
(localhost:5432, localhost:22, internal admin URLs). On every push the
node fans out signed sync-notify POSTs to all peers — turning this into
an SSRF probe and burying the real peers under junk so node-origin repos
stopped replicating. Observed on node.gitlawb.com: 126/194 peer rows were
injected junk (fake DIDs z6MkScan…/z6MkAdmin…, localhost targets).
- is_public_http_url(): reject non-http(s), loopback, unspecified, private
(RFC1918 + ULA), link-local, and localhost/.local/.internal hosts.
Applied at announce time.
- db.prune_non_public_peers(): boot-time sweep removing already-injected
rows, run from main() alongside prune_self_peers — self-heals poisoned
tables on deploy.
- tests for the validator (accept public, reject loopback/private/internal,
incl. IPv6 [::1]).
Co-Authored-By: OpenClaude <[email protected]>
* fix(security): address PR #78 review — build, redirect SSRF, validation gaps
Resolves the P1/P2 review feedback on the peer-announce SSRF fix:
- main.rs: revert the unrelated, incomplete `storage::build()` refactor back
to the tigris init from main (the `storage` module never existed, breaking
the build). Keeps only the prune_non_public_peers boot call.
- Redirect bypass: set redirect::Policy::none() on all peer-facing reqwest
clients (shared http_client, sync worker, ping_peer) so a peer cannot answer
`302 -> http://127.0.0.1/` and bypass the public-URL check. ping_peer now
reuses the shared no-redirect client instead of bare reqwest::get.
- Move the public-URL guard into db::upsert_peer so the bootstrap announce-back
path (main.rs) can no longer re-poison the table after the prune.
- is_public_http_url: strip trailing-dot FQDNs (`localhost.`), fold
IPv4-mapped/compatible IPv6 (`::ffff:127.0.0.1`, `::127.0.0.1`) to IPv4 before
range checks, and reject CGNAT (100.64.0.0/10). Added tests for each.
- prune_non_public_peers: collect bad DIDs then DELETE ... WHERE did = ANY($1)
in one statement, so a transient error no longer abandons remaining rows.
Follow-up (separate issue): DNS-resolution rebinding guard rejecting peer
hostnames that resolve to private IPs.
Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
* fix(security): satisfy rustfmt + don't drop redirect guard on builder failure
- peers.rs: collapse the CGNAT/private check onto one line (cargo fmt --check).
- sync.rs: replace the unwrap_or_else(reqwest::Client::new()) fallback with
.expect(). The default client follows redirects, so the fallback silently
reintroduced the SSRF vector Policy::none() exists to close (CodeRabbit).
Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
---------
Co-authored-by: OpenClaude <[email protected]>
Co-authored-by: Claude Opus 4.8 (1M context) <[email protected]>
* test(node): add HTTP-API integration harness with CI Postgres
Adds a #[cfg(test)] test-support module: a migrated AppState over a real
#[sqlx::test] pool (test_state), a DB-free variant (test_state_lazy), the
assembled router (app), and signed_request_as for injecting an authenticated
DID without RFC-9421 signing. A test-callable run_migrations reuses the
production migrate() path, and sqlx gains the macros/migrate features so
#[sqlx::test] is available. CI's test job gets a postgres service behind a
pg_isready health gate so the DB-backed tests can run.
The proving test exercises the owner gate on PUT /visibility end to end:
non-owner is rejected with 400, owner succeeds. signed_request_as sets a JSON
content-type so the Json extractor does not 415 before the handler runs.
* feat(node): enforce per-route authorization on REST mutations
Closes the missing-authorization cluster on the signed REST surface, where
'authenticated' meant 'any participant' because identities are permissionless.
Adds two shared helpers in api/mod.rs: require_repo_owner (403) and did_matches,
a method-safe DID comparison that collapses the full did:key vs bare short form
without matching across methods (did:web/did:gitlawb share the base58 space).
Per-route gates:
- merge_pr is owner-only (subsumes branch protection); close_pr and close_issue
allow the repo owner or the PR/issue author (the issue author is read from the
git-JSON blob, with a None author falling back to owner-only).
- create_review, create_comment, create_issue_comment, and create_bounty are
read-gated (participants need read access, not ownership; this also closes the
private-repo bounty leak).
- the task handlers bind the acting DID to the authenticated signer.
- dispute_bounty, previously ungated, now requires the creator or claimant; the
other bounty and replica identity checks switch from raw equality to did_matches.
A source-level guard test asserts every in-scope mutation handler still carries
its expected gate marker, so a removed gate or an unclassified new route fails
CI. DB-backed tests (via the integration harness) cover merge owner-only and the
task signer-binding; did_matches has unit coverage for the cross-method case.
* feat(node): authenticate GraphQL mutations
GraphQL mutations were a fully unauthenticated parallel write path to the task
system (N2): the /graphql route had no auth layer and the resolvers took the
acting DID as plain arguments. Apply optional_signature to the /graphql POST so
a verified DID is attached when a signature is present (queries stay open), and
thread it into request-scoped data. Each mutation now reads the verified signer
via require_signer and binds the acting DID to it (rejecting an unsigned request
or one whose claimed delegator/assignee/by_did differs from the signer), reusing
did_matches for the comparison. Read-only subscriptions (/graphql/ws) are left
open. A resolver-level test covers the unsigned, mismatched, and matching cases.
* fix(node): path-scope the get_tree visibility gate
get_tree gated on the repo root ("/") regardless of the requested subtree, so
a caller denied a withheld subtree could still enumerate its filenames and blob
SHAs (N3). Gate on the requested path instead, mirroring get_blob, and reject
traversal segments. A cross-caller test shows the rejection is path-scoped: a
non-reader is denied the withheld subtree but passes the gate on a non-withheld
path.
* fix(node): authorize task completion and read-gate PR/issue creation
Two authorization gaps surfaced by an adversarial review of the auth stack,
both necessary-but-insufficient siblings of fixes already in it.
complete_task/fail_task bound the acting DID to the signer but never checked
the caller against the task's assignee, and finish_task updated by id alone, so
any authenticated did:key could finish or fail any task in any state. Load the
task and require the caller to be its assignee (REST and GraphQL), and add an
'AND status=claimed' predicate so only a claimed task transitions. The
now-unused by_did request fields are removed.
create_pr/create_issue resolved the repo with get_repo but skipped
authorize_repo_read, so a non-reader could open a PR (firing the owner's
webhooks) or file an issue against a private repo they cannot read. Gate both
on read access like their create_review/create_comment/create_bounty siblings.
Adds DB-backed tests covering non-assignee rejection (including the empty-body
bypass), the claimed-state predicate, and private-repo PR/issue denial.
* test(node): harden the authz drift-guard against false passes
The guard asserted a marker STRING appeared in each handler body, which a bare
identifier in a comment or log line could satisfy even after the real gate was
deleted, and its body slice over-ran on any handler not declared exactly
'pub async fn'.
- Markers are now gate-shaped: a call (require_repo_owner(, did_matches(,
authorize_repo_read() or a binding/comparison expression
(caller != &record.owner_did, let owner_did = auth.0), never a field name.
- Full-line comments are stripped before matching, so a gate that survives only
as a comment no longer counts as enforced.
- The body slice bounds at the next top-level fn item across pub async,
pub(crate) async, async, pub, and bare fn forms.
- Adds rows for the now-read-gated create_pr/create_issue and for fork_repo,
create_repo, set_profile, claim_bounty; register is excluded with a note (it
trusts the body did, tracked as P3 D3-1).
- Adds a self-test proving a comment-only marker does not satisfy a row.
* fix(node): P3 consistency batch from the adversarial review
Five low-severity items from the stack review. The sixth (a did_matches
false-negative on non-canonical multibase encodings of the same key) is left
as-is: it is a self-inflicted owner lockout unreachable with the conforming
client, never an impersonation vector, so decoding keys in the auth path is not
warranted.
- register bound the registered DID to the request body, not the signer, so a
signed caller could create or refresh a trust row under a victim DID. Bind it
to auth.0 (403 on mismatch) and add a drift-guard row now that it is gated.
- Bounty submit/approve/cancel returned 400 for an identity-mismatch denial
where dispute returned 403. Return Forbidden, matching the rest of the surface.
- The visibility/protect owner gates (require_owner and the protect.rs inline
checks) returned 400 for a non-owner; return 403 like require_repo_owner, and
assert the exact code in the harness.
- get_tree rejected ./.. segments but not empty interior segments, so a path
like secret//x could reach git while the gate saw a different string. Reject
empty interior segments too (the empty path remains the root listing).
- Tighten the get_tree path-scope test to assert an exact 200 on the
non-withheld branch so a future upstream 4xx/5xx cannot masquerade as
gate-pass.
Adds a register-binding regression test.
* fix(node): unify protect/visibility owner checks on did_matches
CodeRabbit flagged that protect_branch/unprotect_branch and visibility's
require_owner used a trailing-segment owner compare (split(':').next_back())
distinct from the hardened did_matches, so the two owner-match idioms could
drift. It is not exploitable (the authenticated caller is always the full did:
form while the short form is a bare suffix, so no wrong caller passes), but it
is a real inconsistency, and the trailing-segment form is a false-negative for a
bare-stored owner. Replace both with did_matches (collapses did:key full vs bare
on both sides, never across methods), and extend the drift guard to assert
require_owner itself uses did_matches, not just that it is called.
* feat(node): signature-gated agent self-deregister (#29)
Agents whose keys are rotated or compromised could never be removed from
the registry, so an orphaned DID persisted and could win ?capability=
routing. Give the DID's key holder a way to retire their own agent.
A v5 migration adds an agent lifecycle status (default 'active') plus a
deactivated_at timestamp. list_agents drops non-active agents via a pure
filter_discoverable helper, so a revoked DID no longer appears in
discovery or capability routing; get_agent stays unfiltered so a revoked
DID still resolves and surfaces its status.
DELETE /api/v1/agents/{did} lets an rfc9421-authenticated caller mark its
own agent revoked. Authorization is on the verified AuthenticatedDid via a
pure caller_matches_did gate (a DID can only retire itself), unit-tested
including the cross-DID rejection. Revocation is terminal: register_agent's
ON CONFLICT leaves status untouched, so re-registering a leaked DID cannot
resurrect it.
Authority is team-owned (the key holder); no external contract, chain, or
operator power. Lost-key retirement and any successor pointer are out of
scope.
* fix(node): preserve original deactivated_at on repeat revoke
A retried DELETE /api/v1/agents/{did} re-ran the revoke UPDATE and
overwrote deactivated_at with a fresh timestamp, losing the true
first-retired time. Use COALESCE(deactivated_at, $2) so the original
timestamp is kept; the row still matches so the idempotent 204 is
unchanged.
quinn-proto 0.11.14 (pulled transitively via libp2p's QUIC transport) is
flagged by RUSTSEC-2026-0185. 0.11.15 is a semver-compatible patch that
clears it; cargo audit passes clean after the bump. Lockfile-only, no
manifest or code change, so it carries no ignore entry. Repo-wide:
unblocks cargo audit on main and every open branch.
* fix(docker): reap orphan processes via tini + init, cap pids
The node runs as PID 1 in the container and never reaps reparented
orphans, so git grandchildren (e.g. pack-objects left behind when a
served clone disconnects) accumulate as zombies until fork() fails with
EAGAIN. Run under tini and set compose init: true so reparented orphans
are reaped, and add pids_limit as a blast-radius cap.
Refs #53
* fix(node): reap leaked git child processes (#53)
Serving git over smart-HTTP could leak zombie processes that, over weeks,
exhaust the PID table until fork() fails with EAGAIN and all git
operations break until restart.
The serve path (run_git_service) now runs git in its own process group and
SIGKILLs that group when the request future is dropped (client disconnect)
or returns early, so git and its pack-objects child are torn down together
instead of orphaning the grandchild to PID 1 as a zombie. A drop guard,
disarmed once the request completes cleanly, owns this teardown.
create_issue and build_filtered_pack spawned a std Child and dropped it
unwaited if the stdin write failed (and build_filtered_pack could panic on
a missing stdin handle). Both now always wait the child before surfacing
any error, so the failure path cannot leak.
Adds libc for the group-kill syscall and three regression tests covering
direct-child kill, grandchild reaping via the group, and the disarmed
no-kill path.
Refs #53
* fix(review): reap-then-disarm on serve-path errors; tini/comment nits
Address code-review findings on the #53 fix:
- run_git_service no longer early-returns on a stdin-write error. It captures
the write result, always reaps the child, then disarms the group guard
before surfacing the error. Previously a write/wait error path could drop
the still-armed guard after the child was already reaped, so the group
SIGKILL could in theory land on a reused pgid. (correctness, reliability,
adversarial all flagged this region.)
- Drop tini's -g flag: the node manages its own children and graceful
shutdown, so group-wide signalling would only disturb in-flight git helpers
on shutdown.
- Correct the SAFETY comment (kill(2) is not relevant to async-signal-safety
here) and the alive() test-helper comment (EPERM does not read as alive).
* fix(node): SIGTERM (not SIGKILL) the served-git group on disconnect
SIGKILL gave git no chance to run its signal cleanup, so an aborted
receive-pack could leave a stale .git/*.lock that blocks the next push.
SIGTERM lets git remove its lock files before exiting. Signalling the
whole process group still reaches pack-objects directly, so the grandchild
is terminated rather than orphaned, and a single immediate signal avoids
any delayed-kill pid-reuse window.
Refs #53
* docs(node): correct stale SIGKILL comments to SIGTERM in served-git teardown
* ci: harden PR pipeline (audit, MSRV, release build, beta matrix, docker smoke)
Restructure the single stable-only fmt+clippy+test job into separate jobs:
fmt+clippy (stable), a test matrix on stable+beta (beta non-blocking),
build --release, cargo audit, an MSRV check pinned to Rust 1.91, and a
Docker build smoke test mirroring the release workflow. Add concurrency
cancellation for superseded PR runs and per-job timeouts.
Declare rust-version = "1.91" in [workspace.package] and inherit it in
every member crate so the stated MSRV is a real, CI-enforced contract.
* ci: disable credential persistence on checkout steps
Set persist-credentials: false on all six actions/checkout steps. These
jobs run untrusted PR code via cargo and docker build, and checkout's
default leaves the GITHUB_TOKEN in the local git config where that code
could read it. None of the jobs push or otherwise need git credentials.
* fix(deps): bump rustls-webpki to 0.103.13
Clears three advisories flagged by the new cargo audit job: a reachable
panic in CRL parsing and two name-constraint bypasses (wildcard and URI).
Semver-compatible patch, lockfile only, no manifest or code change.
cargo also corrects a stale lockfile reference while rewriting Cargo.lock:
data-encoding-macro-internal 0.1.17 (a build-time macro crate) requires
syn 1.x, and its dependency entry is re-pinned from syn 2.0.117 to the
1.0.109 it actually resolves to. Unrelated to the bump, unavoidable when
cargo touches the lock, and the more-correct state.
* ci(audit): suppress no-fix advisories with justification and a re-check
The cargo audit job stays hard-fail. The remaining three advisories have
no available upstream fix, so they are suppressed in .cargo/audit.toml
(the location cargo-audit actually reads; a repo-root audit.toml is not
read), each with an inline rationale and removal condition:
- RUSTSEC-2026-0118/0119 (hickory-proto 0.25.2, via libp2p-dns 0.44):
reachable, since the node wraps QUIC in the system DNS resolver. No fix
until libp2p-dns ships hickory 0.26. Accepted, reachable risk.
- RUSTSEC-2023-0071 (rsa Marvin): in the lockfile via sqlx's unconditional
sqlx-mysql, but not linked (sqlx is postgres-only), so no RSA oracle
exists; no upstream fix regardless.
Two controls keep the suppression honest. The PR audit job echoes the
active ignore list to the job summary, so a green check is not read as
advisory-clean. A weekly scheduled job re-runs cargo audit without the
ignores for visibility and hard-fails if hickory-proto moves off 0.25.x
while the ignores remain, so the suppression cannot silently outlive the
upstream fix.
* ci(audit): reference tracking issue #76 in the hickory ignores
* ci(audit): treat hickory-proto removal as drift too
The scheduled drift guard only failed when hickory-proto was present and
off 0.25.x, so removing it from Cargo.lock entirely left the RUSTSEC-2026-0118
/0119 ignores in place with the guard still passing. Treat an absent version
as drift as well, so dead ignore entries are caught whether the dependency is
bumped past 0.25.x or removed.
* feat(node): blind recipient identities at rest and gate B1 by repo readability
The origin no longer stores recipient DIDs. Migration v5 replaces the
encrypted_blobs.recipients column with an opaque, node-keyed recipients_tag
used only to detect a recipient-set change for re-seal. B1 discovery and fetch
are now gated by the same repo-readability check the git read path uses, not by
per-recipient matching; decryption is gated by the envelope crypto, so a
non-recipient who can read the repo sees a blob's {oid, cid} but cannot open it.
encrypt_and_pin keys the tag from the node seed and returns {oid, cid}; the
Arweave manifest tuple drops the now-unused recipient vec.
A DB compromise no longer reveals the reader set; recovering it would require
brute-forcing candidate DID sets against the keyed tag with the node key.
* chore: remove redundant .gitignore entry (covered by local exclude)
* fix(node): gate /replicate by repo readability; harden reseal on DB error
Address review on the at-rest blinding change:
- The encrypted-blobs/replicate listing returned {oid, cid} with no visibility
check, so a non-readable repo's blob index was reachable by an unauthenticated
caller who guessed {owner}/{repo}. Gate it by the same repo-readability check
discovery and fetch use. For the intended case (a public repo with withheld
subtrees) the public root keeps this open to peers; only fully non-readable
repos are withheld, which is the desired behavior.
- encrypt_and_pin treated a recipients_tag DB read error as a cache miss and
resealed, causing avoidable IPFS writes during a partial outage; skip and retry
on the next push instead.
- Correct the get_encrypted_blob doc comment to describe repo-readability access.
* fix(node): enforce path-scoped visibility on the REST read API
The repo-scoped REST read endpoints (blob, tree, commits, refs, changelog,
pulls, pr diff, repo metadata) read straight from disk after a bare repo
lookup, with no visibility check, so private repositories and private
subtrees of public repositories were served to unauthenticated callers,
bypassing the withholding the git and replication paths enforce.
Add a shared authorize_repo_read gate (resolve repo, load rules,
visibility_check, 404 on deny) and call it from every repo-scoped read
handler. get_blob gates at the requested file path; get_pr_diff withholds
the whole diff when it touches a path the caller cannot read. Attach
optional_signature to the read routes so authorized readers authenticate.
Listings stay gated at the repo root, consistent with mode B exposing tree
and commit SHAs over git.
Closes#51
* fix(node): gate PR reviews/comments and fork on repo read visibility
The REST visibility gate added in this PR missed three read surfaces:
list_reviews and list_comments served a private repo's PR reviews and
comments to unauthenticated callers, and fork_repo cloned a full mirror of
the source without checking the caller could read it. Route all three through
authorize_repo_read at the repo root, matching the git-protocol gate.
Also make branch_diff_names fail closed: it ignored git's exit status and
returned an empty path list on error, which would let get_pr_diff serve a diff
it should have withheld. Fix changelog.rs import order to satisfy cargo fmt.
* fix(node): NUL-delimit branch_diff_names to keep path-gate enforcement exact
git diff --name-only quotes and escapes paths containing newlines, so they
no longer match the visibility globs in get_pr_diff. On a public repo where /
is allowed but a subtree is denied, a touched restricted path could be missed
by the deny check and the full PR diff disclosed. Use -z and split on NUL to
preserve exact path bytes.
The release-binaries job packaged gl, git-remote-gitlawb, and gitlawb-node
into the release tarball but never ran them, so a broken artifact could ship.
This is the follow-up promised in #30 and the gap noted in OSS-READINESS-AUDIT
("release binaries are not smoke-tested after packaging"), now unblocked by
git-remote-gitlawb gaining --version.
Run each binary's --version after build and assert the released version string
is present. Skipped for aarch64-unknown-linux-musl, which is cross-compiled and
can't execute on the x86_64 runner.
Co-authored-by: FrozenRaspberry <[email protected]>
Co-authored-by: Claude Opus 4.8 <[email protected]>
* feat(core): ed25519 to x25519 key conversion for envelope crypto
* feat(core): seal/open per-blob encryption envelopes
* feat(node): resolve per-blob encryption recipients (least-privilege)
* feat(node): encrypted_blobs table and recipient-scoped queries
* feat(node): encrypt-then-pin withheld blobs at push (IPFS)
At the push chokepoint, after pinning withheld objects, resolve each
withheld blob's recipient DIDs and seal it to their Ed25519 keys with
seal_blob, pinning the ciphertext to IPFS and recording it in
encrypted_blobs. Best-effort per blob: failures are logged and skipped,
never pinned in plaintext. Pinata replication is unchanged; B1
encrypts to IPFS only.
Adds ed25519-dalek as a direct dependency of gitlawb-node (it was only
declared in the workspace Cargo.toml).
* feat(node): authenticated discovery and fetch for encrypted blobs
* feat(gl): transparent recovery of authorized encrypted blobs on clone
* style: cargo fmt
* fix(node): re-seal encrypted blob when recipient set changes
Skip re-pinning only when an existing envelope already covers exactly the
current recipients. A reader added to a rule after the first pin now gets
a re-seal on the next push instead of being permanently locked out.
Reader removal stays non-retroactive (the old envelope is already public).
* feat(node): unscoped encrypted-blob listing for replication
* feat(node): encrypted-blobs/replicate endpoint for peer mirrors
* feat(node): route encrypted-blobs/replicate under git_read_routes
* feat(node): peer mirrors replicate encrypted withheld blobs (Option B2)
* feat(node): anchor_encrypted_manifest for Option B3 Arweave manifests
* feat(node): encrypt_and_pin returns the per-push sealed delta
* feat(node): anchor encrypted-blob manifest on push (Option B3)
* feat(gl): Arweave/IPFS gateway recovery for encrypted blobs (Option B3)
* feat(gl): transparent Arweave fallback recovery on clone (Option B3)
* style: cargo fmt
* fix(gl): bound Arweave recovery gateway requests with a 30s timeout
* test(gl): read-path recovery tests for Arweave fallback; quiet promisor present-check
Add two hermetic integration tests for recover_from_arweave that drive the
full read path over mocked Arweave GraphQL + IPFS gateways: discover the
manifest, fetch it, fetch the envelope, decrypt, and install the withheld
blob. One covers an authorized recipient (blob installed), the other a
non-recipient (nothing recovered). Both simulate origin death by removing
the promisor remote and enable uploadpack.allowFilter so the blob is truly
withheld over file://.
Also harden the local presence check in recover_from_arweave with
GIT_NO_LAZY_FETCH=1 and .output() so the expected 'missing object' case does
not trigger a wasted promisor fetch or leak git stderr to the user.
* fix: surface corrupt recipients JSON and silent recovery I/O failures
db: parse_recipients now propagates a descriptive error instead of
defaulting corrupt recipients JSON to an empty list, which would have
denied authorized readers and handed peers incomplete metadata.
gl: clone recovery now warns when the sparse-checkout file cannot be read
or written, or when the post-recovery checkout fails, instead of silently
discarding those errors and claiming files were recovered.
* feat(core): blind recipient identities in withheld-blob envelopes
Envelope v2 drops the cleartext recipient public key (kid) from each
wrapped-key header entry, so a party holding the public IPFS/Arweave copy
can no longer enumerate who is authorized to decrypt a withheld blob. The
version is bumped to 2 and v1 envelopes are rejected. open_blob now selects
the reader's entry by trial decryption (the AEAD tag authenticates exactly
one entry) instead of matching on the public key.
Recipient count is still visible; identity is not. Scope is envelope-only,
the node DB recipients column and peer replication metadata are unchanged.
Co-authored-by: CommandCodeBot <[email protected]>
* fix(core): reject malformed envelope nonces instead of panicking
open_blob fed attacker-controlled envelopes to GenericArray::from_slice and
XNonce::from_slice, which panic on a wrong-length input. Validate both the
per-recipient box nonce and the body nonce to 24 bytes before use: skip a
recipient entry whose nonce is malformed, and return an error for a malformed
body nonce, so the public recovery path surfaces an error rather than panicking.
* feat(node): blind recipient identities on the B2 replication surface
The encrypted-blobs/replicate endpoint shipped every blob's full recipient
DID list to mirrors, which persisted it, so any mirroring peer learned the
reader set. The v2 envelope blinding already removed recipient public keys
from the pinned envelopes, so the comment justifying this (DIDs are already
public) was no longer true.
/replicate now returns {oid, cid} only. Mirrors detect a re-seal by the CID
changing (the OID is stable across re-seals) instead of comparing recipient
sets, and store no recipient identities. Origin-side authz and the at-rest
recipients column are unchanged; this blinds only the peer-facing surface.
* feat(node): stop anchoring recipient identities to Arweave
The B3 encrypted-blob manifest anchored {oid, cid, recipients} to Arweave, a
permanent public record of every blob's reader set. The v2 envelope blinding
already removed recipient keys from the pinned envelopes, so the comment
justifying this (recipient DIDs are already public) was no longer true. The
manifest now anchors {oid, cid} only. The gl reader already ignores the
recipients field (it recovers by trial decryption), so no reader changes.
---------
Co-authored-by: CommandCodeBot <[email protected]>
2026-06-19 22:48:33 +08:00
149 changed files with 58252 additions and 1897 deletions
echo "hickory-proto in Cargo.lock: ${version:-not present}"
ignores_present=false
if grep -q 'RUSTSEC-2026-011' .cargo/audit.toml; then
ignores_present=true
fi
# Drift = the lockfile no longer matches what the ignores assume:
# hickory moved off 0.25.x (fix shipped) OR was removed entirely
# (dead ignore entries left behind). Both must fail.
if [ "$ignores_present" = true ] && { [ -z "${version}" ] || [[ "${version}" != 0.25.* ]]; }; then
echo "::error::hickory-proto is ${version:-absent from Cargo.lock} but the RUSTSEC-2026-0118/0119 ignores are still in .cargo/audit.toml. The upstream fix appears to be available or the dependency is gone; remove the ignores."
exit 1
fi
echo "No drift: ignore list is consistent with the pinned hickory-proto version."
This file carries the rules a patch author cannot recover from the code alone, for humans and coding agents alike. For build, test, and PR basics, read `CONTRIBUTING.md` first.
## Authorization invariants (the rules PRs most often break)
Every repo-scoped endpoint must bind the caller to an authorization decision before serving or mutating anything. The `known_ungated` allowlist in `authz_guard::every_repo_scoped_handler_is_gated` is empty on main: `list_webhooks`, `list_replicas`, and `list_protected_branches` are gated (`list_webhooks` with read-visibility then owner; the other two with `authorize_repo_read`). Do not re-open those surfaces as exceptions.
- Owner-only mutations (visibility, webhooks, protected branches, merges) are gated against the repo owner. Three gate forms are in use and all are recognized: webhooks and merges call `require_repo_owner` (`crates/gitlawb-node/src/api/mod.rs`), visibility uses its module-local `require_owner`, and protected branches compare inline with `did_matches`. Use the form the surrounding module already uses.
- Not every write is owner-only, and that is by design: `star_repo`/`unstar_repo` bind to the signer (they still require repo read access), `register_replica`/`unregister_replica` bind to the replica's own DID, the bounty actions (claim, submit, approve, cancel, dispute) have their own multi-party rules, and closing a PR or issue is owner-or-author. Do not add owner gates to these.
- Content-serving reads (blobs, trees, raw files) gate through `authorize_repo_read` with the specific path being served, so a withheld subtree is denied even on an otherwise-public repo; repo-level reads (listings) pass `"/"` by the helper's own contract. The source scans below cannot check this argument's value, so review will.
- A route that reaches repo data through a global id, with no repo in its path, or through an extractor shape the scans do not recognize still needs the same gate; the scans cannot see those.
- Read-surface denials must not reveal existence: a caller who may not read a repo (or a withheld subtree) gets the same 404 as a missing repo, never a 403. (Owner-gated mutations currently return 403 after the repo lookup; do not widen that shape to read paths.)
- Two tests in module `authz_guard` (`crates/gitlawb-node/src/api/mod.rs`) scan handler sources and fail on a repo-scoped handler without a recognized gate: `every_in_scope_mutation_has_its_gate` and `every_repo_scoped_handler_is_gated`. If they fail on your handler, add the gate. The second test's `known_ungated` allowlist is empty; never add an entry to it without an open issue and review.
## Adding or changing routes
- A new gated handler needs tests proving its route-specific gate fires: exercise an unauthorized authenticated caller and an anonymous caller where applicable, assert the exact denial status each way, and assert the body leaks nothing. The non-owner tests in `crates/gitlawb-node/src/test_support.rs` show the shape for owner-gated routes.
- A new mutation handler also gets a row in `every_in_scope_mutation_has_its_gate` naming its gate type; the per-handler row is the point of that test.
- A table-driven deny suite covering every deny-bearing route is in review (#194, #195). If it is present in your checkout, add a probe row there for any new deny-bearing route.
## Verifying signatures and certificates
- Derive the verifying key from something outside the artifact you are checking: the node's own DID, a pinned or configured value, a peers lookup. A key read from a field of the object being verified (`Did::from_str(&cert.node_did)`) only proves the artifact is self-consistent, and identities here are permissionless, so anyone can mint a keypair, name it in the artifact, self-sign, and be told it is valid. A mismatch against that anchor has to fail the result, not log a warning.
- A handler whose correct answer is "valid" needs a positive case driven by a trusted, independently anchored artifact, and a rejection case for a well-formed, fully populated forged artifact. Invalid-only suites miss an implementation that rejects everything; a forged artifact that still returns valid is the regression that matters.
- The fields a signature covers are a versioned format. Adding one invalidates every artifact already signed, so give the payload a version, keep a path that verifies the older form, and add a test that verifies an artifact signed before your change.
## Removing a serving path
When a fix removes a path that used to serve content, the same PR must add a test asserting the removed path now denies. The deletion is what turns that test green; without it the path can quietly come back.
## Database migrations
Schema changes are code-defined in `MIGRATIONS` (`crates/gitlawb-node/src/db/mod.rs`). Always append a new versioned entry; never edit an entry that has merged (operators deploy from main). Test runs build the schema from scratch, so editing an applied version stays green in CI while breaking every existing deployment.
## Client behavior (`gl`, `git-remote-gitlawb`)
When a node denies a request, surface the denial to the user. Never render a denial as an empty list or a silent success; that turns an authorization boundary into invisible data loss.
## Toolchain and CI reality
- MSRV is Rust 1.91 (`rust-version` in the workspace `Cargo.toml`). The MSRV CI job runs `cargo check --workspace --all-targets` on exactly 1.91, so check that an API you rely on is stable there; the full test suite runs on stable.
- The PR workflow runs these jobs on every PR: `cargo fmt --check`, `cargo clippy --workspace --all-targets -- -D warnings`, `cargo test --workspace` (against Postgres), a release build, `cargo audit`, the MSRV check, and a Docker build. The `main` branch ruleset currently requires one approving review but does not require these checks to pass before merge; a failing job should block review approval in practice, but nothing enforces that mechanically today.
- Conventional commit titles (`feat:`, `fix:`, `docs:`, ...) are required by convention and drive releases, but no CI job checks them; a bad title fails review, not the pipeline.
* **ci:** make the docker dependency cache work and unbreak Dockerfile.bins ([99b2b9a](https://github.com/Gitlawb/node/commit/99b2b9a16662761a3959a2941798e6823621bda5))
* **ci:** make the docker dependency cache work and unbreak Dockerfile.bins ([a49e04d](https://github.com/Gitlawb/node/commit/a49e04ddcd23f3b666421486ae60c3c4329e2242)), closes [#265](https://github.com/Gitlawb/node/issues/265) [#266](https://github.com/Gitlawb/node/issues/266)
* **db:** pin profile DID matching with lone non-key fixture and boundary matrix ([b4016dc](https://github.com/Gitlawb/node/commit/b4016dce9d1fe450603173f9d92bd880127620fa)), closes [#298](https://github.com/Gitlawb/node/issues/298)
* **node:** add a shared owner/name slug validator ([#272](https://github.com/Gitlawb/node/issues/272)) ([51d82de](https://github.com/Gitlawb/node/commit/51d82ded57a776d9c7e2fe65895efdbf9e227dc8))
* **node:** bound the slug owner by NAME_MAX, not the DID limit ([#272](https://github.com/Gitlawb/node/issues/272)) ([27b0495](https://github.com/Gitlawb/node/commit/27b0495a6f7edeecddbe62b0f075a7e3b1ef5053))
* **node:** fail a sync row terminally when a non-directory holds the owner path ([2e664b7](https://github.com/Gitlawb/node/commit/2e664b7885036ee72ce175af95bb567b18222583))
* **node:** make a stalled sync queue visible, and fix two test claims ([9aa4c17](https://github.com/Gitlawb/node/commit/9aa4c17aefd1885d134b1334b54551366cdd8ef0))
* **node:** move the attempted_at migration to v17, clear of two open branches ([fbc5868](https://github.com/Gitlawb/node/commit/fbc5868bd16946ea184dad26abcdb0e89d68e99a))
* **node:** prove the mirror path is inside repos_dir before running git ([#272](https://github.com/Gitlawb/node/issues/272)) ([3ca9473](https://github.com/Gitlawb/node/commit/3ca94739ed7b90e4c265a0947025dc3b50e7583b))
* **node:** reject a malformed slug at the notify boundary ([#272](https://github.com/Gitlawb/node/issues/272)) ([c587a40](https://github.com/Gitlawb/node/commit/c587a40a508acea9f74c3468d42ff037397c75d0))
* **node:** reset a peer's reachability gate when an announce changes its http_url ([#270](https://github.com/Gitlawb/node/issues/270)) ([1964f88](https://github.com/Gitlawb/node/commit/1964f88378fe6942ffca5836de3a092b5b8f67b4))
* **node:** reset a peer's reachability gate when an announce changes its http_url ([#270](https://github.com/Gitlawb/node/issues/270)) ([e2560ee](https://github.com/Gitlawb/node/commit/e2560ee46c28e4e966d87867a86dc4bad4ca7a3a))
* **node:** stamp every dequeued sync row so a stuck one yields the batch ([6e64d0d](https://github.com/Gitlawb/node/commit/6e64d0d0a7fdd38ef3a56820c1808dd7c9bbd658))
* **node:** stop a peer-supplied repo slug from escaping repos_dir ([#272](https://github.com/Gitlawb/node/issues/272)) ([c70b1df](https://github.com/Gitlawb/node/commit/c70b1df42d5c3f70ec4c272886c85fcc16ed92d1))
* **node:** validate the queued slug before the worker derives a path ([#272](https://github.com/Gitlawb/node/issues/272)) ([9f00e24](https://github.com/Gitlawb/node/commit/9f00e24aec217e4b8256dd437cff4741b1478b96))
* **review:** correct two comment claims and guard two more wrong shapes ([#270](https://github.com/Gitlawb/node/issues/270)) ([811ad50](https://github.com/Gitlawb/node/commit/811ad504bc268c8e9cddac93822adcd00bfe9ff8))
* **review:** stop the dequeue overclaiming, and pin the starvation premise ([457be11](https://github.com/Gitlawb/node/commit/457be11d728a2a34775cf96be2bc5af9a58cb250))
* strict DID matching in get_profile ([acb4714](https://github.com/Gitlawb/node/commit/acb47145beec8ce7b20b7a2479f7a55ed68e2a89))
* strict DID matching in get_profile ([13b565b](https://github.com/Gitlawb/node/commit/13b565bcac09f075a7fd555a6e9acdd27de6c7bb))
* target stored did on profile update aliases ([9836c3a](https://github.com/Gitlawb/node/commit/9836c3a21bc96a2ce95cb1fd12bd84b22142787d))
* **gl:** actually verify certificate signatures in gl cert show ([b0a2ed8](https://github.com/Gitlawb/node/commit/b0a2ed830821b5a77b0dafdcf5ce4b1ad904ec25))
* **gl:** doctor warns when a shell alias shadows the gl binary ([0765c42](https://github.com/Gitlawb/node/commit/0765c42feff47b29054b9a277881a7cd5bfd9135))
* **gl:** anchor cert --verify to a trusted issuer; pin canonical payload form ([c681af7](https://github.com/Gitlawb/node/commit/c681af70dba3a76b11795d324f2ca0fecf7e42fd))
* **gl:** detect gitlawb remotes beyond origin in gl status ([2d88937](https://github.com/Gitlawb/node/commit/2d88937fbabf21013f28debb4d4f93270a4bb8cd))
* **gl:** doctor treats a reachable local node as configuration, not failure ([94c61f7](https://github.com/Gitlawb/node/commit/94c61f79421fa26ecc1bdf7c255e0c48e66a2048))
* **gl:** make gl init's push guidance match the repo's actual state ([cb42a48](https://github.com/Gitlawb/node/commit/cb42a4830d0d070165e2212f316d009ec716dbee))
* warn about the oh-my-zsh gl alias at install time ([e3df51d](https://github.com/Gitlawb/node/commit/e3df51d76c0275fde33954ff0af0aab95fd0cb7d))
* **icaptcha-client:** solve the iCaptcha proof-of-work on answer ([#181](https://github.com/Gitlawb/node/issues/181)) ([c98b503](https://github.com/Gitlawb/node/commit/c98b503f90cade54ce9c588bd2373a4e3486f2fc))
* **gl:** point doctor's version check at Gitlawb/node, not the frozen Gitlawb/releases repo ([#198](https://github.com/Gitlawb/node/issues/198)) ([98d728c](https://github.com/Gitlawb/node/commit/98d728c325f68f0ee4a62e473dc484b7dd627504))
* **node:** carry full owner DID on ref-update wire event ([#144](https://github.com/Gitlawb/node/issues/144)) ([#145](https://github.com/Gitlawb/node/issues/145)) ([69a62e5](https://github.com/Gitlawb/node/commit/69a62e51c2d2f090f9cae88f4d8ce3b9017b7149))
* **deps:** bump crossbeam-epoch to 0.9.20 for RUSTSEC-2026-0204 ([#162](https://github.com/Gitlawb/node/issues/162)) ([#163](https://github.com/Gitlawb/node/issues/163)) ([67ad2b8](https://github.com/Gitlawb/node/commit/67ad2b876c8d9a336219d1016968de7a88fc4e75))
* **node,git:** bound a hung served git with a total-duration timeout ([#62](https://github.com/Gitlawb/node/issues/62)) ([#165](https://github.com/Gitlawb/node/issues/165)) ([cd67718](https://github.com/Gitlawb/node/commit/cd67718f49ec38726a40f6bcf36f539ccdb42969))
* **node:** bound list_ref_certificates with LIMIT and add upsert to prevent unbounded growth ([#147](https://github.com/Gitlawb/node/issues/147)) ([#149](https://github.com/Gitlawb/node/issues/149)) ([6b5e5bc](https://github.com/Gitlawb/node/commit/6b5e5bc7aee00a2d03295d3620df1ee4d8c024a2))
* **node:** gate POST /api/v1/sync/trigger and rate-limit the peer-sync routes ([#82](https://github.com/Gitlawb/node/issues/82)) ([#161](https://github.com/Gitlawb/node/issues/161)) ([d00d89a](https://github.com/Gitlawb/node/commit/d00d89ae5be992d1f63e95b714ae1bd3735e8457))
* **node:** rate-limit repo/agent creation per client IP to stop DID-farm spam floods ([#180](https://github.com/Gitlawb/node/issues/180)) ([dfcaa22](https://github.com/Gitlawb/node/commit/dfcaa22b23ec91be4c75926956aa994ca89de8d5))
* **gl:** sign the CLI's /ipfs/pins reads under the [#134](https://github.com/Gitlawb/node/issues/134) auth gate ([#146](https://github.com/Gitlawb/node/issues/146)) ([20d6848](https://github.com/Gitlawb/node/commit/20d6848846b3a988d604208833167a528b7d8820))
* **node,gossip:** route gossip HTTP through the no-redirect client ([#93](https://github.com/Gitlawb/node/issues/93)) ([#140](https://github.com/Gitlawb/node/issues/140)) ([563c456](https://github.com/Gitlawb/node/commit/563c456803bf3e958d63869db424b3940472bc3d))
* **node:** close two spam-vector root causes (trust upsert + ungated push) ([#152](https://github.com/Gitlawb/node/issues/152)) ([2df6ff9](https://github.com/Gitlawb/node/commit/2df6ff9d30de62f754fa41473e85db316021718e))
* **node:** gate GET /ipfs/{cid} on reachable allowed-set, not deny-set ([#126](https://github.com/Gitlawb/node/issues/126)) ([#133](https://github.com/Gitlawb/node/issues/133)) ([466a550](https://github.com/Gitlawb/node/commit/466a550915edd711856ef32035f9f474e2577c4f))
* **node:** gate the ref-updates feeds on read visibility ([#112](https://github.com/Gitlawb/node/issues/112), [#114](https://github.com/Gitlawb/node/issues/114)) ([#143](https://github.com/Gitlawb/node/issues/143)) ([4891db3](https://github.com/Gitlawb/node/commit/4891db38892663326ee0c1417a2db931988be4b5))
* **node:** prefer canonical repo row over mirror row in get_repo ([#124](https://github.com/Gitlawb/node/issues/124)) ([#141](https://github.com/Gitlawb/node/issues/141)) ([6c95592](https://github.com/Gitlawb/node/commit/6c95592d188222ac3446dc23ef8d9befbf82ad6f))
* **remote:** include HTTP error response body ([#137](https://github.com/Gitlawb/node/issues/137)) ([09a0cb2](https://github.com/Gitlawb/node/commit/09a0cb23b9f284ccbd69aca6958b70671f3bfb46))
* **repos:** log the cause when repo create fails ([#103](https://github.com/Gitlawb/node/issues/103)) ([2620e97](https://github.com/Gitlawb/node/commit/2620e973e3cd4835ed42ddd4adcd8183b5b3080e))
* **db:** versioned schema migrations with idempotent backfill ([#21](https://github.com/Gitlawb/node/issues/21)) ([927e4d0](https://github.com/Gitlawb/node/commit/927e4d0cfb4ea11dca6930780939afaa067797ba))
* encrypted replication for private subtrees (B1/B2/B3) for [#18](https://github.com/Gitlawb/node/issues/18) ([#36](https://github.com/Gitlawb/node/issues/36)) ([5ff7af8](https://github.com/Gitlawb/node/commit/5ff7af84fa21fab53a12dfa04a0e7fb7e7d672e6))
* **git-remote-gitlawb:** add --version and --help flags ([#30](https://github.com/Gitlawb/node/issues/30)) ([3a401eb](https://github.com/Gitlawb/node/commit/3a401eb0792f9c5f5a10de6c16861655ab3836e0))
* **gitlawb-attest:** External Attestation v1 for ref-update certs ([#20](https://github.com/Gitlawb/node/issues/20)) ([924bccd](https://github.com/Gitlawb/node/commit/924bccd8e53e9be2dc9d6d4e4f1376952d6462bb))
* **node:** blind recipient identities at rest and gate B1 by repo readability ([#40](https://github.com/Gitlawb/node/issues/40)) ([abdc775](https://github.com/Gitlawb/node/commit/abdc7757708d2bbb2bfda99bb65d50756142a42e))
* **node:** enforce per-route authorization across the REST and GraphQL surface ([#87](https://github.com/Gitlawb/node/issues/87)) ([2202b00](https://github.com/Gitlawb/node/commit/2202b0097fab6976ab366a2cdc385f1146a72f86))
* **node:** peer partial-mirrors for repos with private subtrees ([#35](https://github.com/Gitlawb/node/issues/35)) ([e365a57](https://github.com/Gitlawb/node/commit/e365a57b51c167abf06e87bfeb0565c80ed1b849))
* **node:** pin the per-push object delta instead of re-enumerating the whole repo ([#90](https://github.com/Gitlawb/node/issues/90)) ([1af4fdf](https://github.com/Gitlawb/node/commit/1af4fdf485c7084cd160551b757b1ad1eed65cc6))
* **node:** replication enforcement (Phase 2) for [#18](https://github.com/Gitlawb/node/issues/18) ([#34](https://github.com/Gitlawb/node/issues/34)) ([8680d0f](https://github.com/Gitlawb/node/commit/8680d0f9d6600bba1a52d15624f8a2802a169511))
* path-scoped repository visibility (Phase 1) for [#18](https://github.com/Gitlawb/node/issues/18) ([#25](https://github.com/Gitlawb/node/issues/25)) ([6abaf1d](https://github.com/Gitlawb/node/commit/6abaf1d7ed8fc55c6547568ae7247131311bde98))
* per-DID rate limiting on creation endpoints (10/hour) ([#13](https://github.com/Gitlawb/node/issues/13)) ([b12c6bc](https://github.com/Gitlawb/node/commit/b12c6bc3283c2647224a62fb520a3cb7acf4a747))
* **sync:** auto-register as replica with origin after successful mirror ([#56](https://github.com/Gitlawb/node/issues/56)) ([c03c9af](https://github.com/Gitlawb/node/commit/c03c9af8fadafe262a2bf3cf25e19edf7160d376))
### Bug Fixes
* **api:** blob endpoint returns 400/404 instead of 500 on bad paths ([#37](https://github.com/Gitlawb/node/issues/37)) ([b61a1bd](https://github.com/Gitlawb/node/commit/b61a1bd46ffe78b39c0967e97b0ad349eba0b046))
* **core:** route seed access through the zeroizing wrapper ([#41](https://github.com/Gitlawb/node/issues/41)) ([#64](https://github.com/Gitlawb/node/issues/64)) ([c9f43b0](https://github.com/Gitlawb/node/commit/c9f43b010576edb3c92a3be3d935cad232250344))
* **core:** zeroize the derived X25519 secret ([#65](https://github.com/Gitlawb/node/issues/65)) ([#91](https://github.com/Gitlawb/node/issues/91)) ([2f6611a](https://github.com/Gitlawb/node/commit/2f6611a30fbb66cef9991cf4c6e507548acc5038))
* **gl:** paginate gl-clone Arweave recovery and make /encrypted-blobs parsing schema-strict ([#49](https://github.com/Gitlawb/node/issues/49)) ([#70](https://github.com/Gitlawb/node/issues/70)) ([2153b0b](https://github.com/Gitlawb/node/commit/2153b0b7a67a48c29a33cd70b80cfbb69760805d))
* **infra:** drop fly idle_timeout 600 -> 120 ([#38](https://github.com/Gitlawb/node/issues/38)) ([a2217bf](https://github.com/Gitlawb/node/commit/a2217bff27ff3d6cae02bd8a12f8a0af7ce2b0a1))
* **node:** anchor the real old_sha and issue a per-ref certificate ([#72](https://github.com/Gitlawb/node/issues/72)) ([6809201](https://github.com/Gitlawb/node/commit/6809201daa223d6f833be6e95876a7b4e1f2b0b5))
* **node:** close under-withholding via full ref scope and full-history classification ([#42](https://github.com/Gitlawb/node/issues/42)) ([#84](https://github.com/Gitlawb/node/issues/84)) ([3e1e904](https://github.com/Gitlawb/node/commit/3e1e9045e4e3aa5ea0aee69767ceb01637920d2a))
* **node:** dedupe mirror and canonical repo rows on list surfaces ([#6](https://github.com/Gitlawb/node/issues/6)) ([#73](https://github.com/Gitlawb/node/issues/73)) ([3e8e333](https://github.com/Gitlawb/node/commit/3e8e333aa03d7a2fe455d5d83f23089d58feb8c9))
* **node:** enforce path-scoped visibility on the REST read API ([#52](https://github.com/Gitlawb/node/issues/52)) ([e37ea7f](https://github.com/Gitlawb/node/commit/e37ea7fec6d5a3171b526c84f884670bbbd258fb))
* **node:** fail closed when a recipient DID can't be resolved ([#47](https://github.com/Gitlawb/node/issues/47)) ([#67](https://github.com/Gitlawb/node/issues/67)) ([abc9ad0](https://github.com/Gitlawb/node/commit/abc9ad03acb48e608234650a25049622673fa53a))
* **node:** reject malformed path globs with non-trailing or empty-segment wildcards ([#74](https://github.com/Gitlawb/node/issues/74)) ([#75](https://github.com/Gitlawb/node/issues/75)) ([3d880cd](https://github.com/Gitlawb/node/commit/3d880cd56fed10e4c5ae787b52ce411384950f5e))
* **node:** skip withheld-walk when no path-scoped rule can withhold ([#60](https://github.com/Gitlawb/node/issues/60)) ([338ff83](https://github.com/Gitlawb/node/commit/338ff83584f2ab960be7e42ecec64191f5aaeb95))
* **release:** give crates explicit versions for release-please ([#129](https://github.com/Gitlawb/node/issues/129)) ([788a868](https://github.com/Gitlawb/node/commit/788a8686c1deafe03573703d72eb927cde81f54d))
└── git-remote-gitlawb/ git remote helper for gitlawb:// URLs
├── git-remote-gitlawb/ git remote helper for gitlawb:// URLs
├── gitlawb-attest/ provenance attestations for ref-update certificates
└── icaptcha-client/ client for the iCaptcha proof-of-intelligence service
docs/ Operator guides
scripts/ Build helpers
```
@@ -54,6 +56,26 @@ Smart contracts live in a separate repo: [github.com/Gitlawb/contracts](https://
3.**No breaking changes without discussion.** Open an issue first for protocol-level changes.
4.**Conventional commits.** Use `feat:`, `fix:`, `docs:`, `refactor:`, `test:`, `chore:`. Releases are automated by [release-please](https://github.com/googleapis/release-please) — your commit prefixes drive the next version bump.
5.**Format and lint.** Run `cargo fmt --all` and `cargo clippy --workspace --all-targets -- -D warnings` before submitting. CI will reject anything that fails these.
6.**Security-surface rules.** [`AGENTS.md`](AGENTS.md) carries the authorization invariants PRs most often break; it is written for coding agents and humans alike, so read it before touching endpoints, visibility, or migrations.
## What gets merged, what gets closed
We welcome contributions from humans and agents alike. To keep review sustainable, PRs are
expected to clear a basic quality bar:
- **Link an issue.** Bug fixes and features should reference an issue (`Closes #123`). For
protocol-level changes (identity, signatures, UCAN, ref certs, wire formats), open the
issue *before* writing code.
- **One change per PR.** Unrelated churn slows review and gets sent back.
- **Tests and a green pipeline.** New behavior needs tests; `cargo fmt`, `cargo clippy`,
and the full CI suite must pass.
- **A real description.** Say what changes and why. "Update code" is not a description.
A triage bot labels PRs that are missing these and leaves a short note. Nothing is closed
automatically while you're engaging. A flagged PR still missing a description after 14 days of
inactivity gets a stale warning, and is closed 7 days later if still untouched. A missing issue
link alone never triggers closure; that label stays advisory. Closed PRs can be reopened at any
time once updated.
## Development environment
@@ -86,7 +108,7 @@ cargo test -p gitlawb-node
- **TypeScript SDK** (`@gitlawb/sdk`) — client library for the HTTP API
- **Python SDK** (`gitlawb`) — for ML/agent pipeline integration
- **UCAN chain validation** — complete the auth middleware
Gitlawb Node is the open-source node software behind the Gitlawb network. It lets anyone run a self-hosted node, publish repositories under a DID, sign writes with Ed25519 HTTP signatures, replicate git activity across peers, and move toward a resilient app-delivery network where code and build assets can be served closer to users.
Gitlawb is not trying to be only “another git host.” The long-term direction is:
Gitlawb is not trying to be only "another git host." The long-term direction is:
```txt
Decentralized GitHub
@@ -19,7 +19,7 @@ The mission is simple: once code is pushed to the network, it should not disappe
## What is in this repository?
This is a Rust workspace with four crates:
This is a Rust workspace with six crates:
| Crate | Purpose |
|---|---|
@@ -27,6 +27,8 @@ This is a Rust workspace with four crates:
| `gl` | The Gitlawb CLI for identity, repos, issues, PRs, bounties, tasks, peers, node status, MCP, and setup flows. |
| `git-remote-gitlawb` | Git remote helper for `gitlawb://` URLs, so normal `git clone`, `git fetch`, and `git push` can talk to Gitlawb nodes. |
| `icaptcha-client` | Client for the iCaptcha proof-of-intelligence flow used to protect spam-prone writes. |
---
@@ -34,12 +36,12 @@ This is a Rust workspace with four crates:
Most git hosting today depends on a small number of centralized platforms. Gitlawb Nodes are designed for a different model:
- **Own your identity** — every user, agent, and node is an Ed25519 keypair represented as `did:key:z6Mk...`.
- **Signed writes by default** — write requests use RFC 9421 HTTP Signatures instead of passwords.
- **Git-native transport** — repositories are still real git repositories served over smart HTTP.
- **Agent-native workflows** — the `gl` CLI and MCP server expose repo, issue, task, PR, and UCAN flows to AI agents.
- **Peer-aware delivery** — nodes can announce, discover, gossip, and sync with each other.
- **App CDN direction** — the network can evolve from decentralized code storage into code + asset + app delivery.
- **Own your identity**: every user, agent, and node is an Ed25519 keypair represented as `did:key:z6Mk...`.
- **Signed writes by default**: write requests use RFC 9421 HTTP Signatures instead of passwords.
- **Git-native transport**: repositories are still real git repositories served over smart HTTP.
- **Agent-native workflows**: the `gl` CLI and MCP server expose repo, issue, task, PR, and UCAN flows to AI agents.
- **Peer-aware delivery**: nodes can announce, discover, gossip, and sync with each other.
- **App CDN direction**: the network can evolve from decentralized code storage into code + asset + app delivery.
---
@@ -54,6 +56,7 @@ Good today:
- Bare git repository storage.
- Git smart-HTTP clone/fetch/push.
- RFC 9421-signed writes.
- Repository and path-scoped visibility enforcement for repository and Git content reads, with 404-shaped repository denials.
- DID identities.
-`gl` CLI workflows.
- libp2p peer discovery/gossip foundation.
@@ -63,11 +66,13 @@ Good today:
Known limitations:
-Private repository read enforcement is not wired yet. Treat public nodes as public infrastructure unless you restrict access at your proxy/firewall.
- UCAN chain validation and revocation are not complete.
-Repository write authorization is not capability-complete yet; HTTP signatures prove identity, not full authorization policy.
-Repository write authorization is not secure by default: `GITLAWB_ENFORCE_OWNER_PUSH` defaults to `false` for compatibility, so a valid HTTP Signature identifies a pusher but does not enforce owner-only pushes.
- UCAN proof chains are validated when supplied, but UCAN capabilities are not consulted by write authorization and the root issuer is not independently trust-anchored. UCANs therefore do not yet grant scoped collaborator access.
-Agent lifecycle revocation is not enforced by HTTP Signature authorization; do not rely on removing or revoking an agent record to block a compromised signer.
- Read visibility is not a blanket data-classification boundary: task, IPFS-pin, and Arweave-anchor listings are not repository-gated; withheld path names can be visible to a root reader; and later visibility changes cannot retract content already announced or externally anchored.
- Peer writes are signed by upgraded nodes, but strict signed-peer enforcement is opt-in during rolling upgrades.
- GraphQL mutations need mutation-aware auth before becoming a public write surface.
- Current GraphQL mutations require an authenticated signer, but there is no mutation-specific guardrail that prevents a future mutation from omitting that check.
- Pull-request review comments do not yet have threaded line-level anchors, and merges do not enforce approval requirements.
See:
@@ -119,10 +124,18 @@ docker compose down
## Install the CLI
Use the installer once releases are available for your platform:
git push origin2 main # origin2 = gitlawb://<your-did>/memlawb (signed)
git clone gitlawb://<your-did>/memlawb # public read, no proof needed
gl doctor # preflight: identity, node, version, iCaptcha
```
Notes:
- **`requesterId` is always your DID.** The proof's `sub` claim must equal the
authenticated signer; `gl`/helper set this automatically and the node enforces
`sub == authenticated DID` (so a proof minted for another identity is rejected).
- **Proofs are short-lived (~5 min TTL) and single-use.** If one expires between
solving and use, the client transparently solves a fresh one and retries.
- **What needs what:** create / fork / register are signed **and** iCaptcha-gated;
`git push` is **signed-only** (owner signature is the gate, no per-push challenge);
reads (clone / fetch / `repo info`) need no proof. A non-existent repo returns a
clear `404`, never a placeholder.
- **API-key iCaptcha deployments:** set `GITLAWB_ICAPTCHA_URL` to your iCaptcha
origin and `GITLAWB_ICAPTCHA_API_KEY` to its key. The client only talks to an
`https` origin whose host is allowlisted (that URL or the public default), and
sends the bearer token **only** to your configured origin, never to a URL a
node advertises, so a hostile node can't capture the key or redirect the solve.
---
## Architecture
@@ -255,20 +307,21 @@ POST /api/v1/bounties/{id}/...
POST /{owner}/{repo}/git-receive-pack
```
Peer write routes support staged rollout:
These peer write routes support staged rollout:
```txt
POST /api/v1/peers/announce
POST /api/v1/sync/notify
POST /api/v1/sync/trigger
```
When `GITLAWB_REQUIRE_SIGNED_PEER_WRITES=false`, unsigned legacy peers are accepted, but signed requests are verified when signature headers are present. Once all live peers upgrade, operators can set:
When `GITLAWB_REQUIRE_SIGNED_PEER_WRITES=false`, unsigned legacy peers are accepted on those two routes, but signed requests are verified when signature headers are present. Staged rollout relaxes who may announce, not who owns a peer row. An unsigned announce may register a previously unseen peer, and it may refresh an existing row whose `http_url` is unchanged, but changing an existing peer's `http_url` requires an RFC 9421 signature from that peer's own DID and is refused with 403 otherwise. An unsigned announce can only register a `did:key`, since that is the only method whose verifying key can be resolved and therefore the only kind of row anyone could ever correct through the signed path; any other DID is refused with 400. Once all live peers upgrade, operators can set:
```bash
GITLAWB_REQUIRE_SIGNED_PEER_WRITES=true
```
`POST /api/v1/sync/trigger` is not part of the staged rollout: it always requires a signature in both config modes and returns 401 without one, because each call drives an O(peers) outbound fan-out.
---
## Configuration
@@ -295,6 +348,19 @@ Important node settings:
| `GITLAWB_REQUIRE_SIGNED_PEER_WRITES` | Require signed peer announce/sync writes. |
| `GITLAWB_AUTO_SYNC` | Enable automatic sync from known peers. |
| `GITLAWB_MAX_PACK_BYTES` | Max git pack body size for smart-HTTP routes. |
| `GITLAWB_GIT_SERVICE_TIMEOUT_SECS` | Max seconds a served git upload-pack, receive-pack, or `info/refs` advertisement may run before it is aborted (504). Default 600. Also bounds the withheld-blob classification walk (on both the upload-pack serve and receive-pack replication paths) and the push-side pin-candidate discovery (`rev-list` / `cat-file`), each reaped via process-group teardown at the deadline. On the path-scoped upload-pack path the classification walk and the pack serve share ONE deadline, so this value bounds their combined duration rather than granting each stage a full budget: a walk that consumes it leaves the serve nothing and the clone gets a 504. Serving large path-scoped repos may therefore need a higher value than they did when each stage was budgeted separately. Accepted range is 1 to 3153600000 (100 years), since the node derives deadlines from this value and a larger one cannot be represented. |
| `GITLAWB_GIT_ACQUIRE_TIMEOUT_SECS` | Max seconds the storage-acquisition phase (Tigris HEAD/GET, push advisory-lock) of a served git op may run before the request is shed with a 503, separate from the git-run timeout. The concurrency permit is released on expiry so a stalled backend cannot pin the pool. Default 30. |
| `GITLAWB_MAX_CONCURRENT_GIT_OPS` | Max concurrent served git READ ops (upload-pack and its `info/refs` advertisement) across all callers; over-cap sheds a 503 + Retry-After. Anonymous reads draw from this pool, so pair it with `GITLAWB_MAX_CONCURRENT_READS_PER_CALLER`. Pushes and the receive-pack advertisement have their own pools, so a read flood cannot shed an authenticated push. Default 128. |
| `GITLAWB_MAX_CONCURRENT_GIT_PUSHES` | Max concurrent `git-receive-pack` POST operations, in a pool separate from the read pool. The anon receive-pack `info/refs` advertisement runs in a third pool of the same size, disjoint from both, so an advertisement flood cannot shed a push either. Two per-source push caps are derived from this value (`/8`, floor 1) and have no env var of their own. Over-cap sheds a 503 + Retry-After. Default 32. |
| `GITLAWB_MAX_CONCURRENT_READS_PER_CALLER` | Max concurrent read ops a single caller may hold, so one caller cannot monopolize the read pool. Keyed on the resolved source IP, never the DID, and only as granular as `GITLAWB_TRUSTED_PROXY`: left unset, a node behind an edge or NAT keys every caller on the edge IP and this collapses to one global cap. Default 16. |
| `GITLAWB_MAX_CONCURRENT_PIN_TASKS` | Max post-push pin loops (IPFS + Pinata) running concurrently across all repos. This caps how many loops RUN at once, not how much object-id list memory the node retains: on the local IPFS path a loop parked waiting for a permit still holds its full list. Do not size memory from this knob alone. A loop over cap waits, never drops a pin. Default 8. |
| `GITLAWB_REPO_LEASE_MAX_WAITERS` | Max pushes parked at once waiting for the same repo's write lease. Each waiter pins its buffered pack body, so this bounds that memory for a hot repo; past the cap the newest push sheds a 503 + Retry-After instead of queueing. Pushes to other repos are unaffected, and the lease holder is not counted. Default 8. |
| `GITLAWB_MAX_CONCURRENT_IPFS_WALKS` | Max concurrent `GET /ipfs/{cid}` visibility walks across all callers (own pool, disjoint from the served-git pools); over-cap sheds 503. Default 32. |
| `GITLAWB_IPFS_WALK_PER_SOURCE` | Max concurrent `/ipfs` walks a single source IP may hold. Default 4. |
| `GITLAWB_IPFS_MAX_REPOS_WALKED` | Max expensive path-scope visibility walks per `/ipfs/{cid}` request; over-cap repos are skipped and the scan continues, shedding a retryable 503 (not a false 404) if the object is then found nowhere. Default 64. |
| `GITLAWB_IPFS_MAX_REPO_VISITS` | Ceiling on repos one `/ipfs/{cid}` request may visit (acquire + probe) past the visibility gate. Also the worst-case per-request Tigris fetch count. On exhaustion the scan stops with a retryable 503. Default 1024. |
| `GITLAWB_IPFS_REQUEST_BUDGET_SECS` | Absolute wall-clock budget for one admitted `/ipfs/{cid}` request's acquire+walk lifetime. Per-stage clamps bound the acquire and walk stages to the remaining budget, and no stage starts once it is exhausted; the scan then stops with a retryable 503. The object-type probe and content-read `cat-file` subprocesses are budget-checked before starting and each also run under their own deadline (the lesser of `GITLAWB_GIT_SERVICE_TIMEOUT_SECS` and the remaining budget), reaped via process-group teardown, so a hung `cat-file` cannot hold the request's walk slot past it. One hang path is still unbounded: the probe's object-store readability check is a plain filesystem sweep with nothing to reap, so a wedged filesystem can hold the slot past the deadline. Default 600. Accepted range is 1 to 3153600000 (100 years), since the node derives a deadline from this value and a larger one cannot be represented. |
| `GITLAWB_IPFS_RATE_LIMIT` | Max `/ipfs/{cid}` requests per client IP per hour (route flood brake). 0 disables. Default 600. |
- Bootstrap UCAN tokens are issued at registration.
-A supplied token's signature, audience, expiry, and proof-chain attenuation are validated.
-Tokens use a signed JSON wire format with expiry.
- Capability grants are not yet consulted by repository write authorization; see the limitations below.
**Smart contracts (Base Sepolia testnet)**
-`GitlawbDIDRegistry` — on-chain DID → document registry
@@ -48,21 +49,20 @@ We will acknowledge receipt within 48 hours and aim to release a fix within 14 d
---
## Known Limitations (Planned for v0.2)
## Known Limitations
These are **documented, accepted limitations** for the current live release and should be prioritized without breaking existing nodes during rolling upgrades.
These are documented limitations of the current live release. They should be prioritized without breaking existing nodes during rolling upgrades.
### UCAN chain validation
-The auth middleware verifies HTTP Signatures and token structure, but does not yet walk the full UCAN delegation chain.
- **Impact:** A node cannot yet enforce fine-grained capability delegation. Currently, any registered agent with a valid HTTP Signature can push.
- **Mitigation:** Keep write endpoints signed, treat public nodes as public infrastructure, and treat trust scores as soft rate-limiting signals rather than authorization.
- **Fix target:** v0.2
### Repository write authorization defaults
-`git-receive-pack` verifies HTTP Signatures, but `GITLAWB_ENFORCE_OWNER_PUSH` defaults to `false` for compatibility during rollout.
- **Impact:** With the default setting, a valid signature authenticates the pusher but does not require that DID to be the repository owner.
- **Mitigation:** Set `GITLAWB_ENFORCE_OWNER_PUSH=true` on nodes where owner-only pushes are required. Confirm that every legitimate pusher uses the owner DID before enabling it.
### UCAN revocation
-Issued UCAN tokens cannot be revoked before expiry.
-**Impact:** If a keypair is compromised, the attacker retains access until the UCAN expires (default: 30 days).
- **Mitigation:** Regenerate your identity (`gl identity new --force`) and re-register to issue a new UCAN. Until revocation/blocklisting is implemented, operators should remove compromised DIDs directly from their local database.
- **Fix target:** v0.2
### UCAN delegation and revocation
-The middleware validates a supplied UCAN's complete proof chain, but a root token is accepted without an independently trusted issuer anchor. `Ucan::can` is not yet used by write handlers, so a UCAN does not grant scoped repository access.
-Agent lifecycle revocation is not checked by HTTP Signature authorization. Removing or revoking an agent record does not itself block a compromised DID from authenticating.
- **Impact:** Do not use UCANs for collaborator permissions or rely on agent-record revocation as a key-compromise response.
- **Mitigation:** Keep sensitive deployments behind operational network controls and enable owner-only push enforcement where it fits the deployment until trusted delegation and authorization-aware revocation are implemented.
### git-receive-pack authentication
- The `git-receive-pack` endpoint enforces HTTP Signature auth. Plain Git smart-HTTP clients do not generate those headers, so the `git-remote-gitlawb` helper is required for pushes.
@@ -71,10 +71,20 @@ These are **documented, accepted limitations** for the current live release and
- **Fix target:** v0.2
### Private repository reads
- Repository records have an `is_public` field and the node exposes `GITLAWB_PUBLIC_READ`, but per-repository private-read enforcement is not wired in the current live release.
-**Impact:** Do not store private repositories or secrets on public nodes.
-**Mitigation:** Run isolated nodes for non-public data and restrict network access at the reverse proxy or firewall layer.
-**Fix target:** v0.2
- Repository and path-scoped visibility checks are enforced for repository API and Git content reads. A denied whole-repository or root read returns the same 404 shape as a missing repository, so the denial does not reveal private-repository existence.
-Sparse-clone support exposes withheld path globs to callers who may read the repository root. Do not put sensitive information in withheld path names.
-`GET /api/v1/tasks`, `/api/v1/ipfs/pins`, and `/api/v1/arweave/anchors` are not repository-gated. Task records include a UCAN token; pin and anchor listings expose object and ref metadata.
-Changing a repository's visibility controls future serving, but cannot retract ref metadata or configured external pins and anchors already announced while the repository was public. Do not push secrets to an announceable repository.
- **Impact:** Visibility policies protect the repository and Git content routes they gate, not every metadata endpoint or previously published content.
- **Remaining boundary:** This read control does not address the independent write-authorization and UCAN-delegation limitations described above.
### Pull-request review enforcement
- Pull-request review comments are not yet threaded or line-anchored, and merges do not enforce required approvals.
- **Impact:** Teams must use their own review policy or external controls for merge approval requirements.
### GraphQL mutation coverage
- Existing GraphQL mutations require an authenticated signer, but a mutation-specific source-level guardrail has not yet been added for future mutations.
- **Impact:** A new mutation could accidentally omit its signer check without an explicit test fence.
### Peer route hardening rollout
- Peer announce and sync notification routes accept signed requests and verify DID matches when a signature is present.
@@ -101,7 +111,7 @@ These are **documented, accepted limitations** for the current live release and
File diff suppressed because it is too large.
Load diff
Loaded 100 of 149 files, more files were not shown because too many files have changed in this diff.
Show more
Reference in new issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Vasanthdev2004