Compare commits

...
11 Commits
Author SHA1 Message Date
github-actions[bot] 3ec8cbf1e1 chore(main): release 0.5.1 (#158)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2026-07-10 19:46:16 +08:00
GravireiandGravirei 6b5e5bc7ae fix(node): bound list_ref_certificates with LIMIT and add upsert to prevent unbounded growth (#147) (#149)
* fix(node): bound list_ref_certificates with LIMIT and add upsert to prevent unbounded growth (#147)

* test: add INV-7 v10 upgrade-path dedup test

* fix INV-7 test: correct id-DESC tiebreaker ids and fix fmt

* fix: insert_ref_certificate returns persisted id via RETURNING; issue_ref_certificate uses it

* fix: guard cert upsert on issued_at so older certs cannot regress newer ones

* fix: .max(0) in events limit parsing, re-add lost cert tests after rebase

* address reviewer findings: return full persisted row, add monotonic clock comment, note test drift risk

* fix: add ?prefix= server-side cert lookup for CLI short-ID resolution

---------

Co-authored-by: Gravirei <[email protected]>
2026-07-10 19:44:43 +08:00
dfcaa22b23 fix(node): rate-limit repo/agent creation per client IP to stop DID-farm spam floods (#180)
* fix(node): add per-IP flood brake to repo/agent creation endpoints

The recurring spam-repo floods are not bypassing the iCaptcha gate — they
pass it. The enforced level (3) draws only machine-solvable deterministic
challenges (arithmetic/algebra/sequence), and the /v1/challenge API lets the
caller pin the easy type, so a bot mints a fresh throwaway did:key, solves a
real proof bound to that DID, and creates a repo. The per-DID creation
limiter never trips because each DID makes a single create call.

Add a per-client-IP rate limiter to the creation route group (repos,
register, fork, issues, pulls), reusing the same IpRateLimiter the push path
already uses against the identical DID-farm bypass. It wraps the auth layer
(outermost) so flood traffic is rejected before signature verification.
Default 120/hour/IP, GITLAWB_CREATE_RATE_LIMIT overrides, 0 disables.

* docs(env): document GITLAWB_CREATE_RATE_LIMIT in .env.example

* fix: apply CodeRabbit auto-fixes

Fixed 1 file(s) based on 1 unresolved review comment.

Co-authored-by: CodeRabbit <[email protected]>

* test(node): unit-test the per-IP creation brake runs before auth

The route-level coverage (repo_creation_is_rate_limited_by_ip) is
#[sqlx::test] and needs Postgres, so it is skipped in the plain unit-test
pass. Add a DB-free middleware test that layers the per-IP creation limiter
over an always-401 auth stand-in and asserts an exhausted IP is rejected
with 429 before auth runs — the ordering the DID-farm flood defense relies
on.

---------

Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
Co-authored-by: CodeRabbit <[email protected]>
2026-07-10 19:44:04 +08:00
2109d08e2c Fix/db startup resilience (#170)
* Keep node alive during database outages

* Fix startup-resilience gaps: reachable 503s, bounded retries, seamless handoff

- Route sqlx errors buried in anyhow chains back to AppError::Db and widen
  the unavailable class (Io/Tls/pool), so DB outages return 503 instead of
  500 — the previous 503 arm was unreachable dead code
- Bound each connect+migrate attempt (GITLAWB_DB_CONNECT_TIMEOUT_SECS) so a
  held migration advisory lock cannot wedge the startup retry loop forever
- Share one bound socket between the degraded and full servers and stop
  awaiting the degraded drain: no closed-port window during the handoff and
  no slow client can block the full server from starting
- Probe the database from /ready (2s bound) and gate Fly routing/deploys on
  it, so mid-life outages drain traffic and a bad DATABASE_URL fails the
  deploy visibly instead of going green; classify auth/catalog errors as
  likely-permanent (error-level log, straight to max backoff)
- Start the metrics listener before the DB connects so the degraded window
  is observable; degraded /health now returns 503 so peer pings and uptime
  monitors stop counting a DB-less node as alive
- Move the five GITLAWB_DB_* knobs onto clap Config (validated, in --help),
  drop the redundant outer connect timeout, default the pool to 20 conns,
  fix the backoff exponent cap that defeated GITLAWB_DB_RETRY_MAX_SECS, and
  strip at most one .git suffix in smart-HTTP repo names

* fix: apply CodeRabbit auto-fixes

Fixed 1 file(s) based on 1 unresolved review comment.

Co-authored-by: CodeRabbit <[email protected]>

---------

Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
Co-authored-by: CodeRabbit <[email protected]>
2026-07-10 09:44:02 +08:00
GravireiandGravirei c5e5099def refactor(visibility): route is_owner through the shared did_matches matcher (#153) (#156)
* fix(node): gate GET /ipfs/{cid} on reachable allowed-set, not deny-set (#126)

The IPFS visibility gate used withheld_blob_oids (a deny-set enumerating
only reachable blobs), so a dangling/unreachable blob was absent from the
set and served in cleartext to anonymous callers. Flip to an allowed-set
(allowed_blob_set_for_caller) that enumerates reachable blobs the caller
may read: a dangling blob has no path, is never in the set, and 404s.

* perf(ipfs): check object existence before allowed-blob walk

Move store::read_object before the allowed_blob_set_for_caller
spawn_blocking call so random-CID spray against repos with
path-scoped rules cannot trigger full-history git walks on
repos that don't carry the object.

* refactor(ipfs): improve formatting and readability in get_by_cid function

✓ P3 blocker fixed: cargo fmt applied — the format gate will pass.

  ✓ P3 cleanup resolved: withheld_blob_oids is still used by replication code in repos.rs, so it stays.

  • P2 follow-up: Tree/commit disclosure tracked in #135 — out of scope here.

* refactor(ipfs): streamline object retrieval by separating type and content reading

* docs(ipfs): update get_by_cid comment to reflect split object retrieval

* Run cargo fmt on store.rs

* refactor(visibility): route is_owner through the shared did_matches matcher (#153)

* style(profiles): remove redundant borrow in format arg

---------

Co-authored-by: Gravirei <[email protected]>
2026-07-10 08:55:49 +08:00
beardthelionandt cd67718f49 fix(node,git): bound a hung served git with a total-duration timeout (#62) (#165)
* feat(node,git): bound a hung served git with a total-duration timeout (#62)

Part 1 of #62. A served git upload-pack/receive-pack that neither finishes
nor disconnects parks forever, pinning a git PID (and, for receive-pack,
holding the write lock); on a public repo an anonymous caller can trigger
it. No tower TimeoutLayer exists and axum imposes no default.

Wrap the child interaction in tokio::time::timeout, bounded by a new
GITLAWB_GIT_SERVICE_TIMEOUT_SECS knob (default 600, must be positive).
Drain stdout/stderr concurrently with the stdin write so a large body
can't deadlock. On expiry run_git_service returns a typed GitServiceTimeout
and reaps the WHOLE process group before returning (SIGTERM so git clears
its .git/*.lock, wait for the leader and its grandchildren like index-pack
to exit, escalate to SIGKILL past a grace, hard-capped), so a caller
releasing the receive-pack write lock can't race a still-live git on the
same repo.

The handlers map GitServiceTimeout to a distinct 504 (not the generic 500
git error, and clear of the read-gate 404 / auth 401 the client keys
retries on) via a pure, unit-testable classifier rather than matching the
anyhow string. Surface git's own non-zero exit (its stderr) before a
stdin-write EPIPE so a malformed body is classified 400, not 500.

Out of scope, deferred in #62: the info/refs advertisement and the
withheld-blob path (spawn_blocking, which tokio timeout cannot cancel);
both remain unbounded (noted in the config knob's docs).

Tests: the timeout fires and tears the group down; it reaps the whole
group before returning (RED if the reap is leader-only); the wrap is
load-bearing (RED via an outer bound, not a hang); the error must be the
typed timeout; the 504 mapping and classifier are unit-tested; config
rejects 0. Full suite green.

* fix(review): harden #62 timeout tests and drop a dead reap flag

- test(node): cover the `protocol error` arm of git_service_app_error
  independently (the input has no "bad line length" substring, so it
  isolates the second classifier arm and goes RED if that arm is removed)
- test(node): make the reap-before-return test's grandchild IGNORE SIGTERM
  (`trap "" TERM`) and outlive the ~4s reap cap, so the SIGKILL escalation
  is load-bearing. Previously the grandchild self-exited under the cap, so
  neutering the SIGKILL still passed; the escalation was exercised but not
  required. Verified: RED with SIGKILL disabled, GREEN with it.
- test(node): add a receive-pack timeout test. Every prior timeout test used
  an upload-pack fake; this proves the push path (which also holds the repo
  write lock) is bounded too. RED-verified with the internal timeout removed.
- test(node): poll for the pidfile in the reap-before-return timeout test
  so a loaded runner can't false-panic before the fake git writes it
- remove the redundant `sigkilled` guard in reap_group_on_timeout; step
  == 200 fires exactly once in the 0..400 loop, so no re-entry guard is
  needed and the SIGKILL still fires exactly once

* fix(node): log malformed receive-pack as warn, not error (#165)

git_receive_pack's error classifier routed every non-Timeout error,
including client-caused BadRequest (400), through tracing::error!. Mirror
the git_upload_pack arm so a malformed push logs at warn level instead of
as a server error.

* test(node,git): make smart_http timeout tests pass under the parallel runner (#165)

Addresses jatmn's review of 41eca0b.

[P2] The #62 timeout tests failed under `cargo test --workspace`:
- run_git_service_tears_down_group_when_future_dropped re-polled a completed
  future ("async fn resumed after completion"): the advance-until-pids loop
  ignored the timeout's Ok(_) (future-finished) case and polled the resolved
  future again. It now breaks on early completion.
- Under fork-storm load a freshly-written fake `git` transiently fails to exec
  (ETXTBSY) or is timed out before recording its pids. Added a
  fake_git_run_with_pids retry helper (used by the four pid-reading tests) plus
  a matching inline retry on the drop test, with growing backoff for the
  correlated bursts and a 12-attempt cap that still fails loudly on a genuine
  never-spawns regression. Reverted two tests from a speculative 2000ms bound
  back to 300ms/1000ms now that the retry covers the timeout-vs-pidfile race.

[P3] Documented GITLAWB_GIT_SERVICE_TIMEOUT_SECS in .env.example and the README
config table (default 600; bounds upload-pack/receive-pack, not info/refs or the
withheld-blob path).

The tests stay load-bearing: breaking process_group(0), the post-reap disarm,
the timeout-arm reap, or the internal timeout each still turns the corresponding
test red (mutation-verified).

---------

Co-authored-by: t <t@t>
2026-07-10 08:54:59 +08:00
GravireiandGravirei 6bafaa6dc5 fix(bounties): add tests for claim_bounty repo-read gate (#160) (#169)
* fix(node): gate GET /ipfs/{cid} on reachable allowed-set, not deny-set (#126)

The IPFS visibility gate used withheld_blob_oids (a deny-set enumerating
only reachable blobs), so a dangling/unreachable blob was absent from the
set and served in cleartext to anonymous callers. Flip to an allowed-set
(allowed_blob_set_for_caller) that enumerates reachable blobs the caller
may read: a dangling blob has no path, is never in the set, and 404s.

* perf(ipfs): check object existence before allowed-blob walk

Move store::read_object before the allowed_blob_set_for_caller
spawn_blocking call so random-CID spray against repos with
path-scoped rules cannot trigger full-history git walks on
repos that don't carry the object.

* refactor(ipfs): improve formatting and readability in get_by_cid function

✓ P3 blocker fixed: cargo fmt applied — the format gate will pass.

  ✓ P3 cleanup resolved: withheld_blob_oids is still used by replication code in repos.rs, so it stays.

  • P2 follow-up: Tree/commit disclosure tracked in #135 — out of scope here.

* refactor(ipfs): streamline object retrieval by separating type and content reading

* docs(ipfs): update get_by_cid comment to reflect split object retrieval

* Run cargo fmt on store.rs

* fix(bounties): add tests for claim_bounty repo-read gate (#160)

* fix(profiles): remove redundant borrow in format arg

---------

Co-authored-by: Gravirei <[email protected]>
2026-07-10 08:39:10 +08:00
GravireiandGravirei 26bc3f6987 fix(visibility): gate repo-scoped read surfaces on visibility (#120) (#157)
* fix(node): gate GET /ipfs/{cid} on reachable allowed-set, not deny-set (#126)

The IPFS visibility gate used withheld_blob_oids (a deny-set enumerating
only reachable blobs), so a dangling/unreachable blob was absent from the
set and served in cleartext to anonymous callers. Flip to an allowed-set
(allowed_blob_set_for_caller) that enumerates reachable blobs the caller
may read: a dangling blob has no path, is never in the set, and 404s.

* perf(ipfs): check object existence before allowed-blob walk

Move store::read_object before the allowed_blob_set_for_caller
spawn_blocking call so random-CID spray against repos with
path-scoped rules cannot trigger full-history git walks on
repos that don't carry the object.

* refactor(ipfs): improve formatting and readability in get_by_cid function

✓ P3 blocker fixed: cargo fmt applied — the format gate will pass.

  ✓ P3 cleanup resolved: withheld_blob_oids is still used by replication code in repos.rs, so it stays.

  • P2 follow-up: Tree/commit disclosure tracked in #135 — out of scope here.

* refactor(ipfs): streamline object retrieval by separating type and content reading

* docs(ipfs): update get_by_cid comment to reflect split object retrieval

* Run cargo fmt on store.rs

* fix(visibility): gate repo-scoped read surfaces on visibility (#120)

* fix(visibility): enhance repo read authorization checks and improve test coverage

* Fix review feedback: clamp limit, memoize authz, identical 404 body, resolved issue id in comments

* cargo update -p crossbeam-epoch: fix RUSTSEC-2026-0204 (invalid pointer dereference)

* Fix review findings: correct authz memoization, page past hidden bounties, gate star/unstar

* Fix findings: sign CLI/MCP reads, keyset pagination, gate claim_bounty

---------

Co-authored-by: Gravirei <[email protected]>
2026-07-08 08:46:19 +08:00
beardthelionandt d00d89ae5b fix(node): gate POST /api/v1/sync/trigger and rate-limit the peer-sync routes (#82) (#161)
* feat(node): add per-client-IP limiters + knobs for the peer-sync routes (#82)

Plumb two dedicated per-IP RateLimiters into config/state so the peer-sync
enqueue routes can be braked independently: sync_trigger_rate_limiter (tight,
GITLAWB_SYNC_TRIGGER_RATE_LIMIT, default 60/hr) and peer_write_rate_limiter
(generous, GITLAWB_PEER_WRITE_RATE_LIMIT, default 600/hr). Both are bounded and
swept by the periodic cleanup loop, keyed via the node-wide push_limiter_trust.
No route is wired to them yet; that lands next.

* fix(node): require a signature on /sync/trigger and rate-limit the peer-sync routes (#82)

/api/v1/sync/trigger was reachable unauthenticated in the default config and
drove an O(peers) outbound fan-out + per-repo sync_queue enqueue — an anonymous
amplification/DoS. The same enqueue_sink is reachable via /sync/notify, which
also accepts unsigned known-peer requests and had no rate limit.

- Move /sync/trigger into its own always-signed route group (add_auth_layers),
  independent of require_signed_peer_writes, and give it a tight per-IP brake.
- Add a separate, generous per-IP brake to the peer-write group
  (/peers/announce, /sync/notify); notify's signature behavior is unchanged
  (rolling-upgrade compat), it only gains the brake. Separate buckets so an
  unsigned notify flood can't drain the signed trigger caller's quota.
- Per-IP (not per-DID): a did:key farm self-registers, so a signature/per-DID
  gate does not cap cost (INV-10); the brake keys on the client IP before auth.

Adds an adversarial test matrix (unsigned->401 both modes, IP flood->429,
DID-agnostic brake, forged X-Forwarded-For can't bypass, notify flood->429,
distinct-IP not throttled, cross-route non-exhaustion, 0-disables-but-sig-holds,
announce still accepts unsigned).

* fix(gl): surface node denial on sync trigger and sanitize the error body (#82)

Now that the node gates /api/v1/sync/trigger behind a signature and rate limit,
'gl sync trigger' must not render a 401/429 as a fabricated '✓ sync triggered /
0 peers' success. Check resp.status() before parsing; on a non-2xx, extract the
node's message and bail. Strip C0/C1 control bytes from the node-supplied string
before printing (INV-6) — a hostile node could embed ANSI/OSC escapes in the
body. Adds mockito tests: 401/429 surface as errors (RED without the status
check), control bytes are stripped, and the 200 path is unchanged.

* fix(review): bound gl error-body read, strip bidi controls, genericize 429 message (#82)

Code-review follow-ups on the #82 change:
- gl sync trigger: read the node error body with a capped reader
  (read_body_capped) instead of resp.text(), so a hostile/broken node can't
  force an unbounded allocation to surface a denial (INV-6, read half).
- sanitize_node_msg: also strip Unicode bidi/format controls (U+202A-202E,
  U+2066-2069, LRM/RLM/ALM) — is_control() only covers the Cc category, but a
  right-to-left override can reorder the displayed error line.
- too_many_requests(): the shared per-IP 429 middleware now serves the peer-sync
  routes too, so drop the push-specific wording ('rate limit exceeded'); the
  push-specific 429 in the receive-pack handler is unchanged.
Adds tests for the bidi/length sanitize and the oversized-body path.

* test(#82): close reasoned-not-run gaps with load-bearing tests

Executes the branches that were previously only reasoned:
- read_body_capped byte-cap test (RED against resp.text() — 2MB body -> ≤cap):
  vets the bounded-read fix, not just the display cap.
- sync-route 429 body asserts 'rate limit exceeded' and NOT 'push' (RED before
  the message was genericized); also finishes genericizing the warn log, which
  the response-body change had missed.
- real RFC 9421 signature through the full router reaches the handler (200) in
  both config modes — the positive path through the actual gate, not a
  direct-mount with an injected DID.
- /peers/announce flood -> 429 (the peer_write brake covers announce, not just
  notify).
- trigger_counts extraction unit test (happy + malformed -> 0,0), closing the
  count-formatting gap in the 200 path.
No production behavior change beyond the warn-log wording.

* fix(#82): require an identity for sync trigger and fix the rollout docs

Address review feedback on the always-signed /sync/trigger route.

- gl sync trigger now requires a local identity via load_keypair_from_dir and
  fails locally with "identity not found" before issuing a request, instead of
  .ok() silently sending an unsigned request that can only 401 remotely. Add a
  test proving it fails before dialing the node.
- Split /sync/trigger out of the staged unsigned peer-write rollout in the
  README and the OSS-readiness audit; only /peers/announce and /sync/notify keep
  the rolling-upgrade unsigned behavior.
- Reorder the two peer-sync rate-limit vars in .env.example for dotenv-linter.
- Bump crossbeam-epoch to 0.9.20 (RUSTSEC-2026-0204), matching main, so cargo
  audit passes.

---------

Co-authored-by: t <t@t>
2026-07-08 08:44:55 +08:00
beardthelionandt 67ad2b876c fix(deps): bump crossbeam-epoch to 0.9.20 for RUSTSEC-2026-0204 (#162) (#163)
RUSTSEC-2026-0204 (invalid pointer dereference in crossbeam-epoch's
fmt::Pointer impl for Atomic/Shared) hard-fails cargo audit on every branch,
since the advisory's addition to the RustSec db. crossbeam-epoch is a fully
transitive dependency (moka -> hickory-resolver -> libp2p-dns), and the
advisory's prescribed fix is >= 0.9.20; moka 0.12.15 accepts >=0.9.18,<0.10.0
so the bump resolves with no other graph changes.

Lockfile-only, single crate. Verified: cargo audit exits 0 and the workspace
builds clean with 0.9.20.

Closes #162.

Co-authored-by: t <t@t>
2026-07-07 22:46:44 +08:00
Kevin CodexandClaude Fable 5 6cff5286b4 fix(release): build aarch64-musl natively on arm64 runners, replace retired macos-13 (#155)
The aarch64-unknown-linux-musl leg cross-compiled a musl target with the
glibc toolchain (CC=aarch64-linux-gnu-gcc), which aws-lc-sys cannot build
with — the Build step failed on every release and no aarch64-linux asset
has ever shipped. Build it natively on ubuntu-24.04-arm instead (free for
public repos), using the same stock musl-tools recipe as the x86_64 leg,
and drop the smoke-test skip since the binaries now run on their build
runner.

The x86_64-apple-darwin leg targeted macos-13, which GitHub has retired;
the job queues forever and holds the release concurrency group open. Move
it to macos-15-intel.

Both failures also skipped the downstream npm-publish, homebrew-bump, and
web-sync jobs, so v0.4.0 and v0.5.0 were never published there.

Co-authored-by: Claude Fable 5 <[email protected]>
2026-07-06 10:39:12 +08:00
43 changed files with 4691 additions and 335 deletions

No files matched your search

+42
View File
@@ -21,6 +21,23 @@ GITLAWB_REPOS_DIR=/data/repos
# When using the bundled docker-compose, this is wired automatically.
DATABASE_URL=postgresql://gitlawb:changeme@localhost:5432/gitlawb
# ── Database pool & startup resilience ────────────────────────────────────
# Maximum connections in the PostgreSQL pool. A cap, not a floor —
# connections open lazily. Size against the DB server's max_connections,
# remembering admin tooling opens its own pool.
GITLAWB_DB_MAX_CONNECTIONS=20
# Seconds a request waits for a pool connection before failing with 503.
GITLAWB_DB_ACQUIRE_TIMEOUT_SECS=5
# Upper bound on each startup connect+migrate attempt, in seconds. Keep it
# generous enough for a peer instance to finish migrating (the migration
# advisory lock is held across a deploy); on expiry the attempt is retried.
GITLAWB_DB_CONNECT_TIMEOUT_SECS=60
# Startup retry backoff while the database is unavailable: first delay and
# ceiling, in seconds. The node stays up serving 503s (with a truthful
# /ready) instead of crash-looping.
GITLAWB_DB_RETRY_INITIAL_SECS=5
GITLAWB_DB_RETRY_MAX_SECS=60
# ── IPFS pinning (Pinata) ─────────────────────────────────────────────────
# Get a JWT at https://app.pinata.cloud/developers/api-keys
GITLAWB_PINATA_JWT=
@@ -90,11 +107,36 @@ GITLAWB_PUBLIC_READ=true
# Maximum git smart-HTTP pack request size, in bytes.
GITLAWB_MAX_PACK_BYTES=2147483648
# Max seconds a served git upload-pack / receive-pack (clone / push) may run
# before it is aborted with a 504. Bounds a hung git that would otherwise pin a
# worker and, on push, the repo write lock. Does NOT cover the info/refs
# advertisement or the withheld-blob path, which remain unbounded. Default 600.
GITLAWB_GIT_SERVICE_TIMEOUT_SECS=600
# ── Push rate limiting (git-receive-pack flood brake) ─────────────────────
# Max receive-pack requests (info/refs advertisement + push POST) per client
# IP per hour. 0 disables. Default 600.
GITLAWB_PUSH_RATE_LIMIT=600
# ── Creation rate limiting (repo/agent/issue/PR flood brake) ──────────────
# Max creation requests (POST /api/v1/repos, /api/register, fork, issues,
# pulls) per client IP per hour, in addition to the per-DID limit. The per-DID
# limit alone is bypassed by a did:key farm (one throwaway identity per repo),
# which is how spam-repo floods slip past it and the iCaptcha gate; the per-IP
# brake caps a single-source flood. Uses GITLAWB_TRUSTED_PROXY below to resolve
# the client IP. 0 disables. Default 120.
GITLAWB_CREATE_RATE_LIMIT=120
# ── Peer-sync rate limiting (per client IP, uses GITLAWB_TRUSTED_PROXY below) ─
# /api/v1/peers/announce and /api/v1/sync/notify accept unsigned requests from
# known peers and run at higher frequency, so a generous bucket. Separate from
# the trigger bucket so an unsigned notify flood can't drain trigger's quota.
# 0 disables. Default 600.
GITLAWB_PEER_WRITE_RATE_LIMIT=600
# /api/v1/sync/trigger requires a signature and fans out to every peer per call,
# so it gets a tight bucket. 0 disables. Default 60.
GITLAWB_SYNC_TRIGGER_RATE_LIMIT=60
# Which forwarded header the edge is trusted to set, used to resolve the real
# client IP for the push limiter. One of:
# (unset) — no trusted proxy: key on the socket peer address, ignore headers.
+10 -10
View File
@@ -116,10 +116,14 @@ jobs:
include:
- target: x86_64-unknown-linux-musl
os: ubuntu-latest
# Native arm64 runner: cross-compiling this target with the glibc
# toolchain broke aws-lc-sys, and native lets the smoke test run.
- target: aarch64-unknown-linux-musl
os: ubuntu-latest
os: ubuntu-24.04-arm
# macos-13 (the last plain Intel image) is retired and queues forever;
# macos-15-intel is GitHub's supported Intel label.
- target: x86_64-apple-darwin
os: macos-13
os: macos-15-intel
- target: aarch64-apple-darwin
os: macos-14
- target: x86_64-pc-windows-msvc
@@ -137,16 +141,13 @@ jobs:
toolchain: stable
targets: ${{ matrix.target }}
# Both musl targets build natively on a matching-arch runner, so the stock
# musl-gcc wrapper is all that's needed (no cross toolchain).
- name: Install musl tools (linux)
if: contains(matrix.target, 'linux-musl')
run: |
sudo apt-get update
sudo apt-get install -y musl-tools
if [ "${{ matrix.target }}" = "aarch64-unknown-linux-musl" ]; then
sudo apt-get install -y gcc-aarch64-linux-gnu
echo "CARGO_TARGET_AARCH64_UNKNOWN_LINUX_MUSL_LINKER=aarch64-linux-gnu-gcc" >> "$GITHUB_ENV"
echo "CC_aarch64_unknown_linux_musl=aarch64-linux-gnu-gcc" >> "$GITHUB_ENV"
fi
- name: Cache cargo
uses: Swatinem/rust-cache@98c8021b550208e191a6a3145459bfc9fb29c4c0 # v2.8.0
@@ -171,10 +172,9 @@ jobs:
cargo build --release --target ${{ matrix.target }} $args
# Run each packaged binary's --version so a broken release artifact fails the
# build instead of shipping. Skipped for cross-compiled targets that can't run
# on the host runner (aarch64 linux on an x86_64 runner).
# build instead of shipping. Every target builds on a matching-arch runner,
# so all of them can execute here.
- name: Smoke test binaries
if: ${{ matrix.target != 'aarch64-unknown-linux-musl' }}
shell: bash
run: |
set -euo pipefail
+1 -1
View File
@@ -1,3 +1,3 @@
{
".": "0.5.0"
".": "0.5.1"
}
+14
View File
@@ -1,5 +1,19 @@
# Changelog
## [0.5.1](https://github.com/Gitlawb/node/compare/v0.5.0...v0.5.1) (2026-07-10)
### Bug Fixes
* **bounties:** add tests for claim_bounty repo-read gate ([#160](https://github.com/Gitlawb/node/issues/160)) ([#169](https://github.com/Gitlawb/node/issues/169)) ([6bafaa6](https://github.com/Gitlawb/node/commit/6bafaa6dc5f05c0dcd61c708397afddbcf8c2e3f))
* **deps:** bump crossbeam-epoch to 0.9.20 for RUSTSEC-2026-0204 ([#162](https://github.com/Gitlawb/node/issues/162)) ([#163](https://github.com/Gitlawb/node/issues/163)) ([67ad2b8](https://github.com/Gitlawb/node/commit/67ad2b876c8d9a336219d1016968de7a88fc4e75))
* **node,git:** bound a hung served git with a total-duration timeout ([#62](https://github.com/Gitlawb/node/issues/62)) ([#165](https://github.com/Gitlawb/node/issues/165)) ([cd67718](https://github.com/Gitlawb/node/commit/cd67718f49ec38726a40f6bcf36f539ccdb42969))
* **node:** bound list_ref_certificates with LIMIT and add upsert to prevent unbounded growth ([#147](https://github.com/Gitlawb/node/issues/147)) ([#149](https://github.com/Gitlawb/node/issues/149)) ([6b5e5bc](https://github.com/Gitlawb/node/commit/6b5e5bc7aee00a2d03295d3620df1ee4d8c024a2))
* **node:** gate POST /api/v1/sync/trigger and rate-limit the peer-sync routes ([#82](https://github.com/Gitlawb/node/issues/82)) ([#161](https://github.com/Gitlawb/node/issues/161)) ([d00d89a](https://github.com/Gitlawb/node/commit/d00d89ae5be992d1f63e95b714ae1bd3735e8457))
* **node:** rate-limit repo/agent creation per client IP to stop DID-farm spam floods ([#180](https://github.com/Gitlawb/node/issues/180)) ([dfcaa22](https://github.com/Gitlawb/node/commit/dfcaa22b23ec91be4c75926956aa994ca89de8d5))
* **release:** build aarch64-musl natively on arm64 runners, replace retired macos-13 ([#155](https://github.com/Gitlawb/node/issues/155)) ([6cff528](https://github.com/Gitlawb/node/commit/6cff5286b436fdee44ad881999e2a5f4bdba18f9))
* **visibility:** gate repo-scoped read surfaces on visibility ([#120](https://github.com/Gitlawb/node/issues/120)) ([#157](https://github.com/Gitlawb/node/issues/157)) ([26bc3f6](https://github.com/Gitlawb/node/commit/26bc3f69870aa77e43c0d92115a5aa59555b7d88))
## [0.5.0](https://github.com/Gitlawb/node/compare/v0.4.0...v0.5.0) (2026-07-05)
Generated
+7 -7
View File
@@ -2293,9 +2293,9 @@ dependencies = [
[[package]]
name = "crossbeam-epoch"
version = "0.9.18"
version = "0.9.20"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5b82ac4a3c2ca9c3460964f020e1402edd5753411d7737aa39c3714ad1b5420e"
checksum = "2d6914041f254d6e9176c01941b21115dcfb7089e55135a35411081bd106ef3f"
dependencies = [
"crossbeam-utils",
]
@@ -3300,7 +3300,7 @@ dependencies = [
[[package]]
name = "git-remote-gitlawb"
version = "0.4.0"
version = "0.5.0"
dependencies = [
"anyhow",
"gitlawb-core",
@@ -3312,7 +3312,7 @@ dependencies = [
[[package]]
name = "gitlawb-attest"
version = "0.4.0"
version = "0.5.0"
dependencies = [
"base64",
"ed25519-dalek",
@@ -3329,7 +3329,7 @@ dependencies = [
[[package]]
name = "gitlawb-core"
version = "0.4.0"
version = "0.5.0"
dependencies = [
"anyhow",
"base64",
@@ -3356,7 +3356,7 @@ dependencies = [
[[package]]
name = "gitlawb-node"
version = "0.4.0"
version = "0.5.0"
dependencies = [
"alloy",
"anyhow",
@@ -3412,7 +3412,7 @@ dependencies = [
[[package]]
name = "gl"
version = "0.4.0"
version = "0.5.0"
dependencies = [
"alloy",
"anyhow",
+5 -3
View File
@@ -302,20 +302,21 @@ POST /api/v1/bounties/{id}/...
POST /{owner}/{repo}/git-receive-pack
```
Peer write routes support staged rollout:
These peer write routes support staged rollout:
```txt
POST /api/v1/peers/announce
POST /api/v1/sync/notify
POST /api/v1/sync/trigger
```
When `GITLAWB_REQUIRE_SIGNED_PEER_WRITES=false`, unsigned legacy peers are accepted, but signed requests are verified when signature headers are present. Once all live peers upgrade, operators can set:
When `GITLAWB_REQUIRE_SIGNED_PEER_WRITES=false`, unsigned legacy peers are accepted on those two routes, but signed requests are verified when signature headers are present. Once all live peers upgrade, operators can set:
```bash
GITLAWB_REQUIRE_SIGNED_PEER_WRITES=true
```
`POST /api/v1/sync/trigger` is not part of the staged rollout: it always requires a signature in both config modes and returns 401 without one, because each call drives an O(peers) outbound fan-out.
---
## Configuration
@@ -342,6 +343,7 @@ Important node settings:
| `GITLAWB_REQUIRE_SIGNED_PEER_WRITES` | Require signed peer announce/sync writes. |
| `GITLAWB_AUTO_SYNC` | Enable automatic sync from known peers. |
| `GITLAWB_MAX_PACK_BYTES` | Max git pack body size for smart-HTTP routes. |
| `GITLAWB_GIT_SERVICE_TIMEOUT_SECS` | Max seconds a served git upload-pack/receive-pack may run before it is aborted (504). Default 600. Does not bound `info/refs` or the withheld-blob path. |
| `GITLAWB_TIGRIS_BUCKET` | Optional S3/Tigris shared repo storage bucket. |
| `GITLAWB_PINATA_JWT` | Optional Pinata/IPFS warm-storage pinning. |
| `GITLAWB_IRYS_URL` | Optional Irys/Arweave permanent anchoring. |
+1 -1
View File
@@ -1,7 +1,7 @@
[package]
name = "git-remote-gitlawb"
description = "Git remote helper — enables 'git clone gitlawb://did:gitlawb:...'"
version = "0.5.0" # x-release-please-version
version = "0.5.1" # x-release-please-version
edition.workspace = true
rust-version.workspace = true
license.workspace = true
+1 -1
View File
@@ -1,7 +1,7 @@
[package]
name = "gitlawb-attest"
description = "External Attestation v1: pluggable provenance attachments for gitlawb ref-update certs"
version = "0.5.0" # x-release-please-version
version = "0.5.1" # x-release-please-version
edition.workspace = true
rust-version.workspace = true
license.workspace = true
+1 -1
View File
@@ -1,7 +1,7 @@
[package]
name = "gitlawb-core"
description = "Core cryptographic primitives for the gitlawb network: DIDs, CIDs, UCAN, HTTP Signatures, ref-update certificates"
version = "0.5.0" # x-release-please-version
version = "0.5.1" # x-release-please-version
edition.workspace = true
rust-version.workspace = true
license.workspace = true
+1 -1
View File
@@ -1,7 +1,7 @@
[package]
name = "gitlawb-node"
description = "The gitlawb node daemon — git hosting over HTTP with DID auth"
version = "0.5.0" # x-release-please-version
version = "0.5.1" # x-release-please-version
edition.workspace = true
rust-version.workspace = true
license.workspace = true
+103 -16
View File
@@ -110,19 +110,28 @@ pub async fn create_bounty(
Ok((StatusCode::CREATED, Json(bounty)))
}
const MAX_BOUNTY_LIMIT: i64 = 200;
/// GET /api/v1/repos/{owner}/{repo}/bounties
pub async fn list_repo_bounties(
State(state): State<AppState>,
Path((owner, repo)): Path<(String, String)>,
Query(q): Query<ListBountiesQuery>,
auth: Option<Extension<AuthenticatedDid>>,
) -> Result<Json<serde_json::Value>> {
let caller = auth.as_ref().map(|e| e.0 .0.as_str());
crate::api::authorize_repo_read(&state, &owner, &repo, caller, "/").await?;
let limit = q.limit.unwrap_or(50).clamp(1, MAX_BOUNTY_LIMIT);
let bounties = state
.db
.list_bounties(
Some(&owner),
Some(&repo),
q.status.as_deref(),
q.limit.unwrap_or(50),
limit,
None,
None,
)
.await?;
@@ -133,25 +142,92 @@ pub async fn list_repo_bounties(
pub async fn list_all_bounties(
State(state): State<AppState>,
Query(q): Query<ListBountiesQuery>,
auth: Option<Extension<AuthenticatedDid>>,
) -> Result<Json<serde_json::Value>> {
let bounties = state
.db
.list_bounties(None, None, q.status.as_deref(), q.limit.unwrap_or(50))
.await?;
let raw_limit = q.limit.unwrap_or(50);
let limit = raw_limit.clamp(1, MAX_BOUNTY_LIMIT);
Ok(Json(serde_json::json!({ "bounties": bounties })))
let caller = auth.as_ref().map(|e| e.0 .0.as_str());
let mut memo = std::collections::HashMap::new();
let mut allowed: Vec<BountyRecord> = Vec::new();
let page_size = (limit * 5).clamp(1, 200);
let max_scanned = 10_000i64;
let mut total_scanned: i64 = 0;
let mut cursor: Option<(String, String)> = None;
while (allowed.len() as i64) < limit {
let after = cursor.as_ref().map(|(ts, id)| (ts.as_str(), id.as_str()));
let bounties = state
.db
.list_bounties(
None,
None,
q.status.as_deref(),
page_size,
after.map(|(ts, _)| ts),
after.map(|(_, id)| id),
)
.await?;
if bounties.is_empty() {
break;
}
total_scanned += bounties.len() as i64;
if total_scanned > max_scanned {
return Err(AppError::Incomplete(
"too many rows scanned; refine your filters".into(),
));
}
let last = bounties.last().unwrap();
for b in &bounties {
if (allowed.len() as i64) >= limit {
break;
}
let key = format!("{}/{}", b.repo_owner, b.repo_name);
let admitted = if let Some(&result) = memo.get(&key) {
result
} else {
let result = crate::api::authorize_repo_read(
&state,
&b.repo_owner,
&b.repo_name,
caller,
"/",
)
.await
.is_ok();
memo.insert(key, result);
result
};
if admitted {
allowed.push(b.clone());
}
}
cursor = Some((last.created_at.clone(), last.id.clone()));
}
Ok(Json(serde_json::json!({ "bounties": allowed })))
}
/// GET /api/v1/bounties/{id}
pub async fn get_bounty(
State(state): State<AppState>,
Path(id): Path<String>,
auth: Option<Extension<AuthenticatedDid>>,
) -> Result<Json<BountyRecord>> {
let bounty = state
.db
.get_bounty(&id)
.await?
.ok_or_else(|| AppError::NotFound(format!("bounty {id} not found")))?;
let not_found = || AppError::NotFound(format!("bounty {id} not found"));
let bounty = state.db.get_bounty(&id).await?.ok_or_else(not_found)?;
let caller = auth.as_ref().map(|e| e.0 .0.as_str());
if crate::api::authorize_repo_read(&state, &bounty.repo_owner, &bounty.repo_name, caller, "/")
.await
.is_err()
{
return Err(not_found());
}
Ok(Json(bounty))
}
@@ -162,11 +238,22 @@ pub async fn claim_bounty(
Path(id): Path<String>,
Json(req): Json<ClaimBountyRequest>,
) -> Result<Json<BountyRecord>> {
let bounty = state
.db
.get_bounty(&id)
.await?
.ok_or_else(|| AppError::NotFound(format!("bounty {id} not found")))?;
let not_found = || AppError::NotFound(format!("bounty {id} not found"));
let bounty = state.db.get_bounty(&id).await?.ok_or_else(not_found)?;
if crate::api::authorize_repo_read(
&state,
&bounty.repo_owner,
&bounty.repo_name,
Some(auth.0.as_str()),
"/",
)
.await
.is_err()
{
return Err(not_found());
}
if bounty.status != "open" {
return Err(AppError::BadRequest(format!(
+41 -15
View File
@@ -1,23 +1,44 @@
//! API handlers for ref certificates.
use axum::extract::{Path, State};
use std::collections::HashMap;
use axum::extract::{Extension, Path, Query, State};
use axum::Json;
use crate::auth::AuthenticatedDid;
use crate::error::{AppError, Result};
use crate::state::AppState;
/// GET /api/v1/repos/{owner}/{repo}/certs
/// GET /api/v1/repos/{owner}/{repo}/certs?limit=50
pub async fn list_certs(
State(state): State<AppState>,
Path((owner, name)): Path<(String, String)>,
Query(params): Query<HashMap<String, String>>,
auth: Option<Extension<AuthenticatedDid>>,
) -> Result<Json<serde_json::Value>> {
let record = state
.db
.get_repo(&owner, &name)
.await?
.ok_or_else(|| AppError::RepoNotFound(format!("{owner}/{name}")))?;
let limit = params
.get("limit")
.and_then(|v| v.parse::<i64>().ok())
.map(|v| v.max(1))
.unwrap_or(50)
.min(200);
let certs = state.db.list_ref_certificates(&record.id).await?;
let caller = auth.as_ref().map(|e| e.0 .0.as_str());
let (record, _rules) =
crate::api::authorize_repo_read(&state, &owner, &name, caller, "/").await?;
// When a prefix is given (short-ID resolution from the CLI) use a
// generous limit and delegate to the prefix-matched query so the
// caller can resolve IDs regardless of how many certs exist.
let prefix = params.get("prefix").filter(|p| !p.is_empty());
let certs = if let Some(prefix) = prefix {
state
.db
.list_ref_certificates_by_prefix(&record.id, prefix, 200)
.await?
} else {
state.db.list_ref_certificates(&record.id, limit).await?
};
let certs_json: Vec<serde_json::Value> = certs
.iter()
.map(|c| {
@@ -35,20 +56,21 @@ pub async fn list_certs(
})
.collect();
Ok(Json(serde_json::json!({ "certificates": certs_json })))
let count = certs_json.len();
Ok(Json(
serde_json::json!({ "certificates": certs_json, "count": count }),
))
}
/// GET /api/v1/repos/{owner}/{repo}/certs/{id}
pub async fn get_cert(
State(state): State<AppState>,
Path((owner, name, id)): Path<(String, String, String)>,
auth: Option<Extension<AuthenticatedDid>>,
) -> Result<Json<serde_json::Value>> {
// Verify the repo exists
let _record = state
.db
.get_repo(&owner, &name)
.await?
.ok_or_else(|| AppError::RepoNotFound(format!("{owner}/{name}")))?;
let caller = auth.as_ref().map(|e| e.0 .0.as_str());
let (record, _rules) =
crate::api::authorize_repo_read(&state, &owner, &name, caller, "/").await?;
let cert = state
.db
@@ -56,6 +78,10 @@ pub async fn get_cert(
.await?
.ok_or_else(|| AppError::NotFound(format!("certificate {id}")))?;
if cert.repo_id != record.id {
return Err(AppError::NotFound(format!("certificate {id}")));
}
Ok(Json(serde_json::json!({
"id": cert.id,
"repo_id": cert.repo_id,
+3 -1
View File
@@ -129,6 +129,7 @@ pub async fn list_ref_updates(
let limit = params
.get("limit")
.and_then(|v| v.parse::<i64>().ok())
.map(|v| v.max(0))
.unwrap_or(50)
.clamp(0, MAX_VISIBLE_REF_UPDATES);
@@ -177,6 +178,7 @@ pub async fn list_repo_events(
let limit = params
.get("limit")
.and_then(|v| v.parse::<i64>().ok())
.map(|v| v.max(0))
.unwrap_or(50)
.clamp(0, MAX_VISIBLE_REF_UPDATES);
@@ -209,7 +211,7 @@ pub async fn list_repo_events(
// into an empty 200, matching the gossip half below.
let cert_events: Vec<serde_json::Value> = state
.db
.list_ref_certificates(&record.id)
.list_ref_certificates(&record.id, limit)
.await?
.iter()
.map(|c| {
+31 -16
View File
@@ -90,12 +90,11 @@ pub async fn create_issue(
pub async fn list_issues(
State(state): State<AppState>,
Path((owner, repo)): Path<(String, String)>,
auth: Option<Extension<AuthenticatedDid>>,
) -> Result<Json<serde_json::Value>> {
let record = state
.db
.get_repo(&owner, &repo)
.await?
.ok_or_else(|| AppError::RepoNotFound(format!("{owner}/{repo}")))?;
let caller = auth.as_ref().map(|e| e.0 .0.as_str());
let (record, _rules) =
crate::api::authorize_repo_read(&state, &owner, &repo, caller, "/").await?;
let disk_path = state
.repo_store
@@ -120,12 +119,11 @@ pub async fn list_issues(
pub async fn get_issue(
State(state): State<AppState>,
Path((owner, repo, issue_id)): Path<(String, String, String)>,
auth: Option<Extension<AuthenticatedDid>>,
) -> Result<Json<serde_json::Value>> {
let record = state
.db
.get_repo(&owner, &repo)
.await?
.ok_or_else(|| AppError::RepoNotFound(format!("{owner}/{repo}")))?;
let caller = auth.as_ref().map(|e| e.0 .0.as_str());
let (record, _rules) =
crate::api::authorize_repo_read(&state, &owner, &repo, caller, "/").await?;
let disk_path = state
.repo_store
@@ -191,14 +189,31 @@ pub async fn create_issue_comment(
pub async fn list_issue_comments(
State(state): State<AppState>,
Path((owner, repo, issue_id)): Path<(String, String, String)>,
auth: Option<Extension<AuthenticatedDid>>,
) -> Result<Json<serde_json::Value>> {
let _record = state
.db
.get_repo(&owner, &repo)
.await?
.ok_or_else(|| AppError::RepoNotFound(format!("{owner}/{repo}")))?;
let caller = auth.as_ref().map(|e| e.0 .0.as_str());
let (record, _rules) =
crate::api::authorize_repo_read(&state, &owner, &repo, caller, "/").await?;
let comments = state.db.list_issue_comments(&issue_id).await?;
let disk_path = state
.repo_store
.acquire(&record.owner_did, &record.name)
.await
.map_err(|e| AppError::Git(e.to_string()))?;
// Resolve the full issue ID (accepts 8-char prefix) so the DB fetch
// below uses the same canonical id as the git ref.
let full_id = match git_issues::resolve_issue_id(&disk_path, &issue_id)
.map_err(|e| AppError::Git(e.to_string()))?
{
Some(id) => id,
None => {
return Err(AppError::RepoNotFound(format!(
"issue {issue_id} not found"
)))
}
};
let comments = state.db.list_issue_comments(&full_id).await?;
Ok(Json(serde_json::json!({ "comments": comments })))
}
+4 -5
View File
@@ -74,12 +74,11 @@ pub async fn remove_label(
pub async fn list_labels(
State(state): State<AppState>,
Path((owner, name)): Path<(String, String)>,
auth: Option<Extension<AuthenticatedDid>>,
) -> Result<Json<serde_json::Value>> {
let record = state
.db
.get_repo(&owner, &name)
.await?
.ok_or_else(|| AppError::RepoNotFound(format!("{owner}/{name}")))?;
let caller = auth.as_ref().map(|e| e.0 .0.as_str());
let (record, _rules) =
crate::api::authorize_repo_read(&state, &owner, &name, caller, "/").await?;
let labels = state.db.list_labels(&record.id).await?;
Ok(Json(serde_json::json!({ "labels": labels })))
+6 -8
View File
@@ -114,6 +114,12 @@ mod did_tests {
assert!(!did_matches("did:key:zABC", "did:key:zXYZ"));
assert!(!did_matches("zABC", "zXYZ"));
}
#[test]
fn empty_did_matches() {
assert!(did_matches("", ""));
assert!(did_matches("", "did:key:"));
}
}
/// Drift guard (plan 002 §Gate-type table, Step 5). Every in-scope mutation
@@ -504,14 +510,6 @@ mod authz_guard {
// Repo-scoped reads known to be ungated today, each tracked by an issue.
// Remove an entry the moment its gate lands (the staleness assert enforces it).
let known_ungated: &[(&str, &str)] = &[
("list_certs", "#120"),
("get_cert", "#120"),
("list_issues", "#120"),
("get_issue", "#120"),
("list_issue_comments", "#120"),
("list_labels", "#120"),
("list_repo_bounties", "#120"),
("get_star_status", "#120"),
("list_webhooks", "#94 (PR #113)"),
("list_replicas", "PR #113"),
("list_protected_branches", "PR #113"),
+454
View File
@@ -537,4 +537,458 @@ mod tests {
// pins the `o[0] == 0` check against an off-by-one into 1.x.
assert!(is_public_http_url("http://1.0.0.0/"));
}
// ── #82: /sync/trigger signature gate + per-IP brake on the peer-sync routes ──
//
// These drive the FULL production router (crate::server::build_router) so the
// route wiring, layer order, and config-mode branching are all under test.
// The positive-path and DID-agnostic cases mount the handler directly because
// signed_request_as injects only an AuthenticatedDid extension (not a real
// RFC-9421 signature), which require_signature rejects on the full router.
use crate::rate_limit::{IpRateLimiter, RateLimiter, TrustedProxy};
use crate::state::AppState;
use crate::test_support::{signed_request_as, test_state};
use axum::body::Body;
use axum::extract::ConnectInfo;
use axum::http::{header, Method, Request, StatusCode};
use axum::routing::post;
use axum::{middleware, Extension, Router};
use gitlawb_core::http_sig::sign_request;
use gitlawb_core::identity::Keypair;
use sqlx::PgPool;
use std::net::SocketAddr;
use std::sync::Arc;
use std::time::Duration;
use tower::ServiceExt;
fn unsigned_post(uri: &str, body: &str, peer: &str) -> Request<Body> {
let mut req = Request::builder()
.method(Method::POST)
.uri(uri)
.header(header::CONTENT_TYPE, "application/json")
.body(Body::from(body.to_string()))
.unwrap();
req.extensions_mut()
.insert(ConnectInfo(peer.parse::<SocketAddr>().unwrap()));
req
}
fn require_signed_peer_writes(state: &mut AppState) {
let mut cfg = (*state.config).clone();
cfg.require_signed_peer_writes = true;
state.config = Arc::new(cfg);
}
const NOTIFY_BODY: &str = r#"{"repo":"demo","ref_name":"refs/heads/main","new_sha":"0000000000000000000000000000000000000000","node_did":"PEER_DID"}"#;
// ── trigger: mandatory signature ──────────────────────────────────────────
#[sqlx::test]
async fn sync_trigger_rejects_unsigned_in_default_mode(pool: PgPool) {
// The hole: with require_signed_peer_writes=false (default), an anonymous
// caller reaches the fan-out. Must be 401 regardless of the flag.
let state = test_state(pool).await;
let router = crate::server::build_router(state);
let resp = router
.oneshot(unsigned_post(
"/api/v1/sync/trigger",
"{}",
"203.0.113.1:5000",
))
.await
.unwrap();
assert_eq!(resp.status(), StatusCode::UNAUTHORIZED);
}
#[sqlx::test]
async fn sync_trigger_rejects_unsigned_in_signed_writes_mode(pool: PgPool) {
// The signature requirement must not depend on the flag.
let mut state = test_state(pool).await;
require_signed_peer_writes(&mut state);
let router = crate::server::build_router(state);
let resp = router
.oneshot(unsigned_post(
"/api/v1/sync/trigger",
"{}",
"203.0.113.2:5000",
))
.await
.unwrap();
assert_eq!(resp.status(), StatusCode::UNAUTHORIZED);
}
#[sqlx::test]
async fn sync_trigger_admits_signed_caller(pool: PgPool) {
// must-not-over-throttle: a signed caller reaches the handler and (with no
// peers seeded) gets 200, not 429. Direct-mount because a real signature
// cannot be forged through the full router in tests.
let state = test_state(pool).await;
let did = Keypair::generate().did().to_string();
let app = Router::new()
.route("/api/v1/sync/trigger", post(super::trigger_sync))
.layer(middleware::from_fn(crate::rate_limit::rate_limit_by_ip))
.layer(Extension(IpRateLimiter {
limiter: RateLimiter::new(60, Duration::from_secs(3600)),
trust: TrustedProxy::None,
}))
.with_state(state);
let mut req =
signed_request_as(&did, Method::POST, "/api/v1/sync/trigger", Body::from("{}"));
req.extensions_mut().insert(ConnectInfo(
"203.0.113.3:5000".parse::<SocketAddr>().unwrap(),
));
let resp = app.oneshot(req).await.unwrap();
assert_eq!(resp.status(), StatusCode::OK);
}
// ── trigger: IP brake ─────────────────────────────────────────────────────
#[sqlx::test]
async fn sync_trigger_ip_flood_is_throttled(pool: PgPool) {
// The brake is outermost, so an over-limit request 429s before it ever
// reaches require_signature — an unsigned request suffices to prove it.
let mut state = test_state(pool).await;
state.sync_trigger_rate_limiter = RateLimiter::new(1, Duration::from_secs(3600));
state.push_limiter_trust = TrustedProxy::None;
let peer = "203.0.113.4:5000";
// Exhaust the single-request budget up front.
assert!(
state
.sync_trigger_rate_limiter
.check(&peer.parse::<SocketAddr>().unwrap().ip().to_string())
.await
);
let router = crate::server::build_router(state);
let resp = router
.oneshot(unsigned_post("/api/v1/sync/trigger", "{}", peer))
.await
.unwrap();
assert_eq!(resp.status(), StatusCode::TOO_MANY_REQUESTS);
}
#[sqlx::test]
async fn sync_trigger_brake_is_did_agnostic(pool: PgPool) {
// Why per-IP, not per-DID: a DID farm (fresh did:key per request) does not
// bypass an IP-keyed brake. Two DISTINCT DIDs from one IP, brake limit 1 →
// the second is 429. A per-DID limiter would give each DID its own bucket.
let state = test_state(pool).await;
let app = Router::new()
.route("/api/v1/sync/trigger", post(super::trigger_sync))
.layer(middleware::from_fn(crate::rate_limit::rate_limit_by_ip))
.layer(Extension(IpRateLimiter {
limiter: RateLimiter::new(1, Duration::from_secs(3600)),
trust: TrustedProxy::None,
}))
.with_state(state);
let peer = "203.0.113.5:5000".parse::<SocketAddr>().unwrap();
let mk = |did: &str| {
let mut r =
signed_request_as(did, Method::POST, "/api/v1/sync/trigger", Body::from("{}"));
r.extensions_mut().insert(ConnectInfo(peer));
r
};
let first = app
.clone()
.oneshot(mk(&Keypair::generate().did().to_string()))
.await
.unwrap();
assert_eq!(first.status(), StatusCode::OK);
let second = app
.oneshot(mk(&Keypair::generate().did().to_string()))
.await
.unwrap();
assert_eq!(second.status(), StatusCode::TOO_MANY_REQUESTS);
}
#[sqlx::test]
async fn sync_trigger_forged_forwarded_header_cannot_bypass(pool: PgPool) {
// TrustedProxy::None keys on the socket peer, so rotating X-Forwarded-For
// from one socket peer does not refill the bucket. limit 1: first request
// consumes it (then 401 unsigned), second from the same socket → 429
// regardless of a different XFF value.
let mut state = test_state(pool).await;
state.sync_trigger_rate_limiter = RateLimiter::new(1, Duration::from_secs(3600));
state.push_limiter_trust = TrustedProxy::None;
let router = crate::server::build_router(state);
let mut a = unsigned_post("/api/v1/sync/trigger", "{}", "203.0.113.6:5000");
a.headers_mut()
.insert("x-forwarded-for", "1.1.1.1".parse().unwrap());
assert_eq!(
router.clone().oneshot(a).await.unwrap().status(),
StatusCode::UNAUTHORIZED
);
let mut b = unsigned_post("/api/v1/sync/trigger", "{}", "203.0.113.6:5000");
b.headers_mut()
.insert("x-forwarded-for", "2.2.2.2".parse().unwrap());
assert_eq!(
router.oneshot(b).await.unwrap().status(),
StatusCode::TOO_MANY_REQUESTS
);
}
#[sqlx::test]
async fn sync_trigger_rate_limit_zero_disables_but_signature_holds(pool: PgPool) {
// 0 disables the brake (RateLimiter::check early-returns), proving the two
// halves are independent: no 429 even under a flood, but the signature gate
// still 401s every unsigned request.
let mut state = test_state(pool).await;
state.sync_trigger_rate_limiter = RateLimiter::new(0, Duration::from_secs(3600));
state.push_limiter_trust = TrustedProxy::None;
let router = crate::server::build_router(state);
for _ in 0..3 {
let resp = router
.clone()
.oneshot(unsigned_post(
"/api/v1/sync/trigger",
"{}",
"203.0.113.7:5000",
))
.await
.unwrap();
assert_eq!(resp.status(), StatusCode::UNAUTHORIZED);
}
}
// ── notify: braked, but signature behavior unchanged ──────────────────────
async fn seed_peer(state: &AppState) -> String {
let did = Keypair::generate().did().to_string();
state
.db
.upsert_peer(&did, "https://peer.example.com")
.await
.unwrap();
did
}
#[sqlx::test]
async fn sync_notify_unsigned_flood_is_throttled(pool: PgPool) {
// notify still accepts an unsigned known-peer notification (rolling-upgrade
// compat), but a flood from one IP is now braked. brake limit 1: first
// enqueues (200), second from the same IP → 429.
let mut state = test_state(pool).await;
state.peer_write_rate_limiter = RateLimiter::new(1, Duration::from_secs(3600));
state.push_limiter_trust = TrustedProxy::None;
let peer_did = seed_peer(&state).await;
let body = NOTIFY_BODY.replace("PEER_DID", &peer_did);
let router = crate::server::build_router(state);
let first = router
.clone()
.oneshot(unsigned_post(
"/api/v1/sync/notify",
&body,
"203.0.113.8:5000",
))
.await
.unwrap();
assert_eq!(first.status(), StatusCode::OK);
let second = router
.oneshot(unsigned_post(
"/api/v1/sync/notify",
&body,
"203.0.113.8:5000",
))
.await
.unwrap();
assert_eq!(second.status(), StatusCode::TOO_MANY_REQUESTS);
}
#[sqlx::test]
async fn sync_notify_distinct_ips_are_not_collaterally_throttled(pool: PgPool) {
// must-not-over-throttle: per-IP keying means one peer's volume does not
// throttle another. brake limit 1, two DIFFERENT source IPs → both 200.
let mut state = test_state(pool).await;
state.peer_write_rate_limiter = RateLimiter::new(1, Duration::from_secs(3600));
state.push_limiter_trust = TrustedProxy::None;
let peer_did = seed_peer(&state).await;
let body = NOTIFY_BODY.replace("PEER_DID", &peer_did);
let router = crate::server::build_router(state);
for ip in ["203.0.113.20:5000", "203.0.113.21:5000"] {
let resp = router
.clone()
.oneshot(unsigned_post("/api/v1/sync/notify", &body, ip))
.await
.unwrap();
assert_eq!(
resp.status(),
StatusCode::OK,
"distinct IP {ip} must not be throttled"
);
}
}
#[sqlx::test]
async fn notify_flood_does_not_exhaust_trigger_bucket(pool: PgPool) {
// Separate buckets: draining peer_write via an unsigned notify flood must
// NOT throttle the signed trigger caller from the same IP. After the notify
// bucket is spent, an unsigned trigger from that IP hits its own (unspent)
// bucket and is rejected by the SIGNATURE gate (401), not the brake (429).
let mut state = test_state(pool).await;
state.peer_write_rate_limiter = RateLimiter::new(1, Duration::from_secs(3600));
state.sync_trigger_rate_limiter = RateLimiter::new(60, Duration::from_secs(3600));
state.push_limiter_trust = TrustedProxy::None;
let peer_did = seed_peer(&state).await;
let body = NOTIFY_BODY.replace("PEER_DID", &peer_did);
let router = crate::server::build_router(state);
let ip = "203.0.113.9:5000";
assert_eq!(
router
.clone()
.oneshot(unsigned_post("/api/v1/sync/notify", &body, ip))
.await
.unwrap()
.status(),
StatusCode::OK
);
assert_eq!(
router
.clone()
.oneshot(unsigned_post("/api/v1/sync/notify", &body, ip))
.await
.unwrap()
.status(),
StatusCode::TOO_MANY_REQUESTS,
"peer_write bucket should now be exhausted for this IP"
);
let trigger = router
.oneshot(unsigned_post("/api/v1/sync/trigger", "{}", ip))
.await
.unwrap();
assert_eq!(
trigger.status(),
StatusCode::UNAUTHORIZED,
"trigger must hit its own bucket (401 from the sig gate), not the drained peer_write bucket (429)"
);
}
#[sqlx::test]
async fn announce_still_accepts_unsigned_in_default_mode(pool: PgPool) {
// must-not-over-reach: adding the brake must not tighten announce's
// rolling-upgrade behavior — an unsigned announce with a public URL still
// succeeds (it only gains the brake, which a single request is under).
let state = test_state(pool).await;
let did = Keypair::generate().did().to_string();
let body = format!(r#"{{"did":"{did}","http_url":"https://peer.example.com"}}"#);
let router = crate::server::build_router(state);
let resp = router
.oneshot(unsigned_post(
"/api/v1/peers/announce",
&body,
"203.0.113.10:5000",
))
.await
.unwrap();
assert_eq!(resp.status(), StatusCode::OK);
}
// Build a request carrying a REAL RFC 9421 signature (not just an injected
// extension), so it passes require_signature on the full production router.
fn real_signed_trigger(peer: &str) -> Request<Body> {
let kp = Keypair::generate();
let body = b"{}";
let s = sign_request(&kp, "POST", "/api/v1/sync/trigger", body);
let mut req = Request::builder()
.method(Method::POST)
.uri("/api/v1/sync/trigger")
.header(header::CONTENT_TYPE, "application/json")
.header("content-digest", s.content_digest)
.header("signature-input", s.signature_input)
.header("signature", s.signature)
.body(Body::from(body.to_vec()))
.unwrap();
req.extensions_mut()
.insert(ConnectInfo(peer.parse::<SocketAddr>().unwrap()));
req
}
#[sqlx::test]
async fn sync_trigger_admits_real_signature_through_full_router(pool: PgPool) {
// The positive path through the REAL gate: a validly-signed trigger passes
// require_signature on the full router and reaches the handler (no peers
// seeded → 200), in BOTH config modes.
let state = test_state(pool.clone()).await;
let resp = crate::server::build_router(state)
.oneshot(real_signed_trigger("203.0.113.30:5000"))
.await
.unwrap();
assert_eq!(
resp.status(),
StatusCode::OK,
"a real signature must reach the handler (default mode)"
);
let mut state = test_state(pool).await;
require_signed_peer_writes(&mut state);
let resp = crate::server::build_router(state)
.oneshot(real_signed_trigger("203.0.113.30:5001"))
.await
.unwrap();
assert_eq!(
resp.status(),
StatusCode::OK,
"a real signature must reach the handler (signed-writes mode)"
);
}
#[sqlx::test]
async fn announce_flood_is_throttled(pool: PgPool) {
// The peer_write brake covers /peers/announce too (co-benefit): limit 1,
// first unsigned announce from an IP is 200, the second from that IP 429.
let mut state = test_state(pool).await;
state.peer_write_rate_limiter = RateLimiter::new(1, Duration::from_secs(3600));
state.push_limiter_trust = TrustedProxy::None;
let did = Keypair::generate().did().to_string();
let body = format!(r#"{{"did":"{did}","http_url":"https://peer.example.com"}}"#);
let router = crate::server::build_router(state);
let ip = "203.0.113.31:5000";
assert_eq!(
router
.clone()
.oneshot(unsigned_post("/api/v1/peers/announce", &body, ip))
.await
.unwrap()
.status(),
StatusCode::OK
);
assert_eq!(
router
.oneshot(unsigned_post("/api/v1/peers/announce", &body, ip))
.await
.unwrap()
.status(),
StatusCode::TOO_MANY_REQUESTS,
"announce must be braked by the peer_write limiter"
);
}
#[sqlx::test]
async fn sync_trigger_429_body_is_generic_not_push(pool: PgPool) {
// The shared 429 middleware now serves the sync routes, so its body must
// not claim "push" (RED before the message was genericized).
let mut state = test_state(pool).await;
state.sync_trigger_rate_limiter = RateLimiter::new(1, Duration::from_secs(3600));
state.push_limiter_trust = TrustedProxy::None;
let peer = "203.0.113.32:5000";
assert!(
state
.sync_trigger_rate_limiter
.check(&peer.parse::<SocketAddr>().unwrap().ip().to_string())
.await
);
let router = crate::server::build_router(state);
let resp = router
.oneshot(unsigned_post("/api/v1/sync/trigger", "{}", peer))
.await
.unwrap();
assert_eq!(resp.status(), StatusCode::TOO_MANY_REQUESTS);
let bytes = axum::body::to_bytes(resp.into_body(), usize::MAX)
.await
.unwrap();
let s = String::from_utf8_lossy(&bytes);
assert!(s.contains("rate limit exceeded"), "429 body: {s:?}");
assert!(
!s.contains("push"),
"a 429 on a sync route must not say 'push': {s:?}"
);
}
}
+1 -1
View File
@@ -98,7 +98,7 @@ pub async fn set_profile(
&state.http_client,
&state.config.pinata_upload_url,
&state.config.pinata_jwt,
&format!("profile-{}", &did),
&format!("profile-{}", did),
&profile_json,
)
.await
+146 -15
View File
@@ -487,6 +487,17 @@ pub async fn get_tree(
// ── Git smart HTTP endpoints ──────────────────────────────────────────────
fn smart_http_repo_name(repo: &str) -> Result<&str> {
// Strip at most one ".git" suffix: trim_end_matches strips repeatedly,
// which would misdirect a repo literally named "foo.git" (creatable via
// the peer mirror path, which skips API name validation) to repo "foo".
let name = repo.strip_suffix(".git").unwrap_or(repo);
if name.is_empty() {
return Err(AppError::BadRequest("missing repository name".into()));
}
Ok(name)
}
/// GET /:owner/:repo.git/info/refs?service=git-upload-pack|git-receive-pack
pub async fn git_info_refs(
State(state): State<AppState>,
@@ -496,7 +507,7 @@ pub async fn git_info_refs(
headers: axum::http::HeaderMap,
auth: Option<Extension<AuthenticatedDid>>,
) -> Result<Response> {
let name = repo.trim_end_matches(".git");
let name = smart_http_repo_name(&repo)?;
tracing::info!(owner = %owner, repo = %name, "info/refs request");
let record = state
.db
@@ -578,6 +589,26 @@ pub async fn git_info_refs(
})
}
/// Map an error from a `smart_http` git service call to the right `AppError`:
/// [`smart_http::GitServiceTimeout`] to 504, a malformed client request to 400,
/// anything else to a 500 git error. Pure (no logging) so it is unit-testable;
/// callers add their own tracing.
fn git_service_app_error(err: &anyhow::Error) -> AppError {
if err
.downcast_ref::<smart_http::GitServiceTimeout>()
.is_some()
{
AppError::Timeout("git service timed out".into())
} else {
let msg = err.to_string();
if msg.contains("bad line length") || msg.contains("protocol error") {
AppError::BadRequest(msg)
} else {
AppError::Git(msg)
}
}
}
/// POST /:owner/:repo.git/git-upload-pack
pub async fn git_upload_pack(
State(state): State<AppState>,
@@ -585,7 +616,7 @@ pub async fn git_upload_pack(
auth: Option<Extension<AuthenticatedDid>>,
body: Bytes,
) -> Result<Response> {
let name = repo.trim_end_matches(".git");
let name = smart_http_repo_name(&repo)?;
let record = state
.db
.get_repo(&owner, name)
@@ -615,8 +646,9 @@ pub async fn git_upload_pack(
// No path-scoped rule can withhold an individual blob, and the whole-repo
// "/" gate above already enforced repo-level access. Skip the per-blob
// withheld walk and serve the pack directly.
let git_timeout = std::time::Duration::from_secs(state.config.git_service_timeout_secs);
let resp = if !visibility_pack::has_path_scoped_rule(&rules) {
smart_http::upload_pack(&disk_path, body).await
smart_http::upload_pack(&disk_path, body, git_timeout).await
} else {
// withheld_blob_oids walks every ref with blocking `git ls-tree`; keep
// that off the async worker thread.
@@ -641,21 +673,22 @@ pub async fn git_upload_pack(
};
if withheld.is_empty() {
smart_http::upload_pack(&disk_path, body).await
smart_http::upload_pack(&disk_path, body, git_timeout).await
} else {
tracing::info!(repo = %name, caller = ?caller, withheld = withheld.len(), "serving filtered pack");
smart_http::upload_pack_excluding(&disk_path, body, &withheld).await
}
}
.map_err(|e| {
let msg = e.to_string();
if msg.contains("bad line length") || msg.contains("protocol error") {
tracing::warn!(repo = %name, err = %msg, "git-upload-pack: bad client request");
AppError::BadRequest(msg)
} else {
tracing::error!(repo = %name, err = %msg, "git-upload-pack failed");
AppError::Git(msg)
let app = git_service_app_error(&e);
match &app {
AppError::Timeout(_) => tracing::warn!(repo = %name, "git-upload-pack timed out"),
AppError::BadRequest(msg) => {
tracing::warn!(repo = %name, err = %msg, "git-upload-pack: bad client request")
}
_ => tracing::error!(repo = %name, err = %e, "git-upload-pack failed"),
}
app
})?;
crate::metrics::record_fetch(&format!("{owner}/{name}"));
crate::metrics::observe_pack_size(body_len as f64);
@@ -820,7 +853,7 @@ pub async fn git_receive_pack(
Extension(auth): Extension<AuthenticatedDid>,
body: Bytes,
) -> Result<Response> {
let name = repo.trim_end_matches(".git");
let name = smart_http_repo_name(&repo)?;
tracing::info!(owner = %owner, repo = %name, "receive-pack request");
let record = state
.db
@@ -902,7 +935,8 @@ pub async fn git_receive_pack(
let disk_path = guard.path().to_path_buf();
tracing::debug!(repo = %name, path = %disk_path.display(), "running git receive-pack");
let body_len = body.len();
let receive_result = smart_http::receive_pack(&disk_path, body).await;
let git_timeout = std::time::Duration::from_secs(state.config.git_service_timeout_secs);
let receive_result = smart_http::receive_pack(&disk_path, body, git_timeout).await;
// Always release the advisory lock — even on error — to prevent stale locks
// from blocking subsequent pushes. Only upload to Tigris when the push
@@ -910,8 +944,15 @@ pub async fn git_receive_pack(
guard.release(receive_result.is_ok()).await;
let result = receive_result.map_err(|e| {
tracing::error!(repo = %name, err = %e, "git receive-pack failed");
AppError::Git(e.to_string())
let app = git_service_app_error(&e);
match &app {
AppError::Timeout(_) => tracing::warn!(repo = %name, "git receive-pack timed out"),
AppError::BadRequest(msg) => {
tracing::warn!(repo = %name, err = %msg, "git receive-pack: bad client request")
}
_ => tracing::error!(repo = %name, err = %e, "git receive-pack failed"),
}
app
})?;
// Update the repo's updated_at timestamp after a successful push
@@ -1795,6 +1836,32 @@ mod tests {
const OWNER_SHORT: &str = "z6MkpTHR8VNsBxYAAWHut2Geadd9jSwuBV8xRoAnwWsdvktH";
const STRANGER_DID: &str = "did:key:z6Mkffonly5tranger0000000000000000000000000000000";
#[test]
fn git_service_app_error_classifies_timeout_bad_request_and_git() {
// GitServiceTimeout carried through anyhow -> 504 Timeout.
let timeout_err: anyhow::Error = smart_http::GitServiceTimeout.into();
assert!(matches!(
git_service_app_error(&timeout_err),
AppError::Timeout(_)
));
// A malformed client request -> 400.
let bad = anyhow::anyhow!("fatal: bad line length character: 0000");
assert!(matches!(
git_service_app_error(&bad),
AppError::BadRequest(_)
));
// The `protocol error` marker (with no "bad line length" substring) also
// -> 400, exercising the second arm of the classifier independently.
let proto = anyhow::anyhow!("fatal: protocol error: unexpected flush packet");
assert!(matches!(
git_service_app_error(&proto),
AppError::BadRequest(_)
));
// Anything else -> 500 git error.
let other = anyhow::anyhow!("some other git failure");
assert!(matches!(git_service_app_error(&other), AppError::Git(_)));
}
fn repo_owned_by(owner_did: &str) -> crate::db::RepoRecord {
let now = chrono::Utc::now();
crate::db::RepoRecord {
@@ -1847,6 +1914,23 @@ mod tests {
);
}
#[test]
fn smart_http_repo_name_rejects_empty_after_git_suffix() {
assert_eq!(smart_http_repo_name("demo.git").unwrap(), "demo");
assert_eq!(smart_http_repo_name("demo").unwrap(), "demo");
// Only one suffix is stripped: a repo literally named "demo.git"
// stays addressable and never aliases to "demo".
assert_eq!(smart_http_repo_name("demo.git.git").unwrap(), "demo.git");
assert!(matches!(
smart_http_repo_name(".git"),
Err(AppError::BadRequest(_))
));
assert!(matches!(
smart_http_repo_name(""),
Err(AppError::BadRequest(_))
));
}
#[test]
fn enforced_allows_owner_full_did() {
let repo = repo_owned_by(OWNER_DID);
@@ -2494,4 +2578,51 @@ mod tests {
"receive-pack advertisement must be throttled before the Tigris acquire"
);
}
/// Repo creation must be throttled by the per-IP creation limiter BEFORE
/// signature verification — otherwise a DID farm (one throwaway did:key per
/// repo, each carrying a valid but machine-solved iCaptcha proof) walks past
/// the per-DID limiter and floods the network, as in the recurring spam-repo
/// incidents. A 429 (not a 401) on an unsigned request from an exhausted IP
/// proves the IP brake runs outermost, ahead of auth.
#[sqlx::test]
async fn repo_creation_is_rate_limited_by_ip(pool: sqlx::PgPool) {
use axum::body::Body;
use axum::extract::ConnectInfo;
use axum::http::{Method, Request, StatusCode};
use std::net::SocketAddr;
use std::time::Duration;
use tower::ServiceExt;
let mut state = crate::test_support::test_state(pool).await;
// Tiny limit, keyed on the socket peer (no trusted proxy).
state.create_ip_rate_limiter =
crate::rate_limit::RateLimiter::new(1, Duration::from_secs(60));
state.push_limiter_trust = crate::rate_limit::TrustedProxy::None;
let peer: SocketAddr = "203.0.113.77:7000".parse().unwrap();
// Exhaust this peer's single-request budget up front.
assert!(
state
.create_ip_rate_limiter
.check(&peer.ip().to_string())
.await
);
let router = crate::server::build_router(state);
let mut req = Request::builder()
.method(Method::POST)
.uri("/api/v1/repos")
.header("content-type", "application/json")
.body(Body::from(r#"{"name":"flood","is_public":true}"#))
.unwrap();
req.extensions_mut().insert(ConnectInfo(peer));
let status = router.oneshot(req).await.unwrap().status();
assert_eq!(
status,
StatusCode::TOO_MANY_REQUESTS,
"repo creation must be IP-throttled before signature verification"
);
}
}
+11 -18
View File
@@ -2,14 +2,14 @@
//!
//! Any authenticated agent can star or unstar a repo.
//! One agent = one star per repo (enforced by UNIQUE constraint in repo_stars).
//! Star count is public — no auth required on GET.
//! Star count is gated on repo read access.
use axum::extract::{Extension, Path, State};
use axum::http::StatusCode;
use axum::Json;
use crate::auth::AuthenticatedDid;
use crate::error::{AppError, Result};
use crate::error::Result;
use crate::state::AppState;
/// PUT /api/v1/repos/:owner/:repo/star
@@ -19,11 +19,8 @@ pub async fn star_repo(
Extension(auth): Extension<AuthenticatedDid>,
Path((owner, repo)): Path<(String, String)>,
) -> Result<(StatusCode, Json<serde_json::Value>)> {
let record = state
.db
.get_repo(&owner, &repo)
.await?
.ok_or_else(|| AppError::RepoNotFound(format!("{owner}/{repo}")))?;
let (record, _rules) =
crate::api::authorize_repo_read(&state, &owner, &repo, Some(auth.0.as_str()), "/").await?;
let caller = &auth.0;
let inserted = state.db.star_repo(&record.id, caller).await?;
@@ -54,11 +51,8 @@ pub async fn unstar_repo(
Extension(auth): Extension<AuthenticatedDid>,
Path((owner, repo)): Path<(String, String)>,
) -> Result<Json<serde_json::Value>> {
let record = state
.db
.get_repo(&owner, &repo)
.await?
.ok_or_else(|| AppError::RepoNotFound(format!("{owner}/{repo}")))?;
let (record, _rules) =
crate::api::authorize_repo_read(&state, &owner, &repo, Some(auth.0.as_str()), "/").await?;
let caller = &auth.0;
state.db.unstar_repo(&record.id, caller).await?;
@@ -74,16 +68,15 @@ pub async fn unstar_repo(
}
/// GET /api/v1/repos/:owner/:repo/star
/// Returns star count — unauthenticated.
/// Returns star count for callers who can read the repo.
pub async fn get_star_status(
State(state): State<AppState>,
Path((owner, repo)): Path<(String, String)>,
auth: Option<Extension<AuthenticatedDid>>,
) -> Result<Json<serde_json::Value>> {
let record = state
.db
.get_repo(&owner, &repo)
.await?
.ok_or_else(|| AppError::RepoNotFound(format!("{owner}/{repo}")))?;
let caller = auth.as_ref().map(|e| e.0 .0.as_str());
let (record, _rules) =
crate::api::authorize_repo_read(&state, &owner, &repo, caller, "/").await?;
let count = state.db.count_stars(&record.id).await?;
+3
View File
@@ -514,8 +514,11 @@ mod tests {
machine_id: None,
repo_store: crate::git::repo_store::RepoStore::for_testing(PathBuf::from("/tmp"), pool),
rate_limiter: RateLimiter::new(100, Duration::from_secs(60)),
create_ip_rate_limiter: RateLimiter::new(1000, Duration::from_secs(3600)),
push_rate_limiter: RateLimiter::new(600, Duration::from_secs(3600)),
push_limiter_trust: crate::rate_limit::TrustedProxy::None,
sync_trigger_rate_limiter: RateLimiter::new(60, Duration::from_secs(3600)),
peer_write_rate_limiter: RateLimiter::new(600, Duration::from_secs(3600)),
shutdown_tx: tokio::sync::watch::channel(false).0,
}
}
+3 -2
View File
@@ -52,6 +52,7 @@ pub async fn issue_ref_certificate(
issued_at,
};
state.db.insert_ref_certificate(&cert).await?;
Ok(cert)
// Persist and return the row as it exists in the database (on a
// conflict the existing row survives when it is newer).
state.db.insert_ref_certificate(&cert).await
}
+104
View File
@@ -140,6 +140,20 @@ pub struct Config {
#[arg(long, env = "GITLAWB_MAX_PACK_BYTES", default_value_t = 2_147_483_648)]
pub max_pack_bytes: usize,
/// Per-client-IP rate limit for `POST /api/v1/sync/trigger`, in requests per
/// hour. `/sync/trigger` requires a signature and drives an O(peers) outbound
/// fan-out per call, so it gets a tight bucket. `0` disables. Default: 60.
#[arg(long, env = "GITLAWB_SYNC_TRIGGER_RATE_LIMIT", default_value_t = 60)]
pub sync_trigger_rate_limit: usize,
/// Per-client-IP rate limit for the peer-write routes (`/peers/announce`,
/// `/sync/notify`), in requests per hour. These accept unsigned requests from
/// known peers and run at higher frequency, so the bucket is generous. Keeping
/// it separate from the trigger bucket stops an unsigned notify flood from
/// draining the signed trigger caller's quota. `0` disables. Default: 600.
#[arg(long, env = "GITLAWB_PEER_WRITE_RATE_LIMIT", default_value_t = 600)]
pub peer_write_rate_limit: usize,
/// Optional address to bind a Prometheus `/metrics` exposition endpoint on.
/// Example: `127.0.0.1:9091`. Leave empty (default) to disable.
/// Bind to localhost or a private interface — the metrics endpoint is
@@ -152,6 +166,74 @@ pub struct Config {
/// in flight and exits. Default: 30s.
#[arg(long, env = "GITLAWB_SHUTDOWN_GRACE_SECS", default_value_t = 30)]
pub shutdown_grace_secs: u64,
/// Maximum wall-clock time a single served git operation (upload-pack /
/// receive-pack through `run_git_service`) may run before it is aborted and
/// its process group torn down, in seconds. Bounds a git that neither
/// finishes nor disconnects. Must be positive; set it very large to
/// effectively disable the bound. Default: 600s (10 min), generous for large
/// clones. Does not cover the ref advertisement (`info/refs`) or the
/// withheld-blob fetch path (`upload_pack_excluding`, a blocking
/// `spawn_blocking` a tokio timeout cannot cancel); both remain unbounded.
#[arg(
long,
env = "GITLAWB_GIT_SERVICE_TIMEOUT_SECS",
default_value_t = 600,
value_parser = clap::value_parser!(u64).range(1..)
)]
pub git_service_timeout_secs: u64,
/// Maximum connections in the PostgreSQL pool. This is a cap, not a floor
/// (connections open lazily). Size against the database server's
/// max_connections, remembering admin tooling opens its own pool.
#[arg(
long,
env = "GITLAWB_DB_MAX_CONNECTIONS",
default_value_t = 20,
value_parser = clap::value_parser!(u32).range(1..)
)]
pub db_max_connections: u32,
/// Maximum time a request waits for a pool connection before failing with
/// 503, in seconds. Bounds queueing when the database is slow or down.
#[arg(
long,
env = "GITLAWB_DB_ACQUIRE_TIMEOUT_SECS",
default_value_t = 5,
value_parser = clap::value_parser!(u64).range(1..)
)]
pub db_acquire_timeout_secs: u64,
/// Upper bound on each startup connect-and-migrate attempt, in seconds.
/// Migrations wait on a cross-instance advisory lock, so this must be
/// generous enough for a peer instance to finish migrating; on expiry the
/// attempt is retried (migrations are idempotent).
#[arg(
long,
env = "GITLAWB_DB_CONNECT_TIMEOUT_SECS",
default_value_t = 60,
value_parser = clap::value_parser!(u64).range(1..)
)]
pub db_connect_timeout_secs: u64,
/// First retry delay when the database is unavailable at startup, in
/// seconds. Doubles each attempt up to --db-retry-max-secs.
#[arg(
long,
env = "GITLAWB_DB_RETRY_INITIAL_SECS",
default_value_t = 5,
value_parser = clap::value_parser!(u64).range(1..)
)]
pub db_retry_initial_secs: u64,
/// Ceiling on the startup retry delay, in seconds.
#[arg(
long,
env = "GITLAWB_DB_RETRY_MAX_SECS",
default_value_t = 60,
value_parser = clap::value_parser!(u64).range(1..)
)]
pub db_retry_max_secs: u64,
}
impl Config {
@@ -169,3 +251,25 @@ impl Config {
PathBuf::from(&self.key_path)
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn git_service_timeout_defaults_to_600_and_rejects_zero() {
assert_eq!(
Config::parse_from(["gitlawb-node"]).git_service_timeout_secs,
600
);
assert_eq!(
Config::parse_from(["gitlawb-node", "--git-service-timeout-secs", "30"])
.git_service_timeout_secs,
30
);
// 0 is a footgun (immediate-504 on every request); clap must reject it.
assert!(
Config::try_parse_from(["gitlawb-node", "--git-service-timeout-secs", "0"]).is_err()
);
}
}
+674 -11
View File
@@ -1,7 +1,8 @@
use anyhow::{Context, Result};
use chrono::{DateTime, Utc};
use serde::{Deserialize, Serialize};
use sqlx::{PgPool, Row};
use sqlx::{postgres::PgPoolOptions, PgPool, Row};
use std::time::Duration;
use tracing::info;
use uuid::Uuid;
@@ -265,13 +266,42 @@ impl Db {
self.migrate().await
}
pub async fn connect(database_url: &str) -> Result<Self> {
let pool = PgPool::connect(database_url).await?;
/// Connect the pool and run migrations. The initial connection is bounded
/// by `acquire_timeout` (sqlx routes pool connects through the acquire
/// path); migrations are unbounded here — the caller wraps the whole call
/// in its own attempt timeout, since the migration advisory lock can
/// legitimately block while another instance migrates.
pub async fn connect(
database_url: &str,
max_connections: u32,
acquire_timeout: Duration,
) -> Result<Self> {
info!(
max_connections,
acquire_timeout_secs = acquire_timeout.as_secs(),
"connecting to postgres"
);
let pool = PgPoolOptions::new()
.max_connections(max_connections)
.acquire_timeout(acquire_timeout)
.connect(database_url)
.await
.context("connecting to postgres")?;
let db = Self { pool };
db.migrate().await?;
Ok(db)
}
/// Cheap liveness probe against the pool, for readiness checks: one
/// `SELECT 1` that fails fast when the database is unreachable.
pub async fn ping(&self) -> Result<()> {
sqlx::query("SELECT 1")
.execute(&self.pool)
.await
.context("db ping")?;
Ok(())
}
/// Run all pending versioned migrations in order, inside a single
/// transaction per migration. Idempotent — migrations whose version is
/// already recorded in `schema_migrations` are skipped.
@@ -823,6 +853,25 @@ const MIGRATIONS: &[Migration] = &[
"ALTER TABLE repos ADD COLUMN IF NOT EXISTS quarantined BOOLEAN NOT NULL DEFAULT FALSE",
],
},
Migration {
version: 10,
name: "ref_cert_unique_per_ref",
stmts: &[
// Dedup before the unique index: keep only the most recent row per
// (repo_id, ref_name) so the CREATE UNIQUE INDEX below does not fail
// on existing databases that accumulated duplicates.
r#"DELETE FROM ref_certificates
WHERE id IN (
SELECT id FROM (
SELECT id, ROW_NUMBER() OVER (
PARTITION BY repo_id, ref_name ORDER BY issued_at DESC, id DESC
) AS rn
FROM ref_certificates
) dups WHERE dups.rn > 1
)"#,
"CREATE UNIQUE INDEX IF NOT EXISTS idx_ref_certs_repo_ref ON ref_certificates(repo_id, ref_name)",
],
},
];
// ── Repos ─────────────────────────────────────────────────────────────────────
@@ -1921,11 +1970,31 @@ impl Db {
// ── Ref Certificates ──────────────────────────────────────────────────────────
impl Db {
pub async fn insert_ref_certificate(&self, cert: &RefCertificate) -> Result<()> {
sqlx::query(
/// Insert a ref certificate, or update it if a row for `(repo_id, ref_name)`
/// already exists. The update only applies when the incoming row is newer
/// (compared by `issued_at`, which assumes a monotonic wall clock), so a
/// late-landing older cert cannot regress a ref's persisted state. Returns
/// the full row as it now exists in the database (the original row on a
/// rejected upsert; the passed row on insert).
pub async fn insert_ref_certificate(&self, cert: &RefCertificate) -> Result<RefCertificate> {
let row = sqlx::query(
"INSERT INTO ref_certificates
(id, repo_id, ref_name, old_sha, new_sha, pusher_did, node_did, signature, issued_at)
VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9)",
VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9)
ON CONFLICT (repo_id, ref_name) DO UPDATE SET
old_sha = CASE WHEN EXCLUDED.issued_at > ref_certificates.issued_at
THEN EXCLUDED.old_sha ELSE ref_certificates.old_sha END,
new_sha = CASE WHEN EXCLUDED.issued_at > ref_certificates.issued_at
THEN EXCLUDED.new_sha ELSE ref_certificates.new_sha END,
pusher_did = CASE WHEN EXCLUDED.issued_at > ref_certificates.issued_at
THEN EXCLUDED.pusher_did ELSE ref_certificates.pusher_did END,
node_did = CASE WHEN EXCLUDED.issued_at > ref_certificates.issued_at
THEN EXCLUDED.node_did ELSE ref_certificates.node_did END,
signature = CASE WHEN EXCLUDED.issued_at > ref_certificates.issued_at
THEN EXCLUDED.signature ELSE ref_certificates.signature END,
issued_at = CASE WHEN EXCLUDED.issued_at > ref_certificates.issued_at
THEN EXCLUDED.issued_at ELSE ref_certificates.issued_at END
RETURNING id, repo_id, ref_name, old_sha, new_sha, pusher_did, node_did, signature, issued_at",
)
.bind(&cert.id)
.bind(&cert.repo_id)
@@ -1936,17 +2005,49 @@ impl Db {
.bind(&cert.node_did)
.bind(&cert.signature)
.bind(&cert.issued_at)
.execute(&self.pool)
.fetch_one(&self.pool)
.await?;
Ok(())
Ok(row_to_cert(row))
}
pub async fn list_ref_certificates(&self, repo_id: &str) -> Result<Vec<RefCertificate>> {
pub async fn list_ref_certificates(
&self,
repo_id: &str,
limit: i64,
) -> Result<Vec<RefCertificate>> {
// Clamp at the DB boundary so every caller (present and future) stays
// bounded even if a raw/negative value slips through the handler layer.
let limit = limit.max(1);
let rows = sqlx::query(
"SELECT id, repo_id, ref_name, old_sha, new_sha, pusher_did, node_did, signature, issued_at
FROM ref_certificates WHERE repo_id = $1 ORDER BY issued_at DESC",
FROM ref_certificates WHERE repo_id = $1 ORDER BY issued_at DESC LIMIT $2",
)
.bind(repo_id)
.bind(limit)
.fetch_all(&self.pool)
.await?;
Ok(rows.into_iter().map(row_to_cert).collect())
}
/// Look up ref certificates whose id starts with the given prefix.
/// Used by the CLI for short-ID resolution where the caller does not know
/// the full UUID. Bounded by `limit` for safety — the caller should pass a
/// generous cap (e.g. 200) since prefix-matching narrows the result set.
pub async fn list_ref_certificates_by_prefix(
&self,
repo_id: &str,
prefix: &str,
limit: i64,
) -> Result<Vec<RefCertificate>> {
let limit = limit.max(1);
let pattern = format!("{}%", prefix);
let rows = sqlx::query(
"SELECT id, repo_id, ref_name, old_sha, new_sha, pusher_did, node_did, signature, issued_at
FROM ref_certificates WHERE repo_id = $1 AND id LIKE $2 ORDER BY issued_at DESC LIMIT $3",
)
.bind(repo_id)
.bind(&pattern)
.bind(limit)
.fetch_all(&self.pool)
.await?;
Ok(rows.into_iter().map(row_to_cert).collect())
@@ -2935,6 +3036,8 @@ impl Db {
repo_name: Option<&str>,
status: Option<&str>,
limit: i64,
after_created_at: Option<&str>,
after_id: Option<&str>,
) -> Result<Vec<BountyRecord>> {
let mut sql = String::from("SELECT * FROM bounties WHERE 1=1");
let mut binds: Vec<String> = Vec::new();
@@ -2955,7 +3058,15 @@ impl Db {
binds.push(s.to_string());
idx += 1;
}
sql.push_str(&format!(" ORDER BY created_at DESC LIMIT ${idx}"));
if let Some(ts) = after_created_at {
let id = after_id.unwrap_or("");
sql.push_str(&format!(" AND (created_at, id) < (${idx}, ${})", idx + 1));
binds.push(ts.to_string());
idx += 1;
binds.push(id.to_string());
idx += 1;
}
sql.push_str(&format!(" ORDER BY created_at DESC, id DESC LIMIT ${idx}"));
let mut q = sqlx::query(&sql);
for b in &binds {
@@ -4600,3 +4711,555 @@ mod ref_update_keyset_same_timestamp_tests {
);
}
}
#[cfg(test)]
mod ref_certificate_tests {
use super::{Db, RefCertificate, RepoRecord};
use chrono::Utc;
use sqlx::PgPool;
async fn db(pool: PgPool) -> Db {
let db = Db::for_testing(pool);
db.run_migrations().await.unwrap();
db
}
fn make_cert(
id: &str,
repo_id: &str,
ref_name: &str,
old_sha: &str,
new_sha: &str,
issued_at: &str,
) -> RefCertificate {
RefCertificate {
id: id.to_string(),
repo_id: repo_id.to_string(),
ref_name: ref_name.to_string(),
old_sha: old_sha.to_string(),
new_sha: new_sha.to_string(),
pusher_did: "did:key:zPUSHER".to_string(),
node_did: "did:key:zNODE".to_string(),
signature: "sig".to_string(),
issued_at: issued_at.to_string(),
}
}
#[sqlx::test]
async fn list_ref_certificates_respects_limit(pool: PgPool) {
let db = db(pool).await;
let repo_id = uuid::Uuid::new_v4().to_string();
// Create a repo to satisfy FK
db.create_repo(&RepoRecord {
id: repo_id.clone(),
name: "limit-test".into(),
owner_did: "did:key:zOWNER".into(),
description: None,
is_public: true,
default_branch: "main".into(),
created_at: Utc::now(),
updated_at: Utc::now(),
disk_path: "/tmp/limit-test".into(),
forked_from: None,
machine_id: None,
})
.await
.unwrap();
// Insert 5 certs with descending issued_at
for i in 0..5 {
db.insert_ref_certificate(&make_cert(
&format!("cert-{i}"),
&repo_id,
&format!("refs/heads/feature-{i}"),
"0000",
"1111",
&format!("2026-07-03T20:0{i}:00Z"),
))
.await
.unwrap();
}
// limit=2 returns only 2
let certs = db.list_ref_certificates(&repo_id, 2).await.unwrap();
assert_eq!(certs.len(), 2, "LIMIT 2 must return exactly 2 certs");
assert_eq!(certs[0].id, "cert-4", "most recent first");
assert_eq!(certs[1].id, "cert-3", "second most recent");
// limit=10 returns all 5 (no padding)
let all = db.list_ref_certificates(&repo_id, 10).await.unwrap();
assert_eq!(all.len(), 5, "LIMIT >= row count returns all rows");
}
#[sqlx::test]
async fn insert_ref_certificate_upserts_on_repo_ref(pool: PgPool) {
let db = db(pool).await;
let repo_id = uuid::Uuid::new_v4().to_string();
db.create_repo(&RepoRecord {
id: repo_id.clone(),
name: "upsert-test".into(),
owner_did: "did:key:zOWNER".into(),
description: None,
is_public: true,
default_branch: "main".into(),
created_at: Utc::now(),
updated_at: Utc::now(),
disk_path: "/tmp/upsert-test".into(),
forked_from: None,
machine_id: None,
})
.await
.unwrap();
// First insert
db.insert_ref_certificate(&make_cert(
"cert-original",
&repo_id,
"refs/heads/main",
"0000",
"1111",
"2026-07-03T20:00:00Z",
))
.await
.unwrap();
// Upsert same ref with new values
db.insert_ref_certificate(&make_cert(
"cert-upserted",
&repo_id,
"refs/heads/main",
"aaaa",
"bbbb",
"2026-07-03T21:00:00Z",
))
.await
.unwrap();
// Only one row exists for this ref
let certs = db.list_ref_certificates(&repo_id, 10).await.unwrap();
assert_eq!(certs.len(), 1, "upsert must not create a duplicate row");
assert_eq!(
certs[0].id, "cert-original",
"upsert must preserve the original ID across re-pushes"
);
assert_eq!(certs[0].old_sha, "aaaa", "old_sha updated");
assert_eq!(certs[0].new_sha, "bbbb", "new_sha updated");
assert_eq!(
certs[0].issued_at, "2026-07-03T21:00:00Z",
"newer issued_at overwrites older"
);
// Now try to overwrite with an OLDER cert — the guard must reject it.
db.insert_ref_certificate(&make_cert(
"stale-id",
&repo_id,
"refs/heads/main",
"stale",
"stale",
"2026-07-03T19:00:00Z",
))
.await
.unwrap();
let certs = db.list_ref_certificates(&repo_id, 10).await.unwrap();
assert_eq!(certs.len(), 1, "no extra row from stale cert");
assert_eq!(
certs[0].id, "cert-original",
"stale cert does not change the original id"
);
assert_eq!(
certs[0].old_sha, "aaaa",
"stale cert does not regress old_sha"
);
assert_eq!(
certs[0].new_sha, "bbbb",
"stale cert does not regress new_sha"
);
assert_eq!(
certs[0].issued_at, "2026-07-03T21:00:00Z",
"stale cert does not regress issued_at"
);
}
#[sqlx::test]
async fn list_ref_certificates_clamps_negative_limit(pool: PgPool) {
let db = db(pool).await;
let repo_id = uuid::Uuid::new_v4().to_string();
db.create_repo(&RepoRecord {
id: repo_id.clone(),
name: "clamp-test".into(),
owner_did: "did:key:zOWNER".into(),
description: None,
is_public: true,
default_branch: "main".into(),
created_at: Utc::now(),
updated_at: Utc::now(),
disk_path: "/tmp/clamp-test".into(),
forked_from: None,
machine_id: None,
})
.await
.unwrap();
db.insert_ref_certificate(&make_cert(
"clamp-1",
&repo_id,
"refs/heads/main",
"0000",
"1111",
"2026-07-03T20:00:00Z",
))
.await
.unwrap();
// Negative limit is clamped to 1 at the DB boundary
let certs = db.list_ref_certificates(&repo_id, -5).await.unwrap();
assert_eq!(certs.len(), 1, "negative limit clamped to min 1");
assert_eq!(certs[0].id, "clamp-1");
// Zero limit also clamped to 1
let certs = db.list_ref_certificates(&repo_id, 0).await.unwrap();
assert_eq!(certs.len(), 1, "zero limit clamped to min 1");
assert_eq!(certs[0].id, "clamp-1");
}
#[sqlx::test]
async fn list_ref_certificates_empty_repo_returns_empty(pool: PgPool) {
let db = db(pool).await;
let certs = db
.list_ref_certificates("nonexistent-repo-id", 10)
.await
.unwrap();
assert!(certs.is_empty());
}
/// NOTE: this test hand-copies the migration SQL as string literals and will
/// silently drift if the v10 migration block changes. The load-bearing
/// upgrade-path test is `v10_upgrade_dedup_via_migration`, which fires the
/// real MIGRATIONS[v10] entry via run_migrations().
#[sqlx::test]
async fn v10_dedup_removes_old_duplicates(pool: PgPool) {
let db = db(pool.clone()).await;
// Drop the unique index so we can simulate pre-v10 duplicate rows.
sqlx::query("DROP INDEX IF EXISTS idx_ref_certs_repo_ref")
.execute(&pool)
.await
.unwrap();
let repo_id = uuid::Uuid::new_v4().to_string();
db.create_repo(&RepoRecord {
id: repo_id.clone(),
name: "dedup-test".into(),
owner_did: "did:key:zOWNER".into(),
description: None,
is_public: true,
default_branch: "main".into(),
created_at: Utc::now(),
updated_at: Utc::now(),
disk_path: "/tmp/dedup-test".into(),
forked_from: None,
machine_id: None,
})
.await
.unwrap();
// Insert two rows for the same (repo_id, ref_name) with raw INSERT
// (no ON CONFLICT — the unique index was dropped above to simulate a
// pre-v10 database). The second row has the newer timestamp and should
// survive the dedup.
sqlx::query(
"INSERT INTO ref_certificates
(id, repo_id, ref_name, old_sha, new_sha, pusher_did, node_did, signature, issued_at)
VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9)",
)
.bind("keep-id")
.bind(&repo_id)
.bind("refs/heads/main")
.bind("0000")
.bind("1111")
.bind("did:key:zPUSHER")
.bind("did:key:zNODE")
.bind("sig-first")
.bind("2026-07-03T20:00:00Z")
.execute(&pool)
.await
.unwrap();
sqlx::query(
"INSERT INTO ref_certificates
(id, repo_id, ref_name, old_sha, new_sha, pusher_did, node_did, signature, issued_at)
VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9)",
)
.bind("remove-id")
.bind(&repo_id)
.bind("refs/heads/main")
.bind("aaaa")
.bind("bbbb")
.bind("did:key:zPUSHER")
.bind("did:key:zNODE")
.bind("sig-dup")
.bind("2026-07-03T19:00:00Z") // older timestamp → should be removed
.execute(&pool)
.await
.unwrap();
// Apply the v10 dedup logic: keep the most recent per (repo_id, ref_name).
sqlx::query(
"DELETE FROM ref_certificates
WHERE id IN (
SELECT id FROM (
SELECT id, ROW_NUMBER() OVER (
PARTITION BY repo_id, ref_name ORDER BY issued_at DESC, id DESC
) AS rn
FROM ref_certificates
) dups WHERE dups.rn > 1
)",
)
.execute(&pool)
.await
.unwrap();
// Re-create the unique index — must succeed after dedup.
sqlx::query("CREATE UNIQUE INDEX IF NOT EXISTS idx_ref_certs_repo_ref ON ref_certificates(repo_id, ref_name)")
.execute(&pool)
.await
.unwrap();
// Only the most recent row survives.
let certs = db.list_ref_certificates(&repo_id, 10).await.unwrap();
assert_eq!(certs.len(), 1, "dedup leaves one row per ref");
assert_eq!(
certs[0].id, "keep-id",
"dedup keeps the most recent (later issued_at)"
);
}
/// INV-7: upgrade-path test — seed a database at v9 with duplicate
/// ref_certificates, then let the real v10 migration fire via
/// run_migrations(). This exercises the migration code path rather than
/// hand-copying its SQL, so the test stays in sync with MIGRATIONS[v10].
#[sqlx::test]
async fn v10_upgrade_dedup_via_migration(pool: PgPool) {
// 1. Bootstrap schema via the full migration chain.
let db = Db::for_testing(pool.clone());
db.run_migrations().await.unwrap();
// 2. Roll back to v9: remove the v10-unique index and the
// schema_migrations record so that run_migrations() re-applies v10.
sqlx::query("DROP INDEX IF EXISTS idx_ref_certs_repo_ref")
.execute(&pool)
.await
.unwrap();
sqlx::query("DELETE FROM schema_migrations WHERE version = 10")
.execute(&pool)
.await
.unwrap();
// 3. Seed repos and duplicate certs (raw INSERT — no ON CONFLICT
// since the index is gone).
let r1 = uuid::Uuid::new_v4().to_string();
let r2 = uuid::Uuid::new_v4().to_string();
for (id, name) in [(&r1, "upgrade-repo-a"), (&r2, "upgrade-repo-b")] {
db.create_repo(&RepoRecord {
id: id.clone(),
name: name.into(),
owner_did: "did:key:zOWNER".into(),
description: None,
is_public: true,
default_branch: "main".into(),
created_at: Utc::now(),
updated_at: Utc::now(),
disk_path: format!("/tmp/{name}"),
forked_from: None,
machine_id: None,
})
.await
.unwrap();
}
// Helper macro for raw INSERT.
macro_rules! insert_cert {
($id:expr, $repo_id:expr, $ref_name:expr, $old_sha:expr, $new_sha:expr, $issued_at:expr) => {
sqlx::query(
"INSERT INTO ref_certificates
(id, repo_id, ref_name, old_sha, new_sha, pusher_did, node_did, signature, issued_at)
VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9)",
)
.bind($id)
.bind($repo_id)
.bind($ref_name)
.bind($old_sha)
.bind($new_sha)
.bind("did:key:zPUSHER")
.bind("did:key:zNODE")
.bind("sig")
.bind($issued_at)
.execute(&pool)
.await
.unwrap();
};
}
// Repo A, ref "main": two rows with distinct timestamps.
insert_cert!(
"dup-a-old",
&r1,
"refs/heads/main",
"0000",
"1111",
"2026-07-01T10:00:00Z"
);
insert_cert!(
"dup-a-new",
&r1,
"refs/heads/main",
"aaaa",
"bbbb",
"2026-07-02T10:00:00Z"
);
// Repo A, ref "feature": two rows with IDENTICAL timestamps — the
// id-DESC tiebreaker must choose the higher id (alphabetical: "z" > "a").
insert_cert!(
"dup-feat-a",
&r1,
"refs/heads/feature",
"0000",
"1111",
"2026-07-01T10:00:00Z"
);
insert_cert!(
"dup-feat-z",
&r1,
"refs/heads/feature",
"cccc",
"dddd",
"2026-07-01T10:00:00Z"
);
// Repo B, ref "main": two rows with distinct timestamps.
insert_cert!(
"dup-b-old",
&r2,
"refs/heads/main",
"0000",
"1111",
"2026-07-01T10:00:00Z"
);
insert_cert!(
"dup-b-new",
&r2,
"refs/heads/main",
"eeee",
"ffff",
"2026-07-02T10:00:00Z"
);
// A non-duplicate singleton row (single row per ref) — must survive
// untouched.
insert_cert!(
"singleton",
&r2,
"refs/heads/singleton",
"0000",
"1111",
"2026-07-01T10:00:00Z"
);
// 4. Run migrations — the v10 dedup fires inside run_pending_migrations.
db.run_migrations().await.unwrap();
// 5. Assert each ref has exactly one survivor.
let all_r1 = db.list_ref_certificates(&r1, 10).await.unwrap();
assert_eq!(all_r1.len(), 2, "repo A: 2 refs, 1 survivor each");
let r1_main: Vec<_> = all_r1
.iter()
.filter(|c| c.ref_name == "refs/heads/main")
.collect();
assert_eq!(r1_main.len(), 1, "repo A main deduped to one row");
assert_eq!(r1_main[0].id, "dup-a-new", "newer timestamp survives");
assert_eq!(r1_main[0].old_sha, "aaaa");
assert_eq!(r1_main[0].new_sha, "bbbb");
let r1_feat: Vec<_> = all_r1
.iter()
.filter(|c| c.ref_name == "refs/heads/feature")
.collect();
assert_eq!(r1_feat.len(), 1, "repo A feature deduped to one row");
assert_eq!(
r1_feat[0].id, "dup-feat-z",
"same-timestamp tiebreaker: higher id wins (id DESC)"
);
let all_r2 = db.list_ref_certificates(&r2, 10).await.unwrap();
assert_eq!(all_r2.len(), 2, "repo B: 2 refs, 1 survivor each");
let r2_main: Vec<_> = all_r2
.iter()
.filter(|c| c.ref_name == "refs/heads/main")
.collect();
assert_eq!(r2_main.len(), 1, "repo B main deduped to one row");
assert_eq!(r2_main[0].id, "dup-b-new", "newer timestamp survives");
let all_r2 = db.list_ref_certificates(&r2, 10).await.unwrap();
assert_eq!(
all_r2.iter().filter(|c| c.id == "singleton").count(),
1,
"non-duplicate singleton untouched"
);
// 6. Verify the unique index exists: the upsert helper must succeed
// (exercises ON CONFLICT) and a direct duplicate INSERT must fail.
db.insert_ref_certificate(&make_cert(
"post-migration-upsert",
&r1,
"refs/heads/main",
"1111",
"2222",
"2026-07-03T10:00:00Z",
))
.await
.unwrap();
let after_upsert = db.list_ref_certificates(&r1, 10).await.unwrap();
let r1_main_after: Vec<_> = after_upsert
.iter()
.filter(|c| c.ref_name == "refs/heads/main")
.collect();
assert_eq!(
r1_main_after.len(),
1,
"upsert keeps exactly one row for main"
);
assert_eq!(
r1_main_after[0].id, "dup-a-new",
"upsert preserves original id"
);
assert_eq!(r1_main_after[0].old_sha, "1111", "upsert updated old_sha");
// A raw INSERT for the same (repo_id, ref_name) must now fail.
let err = sqlx::query(
"INSERT INTO ref_certificates
(id, repo_id, ref_name, old_sha, new_sha, pusher_did, node_did, signature, issued_at)
VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9)",
)
.bind("should-fail")
.bind(&r1)
.bind("refs/heads/main")
.bind("xxxx")
.bind("yyyy")
.bind("did:key:zPUSHER")
.bind("did:key:zNODE")
.bind("sig")
.bind("2026-07-04T10:00:00Z")
.execute(&pool)
.await;
assert!(
err.is_err(),
"raw duplicate INSERT must be rejected by the unique index"
);
}
}
+67 -1
View File
@@ -38,14 +38,51 @@ pub enum AppError {
#[error("too many requests: {0}")]
TooManyRequests(String),
#[error("incomplete: {0}")]
Incomplete(String),
#[error("git error: {0}")]
Git(String),
#[error("git service timed out: {0}")]
Timeout(String),
#[error("database error: {0}")]
Db(#[from] sqlx::Error),
#[error("internal error: {0}")]
Internal(#[from] anyhow::Error),
Internal(anyhow::Error),
}
/// Shared error code/message for "the database is unreachable", also used by
/// the degraded startup server (main.rs) and the readiness probe (server.rs)
/// so clients see one vocabulary for the condition.
pub const DB_UNAVAILABLE_CODE: &str = "db_unavailable";
pub const DB_UNAVAILABLE_MESSAGE: &str = "database is temporarily unavailable";
/// Connection-level sqlx failures that mean the database is unreachable right
/// now (retryable, 503), as opposed to server-reported query errors.
fn db_unavailable(e: &sqlx::Error) -> bool {
matches!(
e,
sqlx::Error::PoolTimedOut
| sqlx::Error::PoolClosed
| sqlx::Error::Io(_)
| sqlx::Error::Tls(_)
)
}
/// The db layer returns `anyhow::Result`, so sqlx errors reach handlers inside
/// anyhow chains. Downcast them back out so the status mapping below can see
/// them — without this, every database outage surfaces as a 500 instead of a
/// 503. anyhow preserves downcastability through `.context()` layers.
impl From<anyhow::Error> for AppError {
fn from(err: anyhow::Error) -> Self {
match err.downcast::<sqlx::Error>() {
Ok(sql) => AppError::Db(sql),
Err(err) => AppError::Internal(err),
}
}
}
impl IntoResponse for AppError {
@@ -98,7 +135,18 @@ impl IntoResponse for AppError {
AppError::TooManyRequests(msg) => {
(StatusCode::TOO_MANY_REQUESTS, "rate_limited", msg.clone())
}
AppError::Incomplete(msg) => {
(StatusCode::UNPROCESSABLE_ENTITY, "incomplete", msg.clone())
}
AppError::Git(msg) => (StatusCode::INTERNAL_SERVER_ERROR, "git_error", msg.clone()),
// 504, distinct from the 500 git_error and from the read-gate's 404 /
// the auth 401, so the client can tell a deadline from a failure.
AppError::Timeout(msg) => (StatusCode::GATEWAY_TIMEOUT, "git_timeout", msg.clone()),
AppError::Db(e) if db_unavailable(e) => (
StatusCode::SERVICE_UNAVAILABLE,
DB_UNAVAILABLE_CODE,
DB_UNAVAILABLE_MESSAGE.into(),
),
AppError::Db(e) => (StatusCode::INTERNAL_SERVER_ERROR, "db_error", e.to_string()),
AppError::Internal(e) => (
StatusCode::INTERNAL_SERVER_ERROR,
@@ -117,3 +165,21 @@ impl IntoResponse for AppError {
}
pub type Result<T> = std::result::Result<T, AppError>;
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn timeout_maps_to_504_distinct_from_git_500() {
assert_eq!(
AppError::Timeout("x".into()).into_response().status(),
StatusCode::GATEWAY_TIMEOUT
);
// Guard against a swap with the generic git failure (500).
assert_eq!(
AppError::Git("x".into()).into_response().status(),
StatusCode::INTERNAL_SERVER_ERROR
);
}
}
+483 -67
View File
@@ -6,7 +6,8 @@ use bytes::Bytes;
use std::collections::HashSet;
use std::path::Path;
use std::process::Stdio;
use tokio::io::AsyncWriteExt;
use std::time::Duration;
use tokio::io::{AsyncReadExt, AsyncWriteExt};
use tokio::process::Command;
/// Handle `GET /:owner/:repo/info/refs?service=git-upload-pack`
@@ -51,8 +52,13 @@ pub async fn info_refs(repo_path: &Path, service: &str) -> Result<Response> {
///
/// Serves pack data for a clone or fetch. This is stateless — the entire
/// negotiation happens in a single request/response.
pub async fn upload_pack(repo_path: &Path, request_body: Bytes) -> Result<Response> {
let output = run_git_service("git", "git-upload-pack", repo_path, request_body).await?;
pub async fn upload_pack(
repo_path: &Path,
request_body: Bytes,
timeout: Duration,
) -> Result<Response> {
let output =
run_git_service("git", "git-upload-pack", repo_path, request_body, timeout).await?;
Ok(Response::builder()
.status(StatusCode::OK)
@@ -65,8 +71,13 @@ pub async fn upload_pack(repo_path: &Path, request_body: Bytes) -> Result<Respon
///
/// Accepts a push. The caller MUST verify HTTP Signature auth before
/// calling this function.
pub async fn receive_pack(repo_path: &Path, request_body: Bytes) -> Result<Response> {
let output = run_git_service("git", "git-receive-pack", repo_path, request_body).await?;
pub async fn receive_pack(
repo_path: &Path,
request_body: Bytes,
timeout: Duration,
) -> Result<Response> {
let output =
run_git_service("git", "git-receive-pack", repo_path, request_body, timeout).await?;
Ok(Response::builder()
.status(StatusCode::OK)
@@ -112,17 +123,85 @@ impl Drop for KillGroupOnDrop {
}
}
/// Error returned when [`run_git_service`] aborts a git op on its timeout.
///
/// Carried through the `anyhow` chain so the HTTP handler can `downcast_ref` it
/// and map to 504 Gateway Timeout, distinct from the generic 500 git error.
#[derive(Debug, thiserror::Error)]
#[error("git service timed out")]
pub struct GitServiceTimeout;
/// On a served-git timeout, tear the process group down AND reap the leader
/// before returning, so a caller that releases a write lock (receive-pack) can't
/// race a still-live git touching the same repo. SIGTERM first (lets git remove
/// its `.git/*.lock` files), escalate to SIGKILL if the leader lingers past a
/// grace, then reap. Bounded, so a git that ignores SIGTERM can't block the
/// response unboundedly.
#[cfg(unix)]
async fn reap_group_on_timeout(child: &mut tokio::process::Child) {
let Some(pid) = child.id() else {
// Never got a pid; nothing to signal, best-effort reap.
let _ = child.wait().await;
return;
};
let pgid = pid as i32;
// SAFETY: kill(2) takes only integers and borrows no Rust memory; ESRCH on an
// already-gone group is ignored.
unsafe {
libc::kill(-pgid, libc::SIGTERM);
}
// Wait for the WHOLE group to exit before returning — not just the leader but
// grandchildren (index-pack / pack-objects) that reparent to init and can
// still be touching the repo, so a caller releasing a write lock can't race
// them. Reap our direct child along the way (`try_wait`, else it lingers as a
// zombie) and poll the group's liveness with `kill(-pgid, 0)` (ESRCH once
// every member is gone). SIGTERM grace, then SIGKILL, then a hard cap so a
// stuck process can never block the response unboundedly.
for step in 0..400u32 {
let _ = child.try_wait();
if unsafe { libc::kill(-pgid, 0) } != 0 {
return; // ESRCH: every group member has exited
}
if step == 200 {
// ~2s SIGTERM grace elapsed; force the group down. `step == 200` fires
// exactly once in the 0..400 loop, so no re-entry guard is needed.
unsafe {
libc::kill(-pgid, libc::SIGKILL);
}
}
tokio::time::sleep(Duration::from_millis(10)).await;
}
// ~4s hard cap. Only reached if a group member survives SIGKILL, which means
// it is wedged in uninterruptible (D-state) I/O — a fsync on stuck storage, a
// hung mount — that no signal interrupts until the kernel returns. Nothing in
// userspace can kill it; blocking here (or holding the write lock) would just
// re-create the "hung git pins the repo" problem this bounds. Reap best-effort
// (tokio's orphan reaper collects it once the syscall unblocks) and return;
// git-level quarantine/ref-locking still guards the brief window before the
// D-state process finishes.
tracing::warn!(
pid,
"served git survived SIGKILL past the teardown cap (uninterruptible I/O?); \
releasing without a confirmed group reap"
);
let _ = child.try_wait();
}
/// Run a stateless-rpc git service and return its stdout.
///
/// `git_bin` is the git executable to spawn; production callers pass `"git"`
/// (resolved via `PATH`). It is injectable purely so the process-group teardown
/// wiring (`process_group(0)` + [`KillGroupOnDrop`]) can be driven end-to-end by
/// a fake `git` in tests without mutating the process-global `PATH`.
///
/// `timeout` bounds the whole child interaction; on expiry the op is aborted with
/// [`GitServiceTimeout`] and its process group is torn down.
async fn run_git_service(
git_bin: &str,
service: &str,
repo_path: &Path,
input: Bytes,
timeout: Duration,
) -> Result<Vec<u8>> {
let mut command = Command::new(git_bin);
command
@@ -142,33 +221,85 @@ async fn run_git_service(
// Arm the group-kill guard for the lifetime of the request. With
// process_group(0) the child is its own group leader, so pgid == its pid.
// This fires on a client disconnect (the whole future is dropped mid-request).
#[cfg(unix)]
let mut group_guard = KillGroupOnDrop {
pgid: child.id().map(|id| id as i32),
};
// Write the request body to git's stdin, but don't early-return on a write
// error: always reap the child first (below), so the guard only ever fires on
// an actual future-drop (client disconnect), never on a pid we just reaped.
let write_result: std::io::Result<()> = match child.stdin.take() {
Some(mut stdin) => stdin.write_all(&input).await,
None => Ok(()),
// Own the pipes so `child` stays reap-able after a timeout: wait_with_output
// would consume it, but on timeout we must actively reap the group before
// returning (see reap_group_on_timeout).
let mut stdin = child.stdin.take();
let mut stdout = child.stdout.take().context("git stdout was not piped")?;
let mut stderr = child.stderr.take().context("git stderr was not piped")?;
let mut out = Vec::new();
let mut err = Vec::new();
// Bound the whole child interaction: a git that neither finishes nor
// disconnects would otherwise pin the pid forever. Write stdin, drain stdout
// and stderr, and wait for the child all concurrently (draining while writing
// avoids a pipe deadlock on a large body), under one deadline.
let interact = async {
// Don't early-return on a write error: the join still waits on the child,
// so an error is surfaced only after the child has been waited on.
let write = async {
match stdin.take() {
Some(mut s) => s.write_all(&input).await,
None => Ok(()),
}
};
let (write_result, r_out, r_err, status) = tokio::join!(
write,
stdout.read_to_end(&mut out),
stderr.read_to_end(&mut err),
child.wait(),
);
r_out?;
r_err?;
Ok::<_, anyhow::Error>((write_result, status?))
};
let output = child.wait_with_output().await?;
let timed = tokio::time::timeout(timeout, interact).await;
let (write_result, status) = match timed {
Ok(result) => {
// The join runs all arms to completion, so the child is reaped: disarm
// before surfacing any interaction error (a read/wait error), else the
// guard's drop would fire SIGTERM on the reaped, possibly-reused pgid.
#[cfg(unix)]
group_guard.disarm();
result?
}
Err(_elapsed) => {
// Timeout: tear the whole group down and reap it before returning so a
// caller releasing a write lock can't race a still-live git. Then disarm
// the (now redundant) guard so its drop can't hit a reused pgid.
#[cfg(unix)]
{
reap_group_on_timeout(&mut child).await;
group_guard.disarm();
}
#[cfg(not(unix))]
{
let _ = child.start_kill();
let _ = tokio::time::timeout(Duration::from_secs(2), child.wait()).await;
}
return Err(GitServiceTimeout.into());
}
};
// Child reaped, so its group is gone: disarm before surfacing any error.
#[cfg(unix)]
group_guard.disarm();
write_result.context("failed to write to git stdin")?;
if !output.status.success() {
let stderr = String::from_utf8_lossy(&output.stderr);
// Surface git's own failure (its stderr, which the handler may classify as a
// 400) before any stdin-write error: when git rejects a malformed body it
// exits non-zero and closes stdin, so the write's EPIPE would otherwise mask
// the real cause.
if !status.success() {
let stderr = String::from_utf8_lossy(&err);
bail!("{service} failed: {stderr}");
}
Ok(output.stdout)
write_result.context("failed to write to git stdin")?;
Ok(out)
}
fn service_to_command(service: &str) -> &str {
@@ -935,6 +1066,60 @@ mod tests {
Some((leader, grandchild))
}
/// Retry policy for the fake-git spawn race, shared by `fake_git_run_with_pids`
/// and the drop test's inline loop (which can't use the helper — it must keep
/// the winning future pending). `nth` retry waits `n * STEP_MS` ms.
#[cfg(unix)]
const FAKE_GIT_RETRY_ATTEMPTS: u64 = 12;
#[cfg(unix)]
const FAKE_GIT_BACKOFF_STEP_MS: u64 = 100;
/// Run a fake-git-based `run` under the parallel test runner, retrying if the
/// fake transiently fails to record its pids. Under `cargo test --workspace`
/// fork-storm load a freshly-written fake `git` can fail to exec (ETXTBSY: a
/// concurrent worker forked while its write fd was still open) or be killed by
/// the service timeout before it is scheduled to write its pidfile; both leave
/// no pids. These misses are *correlated* (bursty) under fork pressure, so each
/// retry backs off (growing) to let a spike subside rather than burning every
/// attempt inside one. `pidfile` is removed before each attempt. Returns the
/// successful attempt's outcome together with the recorded pids; a genuine
/// never-spawns bug still fails loudly after the cap. Retried attempts don't
/// leak in practice: a miss almost always means the fake never exec'd (nothing
/// to clean up), and a spawn that got far enough is reaped by the timeout/drop
/// path before this returns.
#[cfg(unix)]
async fn fake_git_run_with_pids<R, F, Fut>(
pidfile: &std::path::Path,
mut run: F,
) -> (R, (i32, i32))
where
F: FnMut() -> Fut,
Fut: std::future::Future<Output = R>,
{
for i in 0..FAKE_GIT_RETRY_ATTEMPTS {
let _ = std::fs::remove_file(pidfile);
let outcome = run().await;
// A successful run writes its pids before it returns, so a short poll
// catches them (allowing only for fs-visibility lag).
for _ in 0..50 {
if let Some(p) = read_two_pids(pidfile) {
return (outcome, p);
}
tokio::time::sleep(std::time::Duration::from_millis(10)).await;
}
// Miss: back off before retrying so a bursty fork-pressure spike
// subsides (see the doc note on correlated ETXTBSY/EAGAIN failures).
tokio::time::sleep(std::time::Duration::from_millis(
FAKE_GIT_BACKOFF_STEP_MS * (i + 1),
))
.await;
}
panic!(
"fake git failed to spawn and record its pids after {FAKE_GIT_RETRY_ATTEMPTS} \
attempts (persistent failure, not a transient parallel-runner miss)"
);
}
// Dropping the request future mid-flight (client disconnect) must SIGTERM the
// whole group so git AND its pack-objects grandchild die together. Goes RED
// if `process_group(0)` or the guard-arming is removed: without its own
@@ -952,33 +1137,67 @@ mod tests {
);
let git_bin = write_fake_git(tmp.path(), &body);
let mut fut = Box::pin(run_git_service(
git_bin.to_str().unwrap(),
"git-upload-pack",
tmp.path(),
Bytes::new(),
));
// Retry under fork-storm load: a freshly-written fake `git` can transiently
// fail to exec (ETXTBSY) or exit before recording its pids; both leave no
// pids and are independent per attempt (see fake_git_run_with_pids). The
// winning attempt's future is kept PENDING so the drop below exercises the
// client-disconnect teardown. Dropping a losing attempt's future makes its
// guard reap anything that spawned, so retries don't leak.
let (fut, grandchild) = {
let mut attempt = 0u64;
loop {
attempt += 1;
let _ = std::fs::remove_file(&pidfile);
let mut fut = Box::pin(run_git_service(
git_bin.to_str().unwrap(),
"git-upload-pack",
tmp.path(),
Bytes::new(),
Duration::from_secs(60),
));
// Advance the future until the fake has spawned its grandchild.
let mut pids = None;
for _ in 0..500 {
let _ = tokio::time::timeout(std::time::Duration::from_millis(10), &mut fut).await;
if let Some(p) = read_two_pids(&pidfile) {
pids = Some(p);
break;
}
}
// Dropping the future (below or on the None arm) makes tokio reap the
// fake leader itself, so only the grandchild needs panic-cleanup —
// carrying the already-reaped leader pid risks SIGKILLing a recycled pid
// under parallel test load.
let (_leader, grandchild) = match pids {
Some(p) => p,
// Drop the still-armed future first so its guard reaps the fake
// group, then fail — otherwise a spawn hiccup orphans the processes.
None => {
drop(fut);
panic!("fake git should spawn a grandchild and write its pids");
// Advance the future a slice at a time until the fake records its
// pids. `Ok(_)` means run_git_service returned before the pidfile
// appeared (spawn error / early exit); stop polling then, since
// re-polling a completed future panics with `async fn resumed after
// completion`. Read the pidfile first so a fake that wrote its pids
// and then exited is still captured.
let mut pids = None;
for _ in 0..500 {
let finished =
tokio::time::timeout(std::time::Duration::from_millis(10), &mut fut)
.await
.is_ok();
if let Some(p) = read_two_pids(&pidfile) {
pids = Some(p);
break;
}
if finished {
break;
}
}
// Only the grandchild needs panic-cleanup: dropping the future makes
// tokio reap the fake leader, and carrying the reaped leader pid
// risks SIGKILLing a recycled pid under parallel test load.
match pids {
Some((_leader, g)) => break (fut, g),
None => {
// Transient spawn miss: drop the still-armed future so its
// guard reaps anything that spawned, then back off (growing)
// so a bursty fork-pressure spike subsides before retrying.
drop(fut);
assert!(
attempt < FAKE_GIT_RETRY_ATTEMPTS,
"fake git failed to spawn and record its pids after \
{FAKE_GIT_RETRY_ATTEMPTS} attempts (persistent failure, \
not a transient parallel-runner miss)"
);
tokio::time::sleep(std::time::Duration::from_millis(
FAKE_GIT_BACKOFF_STEP_MS * attempt,
))
.await;
}
}
}
};
let _cleanup = ReapOnPanic(vec![grandchild]);
@@ -1027,19 +1246,20 @@ mod tests {
);
let git_bin = write_fake_git(tmp.path(), &body);
let result = run_git_service(
git_bin.to_str().unwrap(),
"git-upload-pack",
tmp.path(),
Bytes::new(),
)
.await;
// wait_with_output already reaped the fake leader, so only the grandchild
// needs panic-cleanup — SIGKILLing the reaped leader pid could hit a
// recycled pid under parallel test load.
let (_leader, grandchild) =
read_two_pids(&pidfile).expect("fake git should have written its pids");
// recycled pid under parallel test load. Retry the run so a transient spawn
// miss under fork-storm load doesn't fail the test (see fake_git_run_with_pids).
let (result, (_leader, grandchild)) = fake_git_run_with_pids(&pidfile, || {
run_git_service(
git_bin.to_str().unwrap(),
"git-upload-pack",
tmp.path(),
Bytes::new(),
Duration::from_secs(60),
)
})
.await;
let _cleanup = ReapOnPanic(vec![grandchild]);
assert!(result.is_err(), "non-zero git exit must surface as Err");
@@ -1072,16 +1292,18 @@ mod tests {
);
let git_bin = write_fake_git(tmp.path(), &body);
let result = run_git_service(
git_bin.to_str().unwrap(),
"git-upload-pack",
tmp.path(),
Bytes::new(),
)
// Retry the run so a transient spawn miss under fork-storm load doesn't
// fail the test (see fake_git_run_with_pids).
let (result, (_leader, grandchild)) = fake_git_run_with_pids(&pidfile, || {
run_git_service(
git_bin.to_str().unwrap(),
"git-upload-pack",
tmp.path(),
Bytes::new(),
Duration::from_secs(60),
)
})
.await;
let (_leader, grandchild) =
read_two_pids(&pidfile).expect("fake git should have written its pids");
let _cleanup = ReapOnPanic(vec![grandchild]);
assert!(result.is_ok(), "zero-exit git must surface as Ok");
@@ -1097,4 +1319,198 @@ mod tests {
tokio::time::sleep(std::time::Duration::from_millis(10)).await;
}
}
// A git that hangs (never finishes, never disconnects) must be bounded: it
// returns GitServiceTimeout (the handler maps that to 504) and its process
// group is torn down. Goes RED if the timeout wrap is removed — the OUTER
// bound below then fires instead of run_git_service's own, so the test fails
// loudly rather than hanging CI.
#[cfg(unix)]
#[tokio::test]
async fn run_git_service_times_out_and_tears_down_a_hung_git() {
let tmp = tempfile::TempDir::new().unwrap();
let pidfile = tmp.path().join("pids");
// Fork a grandchild, then hang forever.
let body = format!(
"#!/bin/sh\nsleep 300 &\nprintf '%s\\n%s\\n' \"$$\" \"$!\" > \"{}\"\nwait\n",
pidfile.display()
);
let git_bin = write_fake_git(tmp.path(), &body);
// Outer bound well above git_timeout: if run_git_service's own timeout is
// broken it hangs, and this fires instead so we assert-fail, not hang.
// fake_git_run_with_pids retries a transient spawn/schedule miss under
// fork-storm load and returns the pidfile the fake recorded.
let git_timeout = Duration::from_millis(1000);
let (outcome, (_leader, grandchild)) = fake_git_run_with_pids(&pidfile, || {
tokio::time::timeout(
Duration::from_secs(5),
run_git_service(
git_bin.to_str().unwrap(),
"git-upload-pack",
tmp.path(),
Bytes::new(),
git_timeout,
),
)
})
.await;
assert!(
outcome.is_ok(),
"run_git_service must return via its own timeout, not hang"
);
let err = outcome
.unwrap()
.expect_err("a hung git must surface as Err");
assert!(
err.downcast_ref::<GitServiceTimeout>().is_some(),
"the error must be GitServiceTimeout (maps to 504), got: {err:#}"
);
// The hung git's group must be torn down (the armed guard fires on the
// timeout return), not leaked.
let _cleanup = ReapOnPanic(vec![grandchild]);
let mut gone = false;
for _ in 0..500 {
if !alive(grandchild) {
gone = true;
break;
}
tokio::time::sleep(Duration::from_millis(10)).await;
}
assert!(
gone,
"a timed-out git's process group must be torn down, not leaked"
);
}
// On timeout, run_git_service must REAP the group before returning, so a
// caller releasing a write lock (receive-pack) can't race a still-live git.
// The fake traps SIGTERM and lingers ~1s (git-cleanup stand-in): with the
// reap the call waits for it to exit, so the leader is dead by the time we
// return; without it, the call returns while the fake is still mid-trap.
// Goes RED if the timeout arm's reap is removed.
#[cfg(unix)]
#[tokio::test]
async fn run_git_service_reaps_the_group_before_returning_on_timeout() {
let tmp = tempfile::TempDir::new().unwrap();
let pidfile = tmp.path().join("pids");
// Leader traps SIGTERM and exits after ~1s. The grandchild (a sub-shell)
// IGNORES SIGTERM entirely (`trap "" TERM`) and sleeps far past the ~4s reap
// cap, so nothing but the untrappable SIGKILL escalation can kill it within
// the cap. That makes the escalation load-bearing: neuter the SIGKILL and
// this test goes RED. (A grandchild that merely trapped-and-slept UNDER the
// cap would self-exit and hide a broken escalation.) "Wait for the whole
// group" (leader AND grandchild) is observable because both must be dead
// before run_git_service returns.
let body = format!(
"#!/bin/sh\ntrap 'sleep 1; exit 0' TERM\nsh -c 'trap \"\" TERM; sleep 300' >/dev/null 2>&1 &\nprintf '%s\\n%s\\n' \"$$\" \"$!\" > \"{}\"\nwait\n",
pidfile.display()
);
let git_bin = write_fake_git(tmp.path(), &body);
// fake_git_run_with_pids retries a transient spawn/schedule miss under
// fork-storm load and returns the pids the fake recorded; the outer 10s
// bound fires only if run_git_service's own timeout is broken (so we
// assert-fail rather than hang), well above the git_timeout plus the
// SIGKILL-escalation reap of the SIGTERM-ignoring grandchild.
let git_timeout = Duration::from_millis(300);
let (outcome, (leader, grandchild)) = fake_git_run_with_pids(&pidfile, || {
tokio::time::timeout(
Duration::from_secs(10),
run_git_service(
git_bin.to_str().unwrap(),
"git-upload-pack",
tmp.path(),
Bytes::new(),
git_timeout,
),
)
})
.await;
assert!(outcome.is_ok(), "must return via its own timeout, not hang");
assert!(outcome.unwrap().is_err(), "a hung git must surface as Err");
// The WHOLE group (leader AND the lingering grandchild) must be gone by the
// time we get here.
let _cleanup = ReapOnPanic(vec![grandchild]);
assert!(
!alive(leader) && !alive(grandchild),
"run_git_service must reap the whole git process group (leader AND \
grandchildren) before returning on a timeout, so a write-lock release \
can't race a still-live git"
);
}
// The timeout bounds git-receive-pack too, not just upload-pack: on the push
// path a hung git also holds the repo's write lock, so an unbounded receive-pack
// pins the repo until the process dies. run_git_service must return
// Err(GitServiceTimeout) for the receive-pack service exactly as it does for
// upload-pack. Goes RED if the internal timeout is removed (the fake never
// exits, so the outer bound trips and the outcome is not Ok).
#[cfg(unix)]
#[tokio::test]
async fn run_git_service_times_out_a_hung_receive_pack() {
let tmp = tempfile::TempDir::new().unwrap();
let git_bin = write_fake_git(tmp.path(), "#!/bin/sh\nsleep 300\n");
let outcome = tokio::time::timeout(
Duration::from_secs(5),
run_git_service(
git_bin.to_str().unwrap(),
"git-receive-pack",
tmp.path(),
Bytes::new(),
Duration::from_millis(200),
),
)
.await;
assert!(
outcome.is_ok(),
"receive-pack must return via its own timeout, not hang"
);
let err = outcome
.unwrap()
.expect_err("a hung receive-pack must surface as Err");
assert!(
err.downcast_ref::<GitServiceTimeout>().is_some(),
"a hung receive-pack must time out as GitServiceTimeout (maps to 504); \
got: {err:#}"
);
}
// A malformed request: git exits non-zero with a recognizable pkt-line error
// on stderr AND does not read stdin, so a body larger than the pipe buffer
// makes the write EPIPE. run_git_service must surface git's stderr (which the
// handler classifies as a 400), not the EPIPE write error (a generic 500).
// Goes RED if the stdin-write check runs before the exit-status check.
#[cfg(unix)]
#[tokio::test]
async fn run_git_service_surfaces_git_stderr_over_a_stdin_epipe() {
let tmp = tempfile::TempDir::new().unwrap();
let git_bin = write_fake_git(
tmp.path(),
"#!/bin/sh\necho 'fatal: bad line length character: 0000' >&2\nexit 128\n",
);
// Larger than a pipe buffer (~64 KiB) so the write blocks then EPIPEs when
// the fake exits without draining stdin, exercising the ordering.
let big = Bytes::from(vec![0u8; 256 * 1024]);
let result = run_git_service(
git_bin.to_str().unwrap(),
"git-upload-pack",
tmp.path(),
big,
Duration::from_secs(60),
)
.await;
let err = result.expect_err("non-zero git must surface as Err");
let msg = format!("{err:#}");
assert!(
msg.contains("bad line length"),
"run_git_service must surface git's stderr (a classifiable 400), not the \
stdin-write EPIPE (a generic 500); got: {msg}"
);
}
}
+379 -60
View File
@@ -24,8 +24,14 @@ mod test_support;
mod visibility;
mod webhooks;
use anyhow::{Context, Result};
use anyhow::{anyhow, Context, Result};
use axum::extract::State;
use axum::http::StatusCode;
use axum::response::IntoResponse;
use axum::routing::get;
use axum::{Json, Router};
use clap::Parser;
use std::sync::atomic::{AtomicU64, Ordering};
use std::sync::Arc;
use tokio::net::TcpListener;
use tokio::sync::watch;
@@ -38,6 +44,20 @@ use config::Config;
use db::Db;
use state::AppState;
#[derive(Clone)]
struct DegradedState {
node_did: String,
db_startup: Arc<DbStartupStatus>,
}
/// Two independent counters with no cross-field invariant — atomics, not a
/// lock, so the retry loop and the degraded handlers never contend.
#[derive(Default)]
struct DbStartupStatus {
attempts: AtomicU64,
next_retry_secs: AtomicU64,
}
#[tokio::main]
async fn main() -> Result<()> {
tracing_subscriber::fmt()
@@ -76,21 +96,101 @@ async fn main() -> Result<()> {
env!("CARGO_PKG_VERSION")
);
info!("╚══════════════════════════════════════════╝");
info!(did = %node_did, "node identity");
info!(addr = %config.bind_addr(), "listening");
// Process-wide shutdown signal. One sender lives in AppState (cloned
// into every handler); main() keeps a clone and flips it on SIGINT
// or SIGTERM. Tasks that hold a watch::Receiver get notified at
// their next await point.
let (shutdown_tx, _shutdown_rx_for_main) = watch::channel(false);
spawn_shutdown_signal(shutdown_tx.clone());
// Connect to PostgreSQL database
let db = Arc::new(
Db::connect(&config.database_url)
.await
.context("failed to connect to database")?,
);
info!(did = %node_did, "node identity");
info!(addr = %config.bind_addr(), "binding HTTP listener");
// Bind HTTP once, before dependency initialization, and keep this socket
// for the life of the process. The degraded server accepts on a dup of the
// same socket, so the degraded→full handoff never closes the port: while
// the full server initializes, connections queue in the shared backlog
// instead of being refused.
let listener = TcpListener::bind(config.bind_addr())
.await
.with_context(|| format!("failed to bind to {}", config.bind_addr()))?;
let full_std_listener = listener.into_std()?;
let degraded_listener = TcpListener::from_std(
full_std_listener
.try_clone()
.context("failed to clone HTTP listener for degraded server")?,
)?;
// Metrics must stay observable during a database outage — the degraded
// window is exactly when dashboards need data — so this listener starts
// before the DB connects.
let metrics_handle = if !config.metrics_addr.is_empty() {
match spawn_metrics_server(&config.metrics_addr, shutdown_tx.subscribe()).await {
Ok(handle) => {
info!(addr = %config.metrics_addr, "metrics endpoint listening");
Some(handle)
}
Err(e) => {
warn!(err = %e, addr = %config.metrics_addr, "failed to start metrics endpoint — continuing without");
None
}
}
} else {
info!("metrics endpoint disabled (GITLAWB_METRICS_ADDR not set)");
None
};
let db_startup = Arc::new(DbStartupStatus::default());
let (db_ready_tx, db_ready_rx) = watch::channel(false);
let mut degraded_handle = tokio::spawn(run_degraded_server(
degraded_listener,
node_did.to_string(),
Arc::clone(&db_startup),
db_ready_rx,
shutdown_tx.subscribe(),
));
// Connect to PostgreSQL database. A transient outage or bad secret should
// not crash-loop the process and hammer the database provider; permanent
// misconfiguration surfaces through error-level logs and the /ready check.
let db = tokio::select! {
db = connect_db_with_retry(&config, Arc::clone(&db_startup), shutdown_tx.subscribe()) => {
match db {
Some(db) => db,
None => {
// Shutdown requested while waiting for the database. The
// degraded server only serves one-shot 503s — abort it
// rather than drain, so a slow client can't stall exit.
degraded_handle.abort();
return Ok(());
}
}
}
degraded = &mut degraded_handle => {
if *shutdown_tx.borrow() {
return Ok(());
}
return match degraded {
Ok(Ok(())) => Err(anyhow!("degraded HTTP server stopped before database became ready")),
Ok(Err(err)) => Err(err.context("degraded HTTP server failed")),
Err(err) => Err(anyhow!("degraded HTTP server task failed: {err}")),
};
}
};
// Flip the degraded server into graceful shutdown, but do NOT await the
// drain: one slow in-flight request must not delay the full server, and
// the shared socket means there is no port gap to cover. The drain
// finishes (and logs) in the background.
db_ready_tx.send(true).ok();
tokio::spawn(async move {
match degraded_handle.await {
Ok(Ok(())) => {}
Ok(Err(err)) => warn!(err = %err, "degraded HTTP server exited with error"),
Err(err) => warn!(err = %err, "degraded HTTP server task failed"),
}
});
info!(addr = %config.bind_addr(), "database ready; starting full HTTP server");
// Prune peer rows that point back at this node (stale self-loop entries)
if let Some(public_url) = config.public_url.as_deref() {
@@ -187,6 +287,26 @@ async fn main() -> Result<()> {
let rate_limiter =
rate_limit::RateLimiter::new_bounded(10, std::time::Duration::from_secs(3600), 200_000);
// Per-client-IP flood brake for the creation endpoints. The per-DID limiter
// above is bypassed by a DID farm (one throwaway did:key per repo), which is
// exactly how the recurring spam-repo floods get past both it and the
// iCaptcha gate. Keyed on the resolved client IP so a single-source flood is
// capped regardless of how many identities it mints. Sized well above any
// legitimate per-IP creation rate; GITLAWB_CREATE_RATE_LIMIT overrides, 0
// disables. Bounded key set — the key is a client-influenced IP.
let create_limit = std::env::var("GITLAWB_CREATE_RATE_LIMIT")
.ok()
.and_then(|v| v.trim().parse::<usize>().ok())
.unwrap_or(120);
let create_ip_rate_limiter = rate_limit::RateLimiter::new_bounded(
create_limit,
std::time::Duration::from_secs(3600),
200_000,
);
if create_limit == 0 {
tracing::warn!("GITLAWB_CREATE_RATE_LIMIT=0 — per-IP creation rate limiting disabled");
}
// Push-path flood brake: max git-receive-pack requests per client IP per
// hour (counts both the info/refs advertisement and the push POST). Sized
// for heavy agent automation while still stopping flood traffic (the June
@@ -213,6 +333,29 @@ async fn main() -> Result<()> {
);
tracing::info!(trust = ?push_limiter_trust, push_limit, "push rate limiter configured");
// Peer-sync flood brakes, keyed on the resolved client IP (per-DID is useless
// here — a did:key farm self-registers). Two buckets so an unsigned notify
// flood can't drain the signed trigger caller's quota (#82). Bounded key sets
// (the key is a client-influenced IP); 0 disables each.
let sync_trigger_rate_limiter = rate_limit::RateLimiter::new_bounded(
config.sync_trigger_rate_limit,
std::time::Duration::from_secs(3600),
200_000,
);
let peer_write_rate_limiter = rate_limit::RateLimiter::new_bounded(
config.peer_write_rate_limit,
std::time::Duration::from_secs(3600),
200_000,
);
if config.sync_trigger_rate_limit == 0 {
tracing::warn!(
"GITLAWB_SYNC_TRIGGER_RATE_LIMIT=0 — /sync/trigger IP rate limiting disabled"
);
}
if config.peer_write_rate_limit == 0 {
tracing::warn!("GITLAWB_PEER_WRITE_RATE_LIMIT=0 — peer-write IP rate limiting disabled");
}
// Initialize the iCaptcha proof gate (inert unless ICAPTCHA_MODE is set).
icaptcha::init().await;
@@ -229,8 +372,11 @@ async fn main() -> Result<()> {
machine_id,
repo_store,
rate_limiter,
create_ip_rate_limiter,
push_rate_limiter,
push_limiter_trust,
sync_trigger_rate_limiter,
peer_write_rate_limiter,
shutdown_tx: shutdown_tx.clone(),
};
@@ -260,38 +406,13 @@ async fn main() -> Result<()> {
});
}
// Spawn a task that flips the shutdown signal on SIGINT or SIGTERM.
// On Unix, both signals are handled. On Windows, only Ctrl-C is
// supported by tokio::signal::ctrl_c.
{
let tx = shutdown_tx.clone();
tokio::spawn(async move {
#[cfg(unix)]
{
use tokio::signal::unix::{signal as unix_signal, SignalKind};
let mut sigterm =
unix_signal(SignalKind::terminate()).expect("install SIGTERM handler");
let mut sigint =
unix_signal(SignalKind::interrupt()).expect("install SIGINT handler");
tokio::select! {
_ = sigterm.recv() => info!("SIGTERM received, shutting down"),
_ = sigint.recv() => info!("SIGINT received, shutting down"),
}
}
#[cfg(not(unix))]
{
use tokio::signal;
let _ = signal::ctrl_c().await;
info!("Ctrl-C received, shutting down");
}
tx.send(true).ok();
});
}
// Periodic cleanup of expired rate limit entries + consumed-proof ledger
{
let rl = state.rate_limiter.clone();
let create_ip_rl = state.create_ip_rate_limiter.clone();
let push_rl = state.push_rate_limiter.clone();
let sync_trigger_rl = state.sync_trigger_rate_limiter.clone();
let peer_write_rl = state.peer_write_rate_limiter.clone();
let db = state.db.clone();
let mut shutdown_rx = state.subscribe_shutdown();
tokio::spawn(async move {
@@ -299,7 +420,10 @@ async fn main() -> Result<()> {
tokio::select! {
_ = tokio::time::sleep(std::time::Duration::from_secs(300)) => {
rl.cleanup().await;
create_ip_rl.cleanup().await;
push_rl.cleanup().await;
sync_trigger_rl.cleanup().await;
peer_write_rl.cleanup().await;
let now = std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.map(|d| d.as_secs() as i64)
@@ -319,9 +443,10 @@ async fn main() -> Result<()> {
}
let router = server::build_router(state.clone());
let listener = TcpListener::bind(config.bind_addr())
.await
.with_context(|| format!("failed to bind to {}", config.bind_addr()))?;
// Re-register the socket bound at startup — same fd, so there was never a
// moment with the port closed between the degraded and full servers.
let listener = TcpListener::from_std(full_std_listener)
.context("failed to re-register HTTP listener with the runtime")?;
info!("✓ node started — did:{}", node_did);
info!(" repos dir: {}", config.repos_dir.display());
@@ -330,23 +455,6 @@ async fn main() -> Result<()> {
&config.database_url.split('@').next_back().unwrap_or("?")
);
// Optional Prometheus metrics listener on a separate port.
let metrics_handle = if !config.metrics_addr.is_empty() {
match spawn_metrics_server(&config.metrics_addr, state.clone()).await {
Ok(handle) => {
info!(addr = %config.metrics_addr, "metrics endpoint listening");
Some(handle)
}
Err(e) => {
warn!(err = %e, addr = %config.metrics_addr, "failed to start metrics endpoint — continuing without");
None
}
}
} else {
info!("metrics endpoint disabled (GITLAWB_METRICS_ADDR not set)");
None
};
// Publish our DID record to the Kademlia DHT shortly after startup
if let Some(p2p) = &state.p2p {
let did_record = p2p::DidRecord {
@@ -456,12 +564,224 @@ async fn main() -> Result<()> {
Ok(())
}
fn spawn_shutdown_signal(tx: watch::Sender<bool>) {
tokio::spawn(async move {
#[cfg(unix)]
{
use tokio::signal::unix::{signal as unix_signal, SignalKind};
let mut sigterm =
unix_signal(SignalKind::terminate()).expect("install SIGTERM handler");
let mut sigint = unix_signal(SignalKind::interrupt()).expect("install SIGINT handler");
tokio::select! {
_ = sigterm.recv() => info!("SIGTERM received, shutting down"),
_ = sigint.recv() => info!("SIGINT received, shutting down"),
}
}
#[cfg(not(unix))]
{
use tokio::signal;
let _ = signal::ctrl_c().await;
info!("Ctrl-C received, shutting down");
}
tx.send(true).ok();
});
}
async fn connect_db_with_retry(
config: &Config,
db_startup: Arc<DbStartupStatus>,
mut shutdown_rx: watch::Receiver<bool>,
) -> Option<Arc<Db>> {
let initial_retry_secs = config.db_retry_initial_secs;
let max_retry_secs = config.db_retry_max_secs.max(initial_retry_secs);
let acquire_timeout = std::time::Duration::from_secs(config.db_acquire_timeout_secs);
let attempt_timeout = std::time::Duration::from_secs(config.db_connect_timeout_secs);
let mut attempts = 0_u64;
loop {
if *shutdown_rx.borrow() {
return None;
}
attempts = attempts.saturating_add(1);
db_startup.attempts.store(attempts, Ordering::Relaxed);
// Bound the whole attempt, not just the pool connect: migrations
// block on a cross-instance advisory lock, and an unbounded wait
// there would wedge this loop — no retries, no logs, no recovery.
// Timing out and retrying is safe; migrations are idempotent.
let attempt = match tokio::time::timeout(
attempt_timeout,
Db::connect(
&config.database_url,
config.db_max_connections,
acquire_timeout,
),
)
.await
{
Ok(result) => result,
Err(_) => Err(anyhow!(
"connect + migrate attempt exceeded {}s (GITLAWB_DB_CONNECT_TIMEOUT_SECS); \
is another instance holding the migration lock?",
attempt_timeout.as_secs()
)),
};
match attempt {
Ok(db) => {
info!(attempts, "database connection established");
return Some(Arc::new(db));
}
Err(err) => {
// A bad DATABASE_URL or rejected credentials won't heal on
// their own. Still retry (exiting would crash-loop and hammer
// the provider — and take liveness down with it), but log at
// error level and skip straight to the maximum backoff; the
// /ready health check is what surfaces this to deploys.
let permanent = is_likely_permanent_db_error(&err);
let retry_secs = if permanent {
max_retry_secs
} else {
database_retry_delay_secs(initial_retry_secs, max_retry_secs, attempts)
};
db_startup
.next_retry_secs
.store(retry_secs, Ordering::Relaxed);
if permanent {
tracing::error!(
attempts,
retry_secs,
err = %err,
"database rejected our configuration (bad DATABASE_URL or credentials?) — retrying, but operator action is likely required"
);
} else {
warn!(
attempts,
retry_secs,
err = %err,
"database unavailable during startup; retrying"
);
}
tokio::select! {
_ = tokio::time::sleep(std::time::Duration::from_secs(retry_secs)) => {}
changed = shutdown_rx.changed() => {
if changed.is_err() || *shutdown_rx.borrow() {
return None;
}
}
}
}
}
}
}
/// Errors that indicate misconfiguration rather than a transient outage: a
/// malformed DATABASE_URL, or a server that answered and rejected us —
/// Postgres error class 28xxx (invalid authorization) or 3D000 (database
/// does not exist). Best-effort: an error that anyhow can't downcast back to
/// sqlx just counts as transient.
fn is_likely_permanent_db_error(err: &anyhow::Error) -> bool {
match err.downcast_ref::<sqlx::Error>() {
Some(sqlx::Error::Configuration(_)) => true,
Some(sqlx::Error::Database(db)) => db
.code()
.map(|c| c.starts_with("28") || c.starts_with("3D"))
.unwrap_or(false),
_ => false,
}
}
fn database_retry_delay_secs(initial_secs: u64, max_secs: u64, attempts: u64) -> u64 {
// The exponent bound only keeps the u32 cast safe — max_secs is the real
// (operator-configurable) cap, and saturating math handles overflow.
let exponent = attempts.saturating_sub(1).min(63) as u32;
initial_secs
.saturating_mul(2_u64.saturating_pow(exponent))
.min(max_secs)
}
async fn run_degraded_server(
listener: TcpListener,
node_did: String,
db_startup: Arc<DbStartupStatus>,
mut db_ready_rx: watch::Receiver<bool>,
mut shutdown_rx: watch::Receiver<bool>,
) -> Result<()> {
let addr = listener.local_addr().ok();
let router = build_degraded_router(node_did, db_startup);
info!(?addr, "degraded HTTP server ready");
axum::serve(listener, router)
.with_graceful_shutdown(async move {
// wait_for resolves on predicate-true or sender-drop; either way
// this phase is over.
tokio::select! {
_ = db_ready_rx.wait_for(|ready| *ready) => {}
_ = shutdown_rx.wait_for(|stop| *stop) => {}
}
})
.await?;
Ok(())
}
fn build_degraded_router(node_did: String, db_startup: Arc<DbStartupStatus>) -> Router {
let state = DegradedState {
node_did,
db_startup,
};
// Everything answers 503 with the same body — including /health and
// /ready, so peer pings (which treat any 2xx /health as alive) and
// uptime monitors correctly see a node that cannot serve traffic.
// `/` additionally carries the node identity for probing peers.
Router::new()
.route("/", get(degraded_node_info))
.fallback(degraded_unavailable)
.with_state(state)
}
/// One source of truth for the degraded 503 body, sharing the error
/// vocabulary with error.rs so clients see the same code/message for
/// "database unavailable" regardless of which phase produced it.
fn degraded_body(db_startup: &DbStartupStatus) -> serde_json::Value {
serde_json::json!({
"status": "degraded",
"database": "initializing",
"error": error::DB_UNAVAILABLE_CODE,
"message": error::DB_UNAVAILABLE_MESSAGE,
"db_attempts": db_startup.attempts.load(Ordering::Relaxed),
"db_next_retry_secs": db_startup.next_retry_secs.load(Ordering::Relaxed),
})
}
async fn degraded_node_info(State(state): State<DegradedState>) -> impl IntoResponse {
let mut body = degraded_body(&state.db_startup);
if let Some(obj) = body.as_object_mut() {
obj.insert("name".into(), "gitlawb-node".into());
obj.insert("version".into(), env!("CARGO_PKG_VERSION").into());
obj.insert("did".into(), state.node_did.clone().into());
}
(StatusCode::SERVICE_UNAVAILABLE, Json(body))
}
async fn degraded_unavailable(State(state): State<DegradedState>) -> impl IntoResponse {
(
StatusCode::SERVICE_UNAVAILABLE,
Json(degraded_body(&state.db_startup)),
)
}
/// Spawn a small axum router that exposes only `GET /metrics` on its own
/// listener. Returns the JoinHandle so `main()` can abort it on shutdown.
/// This is deliberately separate from the main router so the metrics port
/// can be firewalled differently from the API port — bind to localhost
/// or a private interface only.
async fn spawn_metrics_server(addr: &str, state: AppState) -> Result<tokio::task::JoinHandle<()>> {
async fn spawn_metrics_server(
addr: &str,
mut shutdown_rx: watch::Receiver<bool>,
) -> Result<tokio::task::JoinHandle<()>> {
use axum::{response::IntoResponse, routing::get, Router};
async fn metrics_handler() -> impl IntoResponse {
@@ -485,7 +805,6 @@ async fn spawn_metrics_server(addr: &str, state: AppState) -> Result<tokio::task
}
}
let mut shutdown_rx = state.subscribe_shutdown();
let listener = TcpListener::bind(addr)
.await
.with_context(|| format!("failed to bind metrics listener to {addr}"))?;
+67 -3
View File
@@ -250,12 +250,14 @@ impl<S: Send + Sync> axum::extract::FromRequestParts<S> for PeerAddr {
}
}
/// The shared 429 response for the push flood brake.
/// The shared 429 response for the per-IP flood brakes. Route-agnostic: this
/// middleware now serves the push path AND the peer-sync routes, so the message
/// stays generic (the offending path is recorded in the warn log below).
pub fn too_many_requests() -> Response {
(
StatusCode::TOO_MANY_REQUESTS,
[("retry-after", "60")],
"push rate limit exceeded — try again later",
"rate limit exceeded — try again later",
)
.into_response()
}
@@ -273,7 +275,7 @@ pub async fn rate_limit_by_ip(request: Request, next: Next) -> Response {
if let Some(limiter) = limiter {
if let Some(key) = client_key(request.headers(), peer, limiter.trust) {
if !limiter.limiter.check(&key).await {
tracing::warn!(key = %key, path = %request.uri().path(), "push rate limit exceeded");
tracing::warn!(key = %key, path = %request.uri().path(), "per-IP rate limit exceeded");
return too_many_requests();
}
}
@@ -541,4 +543,66 @@ mod tests {
assert_eq!(post_from(&router, b).await, StatusCode::OK); // independent bucket
assert_eq!(post_from(&router, a).await, StatusCode::TOO_MANY_REQUESTS);
}
// ── creation-route layering: IP brake runs BEFORE auth ──────────────
/// A minimal stand-in for the creation route group: an inner "auth" layer
/// that rejects every request with 401 (as signature verification would for
/// an unsigned caller), wrapped by the per-IP creation limiter as the
/// outermost layer — exactly the ordering `server::build_router` applies to
/// `/api/v1/repos` et al. Lets us assert the security-relevant property
/// (flood traffic is braked before auth burns CPU) without a database, which
/// the full `#[sqlx::test]` route test in `api::repos` cannot cover in the
/// plain unit-test pass.
fn ip_limited_over_auth_router(limiter: IpRateLimiter) -> axum::Router {
async fn always_unauthorized(_req: Request, _next: Next) -> Response {
StatusCode::UNAUTHORIZED.into_response()
}
axum::Router::new()
.route(
"/api/v1/repos",
axum::routing::post(|| async { StatusCode::CREATED }),
)
.layer(axum::middleware::from_fn(always_unauthorized))
.layer(axum::middleware::from_fn(rate_limit_by_ip))
.layer(axum::Extension(limiter))
}
async fn post_repos_from(router: &axum::Router, peer: SocketAddr) -> StatusCode {
use tower::ServiceExt;
let mut req = axum::http::Request::builder()
.method(axum::http::Method::POST)
.uri("/api/v1/repos")
.body(axum::body::Body::empty())
.unwrap();
req.extensions_mut().insert(ConnectInfo(peer));
router.clone().oneshot(req).await.unwrap().status()
}
#[tokio::test]
async fn creation_ip_brake_rejects_before_auth() {
// A DID farm mints a fresh signature per repo, so the per-DID limiter
// never trips; the per-IP brake is what stops the flood. It must run
// outermost, ahead of auth, so a flood is rejected with 429 before
// signature verification — otherwise every request would reach (and pay
// for) auth and merely 401.
let router = ip_limited_over_auth_router(IpRateLimiter {
limiter: RateLimiter::new(1, Duration::from_secs(60)),
trust: TrustedProxy::None,
});
let peer: SocketAddr = "203.0.113.42:7000".parse().unwrap();
// First request passes the IP brake and reaches the (failing) auth layer.
assert_eq!(
post_repos_from(&router, peer).await,
StatusCode::UNAUTHORIZED
);
// Budget now exhausted: the IP brake short-circuits with 429 *before*
// auth — proving the limiter is the outermost layer.
assert_eq!(
post_repos_from(&router, peer).await,
StatusCode::TOO_MANY_REQUESTS,
"an exhausted IP must be braked with 429 before auth runs, not leak to 401"
);
}
}
+66 -4
View File
@@ -81,8 +81,19 @@ pub fn build_router(state: AppState) -> Router {
.route("/api/v1/tasks", get(tasks::list_tasks))
.route("/api/v1/tasks/{id}", get(tasks::get_task));
// ── Rate-limited creation routes — require HTTP Signature + per-DID throttle
// ── Rate-limited creation routes — require HTTP Signature, plus a per-DID
// throttle AND a per-IP flood brake. The per-DID limiter (inner) caps a
// single identity; the per-IP limiter (outer) caps a DID farm that mints a
// fresh throwaway did:key per repo to slip past both the per-DID limit and
// the iCaptcha gate — the mechanism behind the recurring spam-repo floods.
// The per-IP layer wraps the auth layer (outermost = runs first) so flood
// traffic is rejected before signature verification burns CPU, matching the
// push path.
let limiter = state.rate_limiter.clone();
let create_ip_limiter = rate_limit::IpRateLimiter {
limiter: state.create_ip_rate_limiter.clone(),
trust: state.push_limiter_trust,
};
let creation_routes = add_auth_layers(
Router::new()
.route("/api/v1/repos", post(repos::create_repo))
@@ -96,7 +107,9 @@ pub fn build_router(state: AppState) -> Router {
.layer(middleware::from_fn(rate_limit::rate_limit_by_did))
.layer(axum::Extension(limiter)),
state.clone(),
);
)
.layer(middleware::from_fn(rate_limit::rate_limit_by_ip))
.layer(axum::Extension(create_ip_limiter));
// ── Write routes — require HTTP Signature (no rate limit) ─────────────
let write_routes = add_auth_layers(
@@ -248,7 +261,8 @@ pub fn build_router(state: AppState) -> Router {
.route(
"/api/v1/agents/{did}/bounties",
get(bounties::agent_bounty_stats),
);
)
.layer(middleware::from_fn(auth::optional_signature));
// ── Profile routes (write — require HTTP Signature) ─────────────────
let profile_write_routes = add_auth_layers(
@@ -277,15 +291,41 @@ pub fn build_router(state: AppState) -> Router {
.route("/api/v1/peers", get(peers::list_peers))
.route("/api/v1/peers/{did}/ping", get(peers::ping_peer));
// /sync/trigger drives an O(peers) outbound fan-out + per-repo enqueue, so it
// ALWAYS requires a signature (both config modes) and carries a tight per-IP
// brake. A signature alone does not cap cost — a did:key farm self-registers
// (INV-10) — so the IP brake is a separate, load-bearing half. The brake is
// outermost (runs before signature verification burns CPU) and is keyed on the
// client IP before any DID is read, so DID rotation cannot bypass it.
let sync_trigger_routes = add_auth_layers(
Router::new().route("/api/v1/sync/trigger", post(peers::trigger_sync)),
state.clone(),
)
.layer(middleware::from_fn(rate_limit::rate_limit_by_ip))
.layer(axum::Extension(rate_limit::IpRateLimiter {
limiter: state.sync_trigger_rate_limiter.clone(),
trust: state.push_limiter_trust,
}));
// announce + notify keep their rolling-upgrade signature behavior (unsigned
// accepted until all peers upgrade), but both reach peer-write side effects —
// notify hits the same enqueue_sync sink as trigger — so they carry a per-IP
// brake too, on a SEPARATE bucket from trigger's, so an unsigned notify flood
// cannot drain the signed trigger caller's quota.
let mut peer_write_routes = Router::new()
.route("/api/v1/peers/announce", post(peers::announce))
.route("/api/v1/sync/trigger", post(peers::trigger_sync))
.route("/api/v1/sync/notify", post(peers::notify_sync));
peer_write_routes = if state.config.require_signed_peer_writes {
add_auth_layers(peer_write_routes, state.clone())
} else {
peer_write_routes.layer(middleware::from_fn(auth::optional_signature))
};
let peer_write_routes = peer_write_routes
.layer(middleware::from_fn(rate_limit::rate_limit_by_ip))
.layer(axum::Extension(rate_limit::IpRateLimiter {
limiter: state.peer_write_rate_limiter.clone(),
trust: state.push_limiter_trust,
}));
// ── Read routes — open for public repos ───────────────────────────────
let read_routes = Router::new()
@@ -415,6 +455,7 @@ pub fn build_router(state: AppState) -> Router {
let meta_routes = Router::new()
.route("/", get(node_info))
.route("/health", get(health))
.route("/ready", get(ready))
.route("/api/v1/p2p/info", get(p2p_info))
.route("/api/v1/stats", get(stats))
.route("/api/v1/contracts", get(contracts_info));
@@ -434,6 +475,7 @@ pub fn build_router(state: AppState) -> Router {
.merge(read_routes)
.merge(peer_read_routes)
.merge(peer_write_routes)
.merge(sync_trigger_routes)
.merge(ipfs_routes)
.merge(arweave_routes)
.merge(meta_routes)
@@ -456,6 +498,26 @@ async fn health() -> Json<serde_json::Value> {
Json(json!({ "status": "ok" }))
}
/// Readiness = "this node can serve real traffic", and every real endpoint
/// depends on the database, so probe it rather than reporting a constant.
/// The probe gets its own short bound so a wedged pool can't hang the health
/// check for the full acquire timeout.
async fn ready(State(state): State<AppState>) -> axum::response::Response {
let probe = tokio::time::timeout(std::time::Duration::from_secs(2), state.db.ping()).await;
match probe {
Ok(Ok(())) => Json(json!({ "status": "ready" })).into_response(),
_ => (
axum::http::StatusCode::SERVICE_UNAVAILABLE,
Json(json!({
"status": "degraded",
"error": crate::error::DB_UNAVAILABLE_CODE,
"message": crate::error::DB_UNAVAILABLE_MESSAGE,
})),
)
.into_response(),
}
}
async fn node_info(State(state): State<AppState>) -> Json<serde_json::Value> {
let p2p_peer_id = state.p2p.as_ref().map(|h| h.local_peer_id.to_string());
Json(json!({
+22
View File
@@ -51,13 +51,35 @@ pub struct AppState {
pub repo_store: RepoStore,
/// Per-DID rate limiter for creation endpoints (repos, issues, PRs)
pub rate_limiter: RateLimiter,
/// Per-client-IP rate limiter for the same creation endpoints. The per-DID
/// limiter above cannot brake a creation flood from a DID farm — one
/// throwaway `did:key` per repo means each DID makes a single create call
/// and never trips its own bucket. A valid iCaptcha proof does not cap this
/// either: the enforced level draws only machine-solvable deterministic
/// challenges (and the caller can pin the easy type), so a bot mints a fresh
/// DID, solves a proof, and creates a repo unthrottled. Braking on the
/// resolved client IP is what actually stops a single-source flood (same
/// rationale as `push_rate_limiter`). Keyed by `push_limiter_trust`.
pub create_ip_rate_limiter: RateLimiter,
/// Per-client-IP rate limiter for git-receive-pack. Per-DID limits cannot
/// brake a push flood from a DID farm (one throwaway DID per repo), so the
/// push path throttles on the resolved client IP instead.
pub push_rate_limiter: RateLimiter,
/// Which forwarded header (if any) the edge is trusted to set, for
/// resolving the push limiter's client-IP key. See `GITLAWB_TRUSTED_PROXY`.
/// Node-wide; also keys the two peer-sync limiters below.
pub push_limiter_trust: crate::rate_limit::TrustedProxy,
/// Per-client-IP limiter for `POST /api/v1/sync/trigger` (tight). The route
/// requires a signature, but a signature does not cap cost (a did:key farm
/// self-registers), and its per-call cost is an O(peers) fan-out, so the IP
/// brake is a separate, load-bearing half. Its own bucket so an unsigned
/// `/sync/notify` flood cannot drain the signed trigger caller's quota.
pub sync_trigger_rate_limiter: RateLimiter,
/// Per-client-IP limiter for the peer-write routes (`/peers/announce`,
/// `/sync/notify`) (generous). `/sync/notify` reaches the same `enqueue_sync`
/// sink as trigger and accepts unsigned requests from known peers, so it is
/// braked too; each peer's distinct IP gets its own bucket.
pub peer_write_rate_limiter: RateLimiter,
/// Process-wide graceful-shutdown signal. Sending `true` causes every
/// task that holds a `watch::Receiver` to exit at its next await point.
/// Used by:
File diff suppressed because it is too large. Load diff
+34 -8
View File
@@ -25,15 +25,12 @@ fn nfc(s: &str) -> String {
s.nfc().collect()
}
/// True if `caller` is the repo owner. Matches exactly or against the
/// did:key short form — a non-key DID (did:gitlawb:/did:web:) in `owner_did`
/// never matches a bare short segment, and a non-key DID in `caller` never
/// matches a did:key owner's short form. Uses `normalize_owner_key` (not
/// `did_matches`) so the did:key-only rule is stricter: cross-method matching
/// would let a non-key canonical row bypass the #124 visibility gate.
/// True if `caller` is the repo owner. Uses [`crate::api::did_matches`] so the
/// owner check is representation-agnostic within `did:key` (full `did:key:z...`
/// matches bare `z...` stored by mirror rows) while still denying cross-method
/// collisions (`did:gitlawb:z...` vs bare `z...`).
fn is_owner(owner_did: &str, caller: &str) -> bool {
let owner_short = crate::db::normalize_owner_key(owner_did);
caller == owner_did || caller == owner_short
crate::api::did_matches(owner_did, caller)
}
/// The match prefix for a glob: "/" stays "/", "/secret/**" becomes "/secret".
@@ -428,6 +425,35 @@ mod tests {
);
}
// #153 forward guard: is_owner must match across bare-key / full-did:key
// representations should someone revert to single-side normalize_owner_key.
#[test]
fn bare_owner_full_caller_allows_owner() {
let rules = [rule("/", VisibilityMode::A, &[])];
assert_eq!(
visibility_check(&rules, false, "z6MkOwner", Some("did:key:z6MkOwner"), "/"),
Decision::Allow,
"bare owner + full did:key caller must match (guard against single-side revert)"
);
}
// #153 regression: cross-method DID must still be denied even when the
// trailing segment collides with a bare owner key.
#[test]
fn cross_method_did_denied_with_bare_owner() {
let rules = [rule("/", VisibilityMode::A, &[])];
assert_eq!(
visibility_check(&rules, false, "z6MkFoo", Some("did:gitlawb:z6MkFoo"), "/"),
Decision::Deny,
"cross-method DID must not match a bare did:key owner"
);
assert_eq!(
visibility_check(&rules, false, "z6MkFoo", Some("did:web:z6MkFoo"), "/"),
Decision::Deny,
"did:web must not match a bare did:key owner"
);
}
#[test]
fn root_rule_allows_listed_reader() {
let rules = [rule("/", VisibilityMode::A, &["did:key:z6MkFriend"])];
+1 -1
View File
@@ -1,7 +1,7 @@
[package]
name = "gl"
description = "The gitlawb CLI — identity management, node control, MCP server"
version = "0.5.0" # x-release-please-version
version = "0.5.1" # x-release-please-version
edition.workspace = true
rust-version.workspace = true
license.workspace = true
+28 -10
View File
@@ -8,6 +8,10 @@ use std::path::PathBuf;
use crate::http::NodeClient;
use crate::identity::load_keypair_from_dir;
fn signed_client(node: &str, dir: Option<&std::path::Path>) -> NodeClient {
NodeClient::new(node, load_keypair_from_dir(dir).ok())
}
#[derive(Args)]
pub struct BountyArgs {
#[command(subcommand)]
@@ -49,6 +53,8 @@ pub enum BountyCmd {
status: Option<String>,
#[arg(long, default_value = "https://node.gitlawb.com", env = "GITLAWB_NODE")]
node: String,
#[arg(long)]
dir: Option<PathBuf>,
},
/// Show details of a specific bounty
Show {
@@ -56,6 +62,8 @@ pub enum BountyCmd {
id: String,
#[arg(long, default_value = "https://node.gitlawb.com", env = "GITLAWB_NODE")]
node: String,
#[arg(long)]
dir: Option<PathBuf>,
},
/// Claim an open bounty
Claim {
@@ -121,8 +129,13 @@ pub async fn run(args: BountyArgs) -> Result<()> {
node,
dir,
} => cmd_create(repo, title, amount, issue, tx_hash, deadline, node, dir).await,
BountyCmd::List { repo, status, node } => cmd_list(repo, status, node).await,
BountyCmd::Show { id, node } => cmd_show(id, node).await,
BountyCmd::List {
repo,
status,
node,
dir,
} => cmd_list(repo, status, node, dir).await,
BountyCmd::Show { id, node, dir } => cmd_show(id, node, dir).await,
BountyCmd::Claim {
id,
wallet,
@@ -192,8 +205,13 @@ async fn cmd_create(
Ok(())
}
async fn cmd_list(repo: Option<String>, status: Option<String>, node: String) -> Result<()> {
let client = NodeClient::new(&node, None);
async fn cmd_list(
repo: Option<String>,
status: Option<String>,
node: String,
dir: Option<PathBuf>,
) -> Result<()> {
let client = signed_client(&node, dir.as_deref());
let url = if let Some(ref repo) = repo {
let (owner, name) = repo
@@ -214,7 +232,7 @@ async fn cmd_list(repo: Option<String>, status: Option<String>, node: String) ->
};
let resp = client
.get(&url)
.get_authed(&url)
.await
.context("failed to connect to node")?;
let body: Value = resp.json().await.unwrap_or_default();
@@ -237,9 +255,9 @@ async fn cmd_list(repo: Option<String>, status: Option<String>, node: String) ->
Ok(())
}
async fn cmd_show(id: String, node: String) -> Result<()> {
let client = NodeClient::new(&node, None);
let resp = client.get(&format!("/api/v1/bounties/{id}")).await?;
async fn cmd_show(id: String, node: String, dir: Option<PathBuf>) -> Result<()> {
let client = signed_client(&node, dir.as_deref());
let resp = client.get_authed(&format!("/api/v1/bounties/{id}")).await?;
let status = resp.status();
let body: Value = resp.json().await.unwrap_or_default();
@@ -462,7 +480,7 @@ mod tests {
.create_async()
.await;
cmd_list(None, None, server.url()).await.unwrap();
cmd_list(None, None, server.url(), None).await.unwrap();
}
#[tokio::test]
@@ -506,7 +524,7 @@ mod tests {
.create_async()
.await;
let err = cmd_show("nonexistent".to_string(), server.url())
let err = cmd_show("nonexistent".to_string(), server.url(), None)
.await
.unwrap_err();
assert!(err.to_string().contains("not found"));
+34 -17
View File
@@ -5,10 +5,15 @@
use anyhow::{Context, Result};
use clap::{Args, Subcommand};
use serde_json::Value;
use std::path::PathBuf;
use crate::http::NodeClient;
use crate::identity::load_keypair_from_dir;
fn signed_client(node: &str, dir: Option<&std::path::Path>) -> NodeClient {
NodeClient::new(node, load_keypair_from_dir(dir).ok())
}
#[derive(Args)]
pub struct CertArgs {
#[command(subcommand)]
@@ -23,6 +28,8 @@ pub enum CertCmd {
repo: String,
#[arg(long, default_value = "https://node.gitlawb.com", env = "GITLAWB_NODE")]
node: String,
#[arg(long)]
dir: Option<PathBuf>,
},
/// Show a specific ref certificate and verify its signature
Show {
@@ -32,28 +39,39 @@ pub enum CertCmd {
id: String,
#[arg(long, default_value = "https://node.gitlawb.com", env = "GITLAWB_NODE")]
node: String,
#[arg(long)]
dir: Option<PathBuf>,
},
}
pub async fn run(args: CertArgs) -> Result<()> {
match args.cmd {
CertCmd::List { repo, node } => cmd_list(repo, node).await,
CertCmd::Show { repo, id, node } => cmd_show(repo, id, node).await,
CertCmd::List { repo, node, dir } => cmd_list(repo, node, dir).await,
CertCmd::Show {
repo,
id,
node,
dir,
} => cmd_show(repo, id, node, dir).await,
}
}
/// Resolve "repo" into (owner, name) using the caller's DID when no slash is given.
async fn resolve_repo(repo: &str, node: &str) -> Result<(String, String)> {
async fn resolve_repo(
repo: &str,
node: &str,
dir: Option<&std::path::Path>,
) -> Result<(String, String)> {
if let Some((owner, name)) = repo.split_once('/') {
Ok((owner.to_string(), name.to_string()))
} else {
let short = if let Ok(kp) = load_keypair_from_dir(None) {
let short = if let Ok(kp) = load_keypair_from_dir(dir) {
let did = kp.did().to_string();
did.split(':').next_back().unwrap_or(&did).to_string()
} else {
let client = NodeClient::new(node, None);
let client = signed_client(node, dir);
let info: Value = client
.get("/")
.get_authed("/")
.await?
.json()
.await
@@ -65,13 +83,13 @@ async fn resolve_repo(repo: &str, node: &str) -> Result<(String, String)> {
}
}
async fn cmd_list(repo: String, node: String) -> Result<()> {
let (owner, name) = resolve_repo(&repo, &node).await?;
async fn cmd_list(repo: String, node: String, dir: Option<PathBuf>) -> Result<()> {
let (owner, name) = resolve_repo(&repo, &node, dir.as_deref()).await?;
let client = NodeClient::new(&node, None);
let client = signed_client(&node, dir.as_deref());
let path = format!("/api/v1/repos/{owner}/{name}/certs");
let resp: Value = client
.get(&path)
.get_authed(&path)
.await?
.json()
.await
@@ -96,16 +114,16 @@ async fn cmd_list(repo: String, node: String) -> Result<()> {
Ok(())
}
async fn cmd_show(repo: String, id: String, node: String) -> Result<()> {
let (owner, name) = resolve_repo(&repo, &node).await?;
async fn cmd_show(repo: String, id: String, node: String, dir: Option<PathBuf>) -> Result<()> {
let (owner, name) = resolve_repo(&repo, &node, dir.as_deref()).await?;
let client = NodeClient::new(&node, None);
let client = signed_client(&node, dir.as_deref());
let id = resolve_cert_id(&client, &owner, &name, &id).await?;
// Fetch the certificate
let path = format!("/api/v1/repos/{owner}/{name}/certs/{id}");
let resp = client
.get(&path)
.get_authed(&path)
.await?
.error_for_status()
.context("certificate not found")?;
@@ -168,9 +186,9 @@ async fn resolve_cert_id(client: &NodeClient, owner: &str, name: &str, id: &str)
return Ok(id.to_string());
}
let path = format!("/api/v1/repos/{owner}/{name}/certs");
let path = format!("/api/v1/repos/{owner}/{name}/certs?prefix={id}");
let resp: Value = client
.get(&path)
.get_authed(&path)
.await?
.error_for_status()
.context("failed to list certificates")?
@@ -182,7 +200,6 @@ async fn resolve_cert_id(client: &NodeClient, owner: &str, name: &str, id: &str)
let matches: Vec<String> = certs
.iter()
.filter_map(|cert| cert["id"].as_str())
.filter(|cert_id| cert_id.starts_with(id))
.map(ToString::to_string)
.collect();
+9
View File
@@ -45,6 +45,15 @@ impl NodeClient {
.with_context(|| format!("GET {url}"))
}
/// GET that signs when a keypair is available; falls back to unsigned for public repos.
pub async fn get_authed(&self, path: &str) -> Result<reqwest::Response> {
if self.keypair.is_some() {
self.get_signed(path).await
} else {
self.get(path).await
}
}
/// GET with RFC 9421 HTTP Signature auth, for owner-only read endpoints.
/// Signs over the empty body (same shape the node verifies for signed reads).
pub async fn get_signed(&self, path: &str) -> Result<reqwest::Response> {
+10 -6
View File
@@ -10,6 +10,10 @@ use uuid::Uuid;
use crate::http::NodeClient;
use crate::identity::load_keypair_from_dir;
fn signed_client(node: &str, dir: Option<&std::path::Path>) -> NodeClient {
NodeClient::new(node, load_keypair_from_dir(dir).ok())
}
#[derive(Args)]
pub struct IssueArgs {
#[command(subcommand)]
@@ -215,10 +219,10 @@ async fn cmd_create(
async fn cmd_list(repo: String, node: String, dir: Option<PathBuf>) -> Result<()> {
let (owner, name) = resolve_repo(&repo, &node, dir.as_deref()).await?;
let client = NodeClient::new(&node, None);
let client = signed_client(&node, dir.as_deref());
let path = format!("/api/v1/repos/{owner}/{name}/issues");
let resp: Value = client
.get(&path)
.get_authed(&path)
.await?
.json()
.await
@@ -254,10 +258,10 @@ async fn cmd_list(repo: String, node: String, dir: Option<PathBuf>) -> Result<()
async fn cmd_show(repo: String, id: String, node: String, dir: Option<PathBuf>) -> Result<()> {
let (owner, name) = resolve_repo(&repo, &node, dir.as_deref()).await?;
let client = NodeClient::new(&node, None);
let client = signed_client(&node, dir.as_deref());
let path = format!("/api/v1/repos/{owner}/{name}/issues/{id}");
let resp = client
.get(&path)
.get_authed(&path)
.await
.context("failed to connect to node")?;
let status = resp.status();
@@ -347,10 +351,10 @@ async fn cmd_issue_comments(
dir: Option<PathBuf>,
) -> Result<()> {
let (owner, name) = resolve_repo(&repo, &node, dir.as_deref()).await?;
let client = NodeClient::new(&node, None);
let client = signed_client(&node, dir.as_deref());
let resp: Value = client
.get(&format!(
.get_authed(&format!(
"/api/v1/repos/{owner}/{name}/issues/{id}/comments"
))
.await?
+3 -3
View File
@@ -970,14 +970,14 @@ async fn call_tool(
}
u
};
let resp: Value = client.get(&url).await?.json().await?;
let resp: Value = client.get_authed(&url).await?.json().await?;
Ok(serde_json::to_string_pretty(&resp)?)
}
"bounty_show" => {
let id = args["id"].as_str().context("missing 'id'")?;
let resp: Value = client
.get(&format!("/api/v1/bounties/{id}"))
.get_authed(&format!("/api/v1/bounties/{id}"))
.await?
.json()
.await?;
@@ -1165,7 +1165,7 @@ async fn call_tool(
(owner, repo.to_string())
};
let resp: Value = client
.get(&format!("/api/v1/repos/{owner}/{name}/issues"))
.get_authed(&format!("/api/v1/repos/{owner}/{name}/issues"))
.await?
.json()
.await?;
+2 -2
View File
@@ -658,10 +658,10 @@ async fn cmd_label_remove(
async fn cmd_label_list(repo: String, node: String, dir: Option<PathBuf>) -> Result<()> {
let (owner, name) = resolve_owner_repo_pair(&repo, &node, dir.as_deref()).await?;
let client = NodeClient::new(&node, None);
let client = NodeClient::new(&node, load_keypair_from_dir(dir.as_deref()).ok());
let resp: Value = client
.get(&format!("/api/v1/repos/{owner}/{name}/labels"))
.get_authed(&format!("/api/v1/repos/{owner}/{name}/labels"))
.await?
.json()
.await
+15 -9
View File
@@ -34,12 +34,14 @@ pub enum StarCmd {
#[arg(long)]
dir: Option<PathBuf>,
},
/// Show star count for a repository (no auth required)
/// Show star count for a repository
Count {
/// Repository in owner/repo format
repo: String,
#[arg(long, default_value = "https://node.gitlawb.com", env = "GITLAWB_NODE")]
node: String,
#[arg(long)]
dir: Option<PathBuf>,
},
}
@@ -47,7 +49,7 @@ pub async fn run(args: StarArgs) -> Result<()> {
match args.cmd {
StarCmd::Add { repo, node, dir } => cmd_add(repo, node, dir).await,
StarCmd::Remove { repo, node, dir } => cmd_remove(repo, node, dir).await,
StarCmd::Count { repo, node } => cmd_count(repo, node).await,
StarCmd::Count { repo, node, dir } => cmd_count(repo, node, dir).await,
}
}
@@ -110,15 +112,15 @@ async fn cmd_remove(repo: String, node: String, dir: Option<PathBuf>) -> Result<
Ok(())
}
async fn cmd_count(repo: String, node: String) -> Result<()> {
async fn cmd_count(repo: String, node: String, dir: Option<PathBuf>) -> Result<()> {
let (owner, name) = repo
.split_once('/')
.map(|(o, n)| (o.to_string(), n.to_string()))
.context("use owner/repo format for count (e.g. alice/myrepo)")?;
let client = NodeClient::new(&node, None);
let client = NodeClient::new(&node, load_keypair_from_dir(dir.as_deref()).ok());
let resp = client
.get(&format!("/api/v1/repos/{owner}/{name}/star"))
.get_authed(&format!("/api/v1/repos/{owner}/{name}/star"))
.await
.context("failed to connect to node")?;
@@ -310,16 +312,20 @@ mod tests {
.create_async()
.await;
cmd_count("alice/myrepo".to_string(), server.url())
cmd_count("alice/myrepo".to_string(), server.url(), None)
.await
.unwrap();
}
#[tokio::test]
async fn test_cmd_count_requires_slash() {
let err = cmd_count("noslash".to_string(), "http://127.0.0.1:1".to_string())
.await
.unwrap_err();
let err = cmd_count(
"noslash".to_string(),
"http://127.0.0.1:1".to_string(),
None,
)
.await
.unwrap_err();
assert!(err.to_string().contains("owner/repo format"));
}
+285 -10
View File
@@ -1,4 +1,4 @@
use anyhow::Result;
use anyhow::{Context, Result};
use clap::{Args, Subcommand};
use std::path::PathBuf;
@@ -30,15 +30,39 @@ pub enum SyncCmd {
pub async fn run(args: SyncArgs) -> Result<()> {
match args.cmd {
SyncCmd::Trigger => {
let keypair = load_keypair_from_dir(args.dir.as_deref()).ok();
let client = NodeClient::new(&args.node, keypair);
let resp: serde_json::Value = client
.post("/api/v1/sync/trigger", b"{}")
.await?
.json()
.await?;
let peers = resp["peers_reached"].as_u64().unwrap_or(0);
let enqueued = resp["repos_enqueued"].as_u64().unwrap_or(0);
// /api/v1/sync/trigger always requires a signature, so a missing or
// unreadable identity must fail here, locally, rather than sending an
// unsigned request that can only 401 remotely (matches the other
// signed CLI writes).
let keypair = load_keypair_from_dir(args.dir.as_deref())
.context("identity not found — run `gl identity new` first")?;
let client = NodeClient::new(&args.node, Some(keypair));
let resp = client.post("/api/v1/sync/trigger", b"{}").await?;
// The node now requires a signature on this route and rate-limits it,
// so a denial (401/429/…) is expected. Check the status BEFORE parsing:
// otherwise a JSON-ish error body deserializes into a zero-count struct
// and prints a fabricated "✓ sync triggered / 0 peers" success.
let status = resp.status();
if !status.is_success() {
// Bound the read: a hostile or broken node must not force an
// unbounded allocation just to surface a denial (INV-6, read half).
let raw = read_body_capped(resp, 8 * 1024).await;
let msg = serde_json::from_str::<serde_json::Value>(&raw)
.ok()
.and_then(|v| {
v.get("message")
.or_else(|| v.get("error"))
.and_then(|m| m.as_str())
.map(str::to_string)
})
.unwrap_or(raw);
anyhow::bail!(
"sync trigger failed ({status}): {}",
sanitize_node_msg(&msg)
);
}
let resp: serde_json::Value = resp.json().await?;
let (peers, enqueued) = trigger_counts(&resp);
println!("✓ sync triggered");
println!(" peers reached: {peers}");
println!(" repos enqueued: {enqueued}");
@@ -69,3 +93,254 @@ pub async fn run(args: SyncArgs) -> Result<()> {
}
Ok(())
}
/// Extract `(peers_reached, repos_enqueued)` from a successful sync-trigger
/// response. Split out so the extraction is unit-testable (missing or malformed
/// fields default to 0 rather than panicking).
fn trigger_counts(resp: &serde_json::Value) -> (u64, u64) {
(
resp["peers_reached"].as_u64().unwrap_or(0),
resp["repos_enqueued"].as_u64().unwrap_or(0),
)
}
/// Read at most `cap` bytes of a response body. Bounds the allocation from a
/// hostile or broken node returning a huge error body — the display is capped
/// separately, but the read itself must not be unbounded (INV-6, read half).
async fn read_body_capped(mut resp: reqwest::Response, cap: usize) -> String {
let mut buf: Vec<u8> = Vec::new();
while buf.len() < cap {
match resp.chunk().await {
Ok(Some(chunk)) => {
let take = (cap - buf.len()).min(chunk.len());
buf.extend_from_slice(&chunk[..take]);
if take < chunk.len() {
break; // hit the cap mid-chunk
}
}
_ => break, // end of body or read error — return what we have
}
}
String::from_utf8_lossy(&buf).into_owned()
}
/// Strip terminal-dangerous characters from (and cap the length of) a
/// node-supplied error string before surfacing it. The node a caller talks to
/// could be hostile and embed escape sequences in its error body; those must not
/// reach the terminal verbatim (INV-6). We drop the C0/C1 control bytes (which
/// defangs ANSI/OSC escapes) AND the Unicode bidi/format controls (which
/// `char::is_control` does not cover — they can reorder the displayed line).
fn sanitize_node_msg(s: &str) -> String {
s.chars()
.filter(|c| !c.is_control() && !is_bidi_format(*c))
.take(200)
.collect()
}
/// Unicode bidirectional and directional-isolate format characters (category
/// `Cf`). These are not `char::is_control()` (that is category `Cc` only), but a
/// right-to-left override or isolate can visually reorder a terminal line to
/// spoof the error text, so they are stripped alongside the control bytes.
fn is_bidi_format(c: char) -> bool {
matches!(c,
'\u{200E}' | '\u{200F}' | '\u{061C}' // LRM, RLM, ALM
| '\u{202A}'..='\u{202E}' // LRE, RLE, PDF, LRO, RLO
| '\u{2066}'..='\u{2069}' // LRI, RLI, FSI, PDI
)
}
#[cfg(test)]
mod tests {
use super::*;
fn trigger_args(node: String) -> (SyncArgs, tempfile::TempDir) {
// Seed a real identity so `run` gets past the mandatory-keypair check and
// reaches the status-handling path. The mocks below return a fixed status
// regardless of the signature, so these tests exercise the client's
// status-check-before-parse, not signature verification (that is proved
// server-side). Return the TempDir so the caller keeps it alive.
let dir = tempfile::TempDir::new().unwrap();
let kp = gitlawb_core::identity::Keypair::generate();
std::fs::write(
dir.path().join("identity.pem"),
kp.to_pem().unwrap().as_bytes(),
)
.unwrap();
let args = SyncArgs {
cmd: SyncCmd::Trigger,
node,
dir: Some(dir.path().to_path_buf()),
};
(args, dir)
}
#[tokio::test]
async fn trigger_requires_identity_fails_before_request() {
// Empty identity dir → no keypair. `sync trigger` must fail locally with
// a clear identity error BEFORE issuing any request. The node URL points
// at an unreachable port, so a request attempt would surface a different
// (connection) error; getting the identity error proves we never dialed.
let dir = tempfile::TempDir::new().unwrap();
let args = SyncArgs {
cmd: SyncCmd::Trigger,
node: "http://127.0.0.1:1".to_string(),
dir: Some(dir.path().to_path_buf()),
};
let err = run(args).await.unwrap_err();
assert!(
err.to_string().contains("identity not found"),
"expected a local identity error before any request, got: {err}"
);
}
#[tokio::test]
async fn trigger_surfaces_401_as_error_not_fake_success() {
let mut server = mockito::Server::new_async().await;
let _m = server
.mock("POST", "/api/v1/sync/trigger")
.with_status(401)
.with_header("content-type", "application/json")
// Valid JSON: the parse-without-status-check bug deserializes this
// into a zero-count success struct and prints "✓ sync triggered".
.with_body(r#"{"message":"unauthorized"}"#)
.create_async()
.await;
let (args, _dir) = trigger_args(server.url());
let err = run(args).await.unwrap_err();
assert!(
err.to_string().contains("401"),
"expected 401 surfaced, got: {err}"
);
}
#[tokio::test]
async fn trigger_surfaces_429_as_error() {
let mut server = mockito::Server::new_async().await;
let _m = server
.mock("POST", "/api/v1/sync/trigger")
.with_status(429)
.with_header("content-type", "application/json")
.with_body(r#"{"message":"slow down"}"#)
.create_async()
.await;
let (args, _dir) = trigger_args(server.url());
let err = run(args).await.unwrap_err();
assert!(
err.to_string().contains("429"),
"expected 429 surfaced, got: {err}"
);
}
#[tokio::test]
async fn trigger_sanitizes_control_chars_in_node_error() {
// A hostile node embeds an ANSI color escape (ESC) and a bell (BEL) in
// the JSON message field. The surfaced error must contain neither raw
// control byte, while keeping the printable text.
let mut server = mockito::Server::new_async().await;
let _m = server
.mock("POST", "/api/v1/sync/trigger")
.with_status(401)
.with_header("content-type", "application/json")
// Valid JSON whose message carries JSON-escaped ESC (\u001b) and
// BEL (\u0007); serde decodes them to real control bytes a naive
// client would print. (The status-check bug fake-successes here.)
.with_body("{\"message\":\"pwned\\u001b[31m\\u0007bad\"}")
.create_async()
.await;
let (args, _dir) = trigger_args(server.url());
let err = run(args).await.unwrap_err();
let s = err.to_string();
assert!(!s.contains('\u{1b}'), "ESC leaked to terminal: {s:?}");
assert!(!s.contains('\u{07}'), "BEL leaked to terminal: {s:?}");
assert!(
s.contains("pwned") && s.contains("bad"),
"message text dropped: {s:?}"
);
}
#[tokio::test]
async fn trigger_ok_prints_counts() {
let mut server = mockito::Server::new_async().await;
let _m = server
.mock("POST", "/api/v1/sync/trigger")
.with_status(200)
.with_header("content-type", "application/json")
.with_body(r#"{"peers_reached":2,"repos_enqueued":5}"#)
.create_async()
.await;
let (args, _dir) = trigger_args(server.url());
run(args).await.unwrap();
}
#[test]
fn sanitize_strips_controls_bidi_and_caps_length() {
// C0 (ESC/BEL) and the Cf bidi override (U+202E) are both removed; the
// printable text survives. (Note: a stripped ESC leaves any following
// "[31m" as inert literal text — that is the point, so the input here
// avoids that residue to keep the expectation unambiguous.)
let out = sanitize_node_msg("a\u{1b}\u{07}b\u{202e}c");
assert!(
!out.chars().any(|c| c.is_control()),
"control char leaked: {out:?}"
);
assert!(
!out.contains('\u{202e}'),
"RLO bidi override leaked: {out:?}"
);
assert_eq!(out, "abc");
// Length is capped at 200 chars regardless of input size.
let long = "x".repeat(250);
assert_eq!(sanitize_node_msg(&long).chars().count(), 200);
}
#[tokio::test]
async fn trigger_handles_oversized_error_body_without_unbounded_output() {
// A hostile/broken node returns a 2 MB error body. The command must still
// surface the denial with a bounded message, not hang or dump the body.
let mut server = mockito::Server::new_async().await;
let _m = server
.mock("POST", "/api/v1/sync/trigger")
.with_status(401)
.with_body("A".repeat(2_000_000))
.create_async()
.await;
let (args, _dir) = trigger_args(server.url());
let err = run(args).await.unwrap_err();
let s = err.to_string();
assert!(s.contains("401"), "denial not surfaced: {s:.80?}");
assert!(
s.len() < 500,
"error message not bounded: {} chars",
s.len()
);
}
#[tokio::test]
async fn read_body_capped_bounds_the_read() {
// The read must stop at the cap — a 2 MB body yields at most `cap` bytes,
// not the whole thing (which resp.text() would return).
let mut server = mockito::Server::new_async().await;
let _m = server
.mock("GET", "/big")
.with_status(200)
.with_body("A".repeat(2_000_000))
.create_async()
.await;
let resp = reqwest::get(format!("{}/big", server.url())).await.unwrap();
let out = read_body_capped(resp, 8192).await;
assert!(out.len() <= 8192, "read not bounded: {} bytes", out.len());
assert!(!out.is_empty(), "expected some body");
}
#[test]
fn trigger_counts_extracts_both_values() {
let v = serde_json::json!({"peers_reached": 2, "repos_enqueued": 5});
assert_eq!(trigger_counts(&v), (2, 5));
// Missing/malformed fields default to 0, never panic.
assert_eq!(trigger_counts(&serde_json::json!({})), (0, 0));
assert_eq!(
trigger_counts(&serde_json::json!({"peers_reached": "x"})),
(0, 0)
);
}
}
+1 -1
View File
@@ -120,7 +120,7 @@ Positive:
Risks:
- Many commands default to the public node; `git-remote-gitlawb` defaults to localhost. This split should be called out in README examples.
- `gl sync trigger` signs the request when a local identity is available and falls back to the legacy unsigned request for live compatibility.
- `gl sync trigger` requires a local identity and always sends a signed request; the `/api/v1/sync/trigger` route rejects unsigned calls, so the command fails locally when no identity is configured.
- Several CLI commands parse dynamic JSON responses permissively; good for compatibility, but error messages can hide response-shape regressions.
## CI and release readiness
+10
View File
@@ -38,6 +38,16 @@ primary_region = "iad"
# slots (hard_limit) — root factor in the 2026-06-12 prod outage
idle_timeout = 120
[[http_service.checks]]
# Gate routing (and deploys) on real readiness: /ready probes the DB, so a
# node stuck retrying a dead or misconfigured database is never routed to,
# and a deploy with a bad DATABASE_URL fails visibly instead of going green.
interval = "15s"
timeout = "5s"
grace_period = "75s"
method = "GET"
path = "/ready"
[[services]]
protocol = "udp"
internal_port = 7546