Compare commits

...
26 Commits
Author SHA1 Message Date
github-actions[bot] 3ec8cbf1e1 chore(main): release 0.5.1 (#158)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2026-07-10 19:46:16 +08:00
GravireiandGravirei 6b5e5bc7ae fix(node): bound list_ref_certificates with LIMIT and add upsert to prevent unbounded growth (#147) (#149)
* fix(node): bound list_ref_certificates with LIMIT and add upsert to prevent unbounded growth (#147)

* test: add INV-7 v10 upgrade-path dedup test

* fix INV-7 test: correct id-DESC tiebreaker ids and fix fmt

* fix: insert_ref_certificate returns persisted id via RETURNING; issue_ref_certificate uses it

* fix: guard cert upsert on issued_at so older certs cannot regress newer ones

* fix: .max(0) in events limit parsing, re-add lost cert tests after rebase

* address reviewer findings: return full persisted row, add monotonic clock comment, note test drift risk

* fix: add ?prefix= server-side cert lookup for CLI short-ID resolution

---------

Co-authored-by: Gravirei <[email protected]>
2026-07-10 19:44:43 +08:00
dfcaa22b23 fix(node): rate-limit repo/agent creation per client IP to stop DID-farm spam floods (#180)
* fix(node): add per-IP flood brake to repo/agent creation endpoints

The recurring spam-repo floods are not bypassing the iCaptcha gate — they
pass it. The enforced level (3) draws only machine-solvable deterministic
challenges (arithmetic/algebra/sequence), and the /v1/challenge API lets the
caller pin the easy type, so a bot mints a fresh throwaway did:key, solves a
real proof bound to that DID, and creates a repo. The per-DID creation
limiter never trips because each DID makes a single create call.

Add a per-client-IP rate limiter to the creation route group (repos,
register, fork, issues, pulls), reusing the same IpRateLimiter the push path
already uses against the identical DID-farm bypass. It wraps the auth layer
(outermost) so flood traffic is rejected before signature verification.
Default 120/hour/IP, GITLAWB_CREATE_RATE_LIMIT overrides, 0 disables.

* docs(env): document GITLAWB_CREATE_RATE_LIMIT in .env.example

* fix: apply CodeRabbit auto-fixes

Fixed 1 file(s) based on 1 unresolved review comment.

Co-authored-by: CodeRabbit <[email protected]>

* test(node): unit-test the per-IP creation brake runs before auth

The route-level coverage (repo_creation_is_rate_limited_by_ip) is
#[sqlx::test] and needs Postgres, so it is skipped in the plain unit-test
pass. Add a DB-free middleware test that layers the per-IP creation limiter
over an always-401 auth stand-in and asserts an exhausted IP is rejected
with 429 before auth runs — the ordering the DID-farm flood defense relies
on.

---------

Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
Co-authored-by: CodeRabbit <[email protected]>
2026-07-10 19:44:04 +08:00
2109d08e2c Fix/db startup resilience (#170)
* Keep node alive during database outages

* Fix startup-resilience gaps: reachable 503s, bounded retries, seamless handoff

- Route sqlx errors buried in anyhow chains back to AppError::Db and widen
  the unavailable class (Io/Tls/pool), so DB outages return 503 instead of
  500 — the previous 503 arm was unreachable dead code
- Bound each connect+migrate attempt (GITLAWB_DB_CONNECT_TIMEOUT_SECS) so a
  held migration advisory lock cannot wedge the startup retry loop forever
- Share one bound socket between the degraded and full servers and stop
  awaiting the degraded drain: no closed-port window during the handoff and
  no slow client can block the full server from starting
- Probe the database from /ready (2s bound) and gate Fly routing/deploys on
  it, so mid-life outages drain traffic and a bad DATABASE_URL fails the
  deploy visibly instead of going green; classify auth/catalog errors as
  likely-permanent (error-level log, straight to max backoff)
- Start the metrics listener before the DB connects so the degraded window
  is observable; degraded /health now returns 503 so peer pings and uptime
  monitors stop counting a DB-less node as alive
- Move the five GITLAWB_DB_* knobs onto clap Config (validated, in --help),
  drop the redundant outer connect timeout, default the pool to 20 conns,
  fix the backoff exponent cap that defeated GITLAWB_DB_RETRY_MAX_SECS, and
  strip at most one .git suffix in smart-HTTP repo names

* fix: apply CodeRabbit auto-fixes

Fixed 1 file(s) based on 1 unresolved review comment.

Co-authored-by: CodeRabbit <[email protected]>

---------

Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
Co-authored-by: CodeRabbit <[email protected]>
2026-07-10 09:44:02 +08:00
GravireiandGravirei c5e5099def refactor(visibility): route is_owner through the shared did_matches matcher (#153) (#156)
* fix(node): gate GET /ipfs/{cid} on reachable allowed-set, not deny-set (#126)

The IPFS visibility gate used withheld_blob_oids (a deny-set enumerating
only reachable blobs), so a dangling/unreachable blob was absent from the
set and served in cleartext to anonymous callers. Flip to an allowed-set
(allowed_blob_set_for_caller) that enumerates reachable blobs the caller
may read: a dangling blob has no path, is never in the set, and 404s.

* perf(ipfs): check object existence before allowed-blob walk

Move store::read_object before the allowed_blob_set_for_caller
spawn_blocking call so random-CID spray against repos with
path-scoped rules cannot trigger full-history git walks on
repos that don't carry the object.

* refactor(ipfs): improve formatting and readability in get_by_cid function

✓ P3 blocker fixed: cargo fmt applied — the format gate will pass.

  ✓ P3 cleanup resolved: withheld_blob_oids is still used by replication code in repos.rs, so it stays.

  • P2 follow-up: Tree/commit disclosure tracked in #135 — out of scope here.

* refactor(ipfs): streamline object retrieval by separating type and content reading

* docs(ipfs): update get_by_cid comment to reflect split object retrieval

* Run cargo fmt on store.rs

* refactor(visibility): route is_owner through the shared did_matches matcher (#153)

* style(profiles): remove redundant borrow in format arg

---------

Co-authored-by: Gravirei <[email protected]>
2026-07-10 08:55:49 +08:00
beardthelionandt cd67718f49 fix(node,git): bound a hung served git with a total-duration timeout (#62) (#165)
* feat(node,git): bound a hung served git with a total-duration timeout (#62)

Part 1 of #62. A served git upload-pack/receive-pack that neither finishes
nor disconnects parks forever, pinning a git PID (and, for receive-pack,
holding the write lock); on a public repo an anonymous caller can trigger
it. No tower TimeoutLayer exists and axum imposes no default.

Wrap the child interaction in tokio::time::timeout, bounded by a new
GITLAWB_GIT_SERVICE_TIMEOUT_SECS knob (default 600, must be positive).
Drain stdout/stderr concurrently with the stdin write so a large body
can't deadlock. On expiry run_git_service returns a typed GitServiceTimeout
and reaps the WHOLE process group before returning (SIGTERM so git clears
its .git/*.lock, wait for the leader and its grandchildren like index-pack
to exit, escalate to SIGKILL past a grace, hard-capped), so a caller
releasing the receive-pack write lock can't race a still-live git on the
same repo.

The handlers map GitServiceTimeout to a distinct 504 (not the generic 500
git error, and clear of the read-gate 404 / auth 401 the client keys
retries on) via a pure, unit-testable classifier rather than matching the
anyhow string. Surface git's own non-zero exit (its stderr) before a
stdin-write EPIPE so a malformed body is classified 400, not 500.

Out of scope, deferred in #62: the info/refs advertisement and the
withheld-blob path (spawn_blocking, which tokio timeout cannot cancel);
both remain unbounded (noted in the config knob's docs).

Tests: the timeout fires and tears the group down; it reaps the whole
group before returning (RED if the reap is leader-only); the wrap is
load-bearing (RED via an outer bound, not a hang); the error must be the
typed timeout; the 504 mapping and classifier are unit-tested; config
rejects 0. Full suite green.

* fix(review): harden #62 timeout tests and drop a dead reap flag

- test(node): cover the `protocol error` arm of git_service_app_error
  independently (the input has no "bad line length" substring, so it
  isolates the second classifier arm and goes RED if that arm is removed)
- test(node): make the reap-before-return test's grandchild IGNORE SIGTERM
  (`trap "" TERM`) and outlive the ~4s reap cap, so the SIGKILL escalation
  is load-bearing. Previously the grandchild self-exited under the cap, so
  neutering the SIGKILL still passed; the escalation was exercised but not
  required. Verified: RED with SIGKILL disabled, GREEN with it.
- test(node): add a receive-pack timeout test. Every prior timeout test used
  an upload-pack fake; this proves the push path (which also holds the repo
  write lock) is bounded too. RED-verified with the internal timeout removed.
- test(node): poll for the pidfile in the reap-before-return timeout test
  so a loaded runner can't false-panic before the fake git writes it
- remove the redundant `sigkilled` guard in reap_group_on_timeout; step
  == 200 fires exactly once in the 0..400 loop, so no re-entry guard is
  needed and the SIGKILL still fires exactly once

* fix(node): log malformed receive-pack as warn, not error (#165)

git_receive_pack's error classifier routed every non-Timeout error,
including client-caused BadRequest (400), through tracing::error!. Mirror
the git_upload_pack arm so a malformed push logs at warn level instead of
as a server error.

* test(node,git): make smart_http timeout tests pass under the parallel runner (#165)

Addresses jatmn's review of 41eca0b.

[P2] The #62 timeout tests failed under `cargo test --workspace`:
- run_git_service_tears_down_group_when_future_dropped re-polled a completed
  future ("async fn resumed after completion"): the advance-until-pids loop
  ignored the timeout's Ok(_) (future-finished) case and polled the resolved
  future again. It now breaks on early completion.
- Under fork-storm load a freshly-written fake `git` transiently fails to exec
  (ETXTBSY) or is timed out before recording its pids. Added a
  fake_git_run_with_pids retry helper (used by the four pid-reading tests) plus
  a matching inline retry on the drop test, with growing backoff for the
  correlated bursts and a 12-attempt cap that still fails loudly on a genuine
  never-spawns regression. Reverted two tests from a speculative 2000ms bound
  back to 300ms/1000ms now that the retry covers the timeout-vs-pidfile race.

[P3] Documented GITLAWB_GIT_SERVICE_TIMEOUT_SECS in .env.example and the README
config table (default 600; bounds upload-pack/receive-pack, not info/refs or the
withheld-blob path).

The tests stay load-bearing: breaking process_group(0), the post-reap disarm,
the timeout-arm reap, or the internal timeout each still turns the corresponding
test red (mutation-verified).

---------

Co-authored-by: t <t@t>
2026-07-10 08:54:59 +08:00
GravireiandGravirei 6bafaa6dc5 fix(bounties): add tests for claim_bounty repo-read gate (#160) (#169)
* fix(node): gate GET /ipfs/{cid} on reachable allowed-set, not deny-set (#126)

The IPFS visibility gate used withheld_blob_oids (a deny-set enumerating
only reachable blobs), so a dangling/unreachable blob was absent from the
set and served in cleartext to anonymous callers. Flip to an allowed-set
(allowed_blob_set_for_caller) that enumerates reachable blobs the caller
may read: a dangling blob has no path, is never in the set, and 404s.

* perf(ipfs): check object existence before allowed-blob walk

Move store::read_object before the allowed_blob_set_for_caller
spawn_blocking call so random-CID spray against repos with
path-scoped rules cannot trigger full-history git walks on
repos that don't carry the object.

* refactor(ipfs): improve formatting and readability in get_by_cid function

✓ P3 blocker fixed: cargo fmt applied — the format gate will pass.

  ✓ P3 cleanup resolved: withheld_blob_oids is still used by replication code in repos.rs, so it stays.

  • P2 follow-up: Tree/commit disclosure tracked in #135 — out of scope here.

* refactor(ipfs): streamline object retrieval by separating type and content reading

* docs(ipfs): update get_by_cid comment to reflect split object retrieval

* Run cargo fmt on store.rs

* fix(bounties): add tests for claim_bounty repo-read gate (#160)

* fix(profiles): remove redundant borrow in format arg

---------

Co-authored-by: Gravirei <[email protected]>
2026-07-10 08:39:10 +08:00
GravireiandGravirei 26bc3f6987 fix(visibility): gate repo-scoped read surfaces on visibility (#120) (#157)
* fix(node): gate GET /ipfs/{cid} on reachable allowed-set, not deny-set (#126)

The IPFS visibility gate used withheld_blob_oids (a deny-set enumerating
only reachable blobs), so a dangling/unreachable blob was absent from the
set and served in cleartext to anonymous callers. Flip to an allowed-set
(allowed_blob_set_for_caller) that enumerates reachable blobs the caller
may read: a dangling blob has no path, is never in the set, and 404s.

* perf(ipfs): check object existence before allowed-blob walk

Move store::read_object before the allowed_blob_set_for_caller
spawn_blocking call so random-CID spray against repos with
path-scoped rules cannot trigger full-history git walks on
repos that don't carry the object.

* refactor(ipfs): improve formatting and readability in get_by_cid function

✓ P3 blocker fixed: cargo fmt applied — the format gate will pass.

  ✓ P3 cleanup resolved: withheld_blob_oids is still used by replication code in repos.rs, so it stays.

  • P2 follow-up: Tree/commit disclosure tracked in #135 — out of scope here.

* refactor(ipfs): streamline object retrieval by separating type and content reading

* docs(ipfs): update get_by_cid comment to reflect split object retrieval

* Run cargo fmt on store.rs

* fix(visibility): gate repo-scoped read surfaces on visibility (#120)

* fix(visibility): enhance repo read authorization checks and improve test coverage

* Fix review feedback: clamp limit, memoize authz, identical 404 body, resolved issue id in comments

* cargo update -p crossbeam-epoch: fix RUSTSEC-2026-0204 (invalid pointer dereference)

* Fix review findings: correct authz memoization, page past hidden bounties, gate star/unstar

* Fix findings: sign CLI/MCP reads, keyset pagination, gate claim_bounty

---------

Co-authored-by: Gravirei <[email protected]>
2026-07-08 08:46:19 +08:00
beardthelionandt d00d89ae5b fix(node): gate POST /api/v1/sync/trigger and rate-limit the peer-sync routes (#82) (#161)
* feat(node): add per-client-IP limiters + knobs for the peer-sync routes (#82)

Plumb two dedicated per-IP RateLimiters into config/state so the peer-sync
enqueue routes can be braked independently: sync_trigger_rate_limiter (tight,
GITLAWB_SYNC_TRIGGER_RATE_LIMIT, default 60/hr) and peer_write_rate_limiter
(generous, GITLAWB_PEER_WRITE_RATE_LIMIT, default 600/hr). Both are bounded and
swept by the periodic cleanup loop, keyed via the node-wide push_limiter_trust.
No route is wired to them yet; that lands next.

* fix(node): require a signature on /sync/trigger and rate-limit the peer-sync routes (#82)

/api/v1/sync/trigger was reachable unauthenticated in the default config and
drove an O(peers) outbound fan-out + per-repo sync_queue enqueue — an anonymous
amplification/DoS. The same enqueue_sink is reachable via /sync/notify, which
also accepts unsigned known-peer requests and had no rate limit.

- Move /sync/trigger into its own always-signed route group (add_auth_layers),
  independent of require_signed_peer_writes, and give it a tight per-IP brake.
- Add a separate, generous per-IP brake to the peer-write group
  (/peers/announce, /sync/notify); notify's signature behavior is unchanged
  (rolling-upgrade compat), it only gains the brake. Separate buckets so an
  unsigned notify flood can't drain the signed trigger caller's quota.
- Per-IP (not per-DID): a did:key farm self-registers, so a signature/per-DID
  gate does not cap cost (INV-10); the brake keys on the client IP before auth.

Adds an adversarial test matrix (unsigned->401 both modes, IP flood->429,
DID-agnostic brake, forged X-Forwarded-For can't bypass, notify flood->429,
distinct-IP not throttled, cross-route non-exhaustion, 0-disables-but-sig-holds,
announce still accepts unsigned).

* fix(gl): surface node denial on sync trigger and sanitize the error body (#82)

Now that the node gates /api/v1/sync/trigger behind a signature and rate limit,
'gl sync trigger' must not render a 401/429 as a fabricated '✓ sync triggered /
0 peers' success. Check resp.status() before parsing; on a non-2xx, extract the
node's message and bail. Strip C0/C1 control bytes from the node-supplied string
before printing (INV-6) — a hostile node could embed ANSI/OSC escapes in the
body. Adds mockito tests: 401/429 surface as errors (RED without the status
check), control bytes are stripped, and the 200 path is unchanged.

* fix(review): bound gl error-body read, strip bidi controls, genericize 429 message (#82)

Code-review follow-ups on the #82 change:
- gl sync trigger: read the node error body with a capped reader
  (read_body_capped) instead of resp.text(), so a hostile/broken node can't
  force an unbounded allocation to surface a denial (INV-6, read half).
- sanitize_node_msg: also strip Unicode bidi/format controls (U+202A-202E,
  U+2066-2069, LRM/RLM/ALM) — is_control() only covers the Cc category, but a
  right-to-left override can reorder the displayed error line.
- too_many_requests(): the shared per-IP 429 middleware now serves the peer-sync
  routes too, so drop the push-specific wording ('rate limit exceeded'); the
  push-specific 429 in the receive-pack handler is unchanged.
Adds tests for the bidi/length sanitize and the oversized-body path.

* test(#82): close reasoned-not-run gaps with load-bearing tests

Executes the branches that were previously only reasoned:
- read_body_capped byte-cap test (RED against resp.text() — 2MB body -> ≤cap):
  vets the bounded-read fix, not just the display cap.
- sync-route 429 body asserts 'rate limit exceeded' and NOT 'push' (RED before
  the message was genericized); also finishes genericizing the warn log, which
  the response-body change had missed.
- real RFC 9421 signature through the full router reaches the handler (200) in
  both config modes — the positive path through the actual gate, not a
  direct-mount with an injected DID.
- /peers/announce flood -> 429 (the peer_write brake covers announce, not just
  notify).
- trigger_counts extraction unit test (happy + malformed -> 0,0), closing the
  count-formatting gap in the 200 path.
No production behavior change beyond the warn-log wording.

* fix(#82): require an identity for sync trigger and fix the rollout docs

Address review feedback on the always-signed /sync/trigger route.

- gl sync trigger now requires a local identity via load_keypair_from_dir and
  fails locally with "identity not found" before issuing a request, instead of
  .ok() silently sending an unsigned request that can only 401 remotely. Add a
  test proving it fails before dialing the node.
- Split /sync/trigger out of the staged unsigned peer-write rollout in the
  README and the OSS-readiness audit; only /peers/announce and /sync/notify keep
  the rolling-upgrade unsigned behavior.
- Reorder the two peer-sync rate-limit vars in .env.example for dotenv-linter.
- Bump crossbeam-epoch to 0.9.20 (RUSTSEC-2026-0204), matching main, so cargo
  audit passes.

---------

Co-authored-by: t <t@t>
2026-07-08 08:44:55 +08:00
beardthelionandt 67ad2b876c fix(deps): bump crossbeam-epoch to 0.9.20 for RUSTSEC-2026-0204 (#162) (#163)
RUSTSEC-2026-0204 (invalid pointer dereference in crossbeam-epoch's
fmt::Pointer impl for Atomic/Shared) hard-fails cargo audit on every branch,
since the advisory's addition to the RustSec db. crossbeam-epoch is a fully
transitive dependency (moka -> hickory-resolver -> libp2p-dns), and the
advisory's prescribed fix is >= 0.9.20; moka 0.12.15 accepts >=0.9.18,<0.10.0
so the bump resolves with no other graph changes.

Lockfile-only, single crate. Verified: cargo audit exits 0 and the workspace
builds clean with 0.9.20.

Closes #162.

Co-authored-by: t <t@t>
2026-07-07 22:46:44 +08:00
Kevin CodexandClaude Fable 5 6cff5286b4 fix(release): build aarch64-musl natively on arm64 runners, replace retired macos-13 (#155)
The aarch64-unknown-linux-musl leg cross-compiled a musl target with the
glibc toolchain (CC=aarch64-linux-gnu-gcc), which aws-lc-sys cannot build
with — the Build step failed on every release and no aarch64-linux asset
has ever shipped. Build it natively on ubuntu-24.04-arm instead (free for
public repos), using the same stock musl-tools recipe as the x86_64 leg,
and drop the smoke-test skip since the binaries now run on their build
runner.

The x86_64-apple-darwin leg targeted macos-13, which GitHub has retired;
the job queues forever and holds the release concurrency group open. Move
it to macos-15-intel.

Both failures also skipped the downstream npm-publish, homebrew-bump, and
web-sync jobs, so v0.4.0 and v0.5.0 were never published there.

Co-authored-by: Claude Fable 5 <[email protected]>
2026-07-06 10:39:12 +08:00
github-actions[bot] 5f63f28215 chore(main): release 0.5.0 (#142)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2026-07-05 20:07:23 +08:00
Kevin CodexandClaude Opus 4.8 06388ec26a feat(gl): sanctioned iCaptcha client flow + secure git lifecycle (#138)
* feat(gl): sanctioned iCaptcha client flow + secure git lifecycle

Builds the client side of the iCaptcha gate so a legitimate agent passes the
challenge transparently, with no manual headers and no GITLAWB_ICAPTCHA_PROOF
env var. Server enforcement is unchanged.

New crate `icaptcha-client` (blocking, shared by gl + the git helper):
- deterministic solvers for the computational challenge types (arithmetic,
  algebra, sequence) matching the iCaptcha generators; anagram/logic/LLM fall
  back to a solver hook / interactive prompt.
- obtain_proof(): POST /v1/challenge (requesterId = caller DID), solve, POST
  /v1/answer, handle escalation, return the signed proof. Optional bearer auth
  via GITLAWB_ICAPTCHA_API_KEY.

gl (crates/gl):
- http.rs: unify post/put/delete through one signed sender that, on a 403
  iCaptcha challenge (detected via x-icaptcha-url / x-icaptcha-level headers),
  solves it and retries the same signed request with the x-icaptcha-proof
  header (bounded retry absorbs the ~5 min proof TTL). Removes the env hack.
  Emits an actionable hint on 401 not_an_agent (old-CLI / unregistered).
- repo.rs: `repo info` checks status and returns a real 404 instead of a stub
  card with `?` fields; `repo info`/`clone` resolve the owner from the local
  identity or an explicit owner/name — never the node's own DID.
- doctor.rs: warn on gl-vs-node version drift; add an iCaptcha reachability
  check.

git-remote-gitlawb: same shared client — push stays signed (RFC 9421); on a 403
iCaptcha (safety net; push is signed-only, not gated) it solves and retries.
Clone/fetch unsigned; missing repo keeps its clear 404.

gitlawb-node: the 403 icaptcha_proof_required response now advertises the
service url + required level as JSON fields and x-icaptcha-url / x-icaptcha-level
headers (mirroring the human_detected pattern) so clients discover them instead
of scraping the message. The sub == authenticated-DID and expiry/level checks
are unchanged — enforcement is not weakened.

README: documents the full lifecycle (identity -> register -> repo create with
auto-iCaptcha -> push -> clone), the requesterId == DID rule, and proof TTL.

Tested: solver unit tests; full workspace green against Postgres (node still
rejects missing/expired/wrong-subject proofs).

* fix(icaptcha-client,gl): address PR #138 review feedback

Rebased onto main (resolving the git-remote-gitlawb/main.rs conflict from #119)
and addressed the review findings:

- [beardthelion P2 / jatmn P2 / CodeRabbit] Removed the dead iCaptcha retry loop
  from the push path. git_receive_pack never calls verify_request (only
  create/fork/register are iCaptcha-gated), so a push 403 never carries
  x-icaptcha-* headers and the loop could never solve. Taking main's push path
  (via #119) deletes it, which also resolves the pack-clone (retries no longer
  copy the whole pack) and the interactive-prompt-over-git-stream threads, and
  restores the sanitized info/refs error path. git-remote-gitlawb no longer
  depends on icaptcha-client.

- [beardthelion P1] Sanitize and length-bound iCaptcha error bodies before they
  reach the terminal. The origin is only as trusted as the node that advertised
  it, so its non-2xx bodies (and Failed{reason}) are attacker-influenceable; they
  went raw into bail! (CWE-150, the #137 class). Now bounded-read + C0/C1-stripped
  + capped, same shape as #137.

- [beardthelion P2] Do not trust or hand credentials to a node-chosen iCaptcha
  URL. x-icaptcha-url was used as the base with no scheme/host check and the API
  key was sent to it on the first hop (SSRF + key exfiltration). resolve_solver_url
  now honors an advertised URL only when https + host-allowlisted (public default
  or the operator's GITLAWB_ICAPTCHA_URL), else falls back to the trusted origin;
  the API key is attached only to the operator's own configured origin.

- [jatmn P2 / CodeRabbit] gl repo clone with a bare name now derives the short
  owner key via resolve_owner_did, matching the other commands' gitlawb://z.../name
  URL shape instead of the colon-bearing full DID.

Skipped (with reason): jatmn P3 (doctor GITLAWB_ICAPTCHA_URL) is resolved by the
P2 change — that env var is now read by the write path, so the doctor's probe is
accurate. CodeRabbit User-Agent-version nit lived in the push code reverted to
main. README push wording kept ("signed-only, no per-push challenge") — now
accurate since the retry path is gone; the API-key bullet was updated for the new
trust model.

Tests: added sanitize + resolve_solver_url unit tests; full workspace green.

Co-Authored-By: Claude Opus 4.8 <[email protected]>

---------

Co-authored-by: Claude Opus 4.8 <[email protected]>
2026-07-05 19:47:06 +08:00
2df6ff9d30 fix(node): close two spam-vector root causes (trust upsert + ungated push) (#152)
* fix(node): close two spam-vector root causes (trust upsert + ungated push)

The June 2026 spam wave kept re-materializing after cleanup because two
paths let an unauthenticated-by-captcha actor keep operating:

1. `update_trust_score` was an upsert. Any authenticated push/issue/PR
   from a DID with no agent row silently INSERTed one with a fresh
   `registered_at`, bypassing the iCaptcha gate on /api/register — so
   deregistered spam DIDs came back the moment they pushed. Make it an
   UPDATE-only no-op for unregistered DIDs; registration is the sole way
   into the agents table. Regression test added.

2. `git-receive-pack` had no rate limit. The per-DID limiter on the
   creation routes can't brake a push flood from a DID farm (one
   throwaway identity per repo), so the node absorbed several pushes/sec
   per IP, each a Tigris round-trip + peer-notify fan-out. Add a per-
   client-IP limiter (Fly-Client-IP / X-Forwarded-For) on the push path,
   default 600/h, `GITLAWB_PUSH_RATE_LIMIT` override, 0 disables. Fails
   open without a proxy header so self-hosted nodes aren't broken.

Co-Authored-By: Claude Opus 4.8 <[email protected]>

* fix(node): address PR review — advertisement throttle, trusted-proxy IP key, bounded limiter

Resolves the review findings on the push rate limiter:

P1 — advertisement path bypassed the limiter. A push hits
GET info/refs?service=git-receive-pack first, which calls acquire_fresh()
(a Tigris download); it sat in git_read_routes under only optional_signature,
so the flood was reachable via the cheap GET without ever sending the throttled
POST. Now git_info_refs applies the same per-IP brake before the fresh acquire.

P1 — client-supplied forwarded headers were trusted. The key came from
Fly-Client-IP / the first X-Forwarded-For hop, both client-settable, so a
flooder could rotate the header and never fill a bucket. Replaced with a
TrustedProxy policy (GITLAWB_TRUSTED_PROXY = fly | x-forwarded-for | unset):
trust only the operator's edge header, fall back to the real socket peer
(ConnectInfo) otherwise. Fly config trusts Fly-Client-IP; the AWS/Caddy
compose trusts the rightmost X-Forwarded-For hop.

P2 — unbounded limiter key set / insert-before-reject. Added a max_keys cap
(reject-before-insert, inline eviction when full) so a varied key cannot grow
the map, and a rejected request never allocates. Applied to both the per-IP
and per-DID limiters.

P2 — malformed forwarded header disabled the brake. Empty leading XFF hop
returned None and skipped the limiter; empty Fly-Client-IP returned Some("").
client_key now takes the rightmost XFF hop and always falls back to the peer,
so a malformed header can never turn the limiter off.

P2 — missing tests. Added: middleware 429 path, per-peer isolation, key-cap
eviction/rejection, trusted-proxy header resolution, and an integration test
asserting the receive-pack advertisement is throttled before the acquire.

Also serve with into_make_service_with_connect_info so the peer address is
available. New env: GITLAWB_PUSH_RATE_LIMIT, GITLAWB_TRUSTED_PROXY (documented
in .env.example).

Co-Authored-By: Claude Opus 4.8 <[email protected]>

* Update infra/aws/compose.yaml.tftpl

Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>

* fix(node): amortize the rate limiter's capacity eviction scan

`RateLimiter::check` ran a full O(max_keys) `retain` sweep on every new-key
miss once the map was at capacity — which is exactly the distinct-key-flood
state the cap defends against, so the limiter serialized all traffic behind a
200k-entry scan per request under the mutex.

Gate the inline sweep to run at most once per `sweep_interval` (min of the
window and 1s), tracked by `last_sweep`. Between sweeps a new key is rejected
without scanning; the background `cleanup()` loop still reclaims independently.
The fast path for existing keys is unchanged, the key cap invariant holds
(map never exceeds max_keys, rejected keys never allocate), and post-flood
self-healing stays bounded to one interval. Added a test asserting a burst of
capacity misses does not trigger repeated eviction scans.

Co-Authored-By: Claude Opus 4.8 <[email protected]>

---------

Co-authored-by: Claude Opus 4.8 <[email protected]>
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
2026-07-05 17:31:21 +08:00
Mizuki Hayashiandmizuki0x 2620e973e3 fix(repos): log the cause when repo create fails (#103)
create_repo was the one git handler that mapped its error without logging
it (info_refs, receive_pack, acquire all log with repo context). So a
failed init_bare returned an opaque 500 "initializing bare repo" with no
server-side trail to the real cause.

Log the full anyhow chain ({:#}, so the leaf git/filesystem error is kept)
with owner+repo context, matching the other handlers. Response unchanged.

Co-authored-by: mizuki0x <[email protected]>
2026-07-05 17:16:11 +08:00
beardthelionandt e3a20317d5 ci: pin cargo-audit on the blocking gate, float the weekly canary (#151)
The PR-blocking audit job ran `cargo install --locked cargo-audit` with no
version, so the scanner floated to the latest release on every run; a
regressed release could red the PR gate repo-wide with no code change. Pin it
to 0.22.2, the version the unpinned install currently resolves to and the one
the audit-gate learnings were validated against. `cargo install --version X`
is an exact pin (not a caret), so this freezes the binary while the advisory
DB is still fetched fresh at scan time, which is what the gate actually keys on.

Keep a forcing function so the pin cannot silently rot: leave the weekly
Scheduled Audit deliberately unpinned as a drift canary. That job is cron-only
(never pull_request/push-triggered), so it never gates a merge; floating there
surfaces a regressed latest as a red weekly run or a changed report, which is
the drift signal, without ever blocking a PR.

Also assert the installed version after the pinned install so an accidental
future unpin fails loud instead of silently running a different scanner,
include the version in the rust-cache key so cache hits stay deterministic,
and document the yank/bump path in the step comment.

Co-authored-by: t <t@t>
2026-07-05 17:10:45 +08:00
beardthelionandt c296e23678 test(node,git): wire run_git_service teardown end-to-end (#62) (#150)
The #61 tests build KillGroupOnDrop by hand and never call
run_git_service, so dropping process_group(0), the guard-arming, or the
post-reap disarm left them green. Add two tests that drive the real
run_git_service through an injected fake git (a minimal git_bin seam;
production passes "git") and assert the production spawn path actually
wires the teardown:

- future-drop mid-flight SIGTERMs the whole group, reaping git and its
  grandchild (RED without process_group(0) or the guard-arming)
- a completed request disarms the guard, leaving a live grandchild
  untouched (RED without the post-reap disarm)

Each assertion was shown RED against its mutation and green when
restored. Parallel-safe: the fake is reached via the seam, not a PATH
mutation.

Co-authored-by: t <t@t>
2026-07-05 17:10:25 +08:00
beardthelionandt 20d6848846 fix(gl): sign the CLI's /ipfs/pins reads under the #134 auth gate (#146)
* fix(gl,ipfs): sign gl ipfs list so it works under the #134 pins auth gate

/api/v1/ipfs/pins now 401s anonymous callers (#134). Load the caller's
identity (--dir, default keystore) and use NodeClient::get_signed; when no
identity exists, propagate load_keypair_from_dir's existing 'gl identity new'
error instead of a raw 401. Tests: signed-headers present, empty, and
no-identity-issues-no-request (mockito).

* fix(gl,node): sign the gl node status pins panel, degrade gracefully

The pins panel signs its /api/v1/ipfs/pins read (#134 gates it behind auth)
via an injectable fetch_pins helper with four states: pins, empty,
unavailable (signed read rejected/errored), and needs-identity (no keypair,
no request issued). cmd_status loads the identity gracefully so a missing
keystore never aborts status; peers/repos/p2p/events panels stay anonymous.
Tests drive fetch_pins directly against a mock node.

* fix(gl): address review — surface pins HTTP errors, correct empty-panel label

- gl ipfs list: check the /ipfs/pins response status before parsing, so a
  rejected signed read (e.g. 401) surfaces as an error instead of silently
  printing 'No IPFS pins recorded'. Matches the sibling gl ipfs get.
- gl node status: PinsPanel::Empty now renders 'Pinned CIDs: 0' (a reachable
  node with zero pins), reserving 'unavailable' for the failure state; the
  prior label reused 'IPFS not configured' and misreported the zero-pins case.
- Carry the resolved count in PinsPanel::Pins, removing the duplicated
  count-fallback closure between fetch_pins and the render arm.

* test(gl,node): cover fetch_pins error branches and pins-panel rendering

Extract the pins-panel render into a testable pins_status_line helper and
assert all four states' output (closes the untested cmd_status render path,
including the empty->'Pinned CIDs: 0' label). Add fetch_pins tests for the
two error branches that were code-traced but unexecuted: a malformed 200
body and a transport error both degrade to Unavailable without panicking.

* style(gl): rustfmt the ipfs list pins-response parse

* fix(gl,node): honor --dir identity selector in `gl node status` (#146)

`gl node status` always loaded the default keystore via
`load_keypair_from_dir(None)`, so a user who created or selected an identity
with `--dir` saw the status pins panel render "sign in to view" even though
`gl ipfs list --dir` authenticated fine. Add a `--dir` option to
`NodeCmd::Status` and thread it into the signed pins fetch, so both CLI pins
callers select the same identity directory.

An explicit `--dir` that can't be loaded (missing, corrupt, or unreadable) no
longer masquerades as "no identity, run `gl identity new`": the pins identity
is resolved up front into Keyed / Anonymous / DirUnusable, and the unusable
explicit-dir case renders a distinct "no usable identity in <dir>" panel line
that names the path, while a missing default keystore (no `--dir`) still
degrades quietly. A pins failure never aborts the dashboard.

Tests cover both flag branches (Some/None) and the resolver both ways: a good
`--dir` loads Keyed, an unusable one reports DirUnusable/IdentityError without
issuing a request. The new symbols were absent before, so they compile-RED
first.

* test(gl,node): pin the pins-identity resolver to the --dir key

Follow-up test hardening on the #146 --dir wiring after review:

- resolve_pins_auth_loads_keyed_identity_from_explicit_dir now destructures
  Keyed and asserts loaded.did() equals the --dir key's DID. A bare Keyed(_)
  match passed even when the load fell through to the default keystore; the
  DID assertion catches that regression (confirmed red under a simulated
  fallthrough, green on the real code).
- add resolve_pins_auth_none_is_never_dir_unusable: an absent --dir must
  degrade to Anonymous or Keyed, never DirUnusable. It asserts only the
  None to DirUnusable mapping, so it is stable whether or not the test
  runner has a default ~/.gitlawb identity.

Test module only; no production behavior change.

---------

Co-authored-by: t <t@t>
2026-07-05 17:10:02 +08:00
beardthelionandt 4891db3889 fix(node): gate the ref-updates feeds on read visibility (#112, #114) (#143)
* feat(node,visibility): add pure fail-closed feed-visibility filter (#112,#114)

Add ref_update_row_visible: given the deduped local repo set + their
visibility rules, decide whether a received_ref_updates row (keyed by its
peer-supplied slug) is visible to the caller. Matches slug->repo by name
plus prefix-tolerant owner key, and drops any row matching a local repo
the caller cannot read at root; rows matching no local repo (remote/
gossip-only) and malformed slugs pass through. Fail-closed by
construction (no default-keep). Pure/I-O-free so visibility.rs stays
exhaustively unit-tested.

Shared gate for the GraphQL (#112) and REST (#114) ref-updates feeds;
call sites land in the following commits. Includes a DB-layer test
pinning that a private canonical row beats a public mirror row in
list_all_repos_deduped (else the feed gate would over-serve).

* fix(node,graphql): gate refUpdates on read visibility (#112)

The GraphQL ref_updates resolver mapped every received_ref_updates row
straight out with no visibility gate, so an anonymous caller could read
a private repo's ref metadata (refName, SHAs, pusherDid, nodeDid) by
name. Thread the caller DID and retain rows through the shared
ref_update_row_visible filter (deduped local set + rules, same '/'
decision the repos resolver uses); applies to both the repo:Some and
repo:None branches. Remote/gossip-only rows still pass.

Removes the now-live filter's dead_code allow. Tested via the GraphQL
schema: anon leak reproduced before the gate (RED) and closed after
(GREEN); owner still sees their private row; full-DID and truncated-key
slug aliases fail closed; remote rows kept.

* fix(node,events): gate global ref-updates feed on read visibility (#114)

GET /api/v1/events/ref-updates returned every received_ref_updates row
across all repos, including private ones, to any anonymous caller (the
REST analog of #112). Thread the caller DID via optional_signature and
retain rows through the shared ref_update_row_visible filter, deriving
count from the filtered set. Keeps the crate::error::Result return type;
the deduped/rules loads fail closed (500) on a DB error rather than
serving unfiltered. Remote/gossip-only rows still pass.

Tested via the crate's HTTP-API harness: anon leak reproduced before the
gate (RED) and closed after (GREEN); owner still sees their private row;
mixed feed yields only public rows with a filtered count; full-DID and
truncated-key slug aliases fail closed; remote rows kept.

* fix(node,graphql): gate the ref-update subscription broadcast on announce (#112,#114)

Code review surfaced a third reader of the same ref metadata the query
and REST feeds gate: the GraphQL subscription. /graphql/ws is mounted
after the optional_signature layer (no caller to gate against), and the
push handler broadcast every ref update to it unconditionally — outside
the if-announce block that already gates the sibling gossip and Arweave
sends. So an anonymous websocket subscriber received live ref metadata
(repo, refs, SHAs, pusher/node DID) for private repos as they were
pushed — the subscription analog of the #112/#114 leak.

Move the ref_update_tx broadcast inside the existing if-announce guard so
only publicly-readable pushes reach the unauthenticated subscription,
matching gossip and Arweave. Pin the gate decision with a
replication_withheld_set test (announce false for private, true for
public).

Also harden the feed filter's owner-key normalization: normalize the
slug owner to the last ':'-segment (same rule as record_key) instead of
stripping only did🔑, so a multi-segment DID (did:web) and its
full-DID slug match and fail closed rather than over-serving. Regression
test added (RED under the old normalization).

* docs(node,graphql): pin the subscription write-side gate invariant; add did:web keep test

Code review flagged that the unauthenticated /graphql/ws ref_updates
subscription now depends on a single-point invariant: safety rests
entirely on every sender to ref_update_tx being announce-gated, since the
resolver has no caller to gate against. Document that on the resolver so
a future sender can't silently reopen the leak.

Add the symmetric keep-side test for the did:web owner-key normalization
(public multi-segment-DID repo row is kept for anon), so a regression
that over-drops legitimate did:web rows is caught alongside the existing
drop-side test.

* docs(node,visibility): correct ref-update slug normalization rationale (#112,#114)

The comment claimed parity with did_matches/DEDUP_CTE, but the last-segment
normalization deliberately diverges: the emitter (publish_ref_update) broadcasts
the short, method-stripped slug, so a repo's own rows arrive keyed by the
trailing segment and the gate must match on it to stay fail-closed. Reusing the
did:key-aware keep-whole rule would fail open on a did:web repo's short slug.

Add feed_private_didweb_short_slug_dropped_for_anon to pin that load-bearing
canonical case; relabel the full-DID test as the non-canonical hand-forged form.

* fix(node,events): gate ref-update feeds before the limit via paging (#114)

The REST global feed and the GraphQL ref_updates resolver loaded `limit` rows
and dropped private ones in memory, so an anonymous request could get fewer than
`limit` events (or zero) when the newest rows named private local repos, even
though older public ref updates existed.

Add a shared collect_visible_ref_updates helper that applies the fail-closed
gate before the limit: it over-fetches in bounded pages (128 rows, capped at
max(limit, 2048) scanned) until `limit` visible rows are collected or the table
is exhausted. Both feeds route through it so the one gate cannot drift. Back it
with list_ref_updates_page (stable timestamp,id ordering for offset paging) and
drop the now-unused list_ref_updates / list_ref_updates_filtered. The page size
is a parameter on an inner fn so a regression can exercise multi-page paging.

Regressions: newest-private, older-public, small limit still returns the latest
visible rows on both surfaces (RED before, GREEN after); and a page=2 test pins
offset paging across page boundaries (no skip or duplicate).

* docs(node,visibility): correct slug-builder attribution in ref-update gate comment

The comment credited publish_ref_update with building the wire slug, but that
p2p method only broadcasts a pre-built RefUpdateEvent; git_receive_pack assembles
the {last-segment}/{name} slug and hands it over. Fix the attribution.

* fix(node,events): bound feed limit and withhold quarantined mirrors (#114)

Resolve two CodeRabbit findings on the ref-update feeds:

- Clamp the effective limit inside the shared collector (max 200). REST already
  capped, but the GraphQL resolver passed its caller-provided limit uncapped,
  letting a large request return unbounded rows and scan unbounded DB rows. Both
  surfaces are now bounded identically.

- Fold quarantined mirrors into the gate's match universe as unreadable.
  list_all_repos_deduped excludes quarantined rows, so a ref-update naming a
  quarantined local mirror matched nothing, was misclassified as remote, and was
  served to anon, bypassing the withheld-from-every-listing-surface contract. Add
  list_quarantined_repos and drop matching rows fail-closed.

Regressions: oversized-limit clamp (201 rows -> 200) and quarantined-mirror
withheld-from-anon, both RED before, GREEN after.

* fix(node,events): gate the repo-scoped events feed on read visibility (#112, #114)

GET /api/v1/repos/{owner}/{repo}/events (list_repo_events) served a repo's ref
certificates and received gossip ref-updates with no caller identity and no
visibility check, so an anonymous caller who guessed a private repo's owner/name
read its ref metadata. It was the last repo-scoped reader in the #112/#114
ref-metadata leak class.

Gate it in two layers, using the caller DID the optional_signature layer already
supplies:
- The cert half is gated by a repo-root authorize_repo_read (keyed by the unique
  repo record id): a denied, quarantined, or not-hosted repo returns an opaque 404
  (RepoNotFound), never a 403.
- The gossip half is filtered per row through the shared collect_visible_ref_updates.
  received_ref_updates rows are keyed by the lossy, non-unique wire slug
  {last-segment}/{name}, so two owners (did:web:a:alice, did:web:b:alice) collide on
  `alice/name` and a plain slug query would serve a colliding private repo's rows to
  anyone allowed to read this one; the per-row gate drops them fail-closed.

Both halves now propagate DB errors as 500 instead of swallowing them into an empty
200. Removes the now-unused list_repo_ref_updates.

Deliberate behavior change: a repo this node does not host returns 404 (it holds no
visibility record for it and fails closed; remote gossip is read via the global
/api/v1/events/ref-updates feed). Drops list_repo_events from the egress-guard
KNOWN_UNGATED allowlist; the source-scrape staleness assert enforces the removal.

Tests cover anon-private, owner-private (both datasets), anon/authed-public,
quarantined, released-from-quarantine, non-owner, not-hosted, did:web (owner and
anon), the slug collision, oversized-limit clamping, and DB-error-fails-closed on
both the gate and the cert fetch.

* fix(node,events): clamp negative ref-update feed limits to zero (#114)

A caller-supplied `?limit=-1` cleared the `.min(MAX_VISIBLE_REF_UPDATES)`
upper cap unchanged and reached `all_events.truncate(limit as usize)` in
list_repo_events, where `-1 as usize` is usize::MAX, so the truncate was a
no-op and the handler returned the full unbounded local ref-cert set.

Floor both feed handlers' limit parse with `.clamp(0, MAX_VISIBLE_REF_UPDATES)`.
Only the repo handler's clamp is load-bearing: it guards the local `truncate`,
which does not pass through the shared collector. The global feed's clamp is a
consistency measure (its limit is re-clamped in the collector, whose want==0
short-circuits before any scan), kept in lockstep so the two per-handler caps
the module comment calls out don't drift.

Tests, both load-bearing by execution:
- repo_events_negative_limit_clamped: seeds local certs (the vulnerable half),
  RED (3) before the clamp and green (0) after.
- feed_negative_limit_returns_empty: seeds 5 visible rows; green (0) with the
  collector clamp, 5 without it, proving the collector is the global feed's guard.

* fix(node,events): drop quarantined repos before the feed visibility gate (#114)

The ref-update feeds folded quarantined mirrors into the deduped repo set with is_public=false and relied on the row visibility gate to withhold them. That held for anonymous callers but not for one matching the mirror's owner_did: visibility_check short-circuits to Allow for the owner before is_public is read, so an owner-matched quarantined row was served on both the REST global feed and the GraphQL ref_updates resolver.

Drop any row that names a quarantined repo in the shared collector, before the visibility gate runs, so the drop bypasses the owner short-circuit. Quarantine is a status decided at admission and checked separately from the mirror's visibility fields, so it must deny every caller including the owner.

Extract the slug match into one ref_update_row_names_repo predicate reused by both the gate and the quarantine drop, so the two cannot disagree about which rows a repo owns.

Tests cover the owner case at the collector, REST handler, and GraphQL resolver in both bare-key and full-did:key owner forms, plus a must-not case proving the drop withholds only quarantined rows while still serving unrelated visible ones.

* test(node,events): align did:web feed tests with the #141 canonical slug

The rebase onto main adopts #141's normalize_owner_key on the repo-scoped
ref-update read path, which keeps a non-did:key DID intact instead of taking
the last URL segment. Align the two did:web fixtures that still assumed the
old {last-segment}/{name} wire slug:

- repo_events_did_web_owner_reads_own_gossip stored "alice/widget", but the
  emit side (api/repos) publishes "did:web:example.com:alice/widget", so the
  read query no longer matched and the owner saw only the cert half. Store the
  emit-side slug so the gossip KEEP branch is exercised again.

- repo_events_gossip_slug_collision_withheld_from_anon is removed. Its premise
  was that two did:web owners collide on the last-segment slug; #141 gives them
  distinct full-DID slugs, so the repo-scoped query fetches zero rows and the
  per-row drop is never reached (verified by probe: fetched=0). Global-feed
  collision coverage stays in feed_full_did_slug_dropped_for_anon and
  feed_truncated_key_slug_dropped_for_anon.

* docs(node): correct ref-update slug comments for the #141 canonical form

The rebase onto #141 changed the emit-side wire slug from the last-segment
form to normalize_owner_key (bare short key for a single-segment did:key,
full DID otherwise). Four comments still described the old behavior:

- visibility.rs ref_update_row_names_repo: the emitter builds the slug via
  normalize_owner_key, not {last-segment}/{name}. The next_back reduction on
  the matcher is what tolerates the untrusted/crafted short form; it does not
  mirror an already-stripped emit form.
- events.rs list_repo_events gossip half: the residual collision the per-row
  filter still guards is a did:key canonical owner vs its bare short-key
  mirror (both normalize to the same slug), not two did:web owners (#141
  gives those distinct slugs).
- the two did:web feed-visibility tests had their canonical/crafted framing
  backwards: post-#141 the full-DID slug is the emit form and the short slug
  is the crafted one.

Comment-only; no behavior change.

* test(node,events): add load-bearing two-repo owner-key collision deny

A public bare-key mirror (z6MkX) and a private did:key canonical repo
(did:key:z6MkX) normalize to the same owner key post-#141, so their gossip
rows share the z6MkX/... slug space on the untrusted wire. Assert an anon
global-feed read sees the public mirror's row but NOT the private canonical
sibling's: the shared gate keys on the full slug (owner + name), so a
readable public repo under an owner key does not unlock that owner's other
private repos' rows. The removed repo-scoped did:web collision test never
covered this pair.

Verified load-bearing: disabling the per-row visibility drop serves
z6MkX/secret to anon (RED); the gate drops it (GREEN).

* fix(node,events): keyset-paginate the ref-update feed collector

collect_visible_ref_updates paged received_ref_updates with LIMIT/OFFSET,
so a concurrent gossip/notify insert (which sorts to the front under
timestamp DESC) shifted every later offset and let one feed response
duplicate or skip a visible row. Switch the collector to a keyset cursor
on (timestamp, id): each page reads rows strictly older than the last row
fetched, so a newer insert lands above the window and cannot shift it.

One collector feeds the REST global, REST per-repo, and GraphQL paths, so
this covers all three; there is no client-facing cursor, so the internal
cursor tracks the last fetched row. Adds tests that OFFSET duplicates a row
under a concurrent insert and that keyset stays stable.

---------

Co-authored-by: t <t@t>
2026-07-05 17:09:15 +08:00
beardthelionandt 6f36fc07b8 fix(node,git-remote): gate receive-pack advertisement, sign client fetch/push (#119)
* fix(node,git-remote): gate receive-pack advertisement, sign client fetch/push

The info/refs ref advertisement was gated only for service=git-upload-pack
(the visibility check sat inside `if service == "git-upload-pack"`), so an
anonymous GET /{owner}/{repo}.git/info/refs?service=git-receive-pack returned a
private repo's branch names and commit tips, bypassing the read gate. Run the
visibility "/" check for both services.

Now that the advertisement is gated for both services, git-remote-gitlawb has to
authenticate its own requests, or a private repo's owner can neither fetch nor
push it. Sign the Phase-1 advertisement GET and the Phase-2 pack POST for both
services over path_and_query when an identity keypair is present; public repos
stay anonymous. This also repairs a pre-existing break where an owner could not
fetch their own private repo, because the upload-pack POST was sent unsigned.

Tests: node-side gate (anon denied, owner-signed cleared, both services); a
real-signature seam proving the client's own signature verifies under the node's
verification primitives; real served-content withholding against an on-disk repo
(the owner gets the ref bytes, a denied caller's 404 omits them); and mockito
coverage that the client attaches and omits signatures correctly.

* docs(node,git-remote): correct auth-contract comments on receive-pack gating (#119)

receive-pack POST is signature-required and separately owner-gated, not
visibility_check-gated; push implies read, not read-subset-of-push.

* test(node): assert unsigned receive-pack POST is rejected (#119)

No test covered the signature-required half of the receive-pack auth
contract. Wire the POST route behind require_signature as production does and
assert an unsigned POST returns 401 before reaching the handler; 401 (not the
handler's 404/500) proves require_signature rejected it pre-handler.

* test(node): RAII-clean the served-refs test's temp dirs on panic (#119)

advertisement_serves_real_refs_only_to_authorized_callers created /tmp/<slug>/<name>.git and /tmp/gl-seam-src-<short> and only removed them after the final assertion, leaking those dirs on the runner when an earlier assertion failed. Wrap each known path in a DirGuard whose Drop calls remove_dir_all, bound right after creation, so cleanup runs on success or panic. tempfile::TempDir does not fit here because repo_store::for_testing fixes the path layout.

* test(node): drop git_info_refs from KNOWN_UNGATED now that #119 gates it

The repo-scoped egress guard's allowlist (added in #122, merged to main
after this branch was opened) still listed git_info_refs as ungated. This
branch makes the info/refs visibility gate unconditional for both
git-upload-pack and git-receive-pack, so the handler is now gated and the
staleness assert requires removing it from the allowlist.

* style(node,git-remote): tidy #119 review nits (em dashes, dead binding, test header)

Post-rebase ce-code-review pass on #119. No behavior change:
- replace em dashes in the added comments with commas/colons/parentheses
- drop the dead `let _ = COVERED_COMPONENTS;` binding (and its import); the
  `missing_components().is_empty()` assert already covers the intent
- add a section header above the new git-info-refs test block to match the
  module's sectioning convention

* test(node): recurse nested api modules in the egress-gate guard

The completeness scan in every_repo_scoped_handler_is_gated used a flat
read_dir over src/api, so a future api/<module>/mod.rs could add a
repo-scoped handler the scrape never inspects. Walk the api tree recursively
via a new collect_rs_files helper (covered by its own unit test), skipping
only the top-level mod.rs so nested module files are checked too.

* fix(git-remote): keep public fetches anonymous, sign only on a 404 retry (#119)

git-remote-gitlawb signed every git-upload-pack request whenever an identity
existed, disclosing the caller's DID to any node it fetched a public repo from,
even though public repos need no auth. Fetch now sends the advertisement GET
anonymously and escalates to a single signed retry only when the node denies it
with 404 (how it withholds a private repo from an unauthenticated reader); if the
retry was needed, the upload-pack POST is signed too. Push (git-receive-pack) is
unchanged and signs from the first request, since a push can never be anonymous.

Verified against the node by execution: an unsigned private fetch returns 404,
not 401, and a signed retry clears the read gate for the owner and for a granted
non-owner reader, while a signed non-reader stays 404.

Tests drive handle_connect end to end (recording mock): public fetch stays
anonymous for GET and POST, private fetch retries signed once and signs the POST,
a keyless fetch does not retry, a still-denied signed retry surfaces 404 once, and
push signs from the first request.

* fix(node): scan nested api sources by relative path in the egress guard (#119)

The egress-gate completeness scan skipped already-covered files by basename
(listed.contains(fname)), so a nested api/<module>/repos.rs would be skipped
because the top-level repos.rs is listed, letting a future nested module add an
ungated repo-scoped handler the scrape never inspects. Extract the skip logic into
unlisted_source_files and match on the path relative to api_root, so only the exact
top-level files already covered are skipped and a nested file with a colliding
basename is still scanned. Completes CodeRabbit's outstanding request.

---------

Co-authored-by: t <t@t>
2026-07-05 17:08:44 +08:00
beardthelion 563c456803 fix(node,gossip): route gossip HTTP through the no-redirect client (#93) (#140)
gossip_task built its own reqwest::Client::new(), whose default redirect
policy follows up to 10 redirects, and used it for the bootstrap announce
and the periodic peer /health ping. A peer host (public at announce time,
per is_public_http_url) could answer /health with 302 Location:
http://169.254.169.254/... and the node would follow it into cloud
metadata or internal addresses. Blind SSRF: only request status feeds
mark_peer_ping.

Reuse the shared state.http_client (Policy::none, already on AppState),
matching sync.rs and the announce fan-out hardened in #78. Extract
ping_peer_health so the redirect-not-followed behavior is covered by a
mockito test (302 to an internal target registered expect(0), asserted
not hit).
2026-07-03 08:13:20 +08:00
GravireiandGravirei 6c95592d18 fix(node): prefer canonical repo row over mirror row in get_repo (#124) (#141)
* fix(node): prefer canonical repo row over mirror row in get_repo (#124)

get_repo matched both mirror rows (bare short-DID, is_public=true, no
visibility rules) and canonical rows (full did🔑 DID, correct rules),
returning one nondeterministically. When the mirror row won, the
visibility gate saw is_public=true + empty rules, allowing unauthenticated
read of private/withheld content.

Add ORDER BY to prefer canonical rows (UUID id, no slash) over mirror
rows (slash-form id), matching the ranking logic already used by
DEDUP_CTE and dedupe_canonical_repos.

Test seeds mirror FIRST so an unordered fetch_optional returns the
mirror row — proving the test locks in the fix.

* fix(test): date canonical row after mirror to guard CASE term in get_repo fix (#124)

* fix(node): tighten get_repo owner matching to did:key only, not any DID method (#124 P2)

* fix(node): refactor owner DID normalization to a dedicated function in create_repo

* fix(node): align clone URL and slug construction with tightened did:key matching

- Make normalize_owner_key pub(crate) and reuse it in api/repos, api/events, api/peers, and visibility modules.

- Add read-gate authorize_repo_read and to_response clone_url slug tests.

* fix(node): hoist owner-key CASE into shared const + address #124 review feedback

- Hoist 7 verbatim SQL CASE copies into OWNER_KEY_CASE_SQL const
  (dedup_cte(), get_repo, count_repos_deduped), keeping one literal
  copy in the v7 migration with a cross-reference comment.
- Add 9 unit tests for normalize_owner_key over the full boundary set.
- Add LIMIT 1 to get_repo query for single-row intent.
- Add non_key_owner_bare_short_does_not_match is_owner regression test.
- Update is_owner doc to reflect did:key-only rule.
- Document non-key clone_url disk-path constraint in to_response.

* test(node): add SQL round-trip test for normalize_owner_key vs OWNER_KEY_CASE_SQL

Verifies every boundary value produces identical results from the
Rust function and the SQL CASE expression, preventing silent drift.

---------

Co-authored-by: Gravirei <[email protected]>
2026-07-03 08:12:27 +08:00
GravireiandGravirei 466a550915 fix(node): gate GET /ipfs/{cid} on reachable allowed-set, not deny-set (#126) (#133)
* fix(node): gate GET /ipfs/{cid} on reachable allowed-set, not deny-set (#126)

The IPFS visibility gate used withheld_blob_oids (a deny-set enumerating
only reachable blobs), so a dangling/unreachable blob was absent from the
set and served in cleartext to anonymous callers. Flip to an allowed-set
(allowed_blob_set_for_caller) that enumerates reachable blobs the caller
may read: a dangling blob has no path, is never in the set, and 404s.

* perf(ipfs): check object existence before allowed-blob walk

Move store::read_object before the allowed_blob_set_for_caller
spawn_blocking call so random-CID spray against repos with
path-scoped rules cannot trigger full-history git walks on
repos that don't carry the object.

* refactor(ipfs): improve formatting and readability in get_by_cid function

✓ P3 blocker fixed: cargo fmt applied — the format gate will pass.

  ✓ P3 cleanup resolved: withheld_blob_oids is still used by replication code in repos.rs, so it stays.

  • P2 follow-up: Tree/commit disclosure tracked in #135 — out of scope here.

* refactor(ipfs): streamline object retrieval by separating type and content reading

* docs(ipfs): update get_by_cid comment to reflect split object retrieval

* Run cargo fmt on store.rs

---------

Co-authored-by: Gravirei <[email protected]>
2026-07-02 08:11:38 +08:00
Yang Yuandparanoa233 09a0cb23b9 fix(remote): include HTTP error response body (#137)
* fix(remote): include HTTP error response body

* fix(remote): sanitize HTTP error response bodies

---------

Co-authored-by: paranoa233 <[email protected]>
2026-07-01 21:59:06 +08:00
Kevin Codex cd4680ebeb chore(aws): codify iCaptcha enforce in the EC2 node Terraform (#139)
The AWS EC2 node (manila) was upgraded to v0.4.0 and enabled iCaptcha
enforce by hand-editing its running /opt/gitlawb/compose.yaml — which would
be lost on instance replacement or a user-data re-run. Codify it so rebuilds
stay enforced:

- variables.tf: add icaptcha_mode (default "enforce"), icaptcha_pubkey
  (default the pinned live icaptcha.gitlawb.com Ed25519 key), and
  icaptcha_required_level (default 3).
- main.tf: pass the three vars into the compose templatefile render.
- compose.yaml.tftpl: set ICAPTCHA_MODE / ICAPTCHA_PUBKEY /
  ICAPTCHA_REQUIRED_LEVEL on the node service environment.

Operators can override per-deployment via terraform.tfvars.
2026-07-01 09:55:16 +08:00
Kevin Codex 8d96cab4f7 chore(node): seed Gitlawb nodes in bootstrap-peers; ignore node1 fly config (#132)
- bootstrap-peers.json: add node2, node3, and manila (manila.gitlawb.com)
  to the canonical seed list alongside node.gitlawb.com, and normalize the
  Gitlawb operator label to "Gitlawb". DIDs left null — bootstrap merges
  peers on http_url and the parser treats did as optional.
- .gitignore: ignore infra/fly/node1.fly.toml. node1's production Fly
  config is pulled from Fly (fly config save), not tracked in the repo.
2026-06-30 21:59:56 +08:00
64 changed files with 11757 additions and 754 deletions

No files matched your search

+57
View File
@@ -21,6 +21,23 @@ GITLAWB_REPOS_DIR=/data/repos
# When using the bundled docker-compose, this is wired automatically.
DATABASE_URL=postgresql://gitlawb:changeme@localhost:5432/gitlawb
# ── Database pool & startup resilience ────────────────────────────────────
# Maximum connections in the PostgreSQL pool. A cap, not a floor —
# connections open lazily. Size against the DB server's max_connections,
# remembering admin tooling opens its own pool.
GITLAWB_DB_MAX_CONNECTIONS=20
# Seconds a request waits for a pool connection before failing with 503.
GITLAWB_DB_ACQUIRE_TIMEOUT_SECS=5
# Upper bound on each startup connect+migrate attempt, in seconds. Keep it
# generous enough for a peer instance to finish migrating (the migration
# advisory lock is held across a deploy); on expiry the attempt is retried.
GITLAWB_DB_CONNECT_TIMEOUT_SECS=60
# Startup retry backoff while the database is unavailable: first delay and
# ceiling, in seconds. The node stays up serving 503s (with a truthful
# /ready) instead of crash-looping.
GITLAWB_DB_RETRY_INITIAL_SECS=5
GITLAWB_DB_RETRY_MAX_SECS=60
# ── IPFS pinning (Pinata) ─────────────────────────────────────────────────
# Get a JWT at https://app.pinata.cloud/developers/api-keys
GITLAWB_PINATA_JWT=
@@ -90,6 +107,46 @@ GITLAWB_PUBLIC_READ=true
# Maximum git smart-HTTP pack request size, in bytes.
GITLAWB_MAX_PACK_BYTES=2147483648
# Max seconds a served git upload-pack / receive-pack (clone / push) may run
# before it is aborted with a 504. Bounds a hung git that would otherwise pin a
# worker and, on push, the repo write lock. Does NOT cover the info/refs
# advertisement or the withheld-blob path, which remain unbounded. Default 600.
GITLAWB_GIT_SERVICE_TIMEOUT_SECS=600
# ── Push rate limiting (git-receive-pack flood brake) ─────────────────────
# Max receive-pack requests (info/refs advertisement + push POST) per client
# IP per hour. 0 disables. Default 600.
GITLAWB_PUSH_RATE_LIMIT=600
# ── Creation rate limiting (repo/agent/issue/PR flood brake) ──────────────
# Max creation requests (POST /api/v1/repos, /api/register, fork, issues,
# pulls) per client IP per hour, in addition to the per-DID limit. The per-DID
# limit alone is bypassed by a did:key farm (one throwaway identity per repo),
# which is how spam-repo floods slip past it and the iCaptcha gate; the per-IP
# brake caps a single-source flood. Uses GITLAWB_TRUSTED_PROXY below to resolve
# the client IP. 0 disables. Default 120.
GITLAWB_CREATE_RATE_LIMIT=120
# ── Peer-sync rate limiting (per client IP, uses GITLAWB_TRUSTED_PROXY below) ─
# /api/v1/peers/announce and /api/v1/sync/notify accept unsigned requests from
# known peers and run at higher frequency, so a generous bucket. Separate from
# the trigger bucket so an unsigned notify flood can't drain trigger's quota.
# 0 disables. Default 600.
GITLAWB_PEER_WRITE_RATE_LIMIT=600
# /api/v1/sync/trigger requires a signature and fans out to every peer per call,
# so it gets a tight bucket. 0 disables. Default 60.
GITLAWB_SYNC_TRIGGER_RATE_LIMIT=60
# Which forwarded header the edge is trusted to set, used to resolve the real
# client IP for the push limiter. One of:
# (unset) — no trusted proxy: key on the socket peer address, ignore headers.
# fly — behind Fly's edge (trust Fly-Client-IP).
# x-forwarded-for — behind a single reverse proxy like Caddy/NGINX (trust the
# rightmost X-Forwarded-For hop).
# Only set this when a proxy you control actually fronts the node; trusting a
# forwarded header on a directly-exposed node lets clients spoof the key.
GITLAWB_TRUSTED_PROXY=
# ── Sync ─────────────────────────────────────────────────────────────────
# Enable automatic background sync from known peers
GITLAWB_AUTO_SYNC=false
+12 -1
View File
@@ -37,7 +37,18 @@ jobs:
with:
key: audit-schedule
- name: Install cargo-audit
- name: Install cargo-audit (floating; drift canary)
# Deliberately UNPINNED, unlike the blocking gate in pr-checks.yml. This
# job is cron-only (weekly); it is never pull_request- or push-triggered,
# so it NEVER gates a merge no matter what it does. Floating the scanner
# here is the drift canary: if a newer cargo-audit regresses, at worst
# this weekly run reds (the install step below is unguarded) or its
# visibility report changes, and that is exactly the signal that latest
# has moved and the pinned gate should not be bumped yet. The scan result
# itself never fails the run (the "Full audit report" step is `|| true`
# and the only other hard-fail is a Cargo.lock grep). The point is that
# drift surfaces here without ever blocking a PR. Do NOT pin to match
# pr-checks.yml.
run: cargo install --locked cargo-audit
# Full advisory report with NO suppressions. Run from a scratch dir that
+23 -3
View File
@@ -133,10 +133,30 @@ jobs:
- name: Cache cargo
uses: Swatinem/rust-cache@98c8021b550208e191a6a3145459bfc9fb29c4c0 # v2.8.0
with:
key: audit
key: audit-0.22.2
- name: Install cargo-audit
run: cargo install --locked cargo-audit
- name: Install cargo-audit (pinned)
# Pin the scanner on this BLOCKING gate: an unpinned `cargo install`
# floats to the latest release, so a regressed cargo-audit could red CI
# repo-wide with no code change. Advisory-DB freshness is fetched at scan
# time, independent of the binary, so this does not weaken what the gate
# catches. The weekly Scheduled Audit (audit-schedule.yml) deliberately
# floats to latest as the drift canary; when it shows a newer cargo-audit
# behaving differently, bump this version (and the cache key above) as a
# deliberate maintainer action. If 0.22.2 is ever yanked from crates.io
# this step fails loud until the pin is bumped.
run: |
set -euo pipefail
cargo install --locked --version 0.22.2 cargo-audit
# Assert the pin took effect so an accidental future unpin (or a drift
# off 0.22.2) fails loudly here instead of silently running a different
# scanner. `cargo audit --version` prints "cargo-audit-audit 0.22.2".
installed="$(cargo audit --version)"
echo "cargo-audit installed: $installed"
echo "$installed" | grep -qE '(^| )0\.22\.2($| )' || {
echo "::error::cargo-audit is not the pinned 0.22.2 (got: $installed)"
exit 1
}
# Hard-fail gate. Suppressions live in .cargo/audit.toml (read automatically
# from the repo root), each with no available upstream fix. A green check
+10 -10
View File
@@ -116,10 +116,14 @@ jobs:
include:
- target: x86_64-unknown-linux-musl
os: ubuntu-latest
# Native arm64 runner: cross-compiling this target with the glibc
# toolchain broke aws-lc-sys, and native lets the smoke test run.
- target: aarch64-unknown-linux-musl
os: ubuntu-latest
os: ubuntu-24.04-arm
# macos-13 (the last plain Intel image) is retired and queues forever;
# macos-15-intel is GitHub's supported Intel label.
- target: x86_64-apple-darwin
os: macos-13
os: macos-15-intel
- target: aarch64-apple-darwin
os: macos-14
- target: x86_64-pc-windows-msvc
@@ -137,16 +141,13 @@ jobs:
toolchain: stable
targets: ${{ matrix.target }}
# Both musl targets build natively on a matching-arch runner, so the stock
# musl-gcc wrapper is all that's needed (no cross toolchain).
- name: Install musl tools (linux)
if: contains(matrix.target, 'linux-musl')
run: |
sudo apt-get update
sudo apt-get install -y musl-tools
if [ "${{ matrix.target }}" = "aarch64-unknown-linux-musl" ]; then
sudo apt-get install -y gcc-aarch64-linux-gnu
echo "CARGO_TARGET_AARCH64_UNKNOWN_LINUX_MUSL_LINKER=aarch64-linux-gnu-gcc" >> "$GITHUB_ENV"
echo "CC_aarch64_unknown_linux_musl=aarch64-linux-gnu-gcc" >> "$GITHUB_ENV"
fi
- name: Cache cargo
uses: Swatinem/rust-cache@98c8021b550208e191a6a3145459bfc9fb29c4c0 # v2.8.0
@@ -171,10 +172,9 @@ jobs:
cargo build --release --target ${{ matrix.target }} $args
# Run each packaged binary's --version so a broken release artifact fails the
# build instead of shipping. Skipped for cross-compiled targets that can't run
# on the host runner (aarch64 linux on an x86_64 runner).
# build instead of shipping. Every target builds on a matching-arch runner,
# so all of them can execute here.
- name: Smoke test binaries
if: ${{ matrix.target != 'aarch64-unknown-linux-musl' }}
shell: bash
run: |
set -euo pipefail
+3
View File
@@ -23,3 +23,6 @@ keys/
# Logs
*.log
.openclaude-profile.json
# Per-operator Fly config (node1 prod config is pulled from Fly, not tracked)
infra/fly/node1.fly.toml
+1 -1
View File
@@ -1,3 +1,3 @@
{
".": "0.4.0"
".": "0.5.1"
}
+34
View File
@@ -1,5 +1,39 @@
# Changelog
## [0.5.1](https://github.com/Gitlawb/node/compare/v0.5.0...v0.5.1) (2026-07-10)
### Bug Fixes
* **bounties:** add tests for claim_bounty repo-read gate ([#160](https://github.com/Gitlawb/node/issues/160)) ([#169](https://github.com/Gitlawb/node/issues/169)) ([6bafaa6](https://github.com/Gitlawb/node/commit/6bafaa6dc5f05c0dcd61c708397afddbcf8c2e3f))
* **deps:** bump crossbeam-epoch to 0.9.20 for RUSTSEC-2026-0204 ([#162](https://github.com/Gitlawb/node/issues/162)) ([#163](https://github.com/Gitlawb/node/issues/163)) ([67ad2b8](https://github.com/Gitlawb/node/commit/67ad2b876c8d9a336219d1016968de7a88fc4e75))
* **node,git:** bound a hung served git with a total-duration timeout ([#62](https://github.com/Gitlawb/node/issues/62)) ([#165](https://github.com/Gitlawb/node/issues/165)) ([cd67718](https://github.com/Gitlawb/node/commit/cd67718f49ec38726a40f6bcf36f539ccdb42969))
* **node:** bound list_ref_certificates with LIMIT and add upsert to prevent unbounded growth ([#147](https://github.com/Gitlawb/node/issues/147)) ([#149](https://github.com/Gitlawb/node/issues/149)) ([6b5e5bc](https://github.com/Gitlawb/node/commit/6b5e5bc7aee00a2d03295d3620df1ee4d8c024a2))
* **node:** gate POST /api/v1/sync/trigger and rate-limit the peer-sync routes ([#82](https://github.com/Gitlawb/node/issues/82)) ([#161](https://github.com/Gitlawb/node/issues/161)) ([d00d89a](https://github.com/Gitlawb/node/commit/d00d89ae5be992d1f63e95b714ae1bd3735e8457))
* **node:** rate-limit repo/agent creation per client IP to stop DID-farm spam floods ([#180](https://github.com/Gitlawb/node/issues/180)) ([dfcaa22](https://github.com/Gitlawb/node/commit/dfcaa22b23ec91be4c75926956aa994ca89de8d5))
* **release:** build aarch64-musl natively on arm64 runners, replace retired macos-13 ([#155](https://github.com/Gitlawb/node/issues/155)) ([6cff528](https://github.com/Gitlawb/node/commit/6cff5286b436fdee44ad881999e2a5f4bdba18f9))
* **visibility:** gate repo-scoped read surfaces on visibility ([#120](https://github.com/Gitlawb/node/issues/120)) ([#157](https://github.com/Gitlawb/node/issues/157)) ([26bc3f6](https://github.com/Gitlawb/node/commit/26bc3f69870aa77e43c0d92115a5aa59555b7d88))
## [0.5.0](https://github.com/Gitlawb/node/compare/v0.4.0...v0.5.0) (2026-07-05)
### Features
* **gl:** sanctioned iCaptcha client flow + secure git lifecycle ([#138](https://github.com/Gitlawb/node/issues/138)) ([06388ec](https://github.com/Gitlawb/node/commit/06388ec26aa29d356ae311276fdb91be054e9ecc))
### Bug Fixes
* **gl:** sign the CLI's /ipfs/pins reads under the [#134](https://github.com/Gitlawb/node/issues/134) auth gate ([#146](https://github.com/Gitlawb/node/issues/146)) ([20d6848](https://github.com/Gitlawb/node/commit/20d6848846b3a988d604208833167a528b7d8820))
* **node,git-remote:** gate receive-pack advertisement, sign client fetch/push ([#119](https://github.com/Gitlawb/node/issues/119)) ([6f36fc0](https://github.com/Gitlawb/node/commit/6f36fc07b8e10a650c5948b269feac1cb25cae2a))
* **node,gossip:** route gossip HTTP through the no-redirect client ([#93](https://github.com/Gitlawb/node/issues/93)) ([#140](https://github.com/Gitlawb/node/issues/140)) ([563c456](https://github.com/Gitlawb/node/commit/563c456803bf3e958d63869db424b3940472bc3d))
* **node:** close two spam-vector root causes (trust upsert + ungated push) ([#152](https://github.com/Gitlawb/node/issues/152)) ([2df6ff9](https://github.com/Gitlawb/node/commit/2df6ff9d30de62f754fa41473e85db316021718e))
* **node:** gate GET /ipfs/{cid} on reachable allowed-set, not deny-set ([#126](https://github.com/Gitlawb/node/issues/126)) ([#133](https://github.com/Gitlawb/node/issues/133)) ([466a550](https://github.com/Gitlawb/node/commit/466a550915edd711856ef32035f9f474e2577c4f))
* **node:** gate the ref-updates feeds on read visibility ([#112](https://github.com/Gitlawb/node/issues/112), [#114](https://github.com/Gitlawb/node/issues/114)) ([#143](https://github.com/Gitlawb/node/issues/143)) ([4891db3](https://github.com/Gitlawb/node/commit/4891db38892663326ee0c1417a2db931988be4b5))
* **node:** prefer canonical repo row over mirror row in get_repo ([#124](https://github.com/Gitlawb/node/issues/124)) ([#141](https://github.com/Gitlawb/node/issues/141)) ([6c95592](https://github.com/Gitlawb/node/commit/6c95592d188222ac3446dc23ef8d9befbf82ad6f))
* **remote:** include HTTP error response body ([#137](https://github.com/Gitlawb/node/issues/137)) ([09a0cb2](https://github.com/Gitlawb/node/commit/09a0cb23b9f284ccbd69aca6958b70671f3bfb46))
* **repos:** log the cause when repo create fails ([#103](https://github.com/Gitlawb/node/issues/103)) ([2620e97](https://github.com/Gitlawb/node/commit/2620e973e3cd4835ed42ddd4adcd8183b5b3080e))
## [0.4.0](https://github.com/Gitlawb/node/compare/v0.3.9...v0.4.0) (2026-06-30)
Generated
+22 -7
View File
@@ -2293,9 +2293,9 @@ dependencies = [
[[package]]
name = "crossbeam-epoch"
version = "0.9.18"
version = "0.9.20"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5b82ac4a3c2ca9c3460964f020e1402edd5753411d7737aa39c3714ad1b5420e"
checksum = "2d6914041f254d6e9176c01941b21115dcfb7089e55135a35411081bd106ef3f"
dependencies = [
"crossbeam-utils",
]
@@ -3300,10 +3300,11 @@ dependencies = [
[[package]]
name = "git-remote-gitlawb"
version = "0.3.9"
version = "0.5.0"
dependencies = [
"anyhow",
"gitlawb-core",
"mockito",
"reqwest",
"tracing",
"tracing-subscriber",
@@ -3311,7 +3312,7 @@ dependencies = [
[[package]]
name = "gitlawb-attest"
version = "0.3.9"
version = "0.5.0"
dependencies = [
"base64",
"ed25519-dalek",
@@ -3328,7 +3329,7 @@ dependencies = [
[[package]]
name = "gitlawb-core"
version = "0.3.9"
version = "0.5.0"
dependencies = [
"anyhow",
"base64",
@@ -3355,7 +3356,7 @@ dependencies = [
[[package]]
name = "gitlawb-node"
version = "0.3.9"
version = "0.5.0"
dependencies = [
"alloy",
"anyhow",
@@ -3411,7 +3412,7 @@ dependencies = [
[[package]]
name = "gl"
version = "0.3.9"
version = "0.5.0"
dependencies = [
"alloy",
"anyhow",
@@ -3419,6 +3420,7 @@ dependencies = [
"clap",
"dirs",
"gitlawb-core",
"icaptcha-client",
"mockito",
"reqwest",
"serde",
@@ -3813,6 +3815,19 @@ dependencies = [
"cc",
]
[[package]]
name = "icaptcha-client"
version = "0.4.0"
dependencies = [
"anyhow",
"gitlawb-core",
"reqwest",
"serde",
"serde_json",
"thiserror 2.0.18",
"tracing",
]
[[package]]
name = "icu_collections"
version = "2.1.1"
+1
View File
@@ -6,6 +6,7 @@ members = [
"crates/gl",
"crates/git-remote-gitlawb",
"crates/gitlawb-attest",
"crates/icaptcha-client",
]
[workspace.package]
+44 -3
View File
@@ -185,6 +185,45 @@ git clone gitlawb://did:key:z6Mk.../my-repo
For public-network use, make sure `GITLAWB_NODE` points to the node you want. The helper defaults to localhost for local development.
### Full lifecycle against an iCaptcha-enforcing node
Public nodes (e.g. `node.gitlawb.com`) require two things on writes:
1. **RFC 9421 HTTP Signatures** — every write is signed by your identity key. `gl`
and the `git-remote-gitlawb` helper do this automatically. An old/unsigned CLI
fails with `401 not_an_agent`; `gl` will tell you to upgrade and register.
2. **An iCaptcha proof** on the spam-gated writes (**repo create, fork, register**).
`gl` solves this for you: on the node's `403 icaptcha_proof_required` it reads the
`x-icaptcha-url` / `x-icaptcha-level` hints, requests a challenge, solves it
locally (arithmetic / algebra / sequence), and **retries the same signed request**
with the `x-icaptcha-proof` header — no manual steps, no env vars.
```bash
gl identity new # create did:key identity
gl register --node https://node.gitlawb.com # signed + auto-solves iCaptcha
gl repo create memlawb --node https://node.gitlawb.com # signed + auto-solves iCaptcha
git push origin2 main # origin2 = gitlawb://<your-did>/memlawb (signed)
git clone gitlawb://<your-did>/memlawb # public read, no proof needed
gl doctor # preflight: identity, node, version, iCaptcha
```
Notes:
- **`requesterId` is always your DID.** The proof's `sub` claim must equal the
authenticated signer; `gl`/helper set this automatically and the node enforces
`sub == authenticated DID` (so a proof minted for another identity is rejected).
- **Proofs are short-lived (~5 min TTL) and single-use.** If one expires between
solving and use, the client transparently solves a fresh one and retries.
- **What needs what:** create / fork / register are signed **and** iCaptcha-gated;
`git push` is **signed-only** (owner signature is the gate — no per-push challenge);
reads (clone / fetch / `repo info`) need no proof. A non-existent repo returns a
clear `404`, never a placeholder.
- **API-key iCaptcha deployments:** set `GITLAWB_ICAPTCHA_URL` to your iCaptcha
origin and `GITLAWB_ICAPTCHA_API_KEY` to its key. The client only talks to an
`https` origin whose host is allowlisted (that URL or the public default), and
sends the bearer token **only** to your configured origin — never to a URL a
node advertises — so a hostile node can't capture the key or redirect the solve.
---
## Architecture
@@ -263,20 +302,21 @@ POST /api/v1/bounties/{id}/...
POST /{owner}/{repo}/git-receive-pack
```
Peer write routes support staged rollout:
These peer write routes support staged rollout:
```txt
POST /api/v1/peers/announce
POST /api/v1/sync/notify
POST /api/v1/sync/trigger
```
When `GITLAWB_REQUIRE_SIGNED_PEER_WRITES=false`, unsigned legacy peers are accepted, but signed requests are verified when signature headers are present. Once all live peers upgrade, operators can set:
When `GITLAWB_REQUIRE_SIGNED_PEER_WRITES=false`, unsigned legacy peers are accepted on those two routes, but signed requests are verified when signature headers are present. Once all live peers upgrade, operators can set:
```bash
GITLAWB_REQUIRE_SIGNED_PEER_WRITES=true
```
`POST /api/v1/sync/trigger` is not part of the staged rollout: it always requires a signature in both config modes and returns 401 without one, because each call drives an O(peers) outbound fan-out.
---
## Configuration
@@ -303,6 +343,7 @@ Important node settings:
| `GITLAWB_REQUIRE_SIGNED_PEER_WRITES` | Require signed peer announce/sync writes. |
| `GITLAWB_AUTO_SYNC` | Enable automatic sync from known peers. |
| `GITLAWB_MAX_PACK_BYTES` | Max git pack body size for smart-HTTP routes. |
| `GITLAWB_GIT_SERVICE_TIMEOUT_SECS` | Max seconds a served git upload-pack/receive-pack may run before it is aborted (504). Default 600. Does not bound `info/refs` or the withheld-blob path. |
| `GITLAWB_TIGRIS_BUCKET` | Optional S3/Tigris shared repo storage bucket. |
| `GITLAWB_PINATA_JWT` | Optional Pinata/IPFS warm-storage pinning. |
| `GITLAWB_IRYS_URL` | Optional Irys/Arweave permanent anchoring. |
+26 -2
View File
@@ -1,16 +1,40 @@
{
"$comment": "Canonical seed list for the Gitlawb network. Merged with GITLAWB_BOOTSTRAP_PEERS at startup. PRs to add public nodes welcome.",
"version": 1,
"updated": "2026-05-16",
"updated": "2026-06-30",
"peers": [
{
"name": "gitlawb",
"operator": "Gitlawb (Kevin)",
"operator": "Gitlawb",
"did": "did:key:z6MkqDnb7Siv3Cwj7pGJq4T5EsUisECqR8KpnDLwcaZq5TPr",
"http_url": "https://node.gitlawb.com",
"p2p_multiaddr": null,
"added": "2026-04-29"
},
{
"name": "gitlawb-node2",
"operator": "Gitlawb",
"did": null,
"http_url": "https://node2.gitlawb.com",
"p2p_multiaddr": null,
"added": "2026-06-30"
},
{
"name": "gitlawb-node3",
"operator": "Gitlawb",
"did": null,
"http_url": "https://node3.gitlawb.com",
"p2p_multiaddr": null,
"added": "2026-06-30"
},
{
"name": "gitlawb-manila",
"operator": "Gitlawb",
"did": null,
"http_url": "https://manila.gitlawb.com",
"p2p_multiaddr": null,
"added": "2026-06-30"
},
{
"name": "rapybus-gitlawb",
"operator": "RapyBus S.A.C.",
+4 -1
View File
@@ -1,7 +1,7 @@
[package]
name = "git-remote-gitlawb"
description = "Git remote helper — enables 'git clone gitlawb://did:gitlawb:...'"
version = "0.4.0" # x-release-please-version
version = "0.5.1" # x-release-please-version
edition.workspace = true
rust-version.workspace = true
license.workspace = true
@@ -16,3 +16,6 @@ anyhow = { workspace = true }
reqwest = { workspace = true }
tracing = { workspace = true }
tracing-subscriber = { workspace = true }
[dev-dependencies]
mockito = "1"
+747 -35
View File
@@ -14,7 +14,13 @@
//! # Protocol flow (connect capability)
//! capabilities → "connect\n\n"
//! connect git-upload-pack → GET /info/refs | POST /git-upload-pack
//! connect git-receive-pack → GET /info/refs | POST /git-receive-pack (+ auth header)
//! connect git-receive-pack → GET /info/refs | POST /git-receive-pack
//! Push (git-receive-pack) is RFC-9421 signed from the first request when an
//! identity keypair is present, since a push can never be anonymous. Fetch
//! (git-upload-pack) stays anonymous and is signed only on a single retry, after
//! the node denies the anonymous advertisement with 404 — so a public clone never
//! discloses the caller's DID, while a private repo's owner (or an authorized
//! reader) still authenticates when the node demands it.
use anyhow::{bail, Context, Result};
use gitlawb_core::http_sig::sign_request;
@@ -63,7 +69,10 @@ fn main() -> Result<()> {
let repo_base = format!("{}/{}/{}", node_base, short_owner, repo_name);
tracing::debug!("repo_base: {repo_base}");
// Load keypair for signing push requests (optional — push still works unsigned in v0.1)
// Load keypair for signing requests (optional). Push always signs when a key is
// present; fetch signs only if the node 404s the anonymous advertisement, so
// public fetches stay anonymous. Absent a key, public fetch still works and push
// falls back to unsigned (v0.1 local alpha only).
let keypair = load_keypair();
run_helper(&repo_base, keypair.as_ref())
@@ -115,7 +124,7 @@ fn help_text() -> String {
\n\
ENVIRONMENT:\n\
\x20 GITLAWB_NODE Node base URL (default: http://127.0.0.1:7545)\n\
\x20 GITLAWB_KEY Identity PEM path for signed pushes (default: ~/.gitlawb/identity.pem)\n\
\x20 GITLAWB_KEY Identity PEM path for signed fetch/push (default: ~/.gitlawb/identity.pem)\n\
\x20 GITLAWB_LOG Log filter (default: warn)\n\
\n\
FLAGS:\n\
@@ -179,11 +188,11 @@ fn run_helper(repo_base: &str, keypair: Option<&Keypair>) -> Result<()> {
// ── HTTP proxy for git smart protocol ────────────────────────────────────────
fn handle_connect(
fn handle_connect<R: Read>(
repo_base: &str,
service: &str,
keypair: Option<&Keypair>,
stdin: &mut io::BufReader<io::Stdin>,
stdin: &mut R,
) -> Result<()> {
match service {
"git-upload-pack" | "git-receive-pack" => {}
@@ -198,19 +207,52 @@ fn handle_connect(
//
// The server advertises its refs. git reads this to decide what to fetch/push.
// Signing policy for this exchange, carried through to the Phase-2 POST:
// - git-receive-pack (push): sign from the first request; a push is
// signature-gated and can never be anonymous.
// - git-upload-pack (fetch): start anonymous so a public clone never discloses
// the caller's DID, then escalate to a single signed retry only if the node
// denies the anonymous advertisement with 404 (how it withholds a private
// repo from an unauthenticated reader).
let mut signing_key = if service == "git-receive-pack" {
keypair
} else {
None
};
let refs_url = format!("{}/info/refs?service={}", repo_base, service);
tracing::debug!("GET {refs_url}");
let refs_resp = client
.get(&refs_url)
.header("User-Agent", "git/2.0 git-remote-gitlawb/0.1.0")
let mut refs_resp = build_advertisement_request(&client, &refs_url, signing_key)
.send()
.with_context(|| format!("GET {refs_url}"))?;
// Fetch escalation: retry the advertisement signed once when the anonymous try
// is denied and we hold an identity. A public repo answers 200 on the first
// (anonymous) request and never reaches here, so it stays DID-private.
if service == "git-upload-pack"
&& refs_resp.status() == reqwest::StatusCode::NOT_FOUND
&& keypair.is_some()
{
tracing::debug!("anonymous info/refs returned 404; retrying signed");
signing_key = keypair;
refs_resp = build_advertisement_request(&client, &refs_url, signing_key)
.send()
.with_context(|| format!("GET {refs_url} (signed retry)"))?;
}
if !refs_resp.status().is_success() {
let status = refs_resp.status();
let body = read_error_body(refs_resp);
bail!(
"GET /info/refs returned {} — is the repo registered on this node?",
refs_resp.status()
"{}",
http_error_message(
"GET",
"/info/refs",
status,
&body,
Some("— is the repo registered on this node?")
)
);
}
@@ -266,29 +308,7 @@ fn handle_connect(
let post_url = format!("{}/{}", repo_base, service);
tracing::debug!("POST {post_url} ({} bytes)", request_body.len());
// Extract the URL path for signing (e.g., "/z6Mk.../my-repo/git-receive-pack")
let path_for_sig = url_path(&post_url);
let mut req = client
.post(&post_url)
.header("Content-Type", format!("application/x-{}-request", service))
.header("User-Agent", "git/2.0 git-remote-gitlawb/0.1.0");
// Add RFC 9421 HTTP Signature auth on push operations
if service == "git-receive-pack" {
if let Some(kp) = keypair {
let signed = sign_request(kp, "POST", &path_for_sig, &request_body);
req = req
.header("Content-Digest", signed.content_digest)
.header("Signature-Input", signed.signature_input)
.header("Signature", signed.signature);
tracing::debug!("attached RFC 9421 HTTP Signature (DID: {})", kp.did());
} else {
tracing::warn!(
"no identity keypair found — push will be unsigned (v0.1 local alpha only)"
);
}
}
let req = build_pack_post_request(&client, &post_url, service, &request_body, signing_key);
// Attach the body after signing so the pack bytes are moved, not cloned —
// packs can be large and the clone doubled peak memory on push.
@@ -298,7 +318,10 @@ fn handle_connect(
.with_context(|| format!("POST {post_url}"))?;
if !pack_resp.status().is_success() {
bail!("POST /{} returned {}", service, pack_resp.status());
let status = pack_resp.status();
let body = read_error_body(pack_resp);
let path = format!("/{service}");
bail!("{}", http_error_message("POST", &path, status, &body, None));
}
let pack_bytes = pack_resp.bytes().context("reading pack response")?;
@@ -310,6 +333,68 @@ fn handle_connect(
Ok(())
}
// ── Smart-protocol request builders ───────────────────────────────────────────
const USER_AGENT: &str = "git/2.0 git-remote-gitlawb/0.1.0";
/// Build the Phase-1 ref-advertisement GET, signing it when an identity is
/// present. The node gates the ref advertisement on read visibility for BOTH
/// services, so a private repo's advertisement is denied (404) to an
/// unauthenticated caller; without this signature the repo's own owner can
/// neither fetch (upload-pack) nor push (receive-pack) it. Public repos still
/// work anonymously (no keypair present, or the gate admits anonymous). Sign over
/// the path *and* query (?service=...) because the node verifies the signature
/// over its path_and_query.
fn build_advertisement_request(
client: &reqwest::blocking::Client,
refs_url: &str,
keypair: Option<&Keypair>,
) -> reqwest::blocking::RequestBuilder {
let mut req = client.get(refs_url).header("User-Agent", USER_AGENT);
if let Some(kp) = keypair {
let signed = sign_request(kp, "GET", &url_path(refs_url), b"");
req = req
.header("Content-Digest", signed.content_digest)
.header("Signature-Input", signed.signature_input)
.header("Signature", signed.signature);
tracing::debug!("signed info/refs advertisement (DID: {})", kp.did());
}
req
}
/// Build the Phase-2 pack POST, signing it when an identity is present, for BOTH
/// services. The node read-gates the git-upload-pack POST with visibility_check
/// at "/", and separately owner-gates the git-receive-pack POST (signature
/// required, enforced by middleware). The Phase-1 advertisement signature does
/// not carry to this separate request, so a private repo's owner must
/// authenticate here too or their fetch/push is denied.
/// Public-repo fetch still works anonymously when no keypair is present. The body
/// is signed (content-digest) but NOT attached here, so the caller can move the
/// (possibly large) pack bytes into `.body()` rather than clone them.
fn build_pack_post_request(
client: &reqwest::blocking::Client,
post_url: &str,
service: &str,
body: &[u8],
keypair: Option<&Keypair>,
) -> reqwest::blocking::RequestBuilder {
let mut req = client
.post(post_url)
.header("Content-Type", format!("application/x-{}-request", service))
.header("User-Agent", USER_AGENT);
if let Some(kp) = keypair {
let signed = sign_request(kp, "POST", &url_path(post_url), body);
req = req
.header("Content-Digest", signed.content_digest)
.header("Signature-Input", signed.signature_input)
.header("Signature", signed.signature);
tracing::debug!("signed {service} POST (DID: {})", kp.did());
} else if service == "git-receive-pack" {
tracing::warn!("no identity keypair found, push will be unsigned (v0.1 local alpha only)");
}
req
}
// ── URL helpers ───────────────────────────────────────────────────────────────
/// Parse a `gitlawb://did:key:z6Mk.../repo-name` URL.
@@ -347,7 +432,7 @@ fn parse_gitlawb_url(url: &str) -> Result<(String, String, String)> {
///
/// We also handle the flush-only case (`"0000"`) that git sends when it already
/// has everything it needs (up-to-date clone).
fn read_upload_pack_request(stdin: &mut io::BufReader<io::Stdin>) -> Result<Vec<u8>> {
fn read_upload_pack_request<R: Read>(stdin: &mut R) -> Result<Vec<u8>> {
let mut buf = Vec::new();
loop {
@@ -433,6 +518,78 @@ fn url_path(url: &str) -> String {
.unwrap_or_else(|| "/".to_string())
}
const MAX_ERROR_BODY_BYTES: u64 = 4096;
const MAX_ERROR_BODY_CHARS: usize = 2000;
fn read_error_body(mut response: reqwest::blocking::Response) -> String {
let mut body = Vec::new();
let mut limited = (&mut response).take(MAX_ERROR_BODY_BYTES);
if limited.read_to_end(&mut body).is_err() {
return String::new();
}
String::from_utf8_lossy(&body).into_owned()
}
fn http_error_message(
method: &str,
path: &str,
status: reqwest::StatusCode,
body: &str,
empty_body_hint: Option<&str>,
) -> String {
let mut message = format!("{method} {path} returned {status}");
let body = safe_error_body_excerpt(body);
if !body.is_empty() {
message.push_str(": ");
message.push_str(&body);
} else if let Some(hint) = empty_body_hint {
message.push(' ');
message.push_str(hint);
}
message
}
fn safe_error_body_excerpt(body: &str) -> String {
let mut excerpt = String::new();
let mut pending_space = false;
let mut kept_chars = 0;
for ch in body.trim().chars() {
if kept_chars >= MAX_ERROR_BODY_CHARS {
break;
}
if ch.is_control() {
if matches!(ch, '\n' | '\r' | '\t') {
pending_space = true;
}
continue;
}
if ch.is_whitespace() {
pending_space = true;
continue;
}
if pending_space && !excerpt.is_empty() {
excerpt.push(' ');
kept_chars += 1;
if kept_chars >= MAX_ERROR_BODY_CHARS {
break;
}
}
excerpt.push(ch);
kept_chars += 1;
pending_space = false;
}
excerpt
}
// ── Keypair loading ───────────────────────────────────────────────────────────
fn load_keypair() -> Option<Keypair> {
@@ -476,6 +633,467 @@ fn resolve_key_path() -> std::path::PathBuf {
#[cfg(test)]
mod tests {
use super::*;
use std::net::{TcpListener, TcpStream};
use std::thread::JoinHandle;
struct TestResponse {
request_line: &'static str,
status: &'static str,
body: &'static str,
}
fn serve_http(responses: Vec<TestResponse>) -> (String, JoinHandle<Vec<String>>) {
let listener = TcpListener::bind("127.0.0.1:0").unwrap();
let base_url = format!("http://{}", listener.local_addr().unwrap());
let handle = std::thread::spawn(move || {
let mut requests = Vec::new();
for response in responses {
let (mut stream, _) = listener.accept().unwrap();
let request = read_http_request(&mut stream);
assert!(
request.starts_with(response.request_line),
"unexpected request:\n{request}"
);
let response_text = format!(
"HTTP/1.1 {}\r\ncontent-type: application/json\r\ncontent-length: {}\r\nconnection: close\r\n\r\n{}",
response.status,
response.body.len(),
response.body
);
stream.write_all(response_text.as_bytes()).unwrap();
requests.push(request);
}
requests
});
(base_url, handle)
}
fn read_http_request(stream: &mut TcpStream) -> String {
let mut buf = Vec::new();
let mut chunk = [0u8; 512];
let mut header_len = None;
loop {
let n = stream.read(&mut chunk).unwrap();
if n == 0 {
break;
}
buf.extend_from_slice(&chunk[..n]);
if header_len.is_none() {
header_len = buf.windows(4).position(|w| w == b"\r\n\r\n").map(|p| p + 4);
}
if let Some(header_len) = header_len {
let header = String::from_utf8_lossy(&buf[..header_len]);
let content_len = header
.lines()
.find_map(|line| {
let (name, value) = line.split_once(':')?;
name.eq_ignore_ascii_case("content-length")
.then(|| value.trim().parse::<usize>().ok())
.flatten()
})
.unwrap_or(0);
if buf.len() >= header_len + content_len {
break;
}
}
}
String::from_utf8_lossy(&buf).into_owned()
}
/// The regression that round-1 missed: the Phase-2 `git-upload-pack` POST was
/// left unsigned, so an owner's fetch of a private repo cleared the (now signed)
/// advertisement and then 404'd on the pack POST. Drive BOTH request builders
/// against a mock node and assert the RFC-9421 headers actually go out on the
/// wire for BOTH services when a keypair is present (the mock only matches when
/// the headers exist, so `.assert()` fails if any are dropped).
#[test]
fn advertisement_and_pack_post_are_signed_for_both_services_with_keypair() {
let kp = Keypair::generate();
let client = reqwest::blocking::Client::new();
let body = b"0009done\n".to_vec();
for service in ["git-upload-pack", "git-receive-pack"] {
let mut server = mockito::Server::new();
let repo_base = format!("{}/zOwner/myrepo", server.url());
let get_mock = server
.mock("GET", mockito::Matcher::Regex(r"/info/refs".to_string()))
.match_header("signature", mockito::Matcher::Any)
.match_header("signature-input", mockito::Matcher::Any)
.match_header("content-digest", mockito::Matcher::Any)
.with_status(200)
.create();
let refs_url = format!("{repo_base}/info/refs?service={service}");
let resp = build_advertisement_request(&client, &refs_url, Some(&kp))
.send()
.unwrap();
assert!(resp.status().is_success());
get_mock.assert();
let post_mock = server
.mock("POST", mockito::Matcher::Regex(format!("/{service}$")))
.match_header("signature", mockito::Matcher::Any)
.match_header("signature-input", mockito::Matcher::Any)
.match_header("content-digest", mockito::Matcher::Any)
.with_status(200)
.create();
let post_url = format!("{repo_base}/{service}");
let resp = build_pack_post_request(&client, &post_url, service, &body, Some(&kp))
.body(body.clone())
.send()
.unwrap();
assert!(resp.status().is_success());
post_mock.assert();
}
}
/// Without an identity, the pack POST must go out UNSIGNED so public fetch (and
/// alpha unsigned push) still work. `Matcher::Missing` matches only when the
/// header is absent, so the mock is hit only if NO signature was attached, for
/// BOTH services (the receive-pack iteration also exercises the no-keypair warn
/// branch and confirms it does not block the request).
#[test]
fn pack_post_is_unsigned_without_keypair_for_both_services() {
let client = reqwest::blocking::Client::new();
let body = b"0009done\n".to_vec();
for service in ["git-upload-pack", "git-receive-pack"] {
let mut server = mockito::Server::new();
let post_url = format!("{}/zOwner/myrepo/{service}", server.url());
let post_mock = server
.mock("POST", mockito::Matcher::Regex(format!("/{service}$")))
.match_header("signature", mockito::Matcher::Missing)
.match_header("signature-input", mockito::Matcher::Missing)
.match_header("content-digest", mockito::Matcher::Missing)
.with_status(200)
.create();
let resp = build_pack_post_request(&client, &post_url, service, &body, None)
.body(body.clone())
.send()
.unwrap();
assert!(resp.status().is_success());
post_mock.assert();
}
}
/// Symmetric negative case for Phase 1: with no identity the advertisement GET
/// carries no signature, so public clones stay anonymous.
#[test]
fn advertisement_get_is_unsigned_without_keypair() {
let client = reqwest::blocking::Client::new();
let mut server = mockito::Server::new();
let refs_url = format!(
"{}/zOwner/myrepo/info/refs?service=git-upload-pack",
server.url()
);
let get_mock = server
.mock("GET", mockito::Matcher::Regex(r"/info/refs".to_string()))
.match_header("signature", mockito::Matcher::Missing)
.match_header("signature-input", mockito::Matcher::Missing)
.match_header("content-digest", mockito::Matcher::Missing)
.with_status(200)
.create();
let resp = build_advertisement_request(&client, &refs_url, None)
.send()
.unwrap();
assert!(resp.status().is_success());
get_mock.assert();
}
// ── Phase-1 retry-on-404 (P2, #119): fetch stays anonymous until the node
// denies it, then escalates to ONE signed retry. Drives handle_connect end to
// end against the recording mock, asserting what actually goes on the wire.
// A request is "signed" iff it carries the `signature-input` header.
/// Public fetch: the node serves the anonymous advertisement (200), so the
/// client never signs — not the advertisement GET, not the upload-pack POST —
/// even though an identity is present. This is the privacy fix: a public clone
/// must not disclose the caller's DID.
#[test]
fn fetch_public_stays_anonymous_even_with_keypair() {
let kp = Keypair::generate();
let (base_url, server) = serve_http(vec![
TestResponse {
request_line: "GET /zOwner/myrepo/info/refs?service=git-upload-pack",
status: "200 OK",
body: "0000",
},
TestResponse {
request_line: "POST /zOwner/myrepo/git-upload-pack",
status: "200 OK",
body: "",
},
]);
let repo_base = format!("{base_url}/zOwner/myrepo");
let mut stdin = std::io::Cursor::new(b"0009done\n".to_vec());
handle_connect(&repo_base, "git-upload-pack", Some(&kp), &mut stdin)
.expect("public fetch should succeed");
let requests = server.join().unwrap();
assert_eq!(requests.len(), 2, "public fetch must not retry");
assert!(
!requests[0].to_lowercase().contains("signature-input"),
"the advertisement GET must be anonymous for a public fetch"
);
assert!(
!requests[1].to_lowercase().contains("signature-input"),
"the upload-pack POST must be anonymous for a public fetch"
);
}
/// Private fetch: the anonymous advertisement is denied (404); with an identity
/// present the client retries the GET signed and — because Phase 1 needed
/// signing — also signs the Phase-2 upload-pack POST. The FIRST request is still
/// anonymous: the DID is not disclosed until the node demands authentication.
#[test]
fn fetch_private_retries_signed_on_404_and_signs_the_pack_post() {
let kp = Keypair::generate();
let (base_url, server) = serve_http(vec![
TestResponse {
request_line: "GET /zOwner/myrepo/info/refs?service=git-upload-pack",
status: "404 Not Found",
body: r#"{"message":"repository is not registered on this node"}"#,
},
TestResponse {
request_line: "GET /zOwner/myrepo/info/refs?service=git-upload-pack",
status: "200 OK",
body: "0000",
},
TestResponse {
request_line: "POST /zOwner/myrepo/git-upload-pack",
status: "200 OK",
body: "",
},
]);
let repo_base = format!("{base_url}/zOwner/myrepo");
let mut stdin = std::io::Cursor::new(b"0009done\n".to_vec());
handle_connect(&repo_base, "git-upload-pack", Some(&kp), &mut stdin)
.expect("signed retry should succeed");
let requests = server.join().unwrap();
assert_eq!(
requests.len(),
3,
"expected anon GET, signed GET retry, signed POST"
);
assert!(
!requests[0].to_lowercase().contains("signature-input"),
"the first advertisement GET must be anonymous"
);
assert!(
requests[1].to_lowercase().contains("signature-input"),
"the retry advertisement GET must be signed"
);
assert!(
requests[2].to_lowercase().contains("signature-input"),
"the upload-pack POST must be signed once the fetch needed auth"
);
}
/// Private fetch with no identity: the anonymous advertisement 404s and there is
/// no keypair to retry with, so the client surfaces the error after exactly one
/// request — no retry, no hang.
#[test]
fn fetch_without_keypair_does_not_retry_on_404() {
let (base_url, server) = serve_http(vec![TestResponse {
request_line: "GET /zOwner/myrepo/info/refs?service=git-upload-pack",
status: "404 Not Found",
body: r#"{"message":"not found"}"#,
}]);
let repo_base = format!("{base_url}/zOwner/myrepo");
let mut stdin = std::io::Cursor::new(Vec::<u8>::new());
let err = handle_connect(&repo_base, "git-upload-pack", None, &mut stdin).unwrap_err();
assert!(err.to_string().contains("returned 404 Not Found"));
let requests = server.join().unwrap();
assert_eq!(requests.len(), 1, "no keypair means no signed retry");
assert!(!requests[0].to_lowercase().contains("signature-input"));
}
/// Private fetch where the SIGNED retry is also denied (caller is not a reader):
/// the client retries exactly once, then surfaces the 404 rather than looping.
/// Bounds the escalation to a single signed attempt.
#[test]
fn fetch_signed_retry_still_denied_surfaces_error_once() {
let kp = Keypair::generate();
let (base_url, server) = serve_http(vec![
TestResponse {
request_line: "GET /zOwner/myrepo/info/refs?service=git-upload-pack",
status: "404 Not Found",
body: r#"{"message":"not found"}"#,
},
TestResponse {
request_line: "GET /zOwner/myrepo/info/refs?service=git-upload-pack",
status: "404 Not Found",
body: r#"{"message":"not found"}"#,
},
]);
let repo_base = format!("{base_url}/zOwner/myrepo");
let mut stdin = std::io::Cursor::new(Vec::<u8>::new());
let err = handle_connect(&repo_base, "git-upload-pack", Some(&kp), &mut stdin).unwrap_err();
assert!(err.to_string().contains("returned 404 Not Found"));
let requests = server.join().unwrap();
assert_eq!(requests.len(), 2, "exactly one signed retry, then surface");
assert!(!requests[0].to_lowercase().contains("signature-input"));
assert!(requests[1].to_lowercase().contains("signature-input"));
}
/// Push (git-receive-pack) is unchanged: it signs from the FIRST request — you
/// cannot push anonymously — so there is no anonymous-first probe for push.
#[test]
fn push_advertisement_is_signed_from_the_first_request() {
let kp = Keypair::generate();
let (base_url, server) = serve_http(vec![
TestResponse {
request_line: "GET /zOwner/myrepo/info/refs?service=git-receive-pack",
status: "200 OK",
body: "0000",
},
TestResponse {
request_line: "POST /zOwner/myrepo/git-receive-pack",
status: "200 OK",
body: "",
},
]);
let repo_base = format!("{base_url}/zOwner/myrepo");
let mut stdin = std::io::Cursor::new(b"0000".to_vec());
handle_connect(&repo_base, "git-receive-pack", Some(&kp), &mut stdin)
.expect("push should succeed");
let requests = server.join().unwrap();
assert_eq!(requests.len(), 2);
assert!(
requests[0].to_lowercase().contains("signature-input"),
"push advertisement must be signed from the first request"
);
assert!(
requests[1].to_lowercase().contains("signature-input"),
"push POST must be signed"
);
}
/// Cross-crate seam: the signature the CLIENT actually emits (via the real
/// request builders) verifies under the SAME `gitlawb_core::http_sig`
/// primitives the node's `require_signature` uses, over the `@path` the client
/// genuinely transmits (derived from the built reqwest request, exactly as the
/// node derives it from `uri.path_and_query()`). A tampered `@path` must fail:
/// this is the byte-match the whole A1 client fix depends on, now executed end
/// to end (sign here, verify with the node's verifier), not reasoned.
#[test]
fn client_signature_verifies_under_node_verification_for_both_services() {
use gitlawb_core::http_sig::{build_signing_string, compute_content_digest, HttpSignature};
use gitlawb_core::identity::verify;
use std::collections::HashMap;
let kp = Keypair::generate();
let client = reqwest::blocking::Client::new();
let body = b"0009done\n".to_vec();
// Re-implements the node's require_signature verification (auth/mod.rs):
// parse headers, recompute content-digest from the body, rebuild the signing
// string over @method/@path/content-digest, Ed25519-verify. Ok iff the node
// would accept it.
let node_verifies = |method: &str,
path_and_query: &str,
body: &[u8],
sig_input: &str,
sig_header: &str,
content_digest: &str|
-> anyhow::Result<()> {
let sig = HttpSignature::parse(sig_input, sig_header)?;
sig.check_created()?;
assert!(
sig.missing_components().is_empty(),
"signature must cover all required components"
);
assert_eq!(sig.alg, "ed25519");
assert_eq!(
content_digest,
compute_content_digest(body),
"content-digest must match the body"
);
let vk = sig.key_id.to_verifying_key()?;
let mut values = HashMap::new();
values.insert("@method".to_string(), method.to_uppercase());
values.insert("@path".to_string(), path_and_query.to_string());
values.insert("content-digest".to_string(), content_digest.to_string());
let sig_params_value = sig_input.strip_prefix("sig1=").unwrap_or(sig_input);
let components: Vec<&str> = sig.components.iter().map(String::as_str).collect();
let signing_string = build_signing_string(&components, sig_params_value, &values)?;
let sig_array: [u8; 64] = sig.signature_bytes.as_slice().try_into()?;
verify(&vk, signing_string.as_bytes(), &sig_array)?;
Ok(())
};
// @path exactly as the node reconstructs it from the request it receives.
let path_and_query = |req: &reqwest::blocking::Request| match req.url().query() {
Some(q) => format!("{}?{}", req.url().path(), q),
None => req.url().path().to_string(),
};
let header = |req: &reqwest::blocking::Request, name: &str| {
req.headers()
.get(name)
.unwrap_or_else(|| panic!("missing {name}"))
.to_str()
.unwrap()
.to_string()
};
for service in ["git-upload-pack", "git-receive-pack"] {
// Phase-1 advertisement GET (empty body).
let refs_url = format!("http://node.example/zOwner/myrepo/info/refs?service={service}");
let req = build_advertisement_request(&client, &refs_url, Some(&kp))
.build()
.unwrap();
node_verifies(
"GET",
&path_and_query(&req),
b"",
&header(&req, "signature-input"),
&header(&req, "signature"),
&header(&req, "content-digest"),
)
.expect("client GET signature must verify under the node's verifier");
// Phase-2 pack POST (real body).
let post_url = format!("http://node.example/zOwner/myrepo/{service}");
let req = build_pack_post_request(&client, &post_url, service, &body, Some(&kp))
.body(body.clone())
.build()
.unwrap();
let pq = path_and_query(&req);
node_verifies(
"POST",
&pq,
&body,
&header(&req, "signature-input"),
&header(&req, "signature"),
&header(&req, "content-digest"),
)
.expect("client POST signature must verify under the node's verifier");
// Negative: a tampered @path (the A1 attack surface) must NOT verify.
let tampered = pq.replace(service, "git-evil");
assert!(
node_verifies(
"POST",
&tampered,
&body,
&header(&req, "signature-input"),
&header(&req, "signature"),
&header(&req, "content-digest"),
)
.is_err(),
"a tampered @path must fail verification"
);
}
}
#[test]
fn parse_standard_url() {
@@ -524,6 +1142,100 @@ mod tests {
);
}
#[test]
fn http_error_message_sanitizes_response_body_controls() {
let message = http_error_message(
"GET",
"/info/refs",
reqwest::StatusCode::BAD_GATEWAY,
"\u{1b}[2Jrepository\r\nis\tblocked\u{7}",
None,
);
assert!(message.contains("repository is blocked"));
assert!(!message.contains('\u{1b}'));
assert!(!message.contains('\u{7}'));
assert!(!message.contains('\r'));
assert!(!message.contains('\n'));
}
#[test]
fn http_error_message_truncates_long_response_body() {
let body = "x".repeat(MAX_ERROR_BODY_CHARS + 100);
let message = http_error_message(
"POST",
"/git-receive-pack",
reqwest::StatusCode::FORBIDDEN,
&body,
None,
);
let prefix = "POST /git-receive-pack returned 403 Forbidden: ";
assert_eq!(message.len(), prefix.len() + MAX_ERROR_BODY_CHARS);
assert!(message.starts_with(prefix));
}
#[test]
fn connect_get_error_includes_response_body() {
let (base_url, server) = serve_http(vec![TestResponse {
request_line: "GET /z6Mk/myrepo/info/refs?service=git-upload-pack HTTP/1.1",
status: "404 Not Found",
body: r#"{"message":"repository is not registered on this node"}"#,
}]);
let repo_base = format!("{base_url}/z6Mk/myrepo");
let mut stdin = std::io::Cursor::new(Vec::<u8>::new());
let err = handle_connect(&repo_base, "git-upload-pack", None, &mut stdin).unwrap_err();
let message = err.to_string();
assert!(message.contains("GET /info/refs returned 404 Not Found"));
assert!(message.contains("repository is not registered on this node"));
let requests = server.join().unwrap();
assert_eq!(requests.len(), 1);
}
#[test]
fn connect_post_error_includes_response_body() {
let (base_url, server) = serve_http(vec![
TestResponse {
request_line: "GET /z6Mk/myrepo/info/refs?service=git-receive-pack HTTP/1.1",
status: "200 OK",
body: "",
},
TestResponse {
request_line: "POST /z6Mk/myrepo/git-receive-pack HTTP/1.1",
status: "403 Forbidden",
body: r#"{"error":"forbidden","message":"push rejected: only the repo owner may push"}"#,
},
]);
let repo_base = format!("{base_url}/z6Mk/myrepo");
let mut stdin = std::io::Cursor::new(b"0000".to_vec());
let err = handle_connect(&repo_base, "git-receive-pack", None, &mut stdin).unwrap_err();
let message = err.to_string();
assert!(message.contains("POST /git-receive-pack returned 403 Forbidden"));
assert!(message.contains("push rejected: only the repo owner may push"));
let requests = server.join().unwrap();
assert_eq!(requests.len(), 2);
assert!(requests[1].contains("\r\n\r\n0000"));
}
#[test]
fn url_path_preserves_query_for_signed_advertisement() {
// The Phase-1 advertisement GET is signed over url_path(refs_url), and the
// node verifies the signature over its path_and_query, so the ?service=
// query MUST survive verbatim or the signatures will not byte-match.
assert_eq!(
url_path("http://127.0.0.1:7545/z6Mk/myrepo/info/refs?service=git-upload-pack"),
"/z6Mk/myrepo/info/refs?service=git-upload-pack"
);
assert_eq!(
url_path("http://127.0.0.1:7545/z6Mk/myrepo/info/refs?service=git-receive-pack"),
"/z6Mk/myrepo/info/refs?service=git-receive-pack"
);
}
fn args(items: &[&str]) -> Vec<String> {
items.iter().map(|s| s.to_string()).collect()
}
+1 -1
View File
@@ -1,7 +1,7 @@
[package]
name = "gitlawb-attest"
description = "External Attestation v1: pluggable provenance attachments for gitlawb ref-update certs"
version = "0.4.0" # x-release-please-version
version = "0.5.1" # x-release-please-version
edition.workspace = true
rust-version.workspace = true
license.workspace = true
+1 -1
View File
@@ -1,7 +1,7 @@
[package]
name = "gitlawb-core"
description = "Core cryptographic primitives for the gitlawb network: DIDs, CIDs, UCAN, HTTP Signatures, ref-update certificates"
version = "0.4.0" # x-release-please-version
version = "0.5.1" # x-release-please-version
edition.workspace = true
rust-version.workspace = true
license.workspace = true
+1 -1
View File
@@ -1,7 +1,7 @@
[package]
name = "gitlawb-node"
description = "The gitlawb node daemon — git hosting over HTTP with DID auth"
version = "0.4.0" # x-release-please-version
version = "0.5.1" # x-release-please-version
edition.workspace = true
rust-version.workspace = true
license.workspace = true
+103 -16
View File
@@ -110,19 +110,28 @@ pub async fn create_bounty(
Ok((StatusCode::CREATED, Json(bounty)))
}
const MAX_BOUNTY_LIMIT: i64 = 200;
/// GET /api/v1/repos/{owner}/{repo}/bounties
pub async fn list_repo_bounties(
State(state): State<AppState>,
Path((owner, repo)): Path<(String, String)>,
Query(q): Query<ListBountiesQuery>,
auth: Option<Extension<AuthenticatedDid>>,
) -> Result<Json<serde_json::Value>> {
let caller = auth.as_ref().map(|e| e.0 .0.as_str());
crate::api::authorize_repo_read(&state, &owner, &repo, caller, "/").await?;
let limit = q.limit.unwrap_or(50).clamp(1, MAX_BOUNTY_LIMIT);
let bounties = state
.db
.list_bounties(
Some(&owner),
Some(&repo),
q.status.as_deref(),
q.limit.unwrap_or(50),
limit,
None,
None,
)
.await?;
@@ -133,25 +142,92 @@ pub async fn list_repo_bounties(
pub async fn list_all_bounties(
State(state): State<AppState>,
Query(q): Query<ListBountiesQuery>,
auth: Option<Extension<AuthenticatedDid>>,
) -> Result<Json<serde_json::Value>> {
let bounties = state
.db
.list_bounties(None, None, q.status.as_deref(), q.limit.unwrap_or(50))
.await?;
let raw_limit = q.limit.unwrap_or(50);
let limit = raw_limit.clamp(1, MAX_BOUNTY_LIMIT);
Ok(Json(serde_json::json!({ "bounties": bounties })))
let caller = auth.as_ref().map(|e| e.0 .0.as_str());
let mut memo = std::collections::HashMap::new();
let mut allowed: Vec<BountyRecord> = Vec::new();
let page_size = (limit * 5).clamp(1, 200);
let max_scanned = 10_000i64;
let mut total_scanned: i64 = 0;
let mut cursor: Option<(String, String)> = None;
while (allowed.len() as i64) < limit {
let after = cursor.as_ref().map(|(ts, id)| (ts.as_str(), id.as_str()));
let bounties = state
.db
.list_bounties(
None,
None,
q.status.as_deref(),
page_size,
after.map(|(ts, _)| ts),
after.map(|(_, id)| id),
)
.await?;
if bounties.is_empty() {
break;
}
total_scanned += bounties.len() as i64;
if total_scanned > max_scanned {
return Err(AppError::Incomplete(
"too many rows scanned; refine your filters".into(),
));
}
let last = bounties.last().unwrap();
for b in &bounties {
if (allowed.len() as i64) >= limit {
break;
}
let key = format!("{}/{}", b.repo_owner, b.repo_name);
let admitted = if let Some(&result) = memo.get(&key) {
result
} else {
let result = crate::api::authorize_repo_read(
&state,
&b.repo_owner,
&b.repo_name,
caller,
"/",
)
.await
.is_ok();
memo.insert(key, result);
result
};
if admitted {
allowed.push(b.clone());
}
}
cursor = Some((last.created_at.clone(), last.id.clone()));
}
Ok(Json(serde_json::json!({ "bounties": allowed })))
}
/// GET /api/v1/bounties/{id}
pub async fn get_bounty(
State(state): State<AppState>,
Path(id): Path<String>,
auth: Option<Extension<AuthenticatedDid>>,
) -> Result<Json<BountyRecord>> {
let bounty = state
.db
.get_bounty(&id)
.await?
.ok_or_else(|| AppError::NotFound(format!("bounty {id} not found")))?;
let not_found = || AppError::NotFound(format!("bounty {id} not found"));
let bounty = state.db.get_bounty(&id).await?.ok_or_else(not_found)?;
let caller = auth.as_ref().map(|e| e.0 .0.as_str());
if crate::api::authorize_repo_read(&state, &bounty.repo_owner, &bounty.repo_name, caller, "/")
.await
.is_err()
{
return Err(not_found());
}
Ok(Json(bounty))
}
@@ -162,11 +238,22 @@ pub async fn claim_bounty(
Path(id): Path<String>,
Json(req): Json<ClaimBountyRequest>,
) -> Result<Json<BountyRecord>> {
let bounty = state
.db
.get_bounty(&id)
.await?
.ok_or_else(|| AppError::NotFound(format!("bounty {id} not found")))?;
let not_found = || AppError::NotFound(format!("bounty {id} not found"));
let bounty = state.db.get_bounty(&id).await?.ok_or_else(not_found)?;
if crate::api::authorize_repo_read(
&state,
&bounty.repo_owner,
&bounty.repo_name,
Some(auth.0.as_str()),
"/",
)
.await
.is_err()
{
return Err(not_found());
}
if bounty.status != "open" {
return Err(AppError::BadRequest(format!(
+41 -15
View File
@@ -1,23 +1,44 @@
//! API handlers for ref certificates.
use axum::extract::{Path, State};
use std::collections::HashMap;
use axum::extract::{Extension, Path, Query, State};
use axum::Json;
use crate::auth::AuthenticatedDid;
use crate::error::{AppError, Result};
use crate::state::AppState;
/// GET /api/v1/repos/{owner}/{repo}/certs
/// GET /api/v1/repos/{owner}/{repo}/certs?limit=50
pub async fn list_certs(
State(state): State<AppState>,
Path((owner, name)): Path<(String, String)>,
Query(params): Query<HashMap<String, String>>,
auth: Option<Extension<AuthenticatedDid>>,
) -> Result<Json<serde_json::Value>> {
let record = state
.db
.get_repo(&owner, &name)
.await?
.ok_or_else(|| AppError::RepoNotFound(format!("{owner}/{name}")))?;
let limit = params
.get("limit")
.and_then(|v| v.parse::<i64>().ok())
.map(|v| v.max(1))
.unwrap_or(50)
.min(200);
let certs = state.db.list_ref_certificates(&record.id).await?;
let caller = auth.as_ref().map(|e| e.0 .0.as_str());
let (record, _rules) =
crate::api::authorize_repo_read(&state, &owner, &name, caller, "/").await?;
// When a prefix is given (short-ID resolution from the CLI) use a
// generous limit and delegate to the prefix-matched query so the
// caller can resolve IDs regardless of how many certs exist.
let prefix = params.get("prefix").filter(|p| !p.is_empty());
let certs = if let Some(prefix) = prefix {
state
.db
.list_ref_certificates_by_prefix(&record.id, prefix, 200)
.await?
} else {
state.db.list_ref_certificates(&record.id, limit).await?
};
let certs_json: Vec<serde_json::Value> = certs
.iter()
.map(|c| {
@@ -35,20 +56,21 @@ pub async fn list_certs(
})
.collect();
Ok(Json(serde_json::json!({ "certificates": certs_json })))
let count = certs_json.len();
Ok(Json(
serde_json::json!({ "certificates": certs_json, "count": count }),
))
}
/// GET /api/v1/repos/{owner}/{repo}/certs/{id}
pub async fn get_cert(
State(state): State<AppState>,
Path((owner, name, id)): Path<(String, String, String)>,
auth: Option<Extension<AuthenticatedDid>>,
) -> Result<Json<serde_json::Value>> {
// Verify the repo exists
let _record = state
.db
.get_repo(&owner, &name)
.await?
.ok_or_else(|| AppError::RepoNotFound(format!("{owner}/{name}")))?;
let caller = auth.as_ref().map(|e| e.0 .0.as_str());
let (record, _rules) =
crate::api::authorize_repo_read(&state, &owner, &name, caller, "/").await?;
let cert = state
.db
@@ -56,6 +78,10 @@ pub async fn get_cert(
.await?
.ok_or_else(|| AppError::NotFound(format!("certificate {id}")))?;
if cert.repo_id != record.id {
return Err(AppError::NotFound(format!("certificate {id}")));
}
Ok(Json(serde_json::json!({
"id": cert.id,
"repo_id": cert.repo_id,
File diff suppressed because it is too large. Load diff
+80 -47
View File
@@ -27,7 +27,7 @@ use std::str::FromStr;
use crate::auth::AuthenticatedDid;
use crate::error::{AppError, Result};
use crate::git::store;
use crate::git::visibility_pack::{has_path_scoped_rule, withheld_blob_oids};
use crate::git::visibility_pack::{allowed_blob_set_for_caller, has_path_scoped_rule};
use crate::state::AppState;
use crate::visibility::{visibility_check, Decision};
@@ -36,16 +36,24 @@ use crate::visibility::{visibility_check, Decision};
/// Search all repos on the node for a git object whose SHA-256 hash matches
/// the given CIDv1, returning its raw content if the caller may read it.
///
/// Visibility (#110): the object is served only from a repo row the caller
/// passes. For each iterated row we gate against that row's OWN rules
/// Visibility (#110, #126): the object is served only from a repo row the
/// caller passes. For each iterated row we gate against that row's OWN rules
/// (`visibility_check` at `"/"`), never re-resolving via `authorize_repo_read`
/// — `get_repo`'s fuzzy match could otherwise authorize a different physical
/// row than the one read (KTD2a). When the row carries path-scoped rules, a
/// blob withheld from the caller (`withheld_blob_oids`) is skipped. Denial and
/// genuine not-found both fall through to an opaque 404.
/// row than the one read (KTD2a). We check object existence via
/// `store::object_type` *before* the expensive reachability walk so random-CID
/// spray cannot trigger full-history git walks on repos that don't carry the
/// object. When the row carries path-scoped rules (KTD4) the served object
/// must be either a non-blob (trees/commits are structural; KTD3) OR a blob
/// in the caller's *reachable* allowed-set (`allowed_blob_set_for_caller`).
/// The reachable allowed-set excludes dangling blobs — a blob written via
/// `git hash-object -w` and never committed has no path to gate, so it is
/// fail-closed 404'd under path-scoped rules (#126). Denial and genuine
/// not-found both fall through to an opaque 404.
///
/// Scope: this closes the direct unauthenticated scan. A stale-public mirror
/// row still serves withheld content (tracked separately, #124).
/// Scope: this closes the direct unauthenticated scan, including the dangling
/// case. A stale-public mirror row still serves withheld content (tracked
/// separately, #124).
pub async fn get_by_cid(
Path(cid_str): Path<String>,
State(state): State<AppState>,
@@ -85,11 +93,16 @@ pub async fn get_by_cid(
.await
.map_err(AppError::Internal)?;
// Request-scoped memo of the per-repo withheld set (KTD1). The caller is
// constant for one request, so `repo.id` alone is a safe, sufficient key —
// never a coarse caller "class", which `visibility_check`'s exact full-DID
// reader match would make unsafe.
let mut withheld_memo: HashMap<String, HashSet<String>> = HashMap::new();
// Request-scoped memo of the per-repo allowed-blob set (KTD1, #126). The
// caller is constant for one request, so `repo.id` alone is a safe,
// sufficient key — never a coarse caller "class", which
// `visibility_check`'s exact full-DID reader match would make unsafe.
//
// We flipped from a deny-set (`withheld_blob_oids`) to an allowed-set
// (`allowed_blob_set_for_caller`) so dangling blobs — never enumerated by
// the reachable walk — fail closed instead of slipping through an empty
// deny entry (#126).
let mut allowed_memo: HashMap<String, HashSet<String>> = HashMap::new();
for repo in &repos {
// Repo-level read gate against THIS row's own rules (KTD2a).
@@ -106,9 +119,24 @@ pub async fn get_by_cid(
Err(_) => continue,
};
// Per-blob withholding only applies when a path-scoped rule exists (KTD4).
if has_path_scoped_rule(rules) {
if !withheld_memo.contains_key(&repo.id) {
// Check whether the object exists in this repo before any expensive
// reachability walk. This prevents random-CID spray from triggering
// full-history git walks on repos that don't carry the object.
let obj_type = match store::object_type(&repo_path, &sha256_hex) {
Ok(Some(t)) => t,
Ok(None) => continue,
Err(e) => {
tracing::warn!(repo = %repo.name, err = %e, "error checking git object type");
continue;
}
};
// Per-blob gating only applies when a path-scoped rule exists (KTD4).
// Without any path-scoped rule, the "/" gate above is the whole story.
// Trees/commits are always served under path-scoped rules (KTD3).
let path_scoped = has_path_scoped_rule(rules);
if path_scoped && obj_type == "blob" {
if !allowed_memo.contains_key(&repo.id) {
let rp = repo_path.clone();
let r = rules.to_vec();
let is_public = repo.is_public;
@@ -116,7 +144,13 @@ pub async fn get_by_cid(
let caller_for_walk = caller_owned.clone();
// Full-history walk shells out to git — keep it off the async runtime.
let walk = tokio::task::spawn_blocking(move || {
withheld_blob_oids(&rp, &r, is_public, &owner, caller_for_walk.as_deref())
allowed_blob_set_for_caller(
&rp,
&r,
is_public,
&owner,
caller_for_walk.as_deref(),
)
})
.await;
// Fail closed on EITHER a task panic (JoinError) or a walk error:
@@ -125,51 +159,50 @@ pub async fn get_by_cid(
let set = match walk {
Ok(Ok(set)) => set,
Ok(Err(e)) => {
tracing::warn!(repo = %repo.name, err = %e, "withheld walk failed; skipping repo");
tracing::warn!(repo = %repo.name, err = %e, "allowed-blob walk failed; skipping repo");
continue;
}
Err(e) => {
tracing::warn!(repo = %repo.name, err = %e, "withheld walk task panicked; skipping repo");
tracing::warn!(repo = %repo.name, err = %e, "allowed-blob walk task panicked; skipping repo");
continue;
}
};
withheld_memo.insert(repo.id.clone(), set);
allowed_memo.insert(repo.id.clone(), set);
}
if withheld_memo
let in_allowed = allowed_memo
.get(&repo.id)
.is_some_and(|set| set.contains(&sha256_hex))
{
.is_some_and(|set| set.contains(&sha256_hex));
if !in_allowed {
continue;
}
}
match store::read_object(&repo_path, &sha256_hex) {
Ok(Some((_obj_type, content))) => {
// 3. Return the content with IPFS-style headers
let mut headers = HeaderMap::new();
headers.insert(
HeaderName::from_static("content-type"),
HeaderValue::from_static("application/octet-stream"),
);
headers.insert(
HeaderName::from_static("x-content-cid"),
HeaderValue::from_str(&cid_str)
.unwrap_or_else(|_| HeaderValue::from_static("invalid")),
);
headers.insert(
HeaderName::from_static("x-git-hash"),
HeaderValue::from_str(&sha256_hex)
.unwrap_or_else(|_| HeaderValue::from_static("invalid")),
);
return Ok((StatusCode::OK, headers, content).into_response());
}
Ok(None) => continue,
// Now that we've passed the gate, read the content.
let content = match store::read_object_content(&repo_path, &sha256_hex, &obj_type) {
Ok(c) => c,
Err(e) => {
tracing::warn!(repo = %repo.name, err = %e, "error reading git object");
tracing::warn!(repo = %repo.name, err = %e, "error reading git object content");
continue;
}
}
};
// 3. Return the content with IPFS-style headers
let mut headers = HeaderMap::new();
headers.insert(
HeaderName::from_static("content-type"),
HeaderValue::from_static("application/octet-stream"),
);
headers.insert(
HeaderName::from_static("x-content-cid"),
HeaderValue::from_str(&cid_str).unwrap_or_else(|_| HeaderValue::from_static("invalid")),
);
headers.insert(
HeaderName::from_static("x-git-hash"),
HeaderValue::from_str(&sha256_hex)
.unwrap_or_else(|_| HeaderValue::from_static("invalid")),
);
return Ok((StatusCode::OK, headers, content).into_response());
}
// Not found in any repo
+31 -16
View File
@@ -90,12 +90,11 @@ pub async fn create_issue(
pub async fn list_issues(
State(state): State<AppState>,
Path((owner, repo)): Path<(String, String)>,
auth: Option<Extension<AuthenticatedDid>>,
) -> Result<Json<serde_json::Value>> {
let record = state
.db
.get_repo(&owner, &repo)
.await?
.ok_or_else(|| AppError::RepoNotFound(format!("{owner}/{repo}")))?;
let caller = auth.as_ref().map(|e| e.0 .0.as_str());
let (record, _rules) =
crate::api::authorize_repo_read(&state, &owner, &repo, caller, "/").await?;
let disk_path = state
.repo_store
@@ -120,12 +119,11 @@ pub async fn list_issues(
pub async fn get_issue(
State(state): State<AppState>,
Path((owner, repo, issue_id)): Path<(String, String, String)>,
auth: Option<Extension<AuthenticatedDid>>,
) -> Result<Json<serde_json::Value>> {
let record = state
.db
.get_repo(&owner, &repo)
.await?
.ok_or_else(|| AppError::RepoNotFound(format!("{owner}/{repo}")))?;
let caller = auth.as_ref().map(|e| e.0 .0.as_str());
let (record, _rules) =
crate::api::authorize_repo_read(&state, &owner, &repo, caller, "/").await?;
let disk_path = state
.repo_store
@@ -191,14 +189,31 @@ pub async fn create_issue_comment(
pub async fn list_issue_comments(
State(state): State<AppState>,
Path((owner, repo, issue_id)): Path<(String, String, String)>,
auth: Option<Extension<AuthenticatedDid>>,
) -> Result<Json<serde_json::Value>> {
let _record = state
.db
.get_repo(&owner, &repo)
.await?
.ok_or_else(|| AppError::RepoNotFound(format!("{owner}/{repo}")))?;
let caller = auth.as_ref().map(|e| e.0 .0.as_str());
let (record, _rules) =
crate::api::authorize_repo_read(&state, &owner, &repo, caller, "/").await?;
let comments = state.db.list_issue_comments(&issue_id).await?;
let disk_path = state
.repo_store
.acquire(&record.owner_did, &record.name)
.await
.map_err(|e| AppError::Git(e.to_string()))?;
// Resolve the full issue ID (accepts 8-char prefix) so the DB fetch
// below uses the same canonical id as the git ref.
let full_id = match git_issues::resolve_issue_id(&disk_path, &issue_id)
.map_err(|e| AppError::Git(e.to_string()))?
{
Some(id) => id,
None => {
return Err(AppError::RepoNotFound(format!(
"issue {issue_id} not found"
)))
}
};
let comments = state.db.list_issue_comments(&full_id).await?;
Ok(Json(serde_json::json!({ "comments": comments })))
}
+4 -5
View File
@@ -74,12 +74,11 @@ pub async fn remove_label(
pub async fn list_labels(
State(state): State<AppState>,
Path((owner, name)): Path<(String, String)>,
auth: Option<Extension<AuthenticatedDid>>,
) -> Result<Json<serde_json::Value>> {
let record = state
.db
.get_repo(&owner, &name)
.await?
.ok_or_else(|| AppError::RepoNotFound(format!("{owner}/{name}")))?;
let caller = auth.as_ref().map(|e| e.0 .0.as_str());
let (record, _rules) =
crate::api::authorize_repo_read(&state, &owner, &name, caller, "/").await?;
let labels = state.db.list_labels(&record.id).await?;
Ok(Json(serde_json::json!({ "labels": labels })))
+122 -20
View File
@@ -114,6 +114,12 @@ mod did_tests {
assert!(!did_matches("did:key:zABC", "did:key:zXYZ"));
assert!(!did_matches("zABC", "zXYZ"));
}
#[test]
fn empty_did_matches() {
assert!(did_matches("", ""));
assert!(did_matches("", "did:key:"));
}
}
/// Drift guard (plan 002 §Gate-type table, Step 5). Every in-scope mutation
@@ -324,6 +330,118 @@ mod authz_guard {
})
}
/// Collect `.rs` source files under `dir`. Recursive so the completeness scan
/// covers nested API modules (`api/<module>/mod.rs` and deeper), not only the
/// immediate `api/*.rs` children.
fn collect_rs_files(dir: &std::path::Path) -> Vec<std::path::PathBuf> {
let mut out = Vec::new();
for entry in std::fs::read_dir(dir).expect("read api dir") {
let path = entry.expect("dir entry").path();
if path.is_dir() {
out.extend(collect_rs_files(&path));
} else if path.extension().is_some_and(|e| e == "rs") {
out.push(path);
}
}
out
}
/// The `.rs` files under `api_root` (as paths RELATIVE to it) that the
/// completeness scan must inspect: everything except the top-level `mod.rs` (the
/// guard file itself) and the top-level files already covered by the per-handler
/// `sources` loop. A nested `api/<module>/<name>.rs` is a distinct source file
/// even when its basename matches a listed top-level file, so it stays in scope.
fn unlisted_source_files(
api_root: &std::path::Path,
listed: &std::collections::HashSet<&str>,
) -> Vec<String> {
collect_rs_files(api_root)
.iter()
.filter_map(|path| {
let rel = path
.strip_prefix(api_root)
.ok()?
.to_string_lossy()
.replace('\\', "/");
if rel == "mod.rs" || listed.contains(rel.as_str()) {
None
} else {
Some(rel)
}
})
.collect()
}
#[test]
fn collect_rs_files_recurses_subdirs() {
let tmp = tempfile::TempDir::new().unwrap();
let root = tmp.path();
std::fs::write(root.join("a.rs"), "").unwrap();
std::fs::write(root.join("note.txt"), "").unwrap();
std::fs::create_dir_all(root.join("sub/deep")).unwrap();
std::fs::write(root.join("sub/mod.rs"), "").unwrap();
std::fs::write(root.join("sub/deep/c.rs"), "").unwrap();
let names: std::collections::HashSet<String> = collect_rs_files(root)
.iter()
.map(|p| {
p.strip_prefix(root)
.unwrap()
.to_string_lossy()
.replace('\\', "/")
})
.collect();
assert!(names.contains("a.rs"));
assert!(
names.contains("sub/mod.rs"),
"nested module file must be collected"
);
assert!(
names.contains("sub/deep/c.rs"),
"deeply nested file must be collected"
);
assert!(
!names.iter().any(|n| n.ends_with(".txt")),
"non-rs files excluded"
);
assert_eq!(names.len(), 3);
}
// P3 (#119): the completeness scan must skip already-covered files by their path
// RELATIVE to api_root, not by basename. A nested api/<module>/repos.rs is a
// distinct source file from the covered top-level repos.rs and must still be
// scanned, or a new nested module could smuggle in an ungated repo-scoped handler
// behind a colliding filename.
#[test]
fn unlisted_source_files_scans_nested_file_with_colliding_basename() {
let tmp = tempfile::TempDir::new().unwrap();
let root = tmp.path();
std::fs::write(root.join("repos.rs"), "").unwrap();
std::fs::write(root.join("mod.rs"), "").unwrap();
std::fs::create_dir_all(root.join("sub")).unwrap();
std::fs::write(root.join("sub/repos.rs"), "").unwrap();
std::fs::write(root.join("sub/fresh.rs"), "").unwrap();
let listed: std::collections::HashSet<&str> = ["repos.rs"].into_iter().collect();
let unlisted = unlisted_source_files(root, &listed);
assert!(
unlisted.contains(&"sub/repos.rs".to_string()),
"a nested file whose basename matches a listed top-level file must still be scanned"
);
assert!(
unlisted.contains(&"sub/fresh.rs".to_string()),
"a nested file with a unique name must be scanned"
);
assert!(
!unlisted.contains(&"repos.rs".to_string()),
"the listed top-level file is covered by the per-handler loop"
);
assert!(
!unlisted.contains(&"mod.rs".to_string()),
"the top-level guard file is skipped"
);
}
/// Egress gate guard: every repo-scoped handler (`Path<(String, String..)>`)
/// must carry an authz marker — a read gate (`authorize_repo_read` /
/// `visibility_check`), or a write gate (`require_repo_owner` / `require_owner`
@@ -376,19 +494,15 @@ mod authz_guard {
"read-guard `sources` lists {f} but the file does not exist"
);
}
for entry in std::fs::read_dir(api_dir).expect("read src/api") {
let path = entry.expect("dir entry").path();
let fname = path.file_name().unwrap().to_string_lossy().into_owned();
if !fname.ends_with(".rs") || fname == "mod.rs" || listed.contains(fname.as_str()) {
continue;
}
let src = std::fs::read_to_string(&path).expect("read api file");
let api_root = std::path::Path::new(api_dir);
for rel in unlisted_source_files(api_root, &listed) {
let src = std::fs::read_to_string(api_root.join(&rel)).expect("read api file");
let has_repo_handler = handler_names(&src)
.iter()
.any(|n| is_repo_scoped(&fn_body(&src, n)));
assert!(
!has_repo_handler,
"api/{fname} declares a repo-scoped handler but is not in the egress \
"api/{rel} declares a repo-scoped handler but is not in the egress \
guard `sources` list — add it so its handlers are gate-checked"
);
}
@@ -396,18 +510,6 @@ mod authz_guard {
// Repo-scoped reads known to be ungated today, each tracked by an issue.
// Remove an entry the moment its gate lands (the staleness assert enforces it).
let known_ungated: &[(&str, &str)] = &[
// info/refs gates only git-upload-pack today; git-receive-pack
// advertisement is ungated until #119 makes the gate unconditional.
("git_info_refs", "#119"),
("list_certs", "#120"),
("get_cert", "#120"),
("list_issues", "#120"),
("get_issue", "#120"),
("list_issue_comments", "#120"),
("list_labels", "#120"),
("list_repo_bounties", "#120"),
("get_star_status", "#120"),
("list_repo_events", "#94 (PR #113)"),
("list_webhooks", "#94 (PR #113)"),
("list_replicas", "PR #113"),
("list_protected_branches", "PR #113"),
+456 -2
View File
@@ -296,8 +296,8 @@ pub async fn trigger_sync(State(state): State<AppState>) -> Result<Json<serde_js
repo.get("name").and_then(|v| v.as_str()),
) {
(Some(owner), Some(name)) => {
// Use short owner (last colon segment) matching DB convention
let short = owner.split(':').next_back().unwrap_or(owner);
// Use short owner (did:key-only normalization) matching DB convention
let short = crate::db::normalize_owner_key(owner);
format!("{short}/{name}")
}
_ => continue,
@@ -537,4 +537,458 @@ mod tests {
// pins the `o[0] == 0` check against an off-by-one into 1.x.
assert!(is_public_http_url("http://1.0.0.0/"));
}
// ── #82: /sync/trigger signature gate + per-IP brake on the peer-sync routes ──
//
// These drive the FULL production router (crate::server::build_router) so the
// route wiring, layer order, and config-mode branching are all under test.
// The positive-path and DID-agnostic cases mount the handler directly because
// signed_request_as injects only an AuthenticatedDid extension (not a real
// RFC-9421 signature), which require_signature rejects on the full router.
use crate::rate_limit::{IpRateLimiter, RateLimiter, TrustedProxy};
use crate::state::AppState;
use crate::test_support::{signed_request_as, test_state};
use axum::body::Body;
use axum::extract::ConnectInfo;
use axum::http::{header, Method, Request, StatusCode};
use axum::routing::post;
use axum::{middleware, Extension, Router};
use gitlawb_core::http_sig::sign_request;
use gitlawb_core::identity::Keypair;
use sqlx::PgPool;
use std::net::SocketAddr;
use std::sync::Arc;
use std::time::Duration;
use tower::ServiceExt;
fn unsigned_post(uri: &str, body: &str, peer: &str) -> Request<Body> {
let mut req = Request::builder()
.method(Method::POST)
.uri(uri)
.header(header::CONTENT_TYPE, "application/json")
.body(Body::from(body.to_string()))
.unwrap();
req.extensions_mut()
.insert(ConnectInfo(peer.parse::<SocketAddr>().unwrap()));
req
}
fn require_signed_peer_writes(state: &mut AppState) {
let mut cfg = (*state.config).clone();
cfg.require_signed_peer_writes = true;
state.config = Arc::new(cfg);
}
const NOTIFY_BODY: &str = r#"{"repo":"demo","ref_name":"refs/heads/main","new_sha":"0000000000000000000000000000000000000000","node_did":"PEER_DID"}"#;
// ── trigger: mandatory signature ──────────────────────────────────────────
#[sqlx::test]
async fn sync_trigger_rejects_unsigned_in_default_mode(pool: PgPool) {
// The hole: with require_signed_peer_writes=false (default), an anonymous
// caller reaches the fan-out. Must be 401 regardless of the flag.
let state = test_state(pool).await;
let router = crate::server::build_router(state);
let resp = router
.oneshot(unsigned_post(
"/api/v1/sync/trigger",
"{}",
"203.0.113.1:5000",
))
.await
.unwrap();
assert_eq!(resp.status(), StatusCode::UNAUTHORIZED);
}
#[sqlx::test]
async fn sync_trigger_rejects_unsigned_in_signed_writes_mode(pool: PgPool) {
// The signature requirement must not depend on the flag.
let mut state = test_state(pool).await;
require_signed_peer_writes(&mut state);
let router = crate::server::build_router(state);
let resp = router
.oneshot(unsigned_post(
"/api/v1/sync/trigger",
"{}",
"203.0.113.2:5000",
))
.await
.unwrap();
assert_eq!(resp.status(), StatusCode::UNAUTHORIZED);
}
#[sqlx::test]
async fn sync_trigger_admits_signed_caller(pool: PgPool) {
// must-not-over-throttle: a signed caller reaches the handler and (with no
// peers seeded) gets 200, not 429. Direct-mount because a real signature
// cannot be forged through the full router in tests.
let state = test_state(pool).await;
let did = Keypair::generate().did().to_string();
let app = Router::new()
.route("/api/v1/sync/trigger", post(super::trigger_sync))
.layer(middleware::from_fn(crate::rate_limit::rate_limit_by_ip))
.layer(Extension(IpRateLimiter {
limiter: RateLimiter::new(60, Duration::from_secs(3600)),
trust: TrustedProxy::None,
}))
.with_state(state);
let mut req =
signed_request_as(&did, Method::POST, "/api/v1/sync/trigger", Body::from("{}"));
req.extensions_mut().insert(ConnectInfo(
"203.0.113.3:5000".parse::<SocketAddr>().unwrap(),
));
let resp = app.oneshot(req).await.unwrap();
assert_eq!(resp.status(), StatusCode::OK);
}
// ── trigger: IP brake ─────────────────────────────────────────────────────
#[sqlx::test]
async fn sync_trigger_ip_flood_is_throttled(pool: PgPool) {
// The brake is outermost, so an over-limit request 429s before it ever
// reaches require_signature — an unsigned request suffices to prove it.
let mut state = test_state(pool).await;
state.sync_trigger_rate_limiter = RateLimiter::new(1, Duration::from_secs(3600));
state.push_limiter_trust = TrustedProxy::None;
let peer = "203.0.113.4:5000";
// Exhaust the single-request budget up front.
assert!(
state
.sync_trigger_rate_limiter
.check(&peer.parse::<SocketAddr>().unwrap().ip().to_string())
.await
);
let router = crate::server::build_router(state);
let resp = router
.oneshot(unsigned_post("/api/v1/sync/trigger", "{}", peer))
.await
.unwrap();
assert_eq!(resp.status(), StatusCode::TOO_MANY_REQUESTS);
}
#[sqlx::test]
async fn sync_trigger_brake_is_did_agnostic(pool: PgPool) {
// Why per-IP, not per-DID: a DID farm (fresh did:key per request) does not
// bypass an IP-keyed brake. Two DISTINCT DIDs from one IP, brake limit 1 →
// the second is 429. A per-DID limiter would give each DID its own bucket.
let state = test_state(pool).await;
let app = Router::new()
.route("/api/v1/sync/trigger", post(super::trigger_sync))
.layer(middleware::from_fn(crate::rate_limit::rate_limit_by_ip))
.layer(Extension(IpRateLimiter {
limiter: RateLimiter::new(1, Duration::from_secs(3600)),
trust: TrustedProxy::None,
}))
.with_state(state);
let peer = "203.0.113.5:5000".parse::<SocketAddr>().unwrap();
let mk = |did: &str| {
let mut r =
signed_request_as(did, Method::POST, "/api/v1/sync/trigger", Body::from("{}"));
r.extensions_mut().insert(ConnectInfo(peer));
r
};
let first = app
.clone()
.oneshot(mk(&Keypair::generate().did().to_string()))
.await
.unwrap();
assert_eq!(first.status(), StatusCode::OK);
let second = app
.oneshot(mk(&Keypair::generate().did().to_string()))
.await
.unwrap();
assert_eq!(second.status(), StatusCode::TOO_MANY_REQUESTS);
}
#[sqlx::test]
async fn sync_trigger_forged_forwarded_header_cannot_bypass(pool: PgPool) {
// TrustedProxy::None keys on the socket peer, so rotating X-Forwarded-For
// from one socket peer does not refill the bucket. limit 1: first request
// consumes it (then 401 unsigned), second from the same socket → 429
// regardless of a different XFF value.
let mut state = test_state(pool).await;
state.sync_trigger_rate_limiter = RateLimiter::new(1, Duration::from_secs(3600));
state.push_limiter_trust = TrustedProxy::None;
let router = crate::server::build_router(state);
let mut a = unsigned_post("/api/v1/sync/trigger", "{}", "203.0.113.6:5000");
a.headers_mut()
.insert("x-forwarded-for", "1.1.1.1".parse().unwrap());
assert_eq!(
router.clone().oneshot(a).await.unwrap().status(),
StatusCode::UNAUTHORIZED
);
let mut b = unsigned_post("/api/v1/sync/trigger", "{}", "203.0.113.6:5000");
b.headers_mut()
.insert("x-forwarded-for", "2.2.2.2".parse().unwrap());
assert_eq!(
router.oneshot(b).await.unwrap().status(),
StatusCode::TOO_MANY_REQUESTS
);
}
#[sqlx::test]
async fn sync_trigger_rate_limit_zero_disables_but_signature_holds(pool: PgPool) {
// 0 disables the brake (RateLimiter::check early-returns), proving the two
// halves are independent: no 429 even under a flood, but the signature gate
// still 401s every unsigned request.
let mut state = test_state(pool).await;
state.sync_trigger_rate_limiter = RateLimiter::new(0, Duration::from_secs(3600));
state.push_limiter_trust = TrustedProxy::None;
let router = crate::server::build_router(state);
for _ in 0..3 {
let resp = router
.clone()
.oneshot(unsigned_post(
"/api/v1/sync/trigger",
"{}",
"203.0.113.7:5000",
))
.await
.unwrap();
assert_eq!(resp.status(), StatusCode::UNAUTHORIZED);
}
}
// ── notify: braked, but signature behavior unchanged ──────────────────────
async fn seed_peer(state: &AppState) -> String {
let did = Keypair::generate().did().to_string();
state
.db
.upsert_peer(&did, "https://peer.example.com")
.await
.unwrap();
did
}
#[sqlx::test]
async fn sync_notify_unsigned_flood_is_throttled(pool: PgPool) {
// notify still accepts an unsigned known-peer notification (rolling-upgrade
// compat), but a flood from one IP is now braked. brake limit 1: first
// enqueues (200), second from the same IP → 429.
let mut state = test_state(pool).await;
state.peer_write_rate_limiter = RateLimiter::new(1, Duration::from_secs(3600));
state.push_limiter_trust = TrustedProxy::None;
let peer_did = seed_peer(&state).await;
let body = NOTIFY_BODY.replace("PEER_DID", &peer_did);
let router = crate::server::build_router(state);
let first = router
.clone()
.oneshot(unsigned_post(
"/api/v1/sync/notify",
&body,
"203.0.113.8:5000",
))
.await
.unwrap();
assert_eq!(first.status(), StatusCode::OK);
let second = router
.oneshot(unsigned_post(
"/api/v1/sync/notify",
&body,
"203.0.113.8:5000",
))
.await
.unwrap();
assert_eq!(second.status(), StatusCode::TOO_MANY_REQUESTS);
}
#[sqlx::test]
async fn sync_notify_distinct_ips_are_not_collaterally_throttled(pool: PgPool) {
// must-not-over-throttle: per-IP keying means one peer's volume does not
// throttle another. brake limit 1, two DIFFERENT source IPs → both 200.
let mut state = test_state(pool).await;
state.peer_write_rate_limiter = RateLimiter::new(1, Duration::from_secs(3600));
state.push_limiter_trust = TrustedProxy::None;
let peer_did = seed_peer(&state).await;
let body = NOTIFY_BODY.replace("PEER_DID", &peer_did);
let router = crate::server::build_router(state);
for ip in ["203.0.113.20:5000", "203.0.113.21:5000"] {
let resp = router
.clone()
.oneshot(unsigned_post("/api/v1/sync/notify", &body, ip))
.await
.unwrap();
assert_eq!(
resp.status(),
StatusCode::OK,
"distinct IP {ip} must not be throttled"
);
}
}
#[sqlx::test]
async fn notify_flood_does_not_exhaust_trigger_bucket(pool: PgPool) {
// Separate buckets: draining peer_write via an unsigned notify flood must
// NOT throttle the signed trigger caller from the same IP. After the notify
// bucket is spent, an unsigned trigger from that IP hits its own (unspent)
// bucket and is rejected by the SIGNATURE gate (401), not the brake (429).
let mut state = test_state(pool).await;
state.peer_write_rate_limiter = RateLimiter::new(1, Duration::from_secs(3600));
state.sync_trigger_rate_limiter = RateLimiter::new(60, Duration::from_secs(3600));
state.push_limiter_trust = TrustedProxy::None;
let peer_did = seed_peer(&state).await;
let body = NOTIFY_BODY.replace("PEER_DID", &peer_did);
let router = crate::server::build_router(state);
let ip = "203.0.113.9:5000";
assert_eq!(
router
.clone()
.oneshot(unsigned_post("/api/v1/sync/notify", &body, ip))
.await
.unwrap()
.status(),
StatusCode::OK
);
assert_eq!(
router
.clone()
.oneshot(unsigned_post("/api/v1/sync/notify", &body, ip))
.await
.unwrap()
.status(),
StatusCode::TOO_MANY_REQUESTS,
"peer_write bucket should now be exhausted for this IP"
);
let trigger = router
.oneshot(unsigned_post("/api/v1/sync/trigger", "{}", ip))
.await
.unwrap();
assert_eq!(
trigger.status(),
StatusCode::UNAUTHORIZED,
"trigger must hit its own bucket (401 from the sig gate), not the drained peer_write bucket (429)"
);
}
#[sqlx::test]
async fn announce_still_accepts_unsigned_in_default_mode(pool: PgPool) {
// must-not-over-reach: adding the brake must not tighten announce's
// rolling-upgrade behavior — an unsigned announce with a public URL still
// succeeds (it only gains the brake, which a single request is under).
let state = test_state(pool).await;
let did = Keypair::generate().did().to_string();
let body = format!(r#"{{"did":"{did}","http_url":"https://peer.example.com"}}"#);
let router = crate::server::build_router(state);
let resp = router
.oneshot(unsigned_post(
"/api/v1/peers/announce",
&body,
"203.0.113.10:5000",
))
.await
.unwrap();
assert_eq!(resp.status(), StatusCode::OK);
}
// Build a request carrying a REAL RFC 9421 signature (not just an injected
// extension), so it passes require_signature on the full production router.
fn real_signed_trigger(peer: &str) -> Request<Body> {
let kp = Keypair::generate();
let body = b"{}";
let s = sign_request(&kp, "POST", "/api/v1/sync/trigger", body);
let mut req = Request::builder()
.method(Method::POST)
.uri("/api/v1/sync/trigger")
.header(header::CONTENT_TYPE, "application/json")
.header("content-digest", s.content_digest)
.header("signature-input", s.signature_input)
.header("signature", s.signature)
.body(Body::from(body.to_vec()))
.unwrap();
req.extensions_mut()
.insert(ConnectInfo(peer.parse::<SocketAddr>().unwrap()));
req
}
#[sqlx::test]
async fn sync_trigger_admits_real_signature_through_full_router(pool: PgPool) {
// The positive path through the REAL gate: a validly-signed trigger passes
// require_signature on the full router and reaches the handler (no peers
// seeded → 200), in BOTH config modes.
let state = test_state(pool.clone()).await;
let resp = crate::server::build_router(state)
.oneshot(real_signed_trigger("203.0.113.30:5000"))
.await
.unwrap();
assert_eq!(
resp.status(),
StatusCode::OK,
"a real signature must reach the handler (default mode)"
);
let mut state = test_state(pool).await;
require_signed_peer_writes(&mut state);
let resp = crate::server::build_router(state)
.oneshot(real_signed_trigger("203.0.113.30:5001"))
.await
.unwrap();
assert_eq!(
resp.status(),
StatusCode::OK,
"a real signature must reach the handler (signed-writes mode)"
);
}
#[sqlx::test]
async fn announce_flood_is_throttled(pool: PgPool) {
// The peer_write brake covers /peers/announce too (co-benefit): limit 1,
// first unsigned announce from an IP is 200, the second from that IP 429.
let mut state = test_state(pool).await;
state.peer_write_rate_limiter = RateLimiter::new(1, Duration::from_secs(3600));
state.push_limiter_trust = TrustedProxy::None;
let did = Keypair::generate().did().to_string();
let body = format!(r#"{{"did":"{did}","http_url":"https://peer.example.com"}}"#);
let router = crate::server::build_router(state);
let ip = "203.0.113.31:5000";
assert_eq!(
router
.clone()
.oneshot(unsigned_post("/api/v1/peers/announce", &body, ip))
.await
.unwrap()
.status(),
StatusCode::OK
);
assert_eq!(
router
.oneshot(unsigned_post("/api/v1/peers/announce", &body, ip))
.await
.unwrap()
.status(),
StatusCode::TOO_MANY_REQUESTS,
"announce must be braked by the peer_write limiter"
);
}
#[sqlx::test]
async fn sync_trigger_429_body_is_generic_not_push(pool: PgPool) {
// The shared 429 middleware now serves the sync routes, so its body must
// not claim "push" (RED before the message was genericized).
let mut state = test_state(pool).await;
state.sync_trigger_rate_limiter = RateLimiter::new(1, Duration::from_secs(3600));
state.push_limiter_trust = TrustedProxy::None;
let peer = "203.0.113.32:5000";
assert!(
state
.sync_trigger_rate_limiter
.check(&peer.parse::<SocketAddr>().unwrap().ip().to_string())
.await
);
let router = crate::server::build_router(state);
let resp = router
.oneshot(unsigned_post("/api/v1/sync/trigger", "{}", peer))
.await
.unwrap();
assert_eq!(resp.status(), StatusCode::TOO_MANY_REQUESTS);
let bytes = axum::body::to_bytes(resp.into_body(), usize::MAX)
.await
.unwrap();
let s = String::from_utf8_lossy(&bytes);
assert!(s.contains("rate limit exceeded"), "429 body: {s:?}");
assert!(
!s.contains("push"),
"a 429 on a sync route must not say 'push': {s:?}"
);
}
}
+1 -1
View File
@@ -98,7 +98,7 @@ pub async fn set_profile(
&state.http_client,
&state.config.pinata_upload_url,
&state.config.pinata_jwt,
&format!("profile-{}", &did),
&format!("profile-{}", did),
&profile_json,
)
.await
+334 -49
View File
@@ -211,7 +211,12 @@ pub async fn create_repo(
.repo_store
.init(&owner_did, &req.name)
.await
.map_err(|e| AppError::Git(e.to_string()))?;
.map_err(|e| {
// `{:#}` walks the anyhow chain to the leaf cause; the other git
// handlers log their failures, this one didn't.
tracing::error!(owner = %owner_did, repo = %req.name, err = %format!("{e:#}"), "repo create failed");
AppError::Git(e.to_string())
})?;
let now = Utc::now();
let record = crate::db::RepoRecord {
@@ -482,14 +487,27 @@ pub async fn get_tree(
// ── Git smart HTTP endpoints ──────────────────────────────────────────────
/// GET /:owner/:repo.git/info/refs?service=git-upload-pack
fn smart_http_repo_name(repo: &str) -> Result<&str> {
// Strip at most one ".git" suffix: trim_end_matches strips repeatedly,
// which would misdirect a repo literally named "foo.git" (creatable via
// the peer mirror path, which skips API name validation) to repo "foo".
let name = repo.strip_suffix(".git").unwrap_or(repo);
if name.is_empty() {
return Err(AppError::BadRequest("missing repository name".into()));
}
Ok(name)
}
/// GET /:owner/:repo.git/info/refs?service=git-upload-pack|git-receive-pack
pub async fn git_info_refs(
State(state): State<AppState>,
Path((owner, repo)): Path<(String, String)>,
Query(query): Query<InfoRefsQuery>,
crate::rate_limit::PeerAddr(peer): crate::rate_limit::PeerAddr,
headers: axum::http::HeaderMap,
auth: Option<Extension<AuthenticatedDid>>,
) -> Result<Response> {
let name = repo.trim_end_matches(".git");
let name = smart_http_repo_name(&repo)?;
tracing::info!(owner = %owner, repo = %name, "info/refs request");
let record = state
.db
@@ -508,10 +526,14 @@ pub async fn git_info_refs(
.ok_or_else(|| AppError::BadRequest("missing ?service= parameter".into()))?;
tracing::debug!(service = %service, repo = %name, "info/refs service");
// Enforce read (clone/fetch) visibility. The push advertisement
// (service=git-receive-pack) is authorized separately on the
// git-receive-pack POST, so leave it untouched here.
if service == "git-upload-pack" {
// Enforce read visibility on the ref advertisement, for BOTH services. The
// upload-pack (clone/fetch) and receive-pack (push) advertisements expose the
// same ref metadata (branch/tag names and commit tips), so a private repo's
// advertisement must be withheld from a non-reader regardless of which service
// is requested. The push itself stays separately owner-gated on the
// git-receive-pack POST; push access implies read access here, so a
// legitimate pusher (the owner) always clears this gate.
{
let rules = state.db.list_visibility_rules(&record.id).await?;
let caller = auth.as_ref().map(|e| e.0 .0.as_str());
// Subtree (mode B) rules do not gate the advertisement: refs expose commit
@@ -519,11 +541,28 @@ pub async fn git_info_refs(
if visibility_check(&rules, record.is_public, &record.owner_did, caller, "/")
== Decision::Deny
{
tracing::debug!(repo = %name, caller = ?caller, "info/refs read denied by visibility");
tracing::debug!(repo = %name, caller = ?caller, service = %service, "info/refs read denied by visibility");
return Err(AppError::RepoNotFound(format!("{owner}/{name}")));
}
}
// Push flood brake on the advertisement phase. A push always hits this
// GET first, and for receive-pack it forces a fresh Tigris download below;
// throttling only the receive-pack POST would leave the expensive
// fresh-acquire reachable unauthenticated and unlimited. Applied before the
// acquire so a rejected request does no Tigris work. Same per-IP limiter and
// trusted-proxy policy as the POST middleware (shared buckets).
if service == "git-receive-pack" {
if let Some(key) = crate::rate_limit::client_key(&headers, peer, state.push_limiter_trust) {
if !state.push_rate_limiter.check(&key).await {
tracing::warn!(repo = %name, key = %key, "receive-pack advertisement rate limited");
return Err(AppError::TooManyRequests(
"push rate limit exceeded — try again later".into(),
));
}
}
}
// For receive-pack (push), download the latest from Tigris so the client
// sees the same refs that acquire_write() will operate on.
let disk_path = if service == "git-receive-pack" {
@@ -550,6 +589,26 @@ pub async fn git_info_refs(
})
}
/// Map an error from a `smart_http` git service call to the right `AppError`:
/// [`smart_http::GitServiceTimeout`] to 504, a malformed client request to 400,
/// anything else to a 500 git error. Pure (no logging) so it is unit-testable;
/// callers add their own tracing.
fn git_service_app_error(err: &anyhow::Error) -> AppError {
if err
.downcast_ref::<smart_http::GitServiceTimeout>()
.is_some()
{
AppError::Timeout("git service timed out".into())
} else {
let msg = err.to_string();
if msg.contains("bad line length") || msg.contains("protocol error") {
AppError::BadRequest(msg)
} else {
AppError::Git(msg)
}
}
}
/// POST /:owner/:repo.git/git-upload-pack
pub async fn git_upload_pack(
State(state): State<AppState>,
@@ -557,7 +616,7 @@ pub async fn git_upload_pack(
auth: Option<Extension<AuthenticatedDid>>,
body: Bytes,
) -> Result<Response> {
let name = repo.trim_end_matches(".git");
let name = smart_http_repo_name(&repo)?;
let record = state
.db
.get_repo(&owner, name)
@@ -587,8 +646,9 @@ pub async fn git_upload_pack(
// No path-scoped rule can withhold an individual blob, and the whole-repo
// "/" gate above already enforced repo-level access. Skip the per-blob
// withheld walk and serve the pack directly.
let git_timeout = std::time::Duration::from_secs(state.config.git_service_timeout_secs);
let resp = if !visibility_pack::has_path_scoped_rule(&rules) {
smart_http::upload_pack(&disk_path, body).await
smart_http::upload_pack(&disk_path, body, git_timeout).await
} else {
// withheld_blob_oids walks every ref with blocking `git ls-tree`; keep
// that off the async worker thread.
@@ -613,21 +673,22 @@ pub async fn git_upload_pack(
};
if withheld.is_empty() {
smart_http::upload_pack(&disk_path, body).await
smart_http::upload_pack(&disk_path, body, git_timeout).await
} else {
tracing::info!(repo = %name, caller = ?caller, withheld = withheld.len(), "serving filtered pack");
smart_http::upload_pack_excluding(&disk_path, body, &withheld).await
}
}
.map_err(|e| {
let msg = e.to_string();
if msg.contains("bad line length") || msg.contains("protocol error") {
tracing::warn!(repo = %name, err = %msg, "git-upload-pack: bad client request");
AppError::BadRequest(msg)
} else {
tracing::error!(repo = %name, err = %msg, "git-upload-pack failed");
AppError::Git(msg)
let app = git_service_app_error(&e);
match &app {
AppError::Timeout(_) => tracing::warn!(repo = %name, "git-upload-pack timed out"),
AppError::BadRequest(msg) => {
tracing::warn!(repo = %name, err = %msg, "git-upload-pack: bad client request")
}
_ => tracing::error!(repo = %name, err = %e, "git-upload-pack failed"),
}
app
})?;
crate::metrics::record_fetch(&format!("{owner}/{name}"));
crate::metrics::observe_pack_size(body_len as f64);
@@ -792,7 +853,7 @@ pub async fn git_receive_pack(
Extension(auth): Extension<AuthenticatedDid>,
body: Bytes,
) -> Result<Response> {
let name = repo.trim_end_matches(".git");
let name = smart_http_repo_name(&repo)?;
tracing::info!(owner = %owner, repo = %name, "receive-pack request");
let record = state
.db
@@ -874,7 +935,8 @@ pub async fn git_receive_pack(
let disk_path = guard.path().to_path_buf();
tracing::debug!(repo = %name, path = %disk_path.display(), "running git receive-pack");
let body_len = body.len();
let receive_result = smart_http::receive_pack(&disk_path, body).await;
let git_timeout = std::time::Duration::from_secs(state.config.git_service_timeout_secs);
let receive_result = smart_http::receive_pack(&disk_path, body, git_timeout).await;
// Always release the advisory lock — even on error — to prevent stale locks
// from blocking subsequent pushes. Only upload to Tigris when the push
@@ -882,8 +944,15 @@ pub async fn git_receive_pack(
guard.release(receive_result.is_ok()).await;
let result = receive_result.map_err(|e| {
tracing::error!(repo = %name, err = %e, "git receive-pack failed");
AppError::Git(e.to_string())
let app = git_service_app_error(&e);
match &app {
AppError::Timeout(_) => tracing::warn!(repo = %name, "git receive-pack timed out"),
AppError::BadRequest(msg) => {
tracing::warn!(repo = %name, err = %msg, "git receive-pack: bad client request")
}
_ => tracing::error!(repo = %name, err = %e, "git receive-pack failed"),
}
app
})?;
// Update the repo's updated_at timestamp after a successful push
@@ -945,11 +1014,7 @@ pub async fn git_receive_pack(
.as_deref()
.unwrap_or("http://127.0.0.1:7545")
.trim_end_matches('/');
let owner_short = record
.owner_did
.split(':')
.next_back()
.unwrap_or(&record.owner_did);
let owner_short = crate::db::normalize_owner_key(&record.owner_did);
let clone_url = format!("{}/{}/{}.git", base_url, owner_short, record.name);
for update in &ref_updates {
@@ -1096,7 +1161,7 @@ pub async fn git_receive_pack(
// this push to Arweave, so the oid->cid index survives total
// node loss. Best-effort; never fails the push.
if !delta.is_empty() && !irys_url.is_empty() {
let owner_short = owner_did.split(':').next_back().unwrap_or(&owner_did);
let owner_short = crate::db::normalize_owner_key(&owner_did);
let repo_slug = format!("{owner_short}/{repo_name}");
let ts = chrono::Utc::now().to_rfc3339();
let manifest = crate::arweave::EncryptedManifest {
@@ -1141,11 +1206,7 @@ pub async fn git_receive_pack(
let node_did_str = state.node_did.to_string();
let repo_slug = format!(
"{}/{}",
record
.owner_did
.split(':')
.next_back()
.unwrap_or(&record.owner_did),
crate::db::normalize_owner_key(&record.owner_did),
record.name
);
let ref_updates_clone = ref_updates
@@ -1213,17 +1274,26 @@ pub async fn git_receive_pack(
}
// Broadcast ref update to GraphQL subscription listeners — one per ref.
// Gated on `announce`: /graphql/ws is unauthenticated (mounted after
// the optional_signature layer), and the subscription resolver has no
// caller to gate against, so only publicly-readable ref updates may
// reach anonymous subscribers. Mirrors the gossip (above) and Arweave
// (below) sends, which are already `announce`-gated. Without this a
// private-repo push would leak live ref metadata over the socket —
// the subscription analog of #112/#114.
let now_ts = chrono::Utc::now().to_rfc3339();
for (ref_name, old_sha, new_sha) in &ref_updates_clone {
let _ = ref_update_tx.send(crate::state::RefUpdateBroadcast {
repo: repo_slug.clone(),
ref_name: ref_name.clone(),
old_sha: old_sha.clone(),
new_sha: new_sha.clone(),
pusher_did: pusher_did_clone.clone(),
node_did: node_did_str.clone(),
timestamp: now_ts.clone(),
});
if announce {
for (ref_name, old_sha, new_sha) in &ref_updates_clone {
let _ = ref_update_tx.send(crate::state::RefUpdateBroadcast {
repo: repo_slug.clone(),
ref_name: ref_name.clone(),
old_sha: old_sha.clone(),
new_sha: new_sha.clone(),
pusher_did: pusher_did_clone.clone(),
node_did: node_did_str.clone(),
timestamp: now_ts.clone(),
});
}
}
// Arweave permanent anchoring — fire for each ref update.
@@ -1474,7 +1544,7 @@ pub async fn fork_repo(
}
// Check no name conflict under the forker's ownership
let forker_short = forker_did.split(':').next_back().unwrap_or(&forker_did);
let forker_short = crate::db::normalize_owner_key(&forker_did);
if state.db.get_repo(forker_short, &fork_name).await?.is_some() {
return Err(AppError::BadRequest(format!(
"you already have a repo named {fork_name}"
@@ -1630,13 +1700,17 @@ fn parse_ref_updates(body: &[u8]) -> Vec<RefUpdate> {
}
// ── Helpers ───────────────────────────────────────────────────────────────
//
// For a non-key DID owner, `normalize_owner_key` returns the full DID, so
// `clone_url` becomes `/did:gitlawb:z6.../repo.git`. That resolves through
// `get_repo`, but the colon-bearing path segment would break the `sync.rs`
// disk-path join (`owner_slug/repo`). Not reachable today (auth is
// did:key-only), so this is a forward constraint to handle before non-key
// ownership lands: the owner-first disk layout must either reject colons or
// encode them.
fn to_response(record: &crate::db::RepoRecord, state: &AppState, star_count: i64) -> RepoResponse {
let owner_short = record
.owner_did
.split(':')
.next_back()
.unwrap_or(&record.owner_did);
let owner_short = crate::db::normalize_owner_key(&record.owner_did);
let base_url = state
.config
@@ -1762,6 +1836,32 @@ mod tests {
const OWNER_SHORT: &str = "z6MkpTHR8VNsBxYAAWHut2Geadd9jSwuBV8xRoAnwWsdvktH";
const STRANGER_DID: &str = "did:key:z6Mkffonly5tranger0000000000000000000000000000000";
#[test]
fn git_service_app_error_classifies_timeout_bad_request_and_git() {
// GitServiceTimeout carried through anyhow -> 504 Timeout.
let timeout_err: anyhow::Error = smart_http::GitServiceTimeout.into();
assert!(matches!(
git_service_app_error(&timeout_err),
AppError::Timeout(_)
));
// A malformed client request -> 400.
let bad = anyhow::anyhow!("fatal: bad line length character: 0000");
assert!(matches!(
git_service_app_error(&bad),
AppError::BadRequest(_)
));
// The `protocol error` marker (with no "bad line length" substring) also
// -> 400, exercising the second arm of the classifier independently.
let proto = anyhow::anyhow!("fatal: protocol error: unexpected flush packet");
assert!(matches!(
git_service_app_error(&proto),
AppError::BadRequest(_)
));
// Anything else -> 500 git error.
let other = anyhow::anyhow!("some other git failure");
assert!(matches!(git_service_app_error(&other), AppError::Git(_)));
}
fn repo_owned_by(owner_did: &str) -> crate::db::RepoRecord {
let now = chrono::Utc::now();
crate::db::RepoRecord {
@@ -1779,6 +1879,32 @@ mod tests {
}
}
/// `announce` is the single boolean that gates every network-facing emission
/// of a push: gossip, Arweave anchoring, and the GraphQL subscription
/// broadcast (the last one added in this change). It must be false for a repo
/// the anonymous public cannot read, or the unauthenticated `/graphql/ws`
/// subscription leaks live private-repo ref metadata. Pin both directions of
/// the decision the broadcast now sits behind. No disk access: a non-announce
/// repo returns early, and a public repo with no path-scoped rule skips the
/// withheld walk.
#[tokio::test]
async fn replication_announce_false_for_private_true_for_public() {
let dummy = std::path::PathBuf::from("/nonexistent");
// Private: no rules at all.
let (announce, _) = replication_withheld_set(None, OWNER_DID, false, dummy.clone()).await;
assert!(!announce, "private repo (no rules) must not announce");
// Private: empty rule set, is_public=false → still not listable at root.
let (announce, _) =
replication_withheld_set(Some(vec![]), OWNER_DID, false, dummy.clone()).await;
assert!(!announce, "private repo (empty rules) must not announce");
// Public: empty rule set, is_public=true → listable at root, announces.
let (announce, _) = replication_withheld_set(Some(vec![]), OWNER_DID, true, dummy).await;
assert!(announce, "public repo must announce");
}
/// A rejection must be a 403 Forbidden (authenticated but not authorized),
/// not a 400 — some git/CI clients retry 400s.
fn assert_forbidden(rejection: Option<AppError>) {
@@ -1788,6 +1914,23 @@ mod tests {
);
}
#[test]
fn smart_http_repo_name_rejects_empty_after_git_suffix() {
assert_eq!(smart_http_repo_name("demo.git").unwrap(), "demo");
assert_eq!(smart_http_repo_name("demo").unwrap(), "demo");
// Only one suffix is stripped: a repo literally named "demo.git"
// stays addressable and never aliases to "demo".
assert_eq!(smart_http_repo_name("demo.git.git").unwrap(), "demo.git");
assert!(matches!(
smart_http_repo_name(".git"),
Err(AppError::BadRequest(_))
));
assert!(matches!(
smart_http_repo_name(""),
Err(AppError::BadRequest(_))
));
}
#[test]
fn enforced_allows_owner_full_did() {
let repo = repo_owned_by(OWNER_DID);
@@ -2340,4 +2483,146 @@ mod tests {
_mock.assert_async().await;
}
#[tokio::test]
async fn to_response_generates_correct_clone_url_slug() {
let state = crate::test_support::test_state_lazy();
let now = chrono::Utc::now();
// 1. did:key owner (should strip did:key: prefix)
let repo_key = crate::db::RepoRecord {
id: "uuid-1".into(),
name: "my-repo".into(),
owner_did: "did:key:z6Mkwbud".into(),
description: None,
is_public: true,
default_branch: "main".into(),
created_at: now,
updated_at: now,
disk_path: "/tmp/my-repo".into(),
forked_from: None,
machine_id: None,
};
let response_key = to_response(&repo_key, &state, 5);
assert!(
response_key.clone_url.contains("/z6Mkwbud/my-repo.git"),
"clone_url should use the bare did:key ID. got: {}",
response_key.clone_url
);
// 2. did:gitlawb owner (non-key DID method, should NOT strip)
let repo_non_key = crate::db::RepoRecord {
id: "uuid-2".into(),
name: "other-repo".into(),
owner_did: "did:gitlawb:z6Mkwbud".into(),
description: None,
is_public: true,
default_branch: "main".into(),
created_at: now,
updated_at: now,
disk_path: "/tmp/other-repo".into(),
forked_from: None,
machine_id: None,
};
let response_non_key = to_response(&repo_non_key, &state, 10);
assert!(
response_non_key
.clone_url
.contains("/did:gitlawb:z6Mkwbud/other-repo.git"),
"clone_url should preserve the full non-key owner DID. got: {}",
response_non_key.clone_url
);
}
/// The receive-pack *advertisement* (`GET info/refs?service=git-receive-pack`)
/// must be throttled by the per-IP push limiter BEFORE it does the fresh
/// Tigris acquire — otherwise the flood brake on the POST is bypassable via
/// the cheaper unauthenticated GET (PR #152 review P1). Pre-filling the
/// bucket makes the assertion deterministic and keeps the test off the
/// acquire path entirely.
#[sqlx::test]
async fn receive_pack_advertisement_is_rate_limited(pool: sqlx::PgPool) {
use axum::body::Body;
use axum::extract::ConnectInfo;
use axum::http::{Method, Request, StatusCode};
use std::net::SocketAddr;
use std::time::Duration;
use tower::ServiceExt;
let mut state = crate::test_support::test_state(pool).await;
// Tiny limit, keyed on the socket peer (no trusted proxy).
state.push_rate_limiter = crate::rate_limit::RateLimiter::new(1, Duration::from_secs(60));
state.push_limiter_trust = crate::rate_limit::TrustedProxy::None;
state
.db
.upsert_mirror_repo("z6advowner", "adv", "/tmp/adv", None, false)
.await
.unwrap();
let peer: SocketAddr = "203.0.113.55:6000".parse().unwrap();
// Exhaust this peer's single-request budget up front.
assert!(state.push_rate_limiter.check(&peer.ip().to_string()).await);
let router = crate::server::build_router(state);
let mut req = Request::builder()
.method(Method::GET)
.uri("/z6advowner/adv/info/refs?service=git-receive-pack")
.body(Body::empty())
.unwrap();
req.extensions_mut().insert(ConnectInfo(peer));
let status = router.oneshot(req).await.unwrap().status();
assert_eq!(
status,
StatusCode::TOO_MANY_REQUESTS,
"receive-pack advertisement must be throttled before the Tigris acquire"
);
}
/// Repo creation must be throttled by the per-IP creation limiter BEFORE
/// signature verification — otherwise a DID farm (one throwaway did:key per
/// repo, each carrying a valid but machine-solved iCaptcha proof) walks past
/// the per-DID limiter and floods the network, as in the recurring spam-repo
/// incidents. A 429 (not a 401) on an unsigned request from an exhausted IP
/// proves the IP brake runs outermost, ahead of auth.
#[sqlx::test]
async fn repo_creation_is_rate_limited_by_ip(pool: sqlx::PgPool) {
use axum::body::Body;
use axum::extract::ConnectInfo;
use axum::http::{Method, Request, StatusCode};
use std::net::SocketAddr;
use std::time::Duration;
use tower::ServiceExt;
let mut state = crate::test_support::test_state(pool).await;
// Tiny limit, keyed on the socket peer (no trusted proxy).
state.create_ip_rate_limiter =
crate::rate_limit::RateLimiter::new(1, Duration::from_secs(60));
state.push_limiter_trust = crate::rate_limit::TrustedProxy::None;
let peer: SocketAddr = "203.0.113.77:7000".parse().unwrap();
// Exhaust this peer's single-request budget up front.
assert!(
state
.create_ip_rate_limiter
.check(&peer.ip().to_string())
.await
);
let router = crate::server::build_router(state);
let mut req = Request::builder()
.method(Method::POST)
.uri("/api/v1/repos")
.header("content-type", "application/json")
.body(Body::from(r#"{"name":"flood","is_public":true}"#))
.unwrap();
req.extensions_mut().insert(ConnectInfo(peer));
let status = router.oneshot(req).await.unwrap().status();
assert_eq!(
status,
StatusCode::TOO_MANY_REQUESTS,
"repo creation must be IP-throttled before signature verification"
);
}
}
+11 -18
View File
@@ -2,14 +2,14 @@
//!
//! Any authenticated agent can star or unstar a repo.
//! One agent = one star per repo (enforced by UNIQUE constraint in repo_stars).
//! Star count is public — no auth required on GET.
//! Star count is gated on repo read access.
use axum::extract::{Extension, Path, State};
use axum::http::StatusCode;
use axum::Json;
use crate::auth::AuthenticatedDid;
use crate::error::{AppError, Result};
use crate::error::Result;
use crate::state::AppState;
/// PUT /api/v1/repos/:owner/:repo/star
@@ -19,11 +19,8 @@ pub async fn star_repo(
Extension(auth): Extension<AuthenticatedDid>,
Path((owner, repo)): Path<(String, String)>,
) -> Result<(StatusCode, Json<serde_json::Value>)> {
let record = state
.db
.get_repo(&owner, &repo)
.await?
.ok_or_else(|| AppError::RepoNotFound(format!("{owner}/{repo}")))?;
let (record, _rules) =
crate::api::authorize_repo_read(&state, &owner, &repo, Some(auth.0.as_str()), "/").await?;
let caller = &auth.0;
let inserted = state.db.star_repo(&record.id, caller).await?;
@@ -54,11 +51,8 @@ pub async fn unstar_repo(
Extension(auth): Extension<AuthenticatedDid>,
Path((owner, repo)): Path<(String, String)>,
) -> Result<Json<serde_json::Value>> {
let record = state
.db
.get_repo(&owner, &repo)
.await?
.ok_or_else(|| AppError::RepoNotFound(format!("{owner}/{repo}")))?;
let (record, _rules) =
crate::api::authorize_repo_read(&state, &owner, &repo, Some(auth.0.as_str()), "/").await?;
let caller = &auth.0;
state.db.unstar_repo(&record.id, caller).await?;
@@ -74,16 +68,15 @@ pub async fn unstar_repo(
}
/// GET /api/v1/repos/:owner/:repo/star
/// Returns star count — unauthenticated.
/// Returns star count for callers who can read the repo.
pub async fn get_star_status(
State(state): State<AppState>,
Path((owner, repo)): Path<(String, String)>,
auth: Option<Extension<AuthenticatedDid>>,
) -> Result<Json<serde_json::Value>> {
let record = state
.db
.get_repo(&owner, &repo)
.await?
.ok_or_else(|| AppError::RepoNotFound(format!("{owner}/{repo}")))?;
let caller = auth.as_ref().map(|e| e.0 .0.as_str());
let (record, _rules) =
crate::api::authorize_repo_read(&state, &owner, &repo, caller, "/").await?;
let count = state.db.count_stars(&record.id).await?;
+5
View File
@@ -514,6 +514,11 @@ mod tests {
machine_id: None,
repo_store: crate::git::repo_store::RepoStore::for_testing(PathBuf::from("/tmp"), pool),
rate_limiter: RateLimiter::new(100, Duration::from_secs(60)),
create_ip_rate_limiter: RateLimiter::new(1000, Duration::from_secs(3600)),
push_rate_limiter: RateLimiter::new(600, Duration::from_secs(3600)),
push_limiter_trust: crate::rate_limit::TrustedProxy::None,
sync_trigger_rate_limiter: RateLimiter::new(60, Duration::from_secs(3600)),
peer_write_rate_limiter: RateLimiter::new(600, Duration::from_secs(3600)),
shutdown_tx: tokio::sync::watch::channel(false).0,
}
}
+3 -2
View File
@@ -52,6 +52,7 @@ pub async fn issue_ref_certificate(
issued_at,
};
state.db.insert_ref_certificate(&cert).await?;
Ok(cert)
// Persist and return the row as it exists in the database (on a
// conflict the existing row survives when it is newer).
state.db.insert_ref_certificate(&cert).await
}
+104
View File
@@ -140,6 +140,20 @@ pub struct Config {
#[arg(long, env = "GITLAWB_MAX_PACK_BYTES", default_value_t = 2_147_483_648)]
pub max_pack_bytes: usize,
/// Per-client-IP rate limit for `POST /api/v1/sync/trigger`, in requests per
/// hour. `/sync/trigger` requires a signature and drives an O(peers) outbound
/// fan-out per call, so it gets a tight bucket. `0` disables. Default: 60.
#[arg(long, env = "GITLAWB_SYNC_TRIGGER_RATE_LIMIT", default_value_t = 60)]
pub sync_trigger_rate_limit: usize,
/// Per-client-IP rate limit for the peer-write routes (`/peers/announce`,
/// `/sync/notify`), in requests per hour. These accept unsigned requests from
/// known peers and run at higher frequency, so the bucket is generous. Keeping
/// it separate from the trigger bucket stops an unsigned notify flood from
/// draining the signed trigger caller's quota. `0` disables. Default: 600.
#[arg(long, env = "GITLAWB_PEER_WRITE_RATE_LIMIT", default_value_t = 600)]
pub peer_write_rate_limit: usize,
/// Optional address to bind a Prometheus `/metrics` exposition endpoint on.
/// Example: `127.0.0.1:9091`. Leave empty (default) to disable.
/// Bind to localhost or a private interface — the metrics endpoint is
@@ -152,6 +166,74 @@ pub struct Config {
/// in flight and exits. Default: 30s.
#[arg(long, env = "GITLAWB_SHUTDOWN_GRACE_SECS", default_value_t = 30)]
pub shutdown_grace_secs: u64,
/// Maximum wall-clock time a single served git operation (upload-pack /
/// receive-pack through `run_git_service`) may run before it is aborted and
/// its process group torn down, in seconds. Bounds a git that neither
/// finishes nor disconnects. Must be positive; set it very large to
/// effectively disable the bound. Default: 600s (10 min), generous for large
/// clones. Does not cover the ref advertisement (`info/refs`) or the
/// withheld-blob fetch path (`upload_pack_excluding`, a blocking
/// `spawn_blocking` a tokio timeout cannot cancel); both remain unbounded.
#[arg(
long,
env = "GITLAWB_GIT_SERVICE_TIMEOUT_SECS",
default_value_t = 600,
value_parser = clap::value_parser!(u64).range(1..)
)]
pub git_service_timeout_secs: u64,
/// Maximum connections in the PostgreSQL pool. This is a cap, not a floor
/// (connections open lazily). Size against the database server's
/// max_connections, remembering admin tooling opens its own pool.
#[arg(
long,
env = "GITLAWB_DB_MAX_CONNECTIONS",
default_value_t = 20,
value_parser = clap::value_parser!(u32).range(1..)
)]
pub db_max_connections: u32,
/// Maximum time a request waits for a pool connection before failing with
/// 503, in seconds. Bounds queueing when the database is slow or down.
#[arg(
long,
env = "GITLAWB_DB_ACQUIRE_TIMEOUT_SECS",
default_value_t = 5,
value_parser = clap::value_parser!(u64).range(1..)
)]
pub db_acquire_timeout_secs: u64,
/// Upper bound on each startup connect-and-migrate attempt, in seconds.
/// Migrations wait on a cross-instance advisory lock, so this must be
/// generous enough for a peer instance to finish migrating; on expiry the
/// attempt is retried (migrations are idempotent).
#[arg(
long,
env = "GITLAWB_DB_CONNECT_TIMEOUT_SECS",
default_value_t = 60,
value_parser = clap::value_parser!(u64).range(1..)
)]
pub db_connect_timeout_secs: u64,
/// First retry delay when the database is unavailable at startup, in
/// seconds. Doubles each attempt up to --db-retry-max-secs.
#[arg(
long,
env = "GITLAWB_DB_RETRY_INITIAL_SECS",
default_value_t = 5,
value_parser = clap::value_parser!(u64).range(1..)
)]
pub db_retry_initial_secs: u64,
/// Ceiling on the startup retry delay, in seconds.
#[arg(
long,
env = "GITLAWB_DB_RETRY_MAX_SECS",
default_value_t = 60,
value_parser = clap::value_parser!(u64).range(1..)
)]
pub db_retry_max_secs: u64,
}
impl Config {
@@ -169,3 +251,25 @@ impl Config {
PathBuf::from(&self.key_path)
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn git_service_timeout_defaults_to_600_and_rejects_zero() {
assert_eq!(
Config::parse_from(["gitlawb-node"]).git_service_timeout_secs,
600
);
assert_eq!(
Config::parse_from(["gitlawb-node", "--git-service-timeout-secs", "30"])
.git_service_timeout_secs,
30
);
// 0 is a footgun (immediate-504 on every request); clap must reject it.
assert!(
Config::try_parse_from(["gitlawb-node", "--git-service-timeout-secs", "0"]).is_err()
);
}
}
File diff suppressed because it is too large. Load diff
+111 -12
View File
@@ -23,24 +23,98 @@ pub enum AppError {
#[allow(dead_code)]
Forbidden(String),
#[error("icaptcha proof required: {0}")]
IcaptchaProofRequired(String),
#[error("icaptcha proof required: {message}")]
IcaptchaProofRequired {
message: String,
/// iCaptcha service base URL the client should solve against.
url: String,
/// Minimum proof level this node requires.
level: u32,
},
#[error("invalid request: {0}")]
BadRequest(String),
#[error("too many requests: {0}")]
TooManyRequests(String),
#[error("incomplete: {0}")]
Incomplete(String),
#[error("git error: {0}")]
Git(String),
#[error("git service timed out: {0}")]
Timeout(String),
#[error("database error: {0}")]
Db(#[from] sqlx::Error),
#[error("internal error: {0}")]
Internal(#[from] anyhow::Error),
Internal(anyhow::Error),
}
/// Shared error code/message for "the database is unreachable", also used by
/// the degraded startup server (main.rs) and the readiness probe (server.rs)
/// so clients see one vocabulary for the condition.
pub const DB_UNAVAILABLE_CODE: &str = "db_unavailable";
pub const DB_UNAVAILABLE_MESSAGE: &str = "database is temporarily unavailable";
/// Connection-level sqlx failures that mean the database is unreachable right
/// now (retryable, 503), as opposed to server-reported query errors.
fn db_unavailable(e: &sqlx::Error) -> bool {
matches!(
e,
sqlx::Error::PoolTimedOut
| sqlx::Error::PoolClosed
| sqlx::Error::Io(_)
| sqlx::Error::Tls(_)
)
}
/// The db layer returns `anyhow::Result`, so sqlx errors reach handlers inside
/// anyhow chains. Downcast them back out so the status mapping below can see
/// them — without this, every database outage surfaces as a 500 instead of a
/// 503. anyhow preserves downcastability through `.context()` layers.
impl From<anyhow::Error> for AppError {
fn from(err: anyhow::Error) -> Self {
match err.downcast::<sqlx::Error>() {
Ok(sql) => AppError::Db(sql),
Err(err) => AppError::Internal(err),
}
}
}
impl IntoResponse for AppError {
fn into_response(self) -> Response {
// iCaptcha challenges carry structured discovery so clients don't have to
// scrape the message: the service URL and required level are returned as
// both JSON fields and `x-icaptcha-url` / `x-icaptcha-level` headers
// (mirroring the header-bearing `human_detected` response in auth/mod.rs).
if let AppError::IcaptchaProofRequired {
message,
url,
level,
} = &self
{
use axum::http::HeaderValue;
let body = Json(json!({
"error": "icaptcha_proof_required",
"message": message,
"icaptcha_url": url,
"required_level": level,
}));
let mut resp = (StatusCode::FORBIDDEN, body).into_response();
let headers = resp.headers_mut();
if let Ok(v) = HeaderValue::from_str(url) {
headers.insert("x-icaptcha-url", v);
}
if let Ok(v) = HeaderValue::from_str(&level.to_string()) {
headers.insert("x-icaptcha-level", v);
}
return resp;
}
let (status, code, message) = match &self {
AppError::RepoNotFound(r) => (
StatusCode::NOT_FOUND,
@@ -55,17 +129,24 @@ impl IntoResponse for AppError {
AppError::NotFound(msg) => (StatusCode::NOT_FOUND, "not_found", msg.clone()),
AppError::Unauthorized(msg) => (StatusCode::UNAUTHORIZED, "not_an_agent", msg.clone()),
AppError::Forbidden(msg) => (StatusCode::FORBIDDEN, "forbidden", msg.clone()),
// 403, not 401: the caller IS an authenticated agent (credentials are
// valid) but is forbidden from this action without a valid, fresh
// iCaptcha proof. The distinct `icaptcha_proof_required` code — which
// clients branch on — keeps it separable from a plain `forbidden`.
AppError::IcaptchaProofRequired(msg) => (
StatusCode::FORBIDDEN,
"icaptcha_proof_required",
msg.clone(),
),
// IcaptchaProofRequired is handled above (it carries extra headers/fields).
AppError::IcaptchaProofRequired { .. } => unreachable!("handled before this match"),
AppError::BadRequest(msg) => (StatusCode::BAD_REQUEST, "bad_request", msg.clone()),
AppError::TooManyRequests(msg) => {
(StatusCode::TOO_MANY_REQUESTS, "rate_limited", msg.clone())
}
AppError::Incomplete(msg) => {
(StatusCode::UNPROCESSABLE_ENTITY, "incomplete", msg.clone())
}
AppError::Git(msg) => (StatusCode::INTERNAL_SERVER_ERROR, "git_error", msg.clone()),
// 504, distinct from the 500 git_error and from the read-gate's 404 /
// the auth 401, so the client can tell a deadline from a failure.
AppError::Timeout(msg) => (StatusCode::GATEWAY_TIMEOUT, "git_timeout", msg.clone()),
AppError::Db(e) if db_unavailable(e) => (
StatusCode::SERVICE_UNAVAILABLE,
DB_UNAVAILABLE_CODE,
DB_UNAVAILABLE_MESSAGE.into(),
),
AppError::Db(e) => (StatusCode::INTERNAL_SERVER_ERROR, "db_error", e.to_string()),
AppError::Internal(e) => (
StatusCode::INTERNAL_SERVER_ERROR,
@@ -84,3 +165,21 @@ impl IntoResponse for AppError {
}
pub type Result<T> = std::result::Result<T, AppError>;
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn timeout_maps_to_504_distinct_from_git_500() {
assert_eq!(
AppError::Timeout("x".into()).into_response().status(),
StatusCode::GATEWAY_TIMEOUT
);
// Guard against a swap with the generic git failure (500).
assert_eq!(
AppError::Git("x".into()).into_response().status(),
StatusCode::INTERNAL_SERVER_ERROR
);
}
}
+660 -23
View File
@@ -6,7 +6,8 @@ use bytes::Bytes;
use std::collections::HashSet;
use std::path::Path;
use std::process::Stdio;
use tokio::io::AsyncWriteExt;
use std::time::Duration;
use tokio::io::{AsyncReadExt, AsyncWriteExt};
use tokio::process::Command;
/// Handle `GET /:owner/:repo/info/refs?service=git-upload-pack`
@@ -51,8 +52,13 @@ pub async fn info_refs(repo_path: &Path, service: &str) -> Result<Response> {
///
/// Serves pack data for a clone or fetch. This is stateless — the entire
/// negotiation happens in a single request/response.
pub async fn upload_pack(repo_path: &Path, request_body: Bytes) -> Result<Response> {
let output = run_git_service("git-upload-pack", repo_path, request_body).await?;
pub async fn upload_pack(
repo_path: &Path,
request_body: Bytes,
timeout: Duration,
) -> Result<Response> {
let output =
run_git_service("git", "git-upload-pack", repo_path, request_body, timeout).await?;
Ok(Response::builder()
.status(StatusCode::OK)
@@ -65,8 +71,13 @@ pub async fn upload_pack(repo_path: &Path, request_body: Bytes) -> Result<Respon
///
/// Accepts a push. The caller MUST verify HTTP Signature auth before
/// calling this function.
pub async fn receive_pack(repo_path: &Path, request_body: Bytes) -> Result<Response> {
let output = run_git_service("git-receive-pack", repo_path, request_body).await?;
pub async fn receive_pack(
repo_path: &Path,
request_body: Bytes,
timeout: Duration,
) -> Result<Response> {
let output =
run_git_service("git", "git-receive-pack", repo_path, request_body, timeout).await?;
Ok(Response::builder()
.status(StatusCode::OK)
@@ -112,8 +123,87 @@ impl Drop for KillGroupOnDrop {
}
}
async fn run_git_service(service: &str, repo_path: &Path, input: Bytes) -> Result<Vec<u8>> {
let mut command = Command::new("git");
/// Error returned when [`run_git_service`] aborts a git op on its timeout.
///
/// Carried through the `anyhow` chain so the HTTP handler can `downcast_ref` it
/// and map to 504 Gateway Timeout, distinct from the generic 500 git error.
#[derive(Debug, thiserror::Error)]
#[error("git service timed out")]
pub struct GitServiceTimeout;
/// On a served-git timeout, tear the process group down AND reap the leader
/// before returning, so a caller that releases a write lock (receive-pack) can't
/// race a still-live git touching the same repo. SIGTERM first (lets git remove
/// its `.git/*.lock` files), escalate to SIGKILL if the leader lingers past a
/// grace, then reap. Bounded, so a git that ignores SIGTERM can't block the
/// response unboundedly.
#[cfg(unix)]
async fn reap_group_on_timeout(child: &mut tokio::process::Child) {
let Some(pid) = child.id() else {
// Never got a pid; nothing to signal, best-effort reap.
let _ = child.wait().await;
return;
};
let pgid = pid as i32;
// SAFETY: kill(2) takes only integers and borrows no Rust memory; ESRCH on an
// already-gone group is ignored.
unsafe {
libc::kill(-pgid, libc::SIGTERM);
}
// Wait for the WHOLE group to exit before returning — not just the leader but
// grandchildren (index-pack / pack-objects) that reparent to init and can
// still be touching the repo, so a caller releasing a write lock can't race
// them. Reap our direct child along the way (`try_wait`, else it lingers as a
// zombie) and poll the group's liveness with `kill(-pgid, 0)` (ESRCH once
// every member is gone). SIGTERM grace, then SIGKILL, then a hard cap so a
// stuck process can never block the response unboundedly.
for step in 0..400u32 {
let _ = child.try_wait();
if unsafe { libc::kill(-pgid, 0) } != 0 {
return; // ESRCH: every group member has exited
}
if step == 200 {
// ~2s SIGTERM grace elapsed; force the group down. `step == 200` fires
// exactly once in the 0..400 loop, so no re-entry guard is needed.
unsafe {
libc::kill(-pgid, libc::SIGKILL);
}
}
tokio::time::sleep(Duration::from_millis(10)).await;
}
// ~4s hard cap. Only reached if a group member survives SIGKILL, which means
// it is wedged in uninterruptible (D-state) I/O — a fsync on stuck storage, a
// hung mount — that no signal interrupts until the kernel returns. Nothing in
// userspace can kill it; blocking here (or holding the write lock) would just
// re-create the "hung git pins the repo" problem this bounds. Reap best-effort
// (tokio's orphan reaper collects it once the syscall unblocks) and return;
// git-level quarantine/ref-locking still guards the brief window before the
// D-state process finishes.
tracing::warn!(
pid,
"served git survived SIGKILL past the teardown cap (uninterruptible I/O?); \
releasing without a confirmed group reap"
);
let _ = child.try_wait();
}
/// Run a stateless-rpc git service and return its stdout.
///
/// `git_bin` is the git executable to spawn; production callers pass `"git"`
/// (resolved via `PATH`). It is injectable purely so the process-group teardown
/// wiring (`process_group(0)` + [`KillGroupOnDrop`]) can be driven end-to-end by
/// a fake `git` in tests without mutating the process-global `PATH`.
///
/// `timeout` bounds the whole child interaction; on expiry the op is aborted with
/// [`GitServiceTimeout`] and its process group is torn down.
async fn run_git_service(
git_bin: &str,
service: &str,
repo_path: &Path,
input: Bytes,
timeout: Duration,
) -> Result<Vec<u8>> {
let mut command = Command::new(git_bin);
command
.arg(service_to_command(service))
.arg("--stateless-rpc")
@@ -131,33 +221,85 @@ async fn run_git_service(service: &str, repo_path: &Path, input: Bytes) -> Resul
// Arm the group-kill guard for the lifetime of the request. With
// process_group(0) the child is its own group leader, so pgid == its pid.
// This fires on a client disconnect (the whole future is dropped mid-request).
#[cfg(unix)]
let mut group_guard = KillGroupOnDrop {
pgid: child.id().map(|id| id as i32),
};
// Write the request body to git's stdin, but don't early-return on a write
// error: always reap the child first (below), so the guard only ever fires on
// an actual future-drop (client disconnect), never on a pid we just reaped.
let write_result: std::io::Result<()> = match child.stdin.take() {
Some(mut stdin) => stdin.write_all(&input).await,
None => Ok(()),
// Own the pipes so `child` stays reap-able after a timeout: wait_with_output
// would consume it, but on timeout we must actively reap the group before
// returning (see reap_group_on_timeout).
let mut stdin = child.stdin.take();
let mut stdout = child.stdout.take().context("git stdout was not piped")?;
let mut stderr = child.stderr.take().context("git stderr was not piped")?;
let mut out = Vec::new();
let mut err = Vec::new();
// Bound the whole child interaction: a git that neither finishes nor
// disconnects would otherwise pin the pid forever. Write stdin, drain stdout
// and stderr, and wait for the child all concurrently (draining while writing
// avoids a pipe deadlock on a large body), under one deadline.
let interact = async {
// Don't early-return on a write error: the join still waits on the child,
// so an error is surfaced only after the child has been waited on.
let write = async {
match stdin.take() {
Some(mut s) => s.write_all(&input).await,
None => Ok(()),
}
};
let (write_result, r_out, r_err, status) = tokio::join!(
write,
stdout.read_to_end(&mut out),
stderr.read_to_end(&mut err),
child.wait(),
);
r_out?;
r_err?;
Ok::<_, anyhow::Error>((write_result, status?))
};
let output = child.wait_with_output().await?;
let timed = tokio::time::timeout(timeout, interact).await;
let (write_result, status) = match timed {
Ok(result) => {
// The join runs all arms to completion, so the child is reaped: disarm
// before surfacing any interaction error (a read/wait error), else the
// guard's drop would fire SIGTERM on the reaped, possibly-reused pgid.
#[cfg(unix)]
group_guard.disarm();
result?
}
Err(_elapsed) => {
// Timeout: tear the whole group down and reap it before returning so a
// caller releasing a write lock can't race a still-live git. Then disarm
// the (now redundant) guard so its drop can't hit a reused pgid.
#[cfg(unix)]
{
reap_group_on_timeout(&mut child).await;
group_guard.disarm();
}
#[cfg(not(unix))]
{
let _ = child.start_kill();
let _ = tokio::time::timeout(Duration::from_secs(2), child.wait()).await;
}
return Err(GitServiceTimeout.into());
}
};
// Child reaped, so its group is gone: disarm before surfacing any error.
#[cfg(unix)]
group_guard.disarm();
write_result.context("failed to write to git stdin")?;
if !output.status.success() {
let stderr = String::from_utf8_lossy(&output.stderr);
// Surface git's own failure (its stderr, which the handler may classify as a
// 400) before any stdin-write error: when git rejects a malformed body it
// exits non-zero and closes stdin, so the write's EPIPE would otherwise mask
// the real cause.
if !status.success() {
let stderr = String::from_utf8_lossy(&err);
bail!("{service} failed: {stderr}");
}
Ok(output.stdout)
write_result.context("failed to write to git stdin")?;
Ok(out)
}
fn service_to_command(service: &str) -> &str {
@@ -876,4 +1018,499 @@ mod tests {
let _ = child.kill().await;
let _ = child.wait().await;
}
// ── #62 PR1: end-to-end teardown wiring through run_git_service ─────────
//
// The kill_group_guard_* tests above build a KillGroupOnDrop by hand and
// never call run_git_service, so deleting `process_group(0)` (spawn site) or
// the guard-arming, or the post-reap disarm, would leave them green. These
// drive the REAL run_git_service via an injected fake `git` (the `git_bin`
// seam) and assert the production spawn path actually wires the teardown up.
// Faithful to the real invocation: run_git_service spawns
// `<git_bin> <cmd> --stateless-rpc <repo_path>` in its own process group.
/// SIGKILLs the given pids on drop (ignoring all kill errors, e.g. ESRCH for
/// an already-dead pid) so a panicking assertion can't leak the fake `git` or
/// its grandchild onto the test runner.
#[cfg(unix)]
struct ReapOnPanic(Vec<i32>);
#[cfg(unix)]
impl Drop for ReapOnPanic {
fn drop(&mut self) {
for &pid in &self.0 {
unsafe {
libc::kill(pid, libc::SIGKILL);
}
}
}
}
/// Write an executable fake `git` (named `git`) into `dir`; return its path.
#[cfg(unix)]
fn write_fake_git(dir: &std::path::Path, body: &str) -> std::path::PathBuf {
use std::os::unix::fs::PermissionsExt as _;
let path = dir.join("git");
std::fs::write(&path, body).unwrap();
std::fs::set_permissions(&path, std::fs::Permissions::from_mode(0o755)).unwrap();
path
}
/// Read a two-line `leader\ngrandchild` pidfile once both lines are present.
#[cfg(unix)]
fn read_two_pids(pidfile: &std::path::Path) -> Option<(i32, i32)> {
let s = std::fs::read_to_string(pidfile).ok()?;
let mut lines = s.lines();
let leader: i32 = lines.next()?.trim().parse().ok()?;
let grandchild: i32 = lines.next()?.trim().parse().ok()?;
Some((leader, grandchild))
}
/// Retry policy for the fake-git spawn race, shared by `fake_git_run_with_pids`
/// and the drop test's inline loop (which can't use the helper — it must keep
/// the winning future pending). `nth` retry waits `n * STEP_MS` ms.
#[cfg(unix)]
const FAKE_GIT_RETRY_ATTEMPTS: u64 = 12;
#[cfg(unix)]
const FAKE_GIT_BACKOFF_STEP_MS: u64 = 100;
/// Run a fake-git-based `run` under the parallel test runner, retrying if the
/// fake transiently fails to record its pids. Under `cargo test --workspace`
/// fork-storm load a freshly-written fake `git` can fail to exec (ETXTBSY: a
/// concurrent worker forked while its write fd was still open) or be killed by
/// the service timeout before it is scheduled to write its pidfile; both leave
/// no pids. These misses are *correlated* (bursty) under fork pressure, so each
/// retry backs off (growing) to let a spike subside rather than burning every
/// attempt inside one. `pidfile` is removed before each attempt. Returns the
/// successful attempt's outcome together with the recorded pids; a genuine
/// never-spawns bug still fails loudly after the cap. Retried attempts don't
/// leak in practice: a miss almost always means the fake never exec'd (nothing
/// to clean up), and a spawn that got far enough is reaped by the timeout/drop
/// path before this returns.
#[cfg(unix)]
async fn fake_git_run_with_pids<R, F, Fut>(
pidfile: &std::path::Path,
mut run: F,
) -> (R, (i32, i32))
where
F: FnMut() -> Fut,
Fut: std::future::Future<Output = R>,
{
for i in 0..FAKE_GIT_RETRY_ATTEMPTS {
let _ = std::fs::remove_file(pidfile);
let outcome = run().await;
// A successful run writes its pids before it returns, so a short poll
// catches them (allowing only for fs-visibility lag).
for _ in 0..50 {
if let Some(p) = read_two_pids(pidfile) {
return (outcome, p);
}
tokio::time::sleep(std::time::Duration::from_millis(10)).await;
}
// Miss: back off before retrying so a bursty fork-pressure spike
// subsides (see the doc note on correlated ETXTBSY/EAGAIN failures).
tokio::time::sleep(std::time::Duration::from_millis(
FAKE_GIT_BACKOFF_STEP_MS * (i + 1),
))
.await;
}
panic!(
"fake git failed to spawn and record its pids after {FAKE_GIT_RETRY_ATTEMPTS} \
attempts (persistent failure, not a transient parallel-runner miss)"
);
}
// Dropping the request future mid-flight (client disconnect) must SIGTERM the
// whole group so git AND its pack-objects grandchild die together. Goes RED
// if `process_group(0)` or the guard-arming is removed: without its own
// group, kill(-pgid) hits no group and the grandchild survives.
#[cfg(unix)]
#[tokio::test]
async fn run_git_service_tears_down_group_when_future_dropped() {
let tmp = tempfile::TempDir::new().unwrap();
let pidfile = tmp.path().join("pids");
// Fork a grandchild (stands in for pack-objects), record leader+grandchild
// pids, then hang so run_git_service parks in wait_with_output.
let body = format!(
"#!/bin/sh\nsleep 300 &\nprintf '%s\\n%s\\n' \"$$\" \"$!\" > \"{}\"\nwait\n",
pidfile.display()
);
let git_bin = write_fake_git(tmp.path(), &body);
// Retry under fork-storm load: a freshly-written fake `git` can transiently
// fail to exec (ETXTBSY) or exit before recording its pids; both leave no
// pids and are independent per attempt (see fake_git_run_with_pids). The
// winning attempt's future is kept PENDING so the drop below exercises the
// client-disconnect teardown. Dropping a losing attempt's future makes its
// guard reap anything that spawned, so retries don't leak.
let (fut, grandchild) = {
let mut attempt = 0u64;
loop {
attempt += 1;
let _ = std::fs::remove_file(&pidfile);
let mut fut = Box::pin(run_git_service(
git_bin.to_str().unwrap(),
"git-upload-pack",
tmp.path(),
Bytes::new(),
Duration::from_secs(60),
));
// Advance the future a slice at a time until the fake records its
// pids. `Ok(_)` means run_git_service returned before the pidfile
// appeared (spawn error / early exit); stop polling then, since
// re-polling a completed future panics with `async fn resumed after
// completion`. Read the pidfile first so a fake that wrote its pids
// and then exited is still captured.
let mut pids = None;
for _ in 0..500 {
let finished =
tokio::time::timeout(std::time::Duration::from_millis(10), &mut fut)
.await
.is_ok();
if let Some(p) = read_two_pids(&pidfile) {
pids = Some(p);
break;
}
if finished {
break;
}
}
// Only the grandchild needs panic-cleanup: dropping the future makes
// tokio reap the fake leader, and carrying the reaped leader pid
// risks SIGKILLing a recycled pid under parallel test load.
match pids {
Some((_leader, g)) => break (fut, g),
None => {
// Transient spawn miss: drop the still-armed future so its
// guard reaps anything that spawned, then back off (growing)
// so a bursty fork-pressure spike subsides before retrying.
drop(fut);
assert!(
attempt < FAKE_GIT_RETRY_ATTEMPTS,
"fake git failed to spawn and record its pids after \
{FAKE_GIT_RETRY_ATTEMPTS} attempts (persistent failure, \
not a transient parallel-runner miss)"
);
tokio::time::sleep(std::time::Duration::from_millis(
FAKE_GIT_BACKOFF_STEP_MS * attempt,
))
.await;
}
}
}
};
let _cleanup = ReapOnPanic(vec![grandchild]);
assert!(
alive(grandchild),
"grandchild should be running before the drop"
);
// Client disconnect: drop the request future. The armed KillGroupOnDrop
// must SIGTERM the whole group.
drop(fut);
let mut gone = false;
for _ in 0..500 {
if !alive(grandchild) {
gone = true;
break;
}
tokio::time::sleep(std::time::Duration::from_millis(10)).await;
}
assert!(
gone,
"grandchild must be torn down via the process group on future-drop \
(proves run_git_service sets process_group(0) and arms the guard)"
);
}
// A request that runs to completion must DISARM the guard after reaping, so
// no stray group SIGTERM fires. The fake exits non-zero (surfacing as Err)
// but leaves a grandchild alive; the grandchild must survive. Goes RED if the
// post-reap disarm is removed: the guard would then fire on return and, since
// the grandchild still holds the group open, sweep it.
#[cfg(unix)]
#[tokio::test]
async fn run_git_service_disarms_on_completion_leaving_group_alive() {
let tmp = tempfile::TempDir::new().unwrap();
let pidfile = tmp.path().join("pids");
// The grandchild is long-lived with stdio redirected to /dev/null:
// /dev/null so it doesn't inherit (and hold open) run_git_service's stdout
// pipe (which would block wait_with_output until it exits); long-lived so
// the "still alive" assertion below can't race the sleep exiting on its own
// under a starved scheduler. ReapOnPanic cleans it up.
let body = format!(
"#!/bin/sh\nsleep 300 >/dev/null 2>&1 &\nprintf '%s\\n%s\\n' \"$$\" \"$!\" > \"{}\"\nexit 1\n",
pidfile.display()
);
let git_bin = write_fake_git(tmp.path(), &body);
// wait_with_output already reaped the fake leader, so only the grandchild
// needs panic-cleanup — SIGKILLing the reaped leader pid could hit a
// recycled pid under parallel test load. Retry the run so a transient spawn
// miss under fork-storm load doesn't fail the test (see fake_git_run_with_pids).
let (result, (_leader, grandchild)) = fake_git_run_with_pids(&pidfile, || {
run_git_service(
git_bin.to_str().unwrap(),
"git-upload-pack",
tmp.path(),
Bytes::new(),
Duration::from_secs(60),
)
})
.await;
let _cleanup = ReapOnPanic(vec![grandchild]);
assert!(result.is_err(), "non-zero git exit must surface as Err");
// A mutant that left the guard armed fires SIGTERM on return; poll a short
// window so a killed grandchild is reliably observed dead — a single
// alive() check can race the SIGTERM+reap and miss it.
for _ in 0..30 {
assert!(
alive(grandchild),
"grandchild must survive: run_git_service must disarm the guard after \
reaping, not fire a group SIGTERM on the completion path"
);
tokio::time::sleep(std::time::Duration::from_millis(10)).await;
}
}
// The SUCCESS path (exit 0, Ok) must disarm too — the disarm test above only
// exercises the exit-1/Err branch. Goes RED if disarm is gated on failure
// (e.g. moved inside the `!status.success()` branch): the guard would then
// fire after a clean completion and sweep the still-live grandchild.
#[cfg(unix)]
#[tokio::test]
async fn run_git_service_disarms_on_success_leaving_group_alive() {
let tmp = tempfile::TempDir::new().unwrap();
let pidfile = tmp.path().join("pids");
// Long-lived, stdio to /dev/null (same reasons as the exit-1 test).
let body = format!(
"#!/bin/sh\nsleep 300 >/dev/null 2>&1 &\nprintf '%s\\n%s\\n' \"$$\" \"$!\" > \"{}\"\nexit 0\n",
pidfile.display()
);
let git_bin = write_fake_git(tmp.path(), &body);
// Retry the run so a transient spawn miss under fork-storm load doesn't
// fail the test (see fake_git_run_with_pids).
let (result, (_leader, grandchild)) = fake_git_run_with_pids(&pidfile, || {
run_git_service(
git_bin.to_str().unwrap(),
"git-upload-pack",
tmp.path(),
Bytes::new(),
Duration::from_secs(60),
)
})
.await;
let _cleanup = ReapOnPanic(vec![grandchild]);
assert!(result.is_ok(), "zero-exit git must surface as Ok");
// Poll a short window (see the exit-1 test): a mutant that disarms only on
// failure fires the guard on this success path, and a single alive() check
// can race the SIGTERM+reap.
for _ in 0..30 {
assert!(
alive(grandchild),
"grandchild must survive: run_git_service must disarm on the success \
path too, not fire a group SIGTERM after a clean completion"
);
tokio::time::sleep(std::time::Duration::from_millis(10)).await;
}
}
// A git that hangs (never finishes, never disconnects) must be bounded: it
// returns GitServiceTimeout (the handler maps that to 504) and its process
// group is torn down. Goes RED if the timeout wrap is removed — the OUTER
// bound below then fires instead of run_git_service's own, so the test fails
// loudly rather than hanging CI.
#[cfg(unix)]
#[tokio::test]
async fn run_git_service_times_out_and_tears_down_a_hung_git() {
let tmp = tempfile::TempDir::new().unwrap();
let pidfile = tmp.path().join("pids");
// Fork a grandchild, then hang forever.
let body = format!(
"#!/bin/sh\nsleep 300 &\nprintf '%s\\n%s\\n' \"$$\" \"$!\" > \"{}\"\nwait\n",
pidfile.display()
);
let git_bin = write_fake_git(tmp.path(), &body);
// Outer bound well above git_timeout: if run_git_service's own timeout is
// broken it hangs, and this fires instead so we assert-fail, not hang.
// fake_git_run_with_pids retries a transient spawn/schedule miss under
// fork-storm load and returns the pidfile the fake recorded.
let git_timeout = Duration::from_millis(1000);
let (outcome, (_leader, grandchild)) = fake_git_run_with_pids(&pidfile, || {
tokio::time::timeout(
Duration::from_secs(5),
run_git_service(
git_bin.to_str().unwrap(),
"git-upload-pack",
tmp.path(),
Bytes::new(),
git_timeout,
),
)
})
.await;
assert!(
outcome.is_ok(),
"run_git_service must return via its own timeout, not hang"
);
let err = outcome
.unwrap()
.expect_err("a hung git must surface as Err");
assert!(
err.downcast_ref::<GitServiceTimeout>().is_some(),
"the error must be GitServiceTimeout (maps to 504), got: {err:#}"
);
// The hung git's group must be torn down (the armed guard fires on the
// timeout return), not leaked.
let _cleanup = ReapOnPanic(vec![grandchild]);
let mut gone = false;
for _ in 0..500 {
if !alive(grandchild) {
gone = true;
break;
}
tokio::time::sleep(Duration::from_millis(10)).await;
}
assert!(
gone,
"a timed-out git's process group must be torn down, not leaked"
);
}
// On timeout, run_git_service must REAP the group before returning, so a
// caller releasing a write lock (receive-pack) can't race a still-live git.
// The fake traps SIGTERM and lingers ~1s (git-cleanup stand-in): with the
// reap the call waits for it to exit, so the leader is dead by the time we
// return; without it, the call returns while the fake is still mid-trap.
// Goes RED if the timeout arm's reap is removed.
#[cfg(unix)]
#[tokio::test]
async fn run_git_service_reaps_the_group_before_returning_on_timeout() {
let tmp = tempfile::TempDir::new().unwrap();
let pidfile = tmp.path().join("pids");
// Leader traps SIGTERM and exits after ~1s. The grandchild (a sub-shell)
// IGNORES SIGTERM entirely (`trap "" TERM`) and sleeps far past the ~4s reap
// cap, so nothing but the untrappable SIGKILL escalation can kill it within
// the cap. That makes the escalation load-bearing: neuter the SIGKILL and
// this test goes RED. (A grandchild that merely trapped-and-slept UNDER the
// cap would self-exit and hide a broken escalation.) "Wait for the whole
// group" (leader AND grandchild) is observable because both must be dead
// before run_git_service returns.
let body = format!(
"#!/bin/sh\ntrap 'sleep 1; exit 0' TERM\nsh -c 'trap \"\" TERM; sleep 300' >/dev/null 2>&1 &\nprintf '%s\\n%s\\n' \"$$\" \"$!\" > \"{}\"\nwait\n",
pidfile.display()
);
let git_bin = write_fake_git(tmp.path(), &body);
// fake_git_run_with_pids retries a transient spawn/schedule miss under
// fork-storm load and returns the pids the fake recorded; the outer 10s
// bound fires only if run_git_service's own timeout is broken (so we
// assert-fail rather than hang), well above the git_timeout plus the
// SIGKILL-escalation reap of the SIGTERM-ignoring grandchild.
let git_timeout = Duration::from_millis(300);
let (outcome, (leader, grandchild)) = fake_git_run_with_pids(&pidfile, || {
tokio::time::timeout(
Duration::from_secs(10),
run_git_service(
git_bin.to_str().unwrap(),
"git-upload-pack",
tmp.path(),
Bytes::new(),
git_timeout,
),
)
})
.await;
assert!(outcome.is_ok(), "must return via its own timeout, not hang");
assert!(outcome.unwrap().is_err(), "a hung git must surface as Err");
// The WHOLE group (leader AND the lingering grandchild) must be gone by the
// time we get here.
let _cleanup = ReapOnPanic(vec![grandchild]);
assert!(
!alive(leader) && !alive(grandchild),
"run_git_service must reap the whole git process group (leader AND \
grandchildren) before returning on a timeout, so a write-lock release \
can't race a still-live git"
);
}
// The timeout bounds git-receive-pack too, not just upload-pack: on the push
// path a hung git also holds the repo's write lock, so an unbounded receive-pack
// pins the repo until the process dies. run_git_service must return
// Err(GitServiceTimeout) for the receive-pack service exactly as it does for
// upload-pack. Goes RED if the internal timeout is removed (the fake never
// exits, so the outer bound trips and the outcome is not Ok).
#[cfg(unix)]
#[tokio::test]
async fn run_git_service_times_out_a_hung_receive_pack() {
let tmp = tempfile::TempDir::new().unwrap();
let git_bin = write_fake_git(tmp.path(), "#!/bin/sh\nsleep 300\n");
let outcome = tokio::time::timeout(
Duration::from_secs(5),
run_git_service(
git_bin.to_str().unwrap(),
"git-receive-pack",
tmp.path(),
Bytes::new(),
Duration::from_millis(200),
),
)
.await;
assert!(
outcome.is_ok(),
"receive-pack must return via its own timeout, not hang"
);
let err = outcome
.unwrap()
.expect_err("a hung receive-pack must surface as Err");
assert!(
err.downcast_ref::<GitServiceTimeout>().is_some(),
"a hung receive-pack must time out as GitServiceTimeout (maps to 504); \
got: {err:#}"
);
}
// A malformed request: git exits non-zero with a recognizable pkt-line error
// on stderr AND does not read stdin, so a body larger than the pipe buffer
// makes the write EPIPE. run_git_service must surface git's stderr (which the
// handler classifies as a 400), not the EPIPE write error (a generic 500).
// Goes RED if the stdin-write check runs before the exit-status check.
#[cfg(unix)]
#[tokio::test]
async fn run_git_service_surfaces_git_stderr_over_a_stdin_epipe() {
let tmp = tempfile::TempDir::new().unwrap();
let git_bin = write_fake_git(
tmp.path(),
"#!/bin/sh\necho 'fatal: bad line length character: 0000' >&2\nexit 128\n",
);
// Larger than a pipe buffer (~64 KiB) so the write blocks then EPIPEs when
// the fake exits without draining stdin, exercising the ordering.
let big = Bytes::from(vec![0u8; 256 * 1024]);
let result = run_git_service(
git_bin.to_str().unwrap(),
"git-upload-pack",
tmp.path(),
big,
Duration::from_secs(60),
)
.await;
let err = result.expect_err("non-zero git must surface as Err");
let msg = format!("{err:#}");
assert!(
msg.contains("bad line length"),
"run_git_service must surface git's stderr (a classifiable 400), not the \
stdin-write EPIPE (a generic 500); got: {msg}"
);
}
}
+29 -9
View File
@@ -271,9 +271,8 @@ pub struct TreeEntry {
/// `/ipfs/<cid>` is computed from these same content bytes via
/// `gitlawb_core::cid::Cid::from_git_object_bytes`.
///
/// Returns `None` if the object does not exist in this repo.
pub fn read_object(repo_path: &Path, sha256_hex: &str) -> Result<Option<(String, Vec<u8>)>> {
// First check if the object exists and get its type
/// Get just the object type. Returns `None` if the object doesn't exist.
pub fn object_type(repo_path: &Path, sha256_hex: &str) -> Result<Option<String>> {
let type_output = Command::new("git")
.args(["cat-file", "-t", sha256_hex])
.current_dir(repo_path)
@@ -284,13 +283,17 @@ pub fn read_object(repo_path: &Path, sha256_hex: &str) -> Result<Option<(String,
return Ok(None);
}
let obj_type = String::from_utf8_lossy(&type_output.stdout)
.trim()
.to_string();
Ok(Some(
String::from_utf8_lossy(&type_output.stdout)
.trim()
.to_string(),
))
}
// Read the raw content bytes
/// Read an object's content if its type is already known.
pub fn read_object_content(repo_path: &Path, sha256_hex: &str, obj_type: &str) -> Result<Vec<u8>> {
let content_output = Command::new("git")
.args(["cat-file", &obj_type, sha256_hex])
.args(["cat-file", obj_type, sha256_hex])
.current_dir(repo_path)
.output()
.context("failed to run git cat-file <type>")?;
@@ -300,7 +303,24 @@ pub fn read_object(repo_path: &Path, sha256_hex: &str) -> Result<Option<(String,
bail!("git cat-file failed: {stderr}");
}
Ok(Some((obj_type, content_output.stdout)))
Ok(content_output.stdout)
}
/// Read a git object by its SHA-256 hex object ID.
///
/// Returns `(object_type, content_bytes)` where `content_bytes` is the raw
/// object content (without the git framing header). The CID served over
/// `/ipfs/<cid>` is computed from these same content bytes via
/// `gitlawb_core::cid::Cid::from_git_object_bytes`.
///
/// Returns `None` if the object does not exist in this repo.
pub fn read_object(repo_path: &Path, sha256_hex: &str) -> Result<Option<(String, Vec<u8>)>> {
let obj_type = match object_type(repo_path, sha256_hex)? {
Some(t) => t,
None => return Ok(None),
};
let content = read_object_content(repo_path, sha256_hex, &obj_type)?;
Ok(Some((obj_type, content)))
}
/// Get the diff between two branches: changes on source_branch not in target_branch.
+95 -1
View File
@@ -309,11 +309,33 @@ pub fn replicable_blob_set(
rules: &[VisibilityRule],
is_public: bool,
owner_did: &str,
) -> Result<HashSet<String>> {
allowed_blob_set_for_caller(repo_path, rules, is_public, owner_did, None)
}
/// Reachable blob OIDs that visibility ALLOWS `caller` at some path. The
/// caller-aware generalization of `replicable_blob_set` (which is the anonymous
/// `caller = None` case). Used by `GET /ipfs/{cid}` to gate fail-closed against
/// dangling/unreachable blobs (#126): a blob written via `git hash-object -w`
/// but unreferenced is absent from the reachable walk, so it is never in this
/// set and the IPFS serve path drops it — even from the owner, who has no path
/// to authorize the blob at.
///
/// A blob reachable at an allowed path is included even when also denied
/// elsewhere (its content is readable to this caller elsewhere). Trees and
/// commits are NOT included here; the caller decides per object type whether
/// the allow-set applies (it does not for trees/commits — KTD3).
pub fn allowed_blob_set_for_caller(
repo_path: &Path,
rules: &[VisibilityRule],
is_public: bool,
owner_did: &str,
caller: Option<&str>,
) -> Result<HashSet<String>> {
let pairs = blob_paths(repo_path)?;
let mut allowed = HashSet::new();
for (oid, path) in &pairs {
if visibility_check(rules, is_public, owner_did, None, path) == Decision::Allow {
if visibility_check(rules, is_public, owner_did, caller, path) == Decision::Allow {
allowed.insert(oid.clone());
}
}
@@ -743,6 +765,78 @@ mod tests {
);
}
#[test]
fn allowed_set_excludes_dangling_blob_for_every_caller() {
// #126: a blob written via `git hash-object -w` but never referenced has
// no path to gate on, so it is absent from the reachable allowed-set —
// for anonymous callers, listed readers, AND the owner. The IPFS serve
// path relies on this fail-closed property to drop dangling withheld
// blobs that the deny-set model leaked.
let td = TempDir::new().unwrap();
let work = td.path().join("work");
std::fs::create_dir_all(work.join("public")).unwrap();
std::fs::write(work.join("public/a.txt"), b"public bytes\n").unwrap();
let run = |args: &[&str]| {
assert!(
Command::new("git")
.args(args)
.current_dir(&work)
.status()
.unwrap()
.success(),
"git {args:?} failed"
);
};
run(&["init", "-q"]);
run(&["config", "user.email", "t@t"]);
run(&["config", "user.name", "t"]);
run(&["add", "."]);
run(&["commit", "-qm", "init"]);
let oid_of = |rev: &str| {
let out = Command::new("git")
.args(["rev-parse", rev])
.current_dir(&work)
.output()
.unwrap();
String::from_utf8_lossy(&out.stdout).trim().to_string()
};
let public_oid = oid_of("HEAD:public/a.txt");
std::fs::write(work.join("orphan.bin"), b"DANGLING SECRET\n").unwrap();
let dangling_oid = {
let out = Command::new("git")
.args(["hash-object", "-w", "orphan.bin"])
.current_dir(&work)
.output()
.unwrap();
String::from_utf8_lossy(&out.stdout).trim().to_string()
};
assert!(
matches!(dangling_oid.len(), 40 | 64),
"precondition: hash-object stored the dangling blob"
);
// Path-scoped rule: /secret/** denied to anon, allowed to a listed reader.
let reader = "did:key:zReader";
let rules = [rule("/secret/**", &[reader])];
// Every gate-relevant caller: anonymous, listed reader, owner. None of
// them can put the dangling blob in the allowed set — it has no path.
for caller in [None, Some(reader), Some(OWNER)] {
let allowed = allowed_blob_set_for_caller(&work, &rules, true, OWNER, caller).unwrap();
assert!(
!allowed.contains(&dangling_oid),
"dangling blob must be absent from allowed-set (caller={caller:?})"
);
// Sanity: the reachable public blob is still in the set for every
// caller (the rule does not deny /public/**).
assert!(
allowed.contains(&public_oid),
"reachable public blob must be in allowed-set (caller={caller:?})"
);
}
}
#[test]
fn recipients_are_owner_plus_allowed_readers_only() {
let (_td, repo, secret_oid, public_oid) = fixture();
+307 -5
View File
@@ -49,13 +49,30 @@ impl QueryRoot {
&self,
ctx: &Context<'_>,
repo: Option<String>,
#[graphql(default = 20)] limit: i64,
#[graphql(
default = 20,
desc = "Max 200; larger requests return the newest 200 rows (no continuation cursor)."
)]
limit: i64,
) -> Result<Vec<RefUpdateType>> {
let db = ctx.data_unchecked::<Arc<Db>>();
let updates = db
.list_ref_updates_filtered(repo.as_deref(), limit)
.await
.map_err(|e| async_graphql::Error::new(e.to_string()))?;
// Gate each row on the same "/" visibility decision the repos resolver
// uses, so anonymous callers get no row for a local repo they can't read
// (#112). The shared collector applies the fail-closed gate *before* the
// limit (paging past dropped private rows) so a small limit still returns
// the latest visible events, and keeps this surface byte-identical to the
// REST feed (#114). The row slug is peer-supplied, so the pure filter
// treats it as untrusted input; remote (no local match) rows pass.
let caller = ctx
.data::<crate::auth::AuthenticatedDid>()
.ok()
.map(|d| d.0.as_str());
let updates =
crate::api::events::collect_visible_ref_updates(db, repo.as_deref(), limit, caller)
.await
.map_err(|e| async_graphql::Error::new(e.to_string()))?;
Ok(updates
.into_iter()
.map(|u| RefUpdateType {
@@ -94,3 +111,288 @@ impl QueryRoot {
Ok(t.map(AgentTaskType::from))
}
}
#[cfg(test)]
mod tests {
use crate::db::{Db, ReceivedRefUpdate, RepoRecord};
use chrono::Utc;
use sqlx::PgPool;
use std::sync::Arc;
const OWNER: &str = "did:key:z6MkOwner";
async fn db(pool: PgPool) -> Arc<Db> {
let db = Db::for_testing(pool);
db.run_migrations().await.unwrap();
Arc::new(db)
}
fn schema(db: Arc<Db>) -> super::super::GitlawbSchema {
let (ref_tx, _) = tokio::sync::broadcast::channel(16);
let (task_tx, _) = tokio::sync::broadcast::channel(16);
super::super::build_schema(db, ref_tx, task_tx)
}
fn repo(id: &str, owner_did: &str, name: &str, is_public: bool) -> RepoRecord {
RepoRecord {
id: id.into(),
name: name.into(),
owner_did: owner_did.into(),
description: None,
is_public,
default_branch: "main".into(),
created_at: Utc::now(),
updated_at: Utc::now(),
disk_path: format!("/srv/{id}"),
forked_from: None,
machine_id: None,
}
}
fn ref_row(id: &str, slug: &str) -> ReceivedRefUpdate {
ReceivedRefUpdate {
id: id.into(),
node_did: "did:key:z6MkNode".into(),
pusher_did: "did:key:z6MkPusher".into(),
repo: slug.into(),
ref_name: "refs/heads/main".into(),
old_sha: "0".repeat(40),
new_sha: "a".repeat(40),
timestamp: Utc::now().to_rfc3339(),
cert_id: None,
received_at: Utc::now().to_rfc3339(),
from_peer: "peer1".into(),
}
}
/// Count `refUpdates` rows in a GraphQL response.
fn count(resp: &async_graphql::Response) -> usize {
assert!(resp.errors.is_empty(), "graphql errors: {:?}", resp.errors);
let async_graphql::Value::Object(obj) = &resp.data else {
panic!("data not an object: {:?}", resp.data);
};
let async_graphql::Value::List(rows) = obj.get("refUpdates").expect("refUpdates key")
else {
panic!("refUpdates not a list");
};
rows.len()
}
async fn anon(schema: &super::super::GitlawbSchema, query: &str) -> async_graphql::Response {
schema.execute(async_graphql::Request::new(query)).await
}
async fn authed(
schema: &super::super::GitlawbSchema,
query: &str,
did: &str,
) -> async_graphql::Response {
schema
.execute(
async_graphql::Request::new(query)
.data(crate::auth::AuthenticatedDid(did.to_string())),
)
.await
}
// Scenario 1 — anon must not get a private local repo's row on the
// repo:Some branch. This is the load-bearing RED→GREEN case.
#[sqlx::test]
async fn ref_updates_private_repo_dropped_for_anon(pool: PgPool) {
let db = db(pool).await;
db.create_repo(&repo("r1", OWNER, "widget", false))
.await
.unwrap();
db.insert_ref_update(&ref_row("u1", "z6MkOwner/widget"))
.await
.unwrap();
let schema = schema(db);
// The GraphQL `repo` arg is the raw slug DB filter, so it must equal the
// stored slug to select the row at all — this is the exact leak path.
let q = r#"{ refUpdates(repo: "z6MkOwner/widget") { refName newSha pusherDid } }"#;
assert_eq!(count(&anon(&schema, q).await), 0);
}
// Scenario 2 — owner still sees their own private repo's row.
#[sqlx::test]
async fn ref_updates_private_repo_kept_for_owner(pool: PgPool) {
let db = db(pool).await;
db.create_repo(&repo("r1", OWNER, "widget", false))
.await
.unwrap();
db.insert_ref_update(&ref_row("u1", "z6MkOwner/widget"))
.await
.unwrap();
let schema = schema(db);
let q = r#"{ refUpdates(repo: "z6MkOwner/widget") { refName } }"#;
assert_eq!(count(&authed(&schema, q, OWNER).await), 1);
}
// Scenario 3 — unfiltered (repo:None): anon gets only the public row.
#[sqlx::test]
async fn ref_updates_unfiltered_anon_gets_only_public(pool: PgPool) {
let db = db(pool).await;
db.create_repo(&repo("pub", OWNER, "openrepo", true))
.await
.unwrap();
db.create_repo(&repo("priv", OWNER, "secret", false))
.await
.unwrap();
db.insert_ref_update(&ref_row("u_pub", "z6MkOwner/openrepo"))
.await
.unwrap();
db.insert_ref_update(&ref_row("u_priv", "z6MkOwner/secret"))
.await
.unwrap();
let schema = schema(db);
let q = r#"{ refUpdates { repo refName } }"#;
let resp = anon(&schema, q).await;
assert_eq!(count(&resp), 1);
// The one row returned must be the public repo's.
let async_graphql::Value::Object(obj) = &resp.data else {
unreachable!()
};
let async_graphql::Value::List(rows) = obj.get("refUpdates").unwrap() else {
unreachable!()
};
let async_graphql::Value::Object(row) = &rows[0] else {
unreachable!()
};
assert_eq!(
row.get("repo").unwrap(),
&async_graphql::Value::from("z6MkOwner/openrepo")
);
}
// Scenario 4 — alias fail-closed: private repo's row stored full-DID form.
#[sqlx::test]
async fn ref_updates_full_did_slug_dropped_for_anon(pool: PgPool) {
let db = db(pool).await;
db.create_repo(&repo("r1", "did:key:zABC", "widget", false))
.await
.unwrap();
db.insert_ref_update(&ref_row("u1", "did:key:zABC/widget"))
.await
.unwrap();
let schema = schema(db);
// repo:None so the slug is not the DB filter key (which is verbatim);
// the gate must still drop it.
let q = r#"{ refUpdates { repo } }"#;
assert_eq!(count(&anon(&schema, q).await), 0);
}
// Scenario 5 — truncated-key fail-closed: 8-char-prefix owner form.
#[sqlx::test]
async fn ref_updates_truncated_key_slug_dropped_for_anon(pool: PgPool) {
let db = db(pool).await;
db.create_repo(&repo("r1", "did:key:zABCDEFGH", "widget", false))
.await
.unwrap();
db.insert_ref_update(&ref_row("u1", "zABCDEF/widget"))
.await
.unwrap();
let schema = schema(db);
let q = r#"{ refUpdates { repo } }"#;
assert_eq!(count(&anon(&schema, q).await), 0);
}
// Scenario 6 — remote slug (no local match) is returned to anon.
#[sqlx::test]
async fn ref_updates_remote_slug_kept_for_anon(pool: PgPool) {
let db = db(pool).await;
db.create_repo(&repo("r1", OWNER, "widget", false))
.await
.unwrap();
// Row whose slug matches no local repo (different owner + name).
db.insert_ref_update(&ref_row("u1", "zZZZOTHER/gadget"))
.await
.unwrap();
let schema = schema(db);
let q = r#"{ refUpdates { repo } }"#;
assert_eq!(count(&anon(&schema, q).await), 1);
}
// Scenario 7 (#114 P2) — a small limit must page past the newest rows when
// they are private, so the older public rows are still returned. Before the
// gate moved ahead of the limit this returned 0 (the newest `limit` rows were
// all private and got filtered after the SQL LIMIT). RED→GREEN.
#[sqlx::test]
async fn ref_updates_small_limit_pages_past_newest_private(pool: PgPool) {
let db = db(pool).await;
db.create_repo(&repo("pub", OWNER, "openrepo", true))
.await
.unwrap();
db.create_repo(&repo("priv", OWNER, "secret", false))
.await
.unwrap();
// 3 older PUBLIC rows …
for i in 0..3 {
let mut r = ref_row(&format!("pub{i}"), "z6MkOwner/openrepo");
r.timestamp = format!("2026-07-01T10:00:0{i}+00:00");
db.insert_ref_update(&r).await.unwrap();
}
// … then 5 NEWER PRIVATE rows (the newest in the feed).
for i in 0..5 {
let mut r = ref_row(&format!("priv{i}"), "z6MkOwner/secret");
r.timestamp = format!("2026-07-01T10:00:1{i}+00:00");
db.insert_ref_update(&r).await.unwrap();
}
let schema = schema(db);
// limit 3 < the 5 newest (all private): anon must still get 3 public rows.
let q = r#"{ refUpdates(limit: 3) { repo } }"#;
let resp = anon(&schema, q).await;
assert_eq!(count(&resp), 3, "paging must reach the older public rows");
let async_graphql::Value::Object(obj) = &resp.data else {
unreachable!()
};
let async_graphql::Value::List(rows) = obj.get("refUpdates").unwrap() else {
unreachable!()
};
for row in rows {
let async_graphql::Value::Object(r) = row else {
unreachable!()
};
assert_eq!(
r.get("repo").unwrap(),
&async_graphql::Value::from("z6MkOwner/openrepo"),
"every returned row must be the public repo's"
);
}
}
// Scenario 8 — a quarantined mirror is withheld on the GraphQL surface too.
// Guards that the resolver keeps delegating to the shared collector (where the
// quarantine fold lives); a REST-only test would miss a resolver that stopped.
#[sqlx::test]
async fn ref_updates_quarantined_mirror_dropped_for_anon(pool: PgPool) {
let db = db(pool).await;
db.upsert_mirror_repo("z6MkQuar", "secret", "/tmp/q", None, true)
.await
.unwrap();
db.insert_ref_update(&ref_row("u1", "z6MkQuar/secret"))
.await
.unwrap();
let schema = schema(db);
let q = r#"{ refUpdates { repo } }"#;
assert_eq!(count(&anon(&schema, q).await), 0);
}
// Scenario 8b — the GraphQL surface also withholds a quarantined repo from an
// authenticated OWNER, not just anon. Without the collector's quarantine drop
// the owner short-circuit in visibility_check keeps the row on this surface
// too, so the REST owner test alone would not guard the resolver.
#[sqlx::test]
async fn ref_updates_quarantined_repo_dropped_for_owner(pool: PgPool) {
let db = db(pool).await;
db.create_repo(&repo("q1", "did:key:z6MkQuar", "secret", false))
.await
.unwrap();
db.set_repo_quarantine("q1", true).await.unwrap();
db.insert_ref_update(&ref_row("u1", "z6MkQuar/secret"))
.await
.unwrap();
let schema = schema(db);
let q = r#"{ refUpdates { repo } }"#;
assert_eq!(count(&authed(&schema, q, "did:key:z6MkQuar").await), 0);
}
}
@@ -11,6 +11,15 @@ pub struct SubscriptionRoot;
#[Subscription]
impl SubscriptionRoot {
/// Live ref-update stream. `/graphql/ws` is mounted outside the
/// `optional_signature` layer, so this resolver has NO caller identity and
/// cannot gate per-subscriber — it relays whatever enters the broadcast
/// channel to any anonymous client. Its visibility safety therefore rests
/// entirely on the WRITE side: the push handler only sends a
/// `RefUpdateBroadcast` for repos the anonymous public may read (inside its
/// `if announce` block, `api/repos.rs`). This is a single-point invariant —
/// any new sender to `ref_update_tx` MUST be `announce`-gated, or private-repo
/// ref metadata leaks here to unauthenticated subscribers (#112/#114 class).
async fn ref_updates(
&self,
ctx: &Context<'_>,
+24 -7
View File
@@ -237,9 +237,13 @@ impl ProofGuard {
match self.0 {
Some(p) => {
if !db.consume_proof_jti(&p.jti, p.exp).await? {
return Err(AppError::IcaptchaProofRequired(
"iCaptcha proof already used (replay); solve a fresh challenge".to_string(),
));
let (url, level) = url_and_level();
return Err(AppError::IcaptchaProofRequired {
message: "iCaptcha proof already used (replay); solve a fresh challenge"
.to_string(),
url,
level,
});
}
Ok(Some(p))
}
@@ -265,10 +269,23 @@ pub fn verify_request(headers: &HeaderMap, did: &str) -> Result<ProofGuard, AppE
}
fn reject_error(v: &Verifier, reason: &str) -> AppError {
AppError::IcaptchaProofRequired(format!(
"iCaptcha proof required ({reason}). Solve a challenge at {} for level >= {} and resend with the {} header.",
v.url, v.required_level, PROOF_HEADER
))
AppError::IcaptchaProofRequired {
message: format!(
"iCaptcha proof required ({reason}). Solve a challenge at {} for level >= {} and resend with the {} header.",
v.url, v.required_level, PROOF_HEADER
),
url: v.url.clone(),
level: v.required_level,
}
}
/// iCaptcha service url + required level for error responses, read from the
/// initialized verifier (falls back to defaults if somehow uninitialized).
fn url_and_level() -> (String, u32) {
VERIFIER
.get()
.map(|v| (v.url.clone(), v.required_level))
.unwrap_or_else(|| ("https://icaptcha.gitlawb.com".to_string(), 3))
}
/// Mode-aware decision. Pure and IO-free (no DB; clock injected via `now`) so it
+533 -91
View File
@@ -24,8 +24,14 @@ mod test_support;
mod visibility;
mod webhooks;
use anyhow::{Context, Result};
use anyhow::{anyhow, Context, Result};
use axum::extract::State;
use axum::http::StatusCode;
use axum::response::IntoResponse;
use axum::routing::get;
use axum::{Json, Router};
use clap::Parser;
use std::sync::atomic::{AtomicU64, Ordering};
use std::sync::Arc;
use tokio::net::TcpListener;
use tokio::sync::watch;
@@ -38,6 +44,20 @@ use config::Config;
use db::Db;
use state::AppState;
#[derive(Clone)]
struct DegradedState {
node_did: String,
db_startup: Arc<DbStartupStatus>,
}
/// Two independent counters with no cross-field invariant — atomics, not a
/// lock, so the retry loop and the degraded handlers never contend.
#[derive(Default)]
struct DbStartupStatus {
attempts: AtomicU64,
next_retry_secs: AtomicU64,
}
#[tokio::main]
async fn main() -> Result<()> {
tracing_subscriber::fmt()
@@ -76,21 +96,101 @@ async fn main() -> Result<()> {
env!("CARGO_PKG_VERSION")
);
info!("╚══════════════════════════════════════════╝");
info!(did = %node_did, "node identity");
info!(addr = %config.bind_addr(), "listening");
// Process-wide shutdown signal. One sender lives in AppState (cloned
// into every handler); main() keeps a clone and flips it on SIGINT
// or SIGTERM. Tasks that hold a watch::Receiver get notified at
// their next await point.
let (shutdown_tx, _shutdown_rx_for_main) = watch::channel(false);
spawn_shutdown_signal(shutdown_tx.clone());
// Connect to PostgreSQL database
let db = Arc::new(
Db::connect(&config.database_url)
.await
.context("failed to connect to database")?,
);
info!(did = %node_did, "node identity");
info!(addr = %config.bind_addr(), "binding HTTP listener");
// Bind HTTP once, before dependency initialization, and keep this socket
// for the life of the process. The degraded server accepts on a dup of the
// same socket, so the degraded→full handoff never closes the port: while
// the full server initializes, connections queue in the shared backlog
// instead of being refused.
let listener = TcpListener::bind(config.bind_addr())
.await
.with_context(|| format!("failed to bind to {}", config.bind_addr()))?;
let full_std_listener = listener.into_std()?;
let degraded_listener = TcpListener::from_std(
full_std_listener
.try_clone()
.context("failed to clone HTTP listener for degraded server")?,
)?;
// Metrics must stay observable during a database outage — the degraded
// window is exactly when dashboards need data — so this listener starts
// before the DB connects.
let metrics_handle = if !config.metrics_addr.is_empty() {
match spawn_metrics_server(&config.metrics_addr, shutdown_tx.subscribe()).await {
Ok(handle) => {
info!(addr = %config.metrics_addr, "metrics endpoint listening");
Some(handle)
}
Err(e) => {
warn!(err = %e, addr = %config.metrics_addr, "failed to start metrics endpoint — continuing without");
None
}
}
} else {
info!("metrics endpoint disabled (GITLAWB_METRICS_ADDR not set)");
None
};
let db_startup = Arc::new(DbStartupStatus::default());
let (db_ready_tx, db_ready_rx) = watch::channel(false);
let mut degraded_handle = tokio::spawn(run_degraded_server(
degraded_listener,
node_did.to_string(),
Arc::clone(&db_startup),
db_ready_rx,
shutdown_tx.subscribe(),
));
// Connect to PostgreSQL database. A transient outage or bad secret should
// not crash-loop the process and hammer the database provider; permanent
// misconfiguration surfaces through error-level logs and the /ready check.
let db = tokio::select! {
db = connect_db_with_retry(&config, Arc::clone(&db_startup), shutdown_tx.subscribe()) => {
match db {
Some(db) => db,
None => {
// Shutdown requested while waiting for the database. The
// degraded server only serves one-shot 503s — abort it
// rather than drain, so a slow client can't stall exit.
degraded_handle.abort();
return Ok(());
}
}
}
degraded = &mut degraded_handle => {
if *shutdown_tx.borrow() {
return Ok(());
}
return match degraded {
Ok(Ok(())) => Err(anyhow!("degraded HTTP server stopped before database became ready")),
Ok(Err(err)) => Err(err.context("degraded HTTP server failed")),
Err(err) => Err(anyhow!("degraded HTTP server task failed: {err}")),
};
}
};
// Flip the degraded server into graceful shutdown, but do NOT await the
// drain: one slow in-flight request must not delay the full server, and
// the shared socket means there is no port gap to cover. The drain
// finishes (and logs) in the background.
db_ready_tx.send(true).ok();
tokio::spawn(async move {
match degraded_handle.await {
Ok(Ok(())) => {}
Ok(Err(err)) => warn!(err = %err, "degraded HTTP server exited with error"),
Err(err) => warn!(err = %err, "degraded HTTP server task failed"),
}
});
info!(addr = %config.bind_addr(), "database ready; starting full HTTP server");
// Prune peer rows that point back at this node (stale self-loop entries)
if let Some(public_url) = config.public_url.as_deref() {
@@ -145,16 +245,8 @@ async fn main() -> Result<()> {
None
};
// Do not follow redirects: this client fans out to peer-supplied URLs
// (sync trigger, profile/repo fetches). A peer answering `302 Location:
// http://127.0.0.1/` would otherwise bypass the announce-time public-URL
// check and turn the redirect into an SSRF.
let http_client = Arc::new(
reqwest::Client::builder()
.timeout(std::time::Duration::from_secs(10))
.redirect(reqwest::redirect::Policy::none())
.build()?,
);
// Shared no-redirect HTTP client. See build_http_client for the SSRF rationale.
let http_client = Arc::new(build_http_client()?);
let (ref_update_tx, _) = tokio::sync::broadcast::channel::<state::RefUpdateBroadcast>(256);
let (task_event_tx, _) = tokio::sync::broadcast::channel::<state::TaskEventBroadcast>(256);
@@ -190,7 +282,79 @@ async fn main() -> Result<()> {
let repo_store =
git::repo_store::RepoStore::new(config.repos_dir.clone(), tigris, db.pool().clone());
let rate_limiter = rate_limit::RateLimiter::new(10, std::time::Duration::from_secs(3600));
// Per-DID limiter for the creation endpoints. Keyed on the authenticated
// DID (attacker-varied), so bound its key set to cap memory.
let rate_limiter =
rate_limit::RateLimiter::new_bounded(10, std::time::Duration::from_secs(3600), 200_000);
// Per-client-IP flood brake for the creation endpoints. The per-DID limiter
// above is bypassed by a DID farm (one throwaway did:key per repo), which is
// exactly how the recurring spam-repo floods get past both it and the
// iCaptcha gate. Keyed on the resolved client IP so a single-source flood is
// capped regardless of how many identities it mints. Sized well above any
// legitimate per-IP creation rate; GITLAWB_CREATE_RATE_LIMIT overrides, 0
// disables. Bounded key set — the key is a client-influenced IP.
let create_limit = std::env::var("GITLAWB_CREATE_RATE_LIMIT")
.ok()
.and_then(|v| v.trim().parse::<usize>().ok())
.unwrap_or(120);
let create_ip_rate_limiter = rate_limit::RateLimiter::new_bounded(
create_limit,
std::time::Duration::from_secs(3600),
200_000,
);
if create_limit == 0 {
tracing::warn!("GITLAWB_CREATE_RATE_LIMIT=0 — per-IP creation rate limiting disabled");
}
// Push-path flood brake: max git-receive-pack requests per client IP per
// hour (counts both the info/refs advertisement and the push POST). Sized
// for heavy agent automation while still stopping flood traffic (the June
// 2026 attack pushed several times per second per IP). GITLAWB_PUSH_RATE_LIMIT
// overrides; 0 disables. Bounded key set — the key is a client-influenced IP.
let push_limit = std::env::var("GITLAWB_PUSH_RATE_LIMIT")
.ok()
.and_then(|v| v.trim().parse::<usize>().ok())
.unwrap_or(600);
let push_rate_limiter = rate_limit::RateLimiter::new_bounded(
push_limit,
std::time::Duration::from_secs(3600),
200_000,
);
if push_limit == 0 {
tracing::warn!("GITLAWB_PUSH_RATE_LIMIT=0 — per-IP push rate limiting disabled");
}
// Which forwarded header the edge is trusted to set. Default None (trust
// nothing, key on the socket peer). Fly nodes set GITLAWB_TRUSTED_PROXY=fly;
// a node behind Caddy/NGINX sets it to x-forwarded-for.
let push_limiter_trust = rate_limit::TrustedProxy::from_env_value(
&std::env::var("GITLAWB_TRUSTED_PROXY").unwrap_or_default(),
);
tracing::info!(trust = ?push_limiter_trust, push_limit, "push rate limiter configured");
// Peer-sync flood brakes, keyed on the resolved client IP (per-DID is useless
// here — a did:key farm self-registers). Two buckets so an unsigned notify
// flood can't drain the signed trigger caller's quota (#82). Bounded key sets
// (the key is a client-influenced IP); 0 disables each.
let sync_trigger_rate_limiter = rate_limit::RateLimiter::new_bounded(
config.sync_trigger_rate_limit,
std::time::Duration::from_secs(3600),
200_000,
);
let peer_write_rate_limiter = rate_limit::RateLimiter::new_bounded(
config.peer_write_rate_limit,
std::time::Duration::from_secs(3600),
200_000,
);
if config.sync_trigger_rate_limit == 0 {
tracing::warn!(
"GITLAWB_SYNC_TRIGGER_RATE_LIMIT=0 — /sync/trigger IP rate limiting disabled"
);
}
if config.peer_write_rate_limit == 0 {
tracing::warn!("GITLAWB_PEER_WRITE_RATE_LIMIT=0 — peer-write IP rate limiting disabled");
}
// Initialize the iCaptcha proof gate (inert unless ICAPTCHA_MODE is set).
icaptcha::init().await;
@@ -208,6 +372,11 @@ async fn main() -> Result<()> {
machine_id,
repo_store,
rate_limiter,
create_ip_rate_limiter,
push_rate_limiter,
push_limiter_trust,
sync_trigger_rate_limiter,
peer_write_rate_limiter,
shutdown_tx: shutdown_tx.clone(),
};
@@ -237,37 +406,13 @@ async fn main() -> Result<()> {
});
}
// Spawn a task that flips the shutdown signal on SIGINT or SIGTERM.
// On Unix, both signals are handled. On Windows, only Ctrl-C is
// supported by tokio::signal::ctrl_c.
{
let tx = shutdown_tx.clone();
tokio::spawn(async move {
#[cfg(unix)]
{
use tokio::signal::unix::{signal as unix_signal, SignalKind};
let mut sigterm =
unix_signal(SignalKind::terminate()).expect("install SIGTERM handler");
let mut sigint =
unix_signal(SignalKind::interrupt()).expect("install SIGINT handler");
tokio::select! {
_ = sigterm.recv() => info!("SIGTERM received, shutting down"),
_ = sigint.recv() => info!("SIGINT received, shutting down"),
}
}
#[cfg(not(unix))]
{
use tokio::signal;
let _ = signal::ctrl_c().await;
info!("Ctrl-C received, shutting down");
}
tx.send(true).ok();
});
}
// Periodic cleanup of expired rate limit entries + consumed-proof ledger
{
let rl = state.rate_limiter.clone();
let create_ip_rl = state.create_ip_rate_limiter.clone();
let push_rl = state.push_rate_limiter.clone();
let sync_trigger_rl = state.sync_trigger_rate_limiter.clone();
let peer_write_rl = state.peer_write_rate_limiter.clone();
let db = state.db.clone();
let mut shutdown_rx = state.subscribe_shutdown();
tokio::spawn(async move {
@@ -275,6 +420,10 @@ async fn main() -> Result<()> {
tokio::select! {
_ = tokio::time::sleep(std::time::Duration::from_secs(300)) => {
rl.cleanup().await;
create_ip_rl.cleanup().await;
push_rl.cleanup().await;
sync_trigger_rl.cleanup().await;
peer_write_rl.cleanup().await;
let now = std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.map(|d| d.as_secs() as i64)
@@ -294,9 +443,10 @@ async fn main() -> Result<()> {
}
let router = server::build_router(state.clone());
let listener = TcpListener::bind(config.bind_addr())
.await
.with_context(|| format!("failed to bind to {}", config.bind_addr()))?;
// Re-register the socket bound at startup — same fd, so there was never a
// moment with the port closed between the degraded and full servers.
let listener = TcpListener::from_std(full_std_listener)
.context("failed to re-register HTTP listener with the runtime")?;
info!("✓ node started — did:{}", node_did);
info!(" repos dir: {}", config.repos_dir.display());
@@ -305,23 +455,6 @@ async fn main() -> Result<()> {
&config.database_url.split('@').next_back().unwrap_or("?")
);
// Optional Prometheus metrics listener on a separate port.
let metrics_handle = if !config.metrics_addr.is_empty() {
match spawn_metrics_server(&config.metrics_addr, state.clone()).await {
Ok(handle) => {
info!(addr = %config.metrics_addr, "metrics endpoint listening");
Some(handle)
}
Err(e) => {
warn!(err = %e, addr = %config.metrics_addr, "failed to start metrics endpoint — continuing without");
None
}
}
} else {
info!("metrics endpoint disabled (GITLAWB_METRICS_ADDR not set)");
None
};
// Publish our DID record to the Kademlia DHT shortly after startup
if let Some(p2p) = &state.p2p {
let did_record = p2p::DidRecord {
@@ -399,19 +532,25 @@ async fn main() -> Result<()> {
let grace = std::time::Duration::from_secs(config.shutdown_grace_secs);
info!(grace_secs = config.shutdown_grace_secs, "axum server ready");
let serve_result = axum::serve(listener, router)
.with_graceful_shutdown(async move {
let mut rx = shutdown_signal_for_axum;
// Wait until the watcher flips to true, then return so axum
// can begin draining.
while !*rx.borrow_and_update() {
if rx.changed().await.is_err() {
// Sender dropped — treat as shutdown.
break;
}
// `into_make_service_with_connect_info` exposes the socket peer address as
// `ConnectInfo<SocketAddr>` so the push limiter can key on the real client
// when no trusted proxy header applies (see `rate_limit::client_key`).
let serve_result = axum::serve(
listener,
router.into_make_service_with_connect_info::<std::net::SocketAddr>(),
)
.with_graceful_shutdown(async move {
let mut rx = shutdown_signal_for_axum;
// Wait until the watcher flips to true, then return so axum
// can begin draining.
while !*rx.borrow_and_update() {
if rx.changed().await.is_err() {
// Sender dropped — treat as shutdown.
break;
}
})
.await;
}
})
.await;
// Server has stopped accepting new connections and drained in-flight
// requests. Tear the rest of the system down.
@@ -425,12 +564,224 @@ async fn main() -> Result<()> {
Ok(())
}
fn spawn_shutdown_signal(tx: watch::Sender<bool>) {
tokio::spawn(async move {
#[cfg(unix)]
{
use tokio::signal::unix::{signal as unix_signal, SignalKind};
let mut sigterm =
unix_signal(SignalKind::terminate()).expect("install SIGTERM handler");
let mut sigint = unix_signal(SignalKind::interrupt()).expect("install SIGINT handler");
tokio::select! {
_ = sigterm.recv() => info!("SIGTERM received, shutting down"),
_ = sigint.recv() => info!("SIGINT received, shutting down"),
}
}
#[cfg(not(unix))]
{
use tokio::signal;
let _ = signal::ctrl_c().await;
info!("Ctrl-C received, shutting down");
}
tx.send(true).ok();
});
}
async fn connect_db_with_retry(
config: &Config,
db_startup: Arc<DbStartupStatus>,
mut shutdown_rx: watch::Receiver<bool>,
) -> Option<Arc<Db>> {
let initial_retry_secs = config.db_retry_initial_secs;
let max_retry_secs = config.db_retry_max_secs.max(initial_retry_secs);
let acquire_timeout = std::time::Duration::from_secs(config.db_acquire_timeout_secs);
let attempt_timeout = std::time::Duration::from_secs(config.db_connect_timeout_secs);
let mut attempts = 0_u64;
loop {
if *shutdown_rx.borrow() {
return None;
}
attempts = attempts.saturating_add(1);
db_startup.attempts.store(attempts, Ordering::Relaxed);
// Bound the whole attempt, not just the pool connect: migrations
// block on a cross-instance advisory lock, and an unbounded wait
// there would wedge this loop — no retries, no logs, no recovery.
// Timing out and retrying is safe; migrations are idempotent.
let attempt = match tokio::time::timeout(
attempt_timeout,
Db::connect(
&config.database_url,
config.db_max_connections,
acquire_timeout,
),
)
.await
{
Ok(result) => result,
Err(_) => Err(anyhow!(
"connect + migrate attempt exceeded {}s (GITLAWB_DB_CONNECT_TIMEOUT_SECS); \
is another instance holding the migration lock?",
attempt_timeout.as_secs()
)),
};
match attempt {
Ok(db) => {
info!(attempts, "database connection established");
return Some(Arc::new(db));
}
Err(err) => {
// A bad DATABASE_URL or rejected credentials won't heal on
// their own. Still retry (exiting would crash-loop and hammer
// the provider — and take liveness down with it), but log at
// error level and skip straight to the maximum backoff; the
// /ready health check is what surfaces this to deploys.
let permanent = is_likely_permanent_db_error(&err);
let retry_secs = if permanent {
max_retry_secs
} else {
database_retry_delay_secs(initial_retry_secs, max_retry_secs, attempts)
};
db_startup
.next_retry_secs
.store(retry_secs, Ordering::Relaxed);
if permanent {
tracing::error!(
attempts,
retry_secs,
err = %err,
"database rejected our configuration (bad DATABASE_URL or credentials?) — retrying, but operator action is likely required"
);
} else {
warn!(
attempts,
retry_secs,
err = %err,
"database unavailable during startup; retrying"
);
}
tokio::select! {
_ = tokio::time::sleep(std::time::Duration::from_secs(retry_secs)) => {}
changed = shutdown_rx.changed() => {
if changed.is_err() || *shutdown_rx.borrow() {
return None;
}
}
}
}
}
}
}
/// Errors that indicate misconfiguration rather than a transient outage: a
/// malformed DATABASE_URL, or a server that answered and rejected us —
/// Postgres error class 28xxx (invalid authorization) or 3D000 (database
/// does not exist). Best-effort: an error that anyhow can't downcast back to
/// sqlx just counts as transient.
fn is_likely_permanent_db_error(err: &anyhow::Error) -> bool {
match err.downcast_ref::<sqlx::Error>() {
Some(sqlx::Error::Configuration(_)) => true,
Some(sqlx::Error::Database(db)) => db
.code()
.map(|c| c.starts_with("28") || c.starts_with("3D"))
.unwrap_or(false),
_ => false,
}
}
fn database_retry_delay_secs(initial_secs: u64, max_secs: u64, attempts: u64) -> u64 {
// The exponent bound only keeps the u32 cast safe — max_secs is the real
// (operator-configurable) cap, and saturating math handles overflow.
let exponent = attempts.saturating_sub(1).min(63) as u32;
initial_secs
.saturating_mul(2_u64.saturating_pow(exponent))
.min(max_secs)
}
async fn run_degraded_server(
listener: TcpListener,
node_did: String,
db_startup: Arc<DbStartupStatus>,
mut db_ready_rx: watch::Receiver<bool>,
mut shutdown_rx: watch::Receiver<bool>,
) -> Result<()> {
let addr = listener.local_addr().ok();
let router = build_degraded_router(node_did, db_startup);
info!(?addr, "degraded HTTP server ready");
axum::serve(listener, router)
.with_graceful_shutdown(async move {
// wait_for resolves on predicate-true or sender-drop; either way
// this phase is over.
tokio::select! {
_ = db_ready_rx.wait_for(|ready| *ready) => {}
_ = shutdown_rx.wait_for(|stop| *stop) => {}
}
})
.await?;
Ok(())
}
fn build_degraded_router(node_did: String, db_startup: Arc<DbStartupStatus>) -> Router {
let state = DegradedState {
node_did,
db_startup,
};
// Everything answers 503 with the same body — including /health and
// /ready, so peer pings (which treat any 2xx /health as alive) and
// uptime monitors correctly see a node that cannot serve traffic.
// `/` additionally carries the node identity for probing peers.
Router::new()
.route("/", get(degraded_node_info))
.fallback(degraded_unavailable)
.with_state(state)
}
/// One source of truth for the degraded 503 body, sharing the error
/// vocabulary with error.rs so clients see the same code/message for
/// "database unavailable" regardless of which phase produced it.
fn degraded_body(db_startup: &DbStartupStatus) -> serde_json::Value {
serde_json::json!({
"status": "degraded",
"database": "initializing",
"error": error::DB_UNAVAILABLE_CODE,
"message": error::DB_UNAVAILABLE_MESSAGE,
"db_attempts": db_startup.attempts.load(Ordering::Relaxed),
"db_next_retry_secs": db_startup.next_retry_secs.load(Ordering::Relaxed),
})
}
async fn degraded_node_info(State(state): State<DegradedState>) -> impl IntoResponse {
let mut body = degraded_body(&state.db_startup);
if let Some(obj) = body.as_object_mut() {
obj.insert("name".into(), "gitlawb-node".into());
obj.insert("version".into(), env!("CARGO_PKG_VERSION").into());
obj.insert("did".into(), state.node_did.clone().into());
}
(StatusCode::SERVICE_UNAVAILABLE, Json(body))
}
async fn degraded_unavailable(State(state): State<DegradedState>) -> impl IntoResponse {
(
StatusCode::SERVICE_UNAVAILABLE,
Json(degraded_body(&state.db_startup)),
)
}
/// Spawn a small axum router that exposes only `GET /metrics` on its own
/// listener. Returns the JoinHandle so `main()` can abort it on shutdown.
/// This is deliberately separate from the main router so the metrics port
/// can be firewalled differently from the API port — bind to localhost
/// or a private interface only.
async fn spawn_metrics_server(addr: &str, state: AppState) -> Result<tokio::task::JoinHandle<()>> {
async fn spawn_metrics_server(
addr: &str,
mut shutdown_rx: watch::Receiver<bool>,
) -> Result<tokio::task::JoinHandle<()>> {
use axum::{response::IntoResponse, routing::get, Router};
async fn metrics_handler() -> impl IntoResponse {
@@ -454,7 +805,6 @@ async fn spawn_metrics_server(addr: &str, state: AppState) -> Result<tokio::task
}
}
let mut shutdown_rx = state.subscribe_shutdown();
let listener = TcpListener::bind(addr)
.await
.with_context(|| format!("failed to bind metrics listener to {addr}"))?;
@@ -515,7 +865,12 @@ async fn gossip_task(
}
}
let client = reqwest::Client::new();
// Reuse the shared no-redirect client for every gossip outbound call (the
// bootstrap announce POST and the periodic peer /health ping). Peer URLs are
// attacker-influenceable, so a 3xx to a private address must not be followed.
// Do NOT fall back to reqwest::Client::new(): its default follows redirects
// and would reintroduce the SSRF closed here (#93).
let client = state.http_client.clone();
let my_did = state.node_did.to_string();
let my_url = state.config.public_url.clone().unwrap_or_default();
@@ -590,13 +945,7 @@ async fn gossip_task(
Err(_) => continue,
};
for peer in peers {
let url = format!("{}/health", peer.http_url.trim_end_matches('/'));
let ok = client
.get(&url)
.send()
.await
.map(|r| r.status().is_success())
.unwrap_or(false);
let ok = ping_peer_health(&client, &peer.http_url).await;
let _ = state.db.mark_peer_ping(&peer.did, ok).await;
}
}
@@ -610,6 +959,38 @@ async fn gossip_task(
}
}
/// Build the shared node HTTP client used for every outbound fan-out (sync
/// trigger, profile/repo fetches, gossip announce + peer pings).
///
/// No redirects: peer URLs are attacker-influenceable, so a `3xx` to a private
/// address must not be followed (SSRF guard, #78/#93). Do NOT replace with
/// `reqwest::Client::new()` — its default follows redirects. Kept as a named
/// builder so tests bind the redirect guarantee to the real client the node
/// runs, not a hand-rolled equivalent.
fn build_http_client() -> reqwest::Result<reqwest::Client> {
reqwest::Client::builder()
.timeout(std::time::Duration::from_secs(10))
.redirect(reqwest::redirect::Policy::none())
.build()
}
/// Ping a peer's `/health` endpoint and report whether it answered 2xx.
///
/// Takes the client by reference so callers supply the shared, no-redirect
/// `state.http_client`. Peer URLs are attacker-influenceable, so a `3xx` to a
/// private address must not be followed. Do NOT call this with a bare
/// `reqwest::Client::new()`: its default follows redirects and would
/// reintroduce the SSRF this guards against (#93).
async fn ping_peer_health(client: &reqwest::Client, http_url: &str) -> bool {
let url = format!("{}/health", http_url.trim_end_matches('/'));
client
.get(&url)
.send()
.await
.map(|r| r.status().is_success())
.unwrap_or(false)
}
fn load_or_create_keypair(config: &Config) -> Result<Keypair> {
let key_path = config.resolved_key_path();
@@ -642,3 +1023,64 @@ fn load_or_create_keypair(config: &Config) -> Result<Keypair> {
Ok(kp)
}
}
#[cfg(test)]
mod gossip_ssrf_tests {
use super::ping_peer_health;
// Build the client exactly as production does (super::build_http_client) so
// these tests bind the redirect guarantee to the real shared client the
// node runs. A regression that makes build_http_client follow redirects
// fails ping_peer_health_does_not_follow_redirect.
fn production_http_client() -> reqwest::Client {
super::build_http_client().expect("failed to build production http client")
}
// A peer answering `/health` with a 302 toward an internal address must not
// be followed: the redirect target must never be requested (#93).
#[tokio::test]
async fn ping_peer_health_does_not_follow_redirect() {
let mut server = mockito::Server::new_async().await;
let internal = server
.mock("GET", "/internal-metadata")
.with_status(200)
.expect(0)
.create_async()
.await;
let _health = server
.mock("GET", "/health")
.with_status(302)
.with_header("location", &format!("{}/internal-metadata", server.url()))
.create_async()
.await;
let ok = ping_peer_health(&production_http_client(), &server.url()).await;
assert!(!ok, "a 302 must not count as a healthy peer");
// expect(0) is enforced only at assert time; this fails if the redirect
// was followed to the internal target.
internal.assert_async().await;
}
#[tokio::test]
async fn ping_peer_health_reports_success_on_200() {
let mut server = mockito::Server::new_async().await;
let _health = server
.mock("GET", "/health")
.with_status(200)
.create_async()
.await;
let ok = ping_peer_health(&production_http_client(), &server.url()).await;
assert!(ok, "a 200 /health must count as a healthy peer");
}
// A transport error (nothing listening) must map to unhealthy, never a
// spurious healthy — the .unwrap_or(false) arm.
#[tokio::test]
async fn ping_peer_health_reports_unhealthy_on_connection_error() {
let ok = ping_peer_health(&production_http_client(), "http://127.0.0.1:1").await;
assert!(!ok, "a connection error must count as an unhealthy peer");
}
}
+490 -19
View File
@@ -1,15 +1,21 @@
use std::collections::HashMap;
use std::net::SocketAddr;
use std::sync::Arc;
use std::time::{Duration, Instant};
use axum::extract::Request;
use axum::http::StatusCode;
use axum::extract::{ConnectInfo, Request};
use axum::http::{HeaderMap, StatusCode};
use axum::middleware::Next;
use axum::response::{IntoResponse, Response};
use tokio::sync::Mutex;
use crate::auth::AuthenticatedDid;
/// Default ceiling on the number of distinct keys a limiter tracks. Bounds the
/// limiter's own memory so a caller that can vary the key (many DIDs, or a
/// spoofable IP header) cannot grow the map without limit.
const DEFAULT_MAX_KEYS: usize = 100_000;
#[derive(Clone)]
struct Window {
timestamps: Vec<Instant>,
@@ -20,38 +26,98 @@ pub struct RateLimiter {
state: Arc<Mutex<HashMap<String, Window>>>,
max_requests: usize,
window: Duration,
/// Hard cap on tracked keys. When full, expired keys are evicted (at most
/// once per [`sweep_interval`](Self::sweep_interval), see below); if still
/// full, a request under a *new* key is rejected rather than inserted — so
/// the map can never exceed this bound and a rejected request never
/// allocates a new entry.
max_keys: usize,
/// Timestamp of the last inline capacity sweep. The eviction scan is
/// O(max_keys), so under a distinct-key flood (when the map sits at the
/// cap) running it on every miss would serialize all traffic behind a full
/// scan. Gating it to at most once per interval amortizes that cost; the
/// background [`cleanup`](Self::cleanup) loop still reclaims on its own.
last_sweep: Arc<Mutex<Instant>>,
}
impl RateLimiter {
// Retained for tests and callers that don't need a tight key bound;
// production limiters use `new_bounded`.
#[allow(dead_code)]
pub fn new(max_requests: usize, window: Duration) -> Self {
Self::new_bounded(max_requests, window, DEFAULT_MAX_KEYS)
}
/// Like [`new`] but with an explicit cap on the number of distinct keys.
/// Production limiters keyed on client-influenced values (per-DID, per-IP)
/// set this so the limiter's own state cannot be turned into a
/// memory-exhaustion vector.
pub fn new_bounded(max_requests: usize, window: Duration, max_keys: usize) -> Self {
Self {
state: Arc::new(Mutex::new(HashMap::new())),
max_requests,
window,
max_keys: max_keys.max(1),
last_sweep: Arc::new(Mutex::new(Instant::now())),
}
}
async fn check(&self, key: &str) -> bool {
/// How often the inline capacity sweep may run: at most once per second, or
/// once per window when the window is shorter. Bounds post-flood staleness
/// (a full map self-heals within one interval) without paying the O(max_keys)
/// scan on every capacity miss.
fn sweep_interval(&self) -> Duration {
self.window.min(Duration::from_secs(1))
}
pub(crate) async fn check(&self, key: &str) -> bool {
// max_requests == 0 means the limiter is disabled, not "block all".
if self.max_requests == 0 {
return true;
}
let now = Instant::now();
let mut state = self.state.lock().await;
// Look up before inserting so the common case (key already tracked)
// doesn't allocate a String per request.
if !state.contains_key(key) {
state.insert(
key.to_string(),
Window {
timestamps: Vec::new(),
},
);
// Fast path: an already-tracked key never allocates and never grows the
// map, so it is unaffected by the key cap.
if let Some(window) = state.get_mut(key) {
window
.timestamps
.retain(|t| now.duration_since(*t) < self.window);
if window.timestamps.len() >= self.max_requests {
return false;
}
window.timestamps.push(now);
return true;
}
let window = state.get_mut(key).expect("window just ensured");
window
.timestamps
.retain(|t| now.duration_since(*t) < self.window);
if window.timestamps.len() >= self.max_requests {
return false;
// New key. Enforce the cap BEFORE inserting so a flood of distinct keys
// cannot grow the map, and a rejected request never allocates an entry.
if state.len() >= self.max_keys {
// Reclaim expired keys, but at most once per sweep interval — the
// scan is O(max_keys) and the map sits at the cap precisely during a
// distinct-key flood, so sweeping per miss would serialize traffic
// behind a full scan. Between sweeps a new key is simply rejected.
let mut last_sweep = self.last_sweep.lock().await;
if now.duration_since(*last_sweep) >= self.sweep_interval() {
state.retain(|_, w| {
w.timestamps
.retain(|t| now.duration_since(*t) < self.window);
!w.timestamps.is_empty()
});
*last_sweep = now;
}
drop(last_sweep);
if state.len() >= self.max_keys {
return false;
}
}
window.timestamps.push(now);
state.insert(
key.to_string(),
Window {
timestamps: vec![now],
},
);
true
}
@@ -88,6 +154,136 @@ pub async fn rate_limit_by_did(request: Request, next: Next) -> Response {
next.run(request).await
}
/// Which forwarded header (if any) the operator's edge is trusted to set. Only
/// a proxy the operator controls may be believed; a raw client can put any
/// value in `Fly-Client-IP` / `X-Forwarded-For`, so trusting them unconditionally
/// lets a flooder rotate the header and never fill a bucket. Configured via
/// `GITLAWB_TRUSTED_PROXY`.
#[derive(Clone, Copy, PartialEq, Eq, Debug)]
pub enum TrustedProxy {
/// No trusted proxy: ignore forwarded headers, key on the socket peer IP.
/// Safe default for direct/self-hosted nodes.
None,
/// Behind Fly's edge, which sets (and overwrites any client-supplied)
/// `Fly-Client-IP`.
Fly,
/// Behind a single reverse proxy (e.g. Caddy on the AWS image) that appends
/// the real client as the rightmost `X-Forwarded-For` hop.
XForwardedFor,
}
impl TrustedProxy {
/// Parse `GITLAWB_TRUSTED_PROXY`. Unknown/empty → `None` (trust nothing).
pub fn from_env_value(s: &str) -> Self {
match s.trim().to_ascii_lowercase().as_str() {
"fly" | "fly-client-ip" => TrustedProxy::Fly,
"xff" | "x-forwarded-for" | "caddy" => TrustedProxy::XForwardedFor,
_ => TrustedProxy::None,
}
}
}
fn trimmed_nonempty(v: &str) -> Option<String> {
let t = v.trim();
(!t.is_empty()).then(|| t.to_string())
}
/// Resolve the rate-limit key for a request. In a trusted-proxy mode the
/// operator's edge header is preferred; when that header is missing or empty we
/// fall back to the socket peer address rather than skipping the limiter (a
/// malformed header must never disable the brake). With no trusted proxy the
/// socket peer is always used and forwarded headers are ignored entirely.
/// Returns `None` only when neither a trusted header nor a peer address is
/// available (e.g. a synthetic test request with no `ConnectInfo`).
pub fn client_key(
headers: &HeaderMap,
peer: Option<SocketAddr>,
trust: TrustedProxy,
) -> Option<String> {
let from_header = match trust {
TrustedProxy::None => None,
TrustedProxy::Fly => headers
.get("fly-client-ip")
.and_then(|v| v.to_str().ok())
.and_then(trimmed_nonempty),
// Rightmost hop = the value appended by the immediately-upstream trusted
// proxy. The leftmost hop is client-controlled and must not be trusted.
TrustedProxy::XForwardedFor => headers
.get("x-forwarded-for")
.and_then(|v| v.to_str().ok())
.and_then(|v| v.rsplit(',').next())
.and_then(trimmed_nonempty),
};
from_header.or_else(|| peer.map(|p| p.ip().to_string()))
}
/// Per-client-IP limiter for the git push path, carrying the trusted-proxy
/// policy. A newtype so it can live in request extensions (keyed by type)
/// alongside the per-DID [`RateLimiter`]. Per-DID limits are useless against a
/// push flood from a DID farm (one throwaway DID per repo), so the push path
/// throttles on the resolved client IP instead.
#[derive(Clone)]
pub struct IpRateLimiter {
pub limiter: RateLimiter,
pub trust: TrustedProxy,
}
/// Infallible extractor for the socket peer address from `ConnectInfo`. Yields
/// `None` when the server was started without connect-info (e.g. `oneshot` in
/// tests), so a handler never 500s on its absence — the limiter simply falls
/// back per [`client_key`].
pub struct PeerAddr(pub Option<SocketAddr>);
impl<S: Send + Sync> axum::extract::FromRequestParts<S> for PeerAddr {
type Rejection = std::convert::Infallible;
async fn from_request_parts(
parts: &mut axum::http::request::Parts,
_state: &S,
) -> Result<Self, Self::Rejection> {
Ok(PeerAddr(
parts
.extensions
.get::<ConnectInfo<SocketAddr>>()
.map(|c| c.0),
))
}
}
/// The shared 429 response for the per-IP flood brakes. Route-agnostic: this
/// middleware now serves the push path AND the peer-sync routes, so the message
/// stays generic (the offending path is recorded in the warn log below).
pub fn too_many_requests() -> Response {
(
StatusCode::TOO_MANY_REQUESTS,
[("retry-after", "60")],
"rate limit exceeded — try again later",
)
.into_response()
}
/// Throttle the git push path by resolved client IP. The socket peer address is
/// read from `ConnectInfo` (see `into_make_service_with_connect_info` in
/// `main`). Only skips the limiter when no key can be resolved at all.
pub async fn rate_limit_by_ip(request: Request, next: Next) -> Response {
let limiter = request.extensions().get::<IpRateLimiter>().cloned();
let peer = request
.extensions()
.get::<ConnectInfo<SocketAddr>>()
.map(|c| c.0);
if let Some(limiter) = limiter {
if let Some(key) = client_key(request.headers(), peer, limiter.trust) {
if !limiter.limiter.check(&key).await {
tracing::warn!(key = %key, path = %request.uri().path(), "per-IP rate limit exceeded");
return too_many_requests();
}
}
}
next.run(request).await
}
#[cfg(test)]
mod tests {
use super::*;
@@ -134,4 +330,279 @@ mod tests {
let state = limiter.state.lock().await;
assert!(state.is_empty());
}
#[tokio::test]
async fn zero_limit_disables() {
let limiter = RateLimiter::new(0, Duration::from_secs(60));
for _ in 0..1000 {
assert!(limiter.check("k").await);
}
}
// ── key-cap / memory-bound (P2) ──────────────────────────────────────
#[tokio::test]
async fn caps_tracked_keys_and_rejects_new_ones_when_full() {
// Cap of 2 distinct keys; generous request budget so rejection is due to
// the key cap, not the per-key limit.
let limiter = RateLimiter::new_bounded(100, Duration::from_secs(60), 2);
assert!(limiter.check("a").await);
assert!(limiter.check("b").await);
// Third distinct key would grow the map past the cap → rejected.
assert!(!limiter.check("c").await);
// The map never exceeded the cap, and the rejected key was NOT inserted.
let state = limiter.state.lock().await;
assert_eq!(state.len(), 2);
assert!(!state.contains_key("c"));
}
#[tokio::test]
async fn known_key_unaffected_by_cap() {
let limiter = RateLimiter::new_bounded(100, Duration::from_secs(60), 1);
assert!(limiter.check("a").await); // fills the single slot
assert!(limiter.check("a").await); // same key still served
assert!(!limiter.check("b").await); // new key rejected — cap full
}
#[tokio::test]
async fn expired_keys_evicted_to_admit_new_when_full() {
let limiter = RateLimiter::new_bounded(100, Duration::from_millis(40), 1);
assert!(limiter.check("old").await);
tokio::time::sleep(Duration::from_millis(55)).await;
// "old" is now expired; a new key triggers inline eviction and is admitted.
assert!(limiter.check("new").await);
let state = limiter.state.lock().await;
assert!(state.contains_key("new"));
assert!(!state.contains_key("old"));
}
#[tokio::test]
async fn capacity_sweep_is_amortized_within_interval() {
// The O(max_keys) eviction scan must not run on every capacity miss.
// With a 1s sweep interval, the resident (unexpired) key is never
// evicted by a burst of misses, so repeated new keys are all rejected
// without disturbing existing state.
let limiter = RateLimiter::new_bounded(100, Duration::from_secs(1), 1);
assert!(limiter.check("resident").await);
for i in 0..100 {
assert!(
!limiter.check(&format!("flood-{i}")).await,
"new key must be rejected while the single slot is held"
);
}
let state = limiter.state.lock().await;
assert_eq!(state.len(), 1, "no flood key was ever inserted");
assert!(state.contains_key("resident"));
}
// ── client_key / trusted-proxy resolution (P1 + P2) ─────────────────
fn headers(pairs: &[(&str, &str)]) -> HeaderMap {
let mut h = HeaderMap::new();
for (k, v) in pairs {
h.insert(
axum::http::HeaderName::from_bytes(k.as_bytes()).unwrap(),
v.parse().unwrap(),
);
}
h
}
fn peer(s: &str) -> Option<SocketAddr> {
Some(s.parse().unwrap())
}
#[test]
fn none_mode_ignores_headers_and_uses_peer() {
// Even a well-formed Fly header is ignored without a trusted proxy.
let h = headers(&[("fly-client-ip", "203.0.113.7")]);
assert_eq!(
client_key(&h, peer("198.51.100.9:4000"), TrustedProxy::None).as_deref(),
Some("198.51.100.9")
);
}
#[test]
fn fly_mode_trusts_fly_header() {
let h = headers(&[
("fly-client-ip", "203.0.113.7"),
("x-forwarded-for", "1.2.3.4"),
]);
assert_eq!(
client_key(&h, peer("10.0.0.1:1"), TrustedProxy::Fly).as_deref(),
Some("203.0.113.7")
);
}
#[test]
fn fly_mode_empty_header_falls_back_to_peer_not_skip() {
// Empty Fly-Client-IP must NOT collapse traffic onto Some("") nor skip
// the limiter — it falls back to the real peer.
let h = headers(&[("fly-client-ip", "")]);
assert_eq!(
client_key(&h, peer("198.51.100.9:4000"), TrustedProxy::Fly).as_deref(),
Some("198.51.100.9")
);
}
#[test]
fn xff_mode_uses_rightmost_hop_not_client_controlled_first() {
// Client prepends spoofed hops; only the rightmost (proxy-appended) is trusted.
let h = headers(&[("x-forwarded-for", "9.9.9.9, 8.8.8.8, 198.51.100.9")]);
assert_eq!(
client_key(&h, peer("10.0.0.1:1"), TrustedProxy::XForwardedFor).as_deref(),
Some("198.51.100.9")
);
}
#[test]
fn xff_mode_empty_leading_hop_does_not_disable_brake() {
// "X-Forwarded-For: ,1.2.3.4" — rightmost hop is used; never None-skips.
let h = headers(&[("x-forwarded-for", ",1.2.3.4")]);
assert_eq!(
client_key(&h, peer("10.0.0.1:1"), TrustedProxy::XForwardedFor).as_deref(),
Some("1.2.3.4")
);
}
#[test]
fn malformed_xff_falls_back_to_peer() {
let h = headers(&[("x-forwarded-for", " , ")]);
assert_eq!(
client_key(&h, peer("198.51.100.9:4000"), TrustedProxy::XForwardedFor).as_deref(),
Some("198.51.100.9")
);
}
#[test]
fn trusted_proxy_parsing() {
assert_eq!(TrustedProxy::from_env_value("fly"), TrustedProxy::Fly);
assert_eq!(
TrustedProxy::from_env_value("X-Forwarded-For"),
TrustedProxy::XForwardedFor
);
assert_eq!(
TrustedProxy::from_env_value("caddy"),
TrustedProxy::XForwardedFor
);
assert_eq!(TrustedProxy::from_env_value(""), TrustedProxy::None);
assert_eq!(TrustedProxy::from_env_value("garbage"), TrustedProxy::None);
}
// ── middleware 429 path ─────────────────────────────────────────────
/// A minimal router with the push limiter layered over an OK handler,
/// driven via `oneshot`. `ConnectInfo` is attached to each request directly
/// (as the production make-service does) so the middleware resolves a peer.
fn ip_limited_router(limiter: IpRateLimiter) -> axum::Router {
axum::Router::new()
.route(
"/o/r/git-receive-pack",
axum::routing::post(|| async { StatusCode::OK }),
)
.layer(axum::middleware::from_fn(rate_limit_by_ip))
.layer(axum::Extension(limiter))
}
async fn post_from(router: &axum::Router, peer: SocketAddr) -> StatusCode {
use tower::ServiceExt;
let mut req = axum::http::Request::builder()
.method(axum::http::Method::POST)
.uri("/o/r/git-receive-pack")
.body(axum::body::Body::empty())
.unwrap();
req.extensions_mut().insert(ConnectInfo(peer));
router.clone().oneshot(req).await.unwrap().status()
}
#[tokio::test]
async fn middleware_returns_429_over_limit() {
let router = ip_limited_router(IpRateLimiter {
limiter: RateLimiter::new(2, Duration::from_secs(60)),
trust: TrustedProxy::None,
});
let peer: SocketAddr = "203.0.113.7:5000".parse().unwrap();
// Two requests inside the budget, the third over it (shared Arc state).
assert_eq!(post_from(&router, peer).await, StatusCode::OK);
assert_eq!(post_from(&router, peer).await, StatusCode::OK);
assert_eq!(
post_from(&router, peer).await,
StatusCode::TOO_MANY_REQUESTS
);
}
#[tokio::test]
async fn middleware_isolates_distinct_peers() {
let router = ip_limited_router(IpRateLimiter {
limiter: RateLimiter::new(1, Duration::from_secs(60)),
trust: TrustedProxy::None,
});
let a: SocketAddr = "203.0.113.1:1".parse().unwrap();
let b: SocketAddr = "203.0.113.2:1".parse().unwrap();
assert_eq!(post_from(&router, a).await, StatusCode::OK);
assert_eq!(post_from(&router, b).await, StatusCode::OK); // independent bucket
assert_eq!(post_from(&router, a).await, StatusCode::TOO_MANY_REQUESTS);
}
// ── creation-route layering: IP brake runs BEFORE auth ──────────────
/// A minimal stand-in for the creation route group: an inner "auth" layer
/// that rejects every request with 401 (as signature verification would for
/// an unsigned caller), wrapped by the per-IP creation limiter as the
/// outermost layer — exactly the ordering `server::build_router` applies to
/// `/api/v1/repos` et al. Lets us assert the security-relevant property
/// (flood traffic is braked before auth burns CPU) without a database, which
/// the full `#[sqlx::test]` route test in `api::repos` cannot cover in the
/// plain unit-test pass.
fn ip_limited_over_auth_router(limiter: IpRateLimiter) -> axum::Router {
async fn always_unauthorized(_req: Request, _next: Next) -> Response {
StatusCode::UNAUTHORIZED.into_response()
}
axum::Router::new()
.route(
"/api/v1/repos",
axum::routing::post(|| async { StatusCode::CREATED }),
)
.layer(axum::middleware::from_fn(always_unauthorized))
.layer(axum::middleware::from_fn(rate_limit_by_ip))
.layer(axum::Extension(limiter))
}
async fn post_repos_from(router: &axum::Router, peer: SocketAddr) -> StatusCode {
use tower::ServiceExt;
let mut req = axum::http::Request::builder()
.method(axum::http::Method::POST)
.uri("/api/v1/repos")
.body(axum::body::Body::empty())
.unwrap();
req.extensions_mut().insert(ConnectInfo(peer));
router.clone().oneshot(req).await.unwrap().status()
}
#[tokio::test]
async fn creation_ip_brake_rejects_before_auth() {
// A DID farm mints a fresh signature per repo, so the per-DID limiter
// never trips; the per-IP brake is what stops the flood. It must run
// outermost, ahead of auth, so a flood is rejected with 429 before
// signature verification — otherwise every request would reach (and pay
// for) auth and merely 401.
let router = ip_limited_over_auth_router(IpRateLimiter {
limiter: RateLimiter::new(1, Duration::from_secs(60)),
trust: TrustedProxy::None,
});
let peer: SocketAddr = "203.0.113.42:7000".parse().unwrap();
// First request passes the IP brake and reaches the (failing) auth layer.
assert_eq!(
post_repos_from(&router, peer).await,
StatusCode::UNAUTHORIZED
);
// Budget now exhausted: the IP brake short-circuits with 429 *before*
// auth — proving the limiter is the outermost layer.
assert_eq!(
post_repos_from(&router, peer).await,
StatusCode::TOO_MANY_REQUESTS,
"an exhausted IP must be braked with 429 before auth runs, not leak to 401"
);
}
}
+77 -5
View File
@@ -81,8 +81,19 @@ pub fn build_router(state: AppState) -> Router {
.route("/api/v1/tasks", get(tasks::list_tasks))
.route("/api/v1/tasks/{id}", get(tasks::get_task));
// ── Rate-limited creation routes — require HTTP Signature + per-DID throttle
// ── Rate-limited creation routes — require HTTP Signature, plus a per-DID
// throttle AND a per-IP flood brake. The per-DID limiter (inner) caps a
// single identity; the per-IP limiter (outer) caps a DID farm that mints a
// fresh throwaway did:key per repo to slip past both the per-DID limit and
// the iCaptcha gate — the mechanism behind the recurring spam-repo floods.
// The per-IP layer wraps the auth layer (outermost = runs first) so flood
// traffic is rejected before signature verification burns CPU, matching the
// push path.
let limiter = state.rate_limiter.clone();
let create_ip_limiter = rate_limit::IpRateLimiter {
limiter: state.create_ip_rate_limiter.clone(),
trust: state.push_limiter_trust,
};
let creation_routes = add_auth_layers(
Router::new()
.route("/api/v1/repos", post(repos::create_repo))
@@ -96,7 +107,9 @@ pub fn build_router(state: AppState) -> Router {
.layer(middleware::from_fn(rate_limit::rate_limit_by_did))
.layer(axum::Extension(limiter)),
state.clone(),
);
)
.layer(middleware::from_fn(rate_limit::rate_limit_by_ip))
.layer(axum::Extension(create_ip_limiter));
// ── Write routes — require HTTP Signature (no rate limit) ─────────────
let write_routes = add_auth_layers(
@@ -175,6 +188,14 @@ pub fn build_router(state: AppState) -> Router {
// HTTP Signature is enforced on receive-pack (push) — the git-remote-gitlawb
// helper signs requests with RFC 9421 signatures using the agent's keypair.
let pack_limit = state.config.max_pack_bytes;
// Per-IP throttle wraps the auth layer (outermost = runs first): flood
// traffic is rejected before signature verification burns CPU. Per-DID
// limiting is deliberately NOT used here — a DID farm (one throwaway
// identity per repo, as in the June 2026 push flood) never trips it.
let push_limiter = rate_limit::IpRateLimiter {
limiter: state.push_rate_limiter.clone(),
trust: state.push_limiter_trust,
};
let git_write_routes = add_auth_layers(
Router::new()
.route(
@@ -184,7 +205,9 @@ pub fn build_router(state: AppState) -> Router {
.layer(DefaultBodyLimit::disable())
.layer(RequestBodyLimitLayer::new(pack_limit)),
state.clone(),
);
)
.layer(middleware::from_fn(rate_limit::rate_limit_by_ip))
.layer(axum::Extension(push_limiter));
// ── IPFS content-addressed retrieval and pin listing ──────────────────
// `/ipfs/{cid}` carries `optional_signature` so `get_by_cid` sees the caller
@@ -238,7 +261,8 @@ pub fn build_router(state: AppState) -> Router {
.route(
"/api/v1/agents/{did}/bounties",
get(bounties::agent_bounty_stats),
);
)
.layer(middleware::from_fn(auth::optional_signature));
// ── Profile routes (write — require HTTP Signature) ─────────────────
let profile_write_routes = add_auth_layers(
@@ -267,15 +291,41 @@ pub fn build_router(state: AppState) -> Router {
.route("/api/v1/peers", get(peers::list_peers))
.route("/api/v1/peers/{did}/ping", get(peers::ping_peer));
// /sync/trigger drives an O(peers) outbound fan-out + per-repo enqueue, so it
// ALWAYS requires a signature (both config modes) and carries a tight per-IP
// brake. A signature alone does not cap cost — a did:key farm self-registers
// (INV-10) — so the IP brake is a separate, load-bearing half. The brake is
// outermost (runs before signature verification burns CPU) and is keyed on the
// client IP before any DID is read, so DID rotation cannot bypass it.
let sync_trigger_routes = add_auth_layers(
Router::new().route("/api/v1/sync/trigger", post(peers::trigger_sync)),
state.clone(),
)
.layer(middleware::from_fn(rate_limit::rate_limit_by_ip))
.layer(axum::Extension(rate_limit::IpRateLimiter {
limiter: state.sync_trigger_rate_limiter.clone(),
trust: state.push_limiter_trust,
}));
// announce + notify keep their rolling-upgrade signature behavior (unsigned
// accepted until all peers upgrade), but both reach peer-write side effects —
// notify hits the same enqueue_sync sink as trigger — so they carry a per-IP
// brake too, on a SEPARATE bucket from trigger's, so an unsigned notify flood
// cannot drain the signed trigger caller's quota.
let mut peer_write_routes = Router::new()
.route("/api/v1/peers/announce", post(peers::announce))
.route("/api/v1/sync/trigger", post(peers::trigger_sync))
.route("/api/v1/sync/notify", post(peers::notify_sync));
peer_write_routes = if state.config.require_signed_peer_writes {
add_auth_layers(peer_write_routes, state.clone())
} else {
peer_write_routes.layer(middleware::from_fn(auth::optional_signature))
};
let peer_write_routes = peer_write_routes
.layer(middleware::from_fn(rate_limit::rate_limit_by_ip))
.layer(axum::Extension(rate_limit::IpRateLimiter {
limiter: state.peer_write_rate_limiter.clone(),
trust: state.push_limiter_trust,
}));
// ── Read routes — open for public repos ───────────────────────────────
let read_routes = Router::new()
@@ -405,6 +455,7 @@ pub fn build_router(state: AppState) -> Router {
let meta_routes = Router::new()
.route("/", get(node_info))
.route("/health", get(health))
.route("/ready", get(ready))
.route("/api/v1/p2p/info", get(p2p_info))
.route("/api/v1/stats", get(stats))
.route("/api/v1/contracts", get(contracts_info));
@@ -424,6 +475,7 @@ pub fn build_router(state: AppState) -> Router {
.merge(read_routes)
.merge(peer_read_routes)
.merge(peer_write_routes)
.merge(sync_trigger_routes)
.merge(ipfs_routes)
.merge(arweave_routes)
.merge(meta_routes)
@@ -446,6 +498,26 @@ async fn health() -> Json<serde_json::Value> {
Json(json!({ "status": "ok" }))
}
/// Readiness = "this node can serve real traffic", and every real endpoint
/// depends on the database, so probe it rather than reporting a constant.
/// The probe gets its own short bound so a wedged pool can't hang the health
/// check for the full acquire timeout.
async fn ready(State(state): State<AppState>) -> axum::response::Response {
let probe = tokio::time::timeout(std::time::Duration::from_secs(2), state.db.ping()).await;
match probe {
Ok(Ok(())) => Json(json!({ "status": "ready" })).into_response(),
_ => (
axum::http::StatusCode::SERVICE_UNAVAILABLE,
Json(json!({
"status": "degraded",
"error": crate::error::DB_UNAVAILABLE_CODE,
"message": crate::error::DB_UNAVAILABLE_MESSAGE,
})),
)
.into_response(),
}
}
async fn node_info(State(state): State<AppState>) -> Json<serde_json::Value> {
let p2p_peer_id = state.p2p.as_ref().map(|h| h.local_peer_id.to_string());
Json(json!({
+29
View File
@@ -51,6 +51,35 @@ pub struct AppState {
pub repo_store: RepoStore,
/// Per-DID rate limiter for creation endpoints (repos, issues, PRs)
pub rate_limiter: RateLimiter,
/// Per-client-IP rate limiter for the same creation endpoints. The per-DID
/// limiter above cannot brake a creation flood from a DID farm — one
/// throwaway `did:key` per repo means each DID makes a single create call
/// and never trips its own bucket. A valid iCaptcha proof does not cap this
/// either: the enforced level draws only machine-solvable deterministic
/// challenges (and the caller can pin the easy type), so a bot mints a fresh
/// DID, solves a proof, and creates a repo unthrottled. Braking on the
/// resolved client IP is what actually stops a single-source flood (same
/// rationale as `push_rate_limiter`). Keyed by `push_limiter_trust`.
pub create_ip_rate_limiter: RateLimiter,
/// Per-client-IP rate limiter for git-receive-pack. Per-DID limits cannot
/// brake a push flood from a DID farm (one throwaway DID per repo), so the
/// push path throttles on the resolved client IP instead.
pub push_rate_limiter: RateLimiter,
/// Which forwarded header (if any) the edge is trusted to set, for
/// resolving the push limiter's client-IP key. See `GITLAWB_TRUSTED_PROXY`.
/// Node-wide; also keys the two peer-sync limiters below.
pub push_limiter_trust: crate::rate_limit::TrustedProxy,
/// Per-client-IP limiter for `POST /api/v1/sync/trigger` (tight). The route
/// requires a signature, but a signature does not cap cost (a did:key farm
/// self-registers), and its per-call cost is an O(peers) fan-out, so the IP
/// brake is a separate, load-bearing half. Its own bucket so an unsigned
/// `/sync/notify` flood cannot drain the signed trigger caller's quota.
pub sync_trigger_rate_limiter: RateLimiter,
/// Per-client-IP limiter for the peer-write routes (`/peers/announce`,
/// `/sync/notify`) (generous). `/sync/notify` reaches the same `enqueue_sync`
/// sink as trigger and accepts unsigned requests from known peers, so it is
/// braked too; each peer's distinct IP gets its own bucket.
pub peer_write_rate_limiter: RateLimiter,
/// Process-wide graceful-shutdown signal. Sending `true` causes every
/// task that holds a `watch::Receiver` to exit at its next await point.
/// Used by:
File diff suppressed because it is too large. Load diff
+414 -5
View File
@@ -4,7 +4,8 @@
//! based on the repo's visibility rules with a fallback to the legacy
//! `is_public` flag. It performs no I/O so it is exhaustively unit tested.
use crate::db::VisibilityRule;
use crate::db::{RepoRecord, VisibilityRule};
use std::collections::HashMap;
use unicode_normalization::UnicodeNormalization;
#[derive(Debug, PartialEq, Eq)]
@@ -24,11 +25,12 @@ fn nfc(s: &str) -> String {
s.nfc().collect()
}
/// True if `caller` is the repo owner (matches full did:key or its short form),
/// mirroring the owner-match idiom in `api/protect.rs`.
/// True if `caller` is the repo owner. Uses [`crate::api::did_matches`] so the
/// owner check is representation-agnostic within `did:key` (full `did:key:z...`
/// matches bare `z...` stored by mirror rows) while still denying cross-method
/// collisions (`did:gitlawb:z...` vs bare `z...`).
fn is_owner(owner_did: &str, caller: &str) -> bool {
let owner_short = owner_did.split(':').next_back().unwrap_or(owner_did);
caller == owner_did || caller == owner_short
crate::api::did_matches(owner_did, caller)
}
/// The match prefix for a glob: "/" stays "/", "/secret/**" becomes "/secret".
@@ -127,6 +129,105 @@ pub fn listable_at_root(
visibility_check(rules, is_public, owner_did, caller, "/") == Decision::Allow
}
/// Whether a single `received_ref_updates` row (identified by its peer-supplied
/// `row_repo` slug) should be shown to `caller` (None = anonymous) on the
/// cross-repo ref-updates feeds (#112 GraphQL, #114 REST).
///
/// Pure and I/O-free: both call sites load the deduped local repo set and its
/// visibility rules once per request and pass them in, so the gate logic lives
/// here and visibility.rs keeps its "no I/O" property.
///
/// The slug is written verbatim from the inbound gossip/notify message, so it is
/// untrusted input, not a canonical key. The decision is fail-closed by
/// construction: the only KEEP paths are (a) a slug with no `/` (cannot name a
/// local `owner/name` pair, so remote by definition), (b) all matched local
/// records are readable, and (c) no local record matches (remote/gossip-only).
/// Any local match the caller cannot read at root DROPs the row. There is no
/// catch-all keep on unexpected state.
///
/// Slug/record owner keys are matched prefix-tolerantly (one is a prefix of the
/// other), covering the exact short-key, the full `did:key:` form, and the
/// URL-truncated 8-char form. Prefix over-match can only over-drop a genuinely
/// remote row (fail-safe), never over-serve a private local one.
///
/// The live call sites are the #112 (GraphQL) and #114 (REST) feed handlers,
/// added in the following units; exercised by the unit tests below meanwhile.
pub fn ref_update_row_visible(
deduped: &[RepoRecord],
rules_by_repo: &HashMap<String, Vec<VisibilityRule>>,
caller: Option<&str>,
row_repo: &str,
) -> bool {
// A slug with no '/' cannot name a local owner/name pair — remote by
// definition (same branch as "matches nothing local") → KEEP.
if row_repo.rsplit_once('/').is_none() {
return true;
}
// Match each local record with the shared slug predicate (one matcher, so
// this gate and the collector's quarantine drop cannot disagree about which
// rows a repo owns), then fail closed on any matched record the caller
// cannot read at root.
for record in deduped {
if !ref_update_row_names_repo(record, row_repo) {
continue;
}
let rules = rules_by_repo
.get(&record.id)
.map(Vec::as_slice)
.unwrap_or(&[]);
if !listable_at_root(rules, record.is_public, &record.owner_did, caller) {
return false;
}
}
// Reached only if every matched local record is readable, or nothing local
// matched (remote/gossip-only). Both are the KEEP paths; there is no
// default-keep on unexpected state — an unreadable match already returned.
true
}
/// Whether `row_repo`'s peer-supplied slug names the local `record`. The single
/// match predicate shared by the feed gate ([`ref_update_row_visible`]) and the
/// quarantine hard-drop in the ref-update collector, so the two cannot diverge
/// about which rows a repo owns — a second, drifting matcher is exactly the #134
/// slug-collision class.
///
/// A record matches when the slug's `name` half is equal and one owner key is a
/// prefix of the other (prefix-tolerant: exact short key, full `did:key:` form,
/// and the URL-truncated 8-char form). Both the slug owner and the record owner
/// are reduced to their last ':'-delimited segment before comparing. The node's
/// own emitter builds the wire slug via `normalize_owner_key` (bare short key for
/// a single-segment `did:key:`, the full DID for every other method; see
/// api/repos.rs), but the stored slug is UNTRUSTED: a peer can broadcast a row
/// under any owner form, including a method-stripped `user/name`. Reducing both
/// sides to the trailing segment matches a repo's own rows whichever form keyed
/// them, and still catches an attacker-planted short slug.
///
/// This intentionally diverges from `did_matches` / `DEDUP_CTE`, which strip only
/// bare `did:key:` and keep other DID methods whole: those compare trusted,
/// canonical DIDs, while this slug is untrusted. Applying the keep-whole rule here
/// would fail open: a hostile `user/name` slug would not prefix-match a private
/// `did:web:host:user` record and would leak. The price is a fail-SAFE over-match
/// when a remote owner shares both a trailing segment and a repo name with a local
/// private repo (negligible for full did:key ids; only did:web / truncated forms
/// collide): it can hide a genuinely remote row, never serve a private one.
pub fn ref_update_row_names_repo(record: &RepoRecord, row_repo: &str) -> bool {
let Some((owner_part, name)) = row_repo.rsplit_once('/') else {
return false;
};
if record.name != name {
return false;
}
let row_key = owner_part.split(':').next_back().unwrap_or(owner_part);
let record_key = record
.owner_did
.split(':')
.next_back()
.unwrap_or(&record.owner_did);
record_key.starts_with(row_key) || row_key.starts_with(record_key)
}
/// The subtree path globs that `caller` (None = anonymous) may NOT read, given
/// the repo's rules. Whole-repo ("/") rules are excluded: a denied whole-repo
/// read is handled by the 404 gate before a clone ever starts. Each remaining
@@ -299,6 +400,60 @@ mod tests {
);
}
#[test]
fn non_key_owner_bare_short_does_not_match() {
// A non-key DID owner (did:gitlawb:z6MkFoo) probed with the bare last
// segment (z6MkFoo) must NOT match — the old is_owner would return true
// via split(':').next_back(), which this PR tightens.
let rules = [rule("/", VisibilityMode::A, &[])];
assert_eq!(
visibility_check(&rules, false, "did:gitlawb:z6MkFoo", Some("z6MkFoo"), "/"),
Decision::Deny,
"bare short must not match a non-key DID owner"
);
// The full non-key DID still matches itself.
assert_eq!(
visibility_check(
&rules,
false,
"did:gitlawb:z6MkFoo",
Some("did:gitlawb:z6MkFoo"),
"/"
),
Decision::Allow,
"full non-key DID still matches itself"
);
}
// #153 forward guard: is_owner must match across bare-key / full-did:key
// representations should someone revert to single-side normalize_owner_key.
#[test]
fn bare_owner_full_caller_allows_owner() {
let rules = [rule("/", VisibilityMode::A, &[])];
assert_eq!(
visibility_check(&rules, false, "z6MkOwner", Some("did:key:z6MkOwner"), "/"),
Decision::Allow,
"bare owner + full did:key caller must match (guard against single-side revert)"
);
}
// #153 regression: cross-method DID must still be denied even when the
// trailing segment collides with a bare owner key.
#[test]
fn cross_method_did_denied_with_bare_owner() {
let rules = [rule("/", VisibilityMode::A, &[])];
assert_eq!(
visibility_check(&rules, false, "z6MkFoo", Some("did:gitlawb:z6MkFoo"), "/"),
Decision::Deny,
"cross-method DID must not match a bare did:key owner"
);
assert_eq!(
visibility_check(&rules, false, "z6MkFoo", Some("did:web:z6MkFoo"), "/"),
Decision::Deny,
"did:web must not match a bare did:key owner"
);
}
#[test]
fn root_rule_allows_listed_reader() {
let rules = [rule("/", VisibilityMode::A, &["did:key:z6MkFriend"])];
@@ -403,6 +558,260 @@ mod tests {
));
}
// ── ref_update_row_visible (feed gate) ──────────────────────────────────
fn rec(id: &str, owner_did: &str, name: &str, is_public: bool) -> RepoRecord {
RepoRecord {
id: id.into(),
name: name.into(),
owner_did: owner_did.into(),
description: None,
is_public,
default_branch: "main".into(),
created_at: Utc::now(),
updated_at: Utc::now(),
disk_path: format!("/srv/{id}"),
forked_from: None,
machine_id: None,
}
}
fn rules_for(entries: &[(&str, &[VisibilityRule])]) -> HashMap<String, Vec<VisibilityRule>> {
entries
.iter()
.map(|(id, rs)| (id.to_string(), rs.to_vec()))
.collect()
}
#[test]
fn feed_public_local_repo_kept_for_anon() {
let deduped = [rec("r1", "did:key:z6MkOwner", "widget", true)];
let rules = HashMap::new();
assert!(ref_update_row_visible(
&deduped,
&rules,
None,
"z6MkOwner/widget"
));
}
#[test]
fn feed_private_local_repo_dropped_for_anon_kept_for_owner() {
let deduped = [rec("r1", "did:key:z6MkOwner", "widget", false)];
let rules = HashMap::new();
// Anonymous → drop.
assert!(!ref_update_row_visible(
&deduped,
&rules,
None,
"z6MkOwner/widget"
));
// Owner (full DID) → keep.
assert!(ref_update_row_visible(
&deduped,
&rules,
Some("did:key:z6MkOwner"),
"z6MkOwner/widget"
));
}
#[test]
fn feed_root_rule_reader_reincluded() {
// Private repo (is_public=false) with a root rule granting a named
// reader. Delegates to listable_at_root: anon and non-reader denied,
// named reader allowed.
let deduped = [rec("r1", OWNER, "widget", false)];
let root = [rule("/", VisibilityMode::A, &["did:key:z6MkFriend"])];
let rules = rules_for(&[("r1", &root)]);
assert!(!ref_update_row_visible(
&deduped,
&rules,
None,
"z6MkOwner/widget"
));
assert!(!ref_update_row_visible(
&deduped,
&rules,
Some("did:key:z6MkStranger"),
"z6MkOwner/widget"
));
assert!(ref_update_row_visible(
&deduped,
&rules,
Some("did:key:z6MkFriend"),
"z6MkOwner/widget"
));
}
#[test]
fn feed_alias_full_did_slug_dropped_for_anon() {
// Owner stored full-DID; slug also carries the full-DID form. Still
// matches (row_key normalizes to the short key) → drop. Round 1's
// string-match would have leaked this.
let deduped = [rec("r1", "did:key:zABC", "widget", false)];
let rules = HashMap::new();
assert!(!ref_update_row_visible(
&deduped,
&rules,
None,
"did:key:zABC/widget"
));
}
#[test]
fn feed_truncated_key_slug_dropped_for_anon() {
// Slug carries an 8-char URL-truncated prefix of the owner key; still
// matches via prefix tolerance → drop. Round 2's get_repo path leaked.
let deduped = [rec("r1", "did:key:zABCDEFGH", "widget", false)];
let rules = HashMap::new();
assert!(!ref_update_row_visible(
&deduped,
&rules,
None,
"zABCDEF/widget"
));
}
#[test]
fn feed_mirror_coexistence_private_canonical_dropped_for_anon() {
// Pure-level mirror-coexistence: the deduped set contains only the
// private canonical record for (owner,name). A matching slug drops for
// anon. (DB-level dedup survivor property is pinned separately.)
let deduped = [rec("uuid-canonical", "did:key:z6Mkwbud", "nipmod", false)];
let rules = HashMap::new();
assert!(!ref_update_row_visible(
&deduped,
&rules,
None,
"z6Mkwbud/nipmod"
));
}
#[test]
fn feed_empty_owner_slug_matches_and_drops() {
// Slug "/name": empty owner_part → row_key "" → starts_with("") matches
// every same-named record. Fail-safe pin for a private repo.
let deduped = [rec("r1", "did:key:z6MkOwner", "widget", false)];
let rules = HashMap::new();
assert!(!ref_update_row_visible(&deduped, &rules, None, "/widget"));
}
#[test]
fn feed_one_char_owner_slug_matches_and_drops() {
// 1-char owner prefix that the private repo's key starts with → match.
let deduped = [rec("r1", "did:key:z6MkOwner", "widget", false)];
let rules = HashMap::new();
assert!(!ref_update_row_visible(&deduped, &rules, None, "z/widget"));
}
#[test]
fn feed_remote_slug_no_match_kept() {
// Different owner key, no prefix relation → no local match → keep.
let deduped = [rec("r1", "did:key:z6MkOwner", "widget", false)];
let rules = HashMap::new();
assert!(ref_update_row_visible(
&deduped,
&rules,
None,
"zZZZOTHER/widget"
));
}
#[test]
fn feed_private_didweb_short_slug_dropped_for_anon() {
// A private did:web repo must drop its row for anon when the slug arrives
// in the short `{last-segment}/{name}` form. Post-#141 the emitter no longer
// produces this form for did:web (normalize_owner_key keeps the full DID),
// so this is the untrusted/crafted variant a peer can broadcast; the full-DID
// variant the emitter now sends is covered by the next test. A did:key-aware
// (keep-whole) rule would fail to match `alice` against `did:web:host:alice`
// and leak.
let deduped = [rec("r1", "did:web:host:alice", "widget", false)];
let rules = HashMap::new();
assert!(!ref_update_row_visible(
&deduped,
&rules,
None,
"alice/widget"
));
}
#[test]
fn feed_multi_segment_did_slug_dropped_for_anon() {
// A private repo owned by a multi-segment DID must also fail closed under
// the full-DID slug form. Post-#141 this IS the form the emitter broadcasts
// (normalize_owner_key keeps non-did:key methods whole), and a peer can craft
// it too. The gate drops it: both sides reduce to the last ':' segment
// ("user"), so they match. Fail-safe regardless of which form keyed the row.
let deduped = [rec("r1", "did:web:host:user", "widget", false)];
let rules = HashMap::new();
assert!(!ref_update_row_visible(
&deduped,
&rules,
None,
"did:web:host:user/widget"
));
}
#[test]
fn feed_multi_segment_did_slug_kept_for_public() {
// Symmetric keep-side: a PUBLIC multi-segment-DID repo's row must still be
// returned to anon after the last-':'-segment normalization — guards
// against a regression that over-drops legitimate did:web rows.
let deduped = [rec("r1", "did:web:host:user", "widget", true)];
let rules = HashMap::new();
assert!(ref_update_row_visible(
&deduped,
&rules,
None,
"did:web:host:user/widget"
));
}
#[test]
fn feed_malformed_slug_no_slash_kept_no_panic() {
let deduped = [rec("r1", "did:key:z6MkOwner", "widget", false)];
let rules = HashMap::new();
assert!(ref_update_row_visible(&deduped, &rules, None, "noslug"));
}
#[test]
fn feed_empty_deduped_set_keeps_any_slug() {
let deduped: [RepoRecord; 0] = [];
let rules = HashMap::new();
assert!(ref_update_row_visible(
&deduped,
&rules,
None,
"z6MkOwner/widget"
));
assert!(ref_update_row_visible(&deduped, &rules, None, "anything"));
}
// The shared slug matcher underpinning both the feed gate and the collector's
// quarantine drop: it must recognize a repo's own row across every owner-DID
// form (bare short key, full did:key, URL-truncated prefix) and must not
// over-match a different name or an unrelated owner. The quarantine drop calls
// this directly, so its match contract is load-bearing for withholding a
// quarantined mirror from a caller who matches the mirror's owner_did.
#[test]
fn names_repo_matches_owner_forms_and_rejects_mismatches() {
// Bare short owner key (the form upsert_mirror_repo stores).
let bare = rec("m", "z6MkQuar", "secret", false);
assert!(ref_update_row_names_repo(&bare, "z6MkQuar/secret"));
// Full did:key owner; the short-segment slug still matches via last-segment.
let full = rec("c", "did:key:z6MkQuar", "secret", false);
assert!(ref_update_row_names_repo(&full, "z6MkQuar/secret"));
assert!(ref_update_row_names_repo(&full, "did:key:z6MkQuar/secret"));
// URL-truncated 8-char prefix slug → prefix-tolerant match.
let long = rec("l", "did:key:z6MkQuarLONGKEY", "secret", false);
assert!(ref_update_row_names_repo(&long, "z6MkQuar/secret"));
// Different name, different owner, and no-slash → no match.
assert!(!ref_update_row_names_repo(&bare, "z6MkQuar/other"));
assert!(!ref_update_row_names_repo(&bare, "z6MkOther/secret"));
assert!(!ref_update_row_names_repo(&bare, "noslash"));
}
// #101: a deny rule must withhold a path that denotes the same characters in
// a different Unicode normalization form. Without NFC normalization at the
// matcher seam, an NFC-authored rule byte-compares unequal to an NFD-stored
+2 -1
View File
@@ -1,7 +1,7 @@
[package]
name = "gl"
description = "The gitlawb CLI — identity management, node control, MCP server"
version = "0.4.0" # x-release-please-version
version = "0.5.1" # x-release-please-version
edition.workspace = true
rust-version.workspace = true
license.workspace = true
@@ -12,6 +12,7 @@ path = "src/main.rs"
[dependencies]
gitlawb-core = { path = "../gitlawb-core" }
icaptcha-client = { path = "../icaptcha-client" }
tokio = { workspace = true }
serde = { workspace = true }
serde_json = { workspace = true }
+28 -10
View File
@@ -8,6 +8,10 @@ use std::path::PathBuf;
use crate::http::NodeClient;
use crate::identity::load_keypair_from_dir;
fn signed_client(node: &str, dir: Option<&std::path::Path>) -> NodeClient {
NodeClient::new(node, load_keypair_from_dir(dir).ok())
}
#[derive(Args)]
pub struct BountyArgs {
#[command(subcommand)]
@@ -49,6 +53,8 @@ pub enum BountyCmd {
status: Option<String>,
#[arg(long, default_value = "https://node.gitlawb.com", env = "GITLAWB_NODE")]
node: String,
#[arg(long)]
dir: Option<PathBuf>,
},
/// Show details of a specific bounty
Show {
@@ -56,6 +62,8 @@ pub enum BountyCmd {
id: String,
#[arg(long, default_value = "https://node.gitlawb.com", env = "GITLAWB_NODE")]
node: String,
#[arg(long)]
dir: Option<PathBuf>,
},
/// Claim an open bounty
Claim {
@@ -121,8 +129,13 @@ pub async fn run(args: BountyArgs) -> Result<()> {
node,
dir,
} => cmd_create(repo, title, amount, issue, tx_hash, deadline, node, dir).await,
BountyCmd::List { repo, status, node } => cmd_list(repo, status, node).await,
BountyCmd::Show { id, node } => cmd_show(id, node).await,
BountyCmd::List {
repo,
status,
node,
dir,
} => cmd_list(repo, status, node, dir).await,
BountyCmd::Show { id, node, dir } => cmd_show(id, node, dir).await,
BountyCmd::Claim {
id,
wallet,
@@ -192,8 +205,13 @@ async fn cmd_create(
Ok(())
}
async fn cmd_list(repo: Option<String>, status: Option<String>, node: String) -> Result<()> {
let client = NodeClient::new(&node, None);
async fn cmd_list(
repo: Option<String>,
status: Option<String>,
node: String,
dir: Option<PathBuf>,
) -> Result<()> {
let client = signed_client(&node, dir.as_deref());
let url = if let Some(ref repo) = repo {
let (owner, name) = repo
@@ -214,7 +232,7 @@ async fn cmd_list(repo: Option<String>, status: Option<String>, node: String) ->
};
let resp = client
.get(&url)
.get_authed(&url)
.await
.context("failed to connect to node")?;
let body: Value = resp.json().await.unwrap_or_default();
@@ -237,9 +255,9 @@ async fn cmd_list(repo: Option<String>, status: Option<String>, node: String) ->
Ok(())
}
async fn cmd_show(id: String, node: String) -> Result<()> {
let client = NodeClient::new(&node, None);
let resp = client.get(&format!("/api/v1/bounties/{id}")).await?;
async fn cmd_show(id: String, node: String, dir: Option<PathBuf>) -> Result<()> {
let client = signed_client(&node, dir.as_deref());
let resp = client.get_authed(&format!("/api/v1/bounties/{id}")).await?;
let status = resp.status();
let body: Value = resp.json().await.unwrap_or_default();
@@ -462,7 +480,7 @@ mod tests {
.create_async()
.await;
cmd_list(None, None, server.url()).await.unwrap();
cmd_list(None, None, server.url(), None).await.unwrap();
}
#[tokio::test]
@@ -506,7 +524,7 @@ mod tests {
.create_async()
.await;
let err = cmd_show("nonexistent".to_string(), server.url())
let err = cmd_show("nonexistent".to_string(), server.url(), None)
.await
.unwrap_err();
assert!(err.to_string().contains("not found"));
+34 -17
View File
@@ -5,10 +5,15 @@
use anyhow::{Context, Result};
use clap::{Args, Subcommand};
use serde_json::Value;
use std::path::PathBuf;
use crate::http::NodeClient;
use crate::identity::load_keypair_from_dir;
fn signed_client(node: &str, dir: Option<&std::path::Path>) -> NodeClient {
NodeClient::new(node, load_keypair_from_dir(dir).ok())
}
#[derive(Args)]
pub struct CertArgs {
#[command(subcommand)]
@@ -23,6 +28,8 @@ pub enum CertCmd {
repo: String,
#[arg(long, default_value = "https://node.gitlawb.com", env = "GITLAWB_NODE")]
node: String,
#[arg(long)]
dir: Option<PathBuf>,
},
/// Show a specific ref certificate and verify its signature
Show {
@@ -32,28 +39,39 @@ pub enum CertCmd {
id: String,
#[arg(long, default_value = "https://node.gitlawb.com", env = "GITLAWB_NODE")]
node: String,
#[arg(long)]
dir: Option<PathBuf>,
},
}
pub async fn run(args: CertArgs) -> Result<()> {
match args.cmd {
CertCmd::List { repo, node } => cmd_list(repo, node).await,
CertCmd::Show { repo, id, node } => cmd_show(repo, id, node).await,
CertCmd::List { repo, node, dir } => cmd_list(repo, node, dir).await,
CertCmd::Show {
repo,
id,
node,
dir,
} => cmd_show(repo, id, node, dir).await,
}
}
/// Resolve "repo" into (owner, name) using the caller's DID when no slash is given.
async fn resolve_repo(repo: &str, node: &str) -> Result<(String, String)> {
async fn resolve_repo(
repo: &str,
node: &str,
dir: Option<&std::path::Path>,
) -> Result<(String, String)> {
if let Some((owner, name)) = repo.split_once('/') {
Ok((owner.to_string(), name.to_string()))
} else {
let short = if let Ok(kp) = load_keypair_from_dir(None) {
let short = if let Ok(kp) = load_keypair_from_dir(dir) {
let did = kp.did().to_string();
did.split(':').next_back().unwrap_or(&did).to_string()
} else {
let client = NodeClient::new(node, None);
let client = signed_client(node, dir);
let info: Value = client
.get("/")
.get_authed("/")
.await?
.json()
.await
@@ -65,13 +83,13 @@ async fn resolve_repo(repo: &str, node: &str) -> Result<(String, String)> {
}
}
async fn cmd_list(repo: String, node: String) -> Result<()> {
let (owner, name) = resolve_repo(&repo, &node).await?;
async fn cmd_list(repo: String, node: String, dir: Option<PathBuf>) -> Result<()> {
let (owner, name) = resolve_repo(&repo, &node, dir.as_deref()).await?;
let client = NodeClient::new(&node, None);
let client = signed_client(&node, dir.as_deref());
let path = format!("/api/v1/repos/{owner}/{name}/certs");
let resp: Value = client
.get(&path)
.get_authed(&path)
.await?
.json()
.await
@@ -96,16 +114,16 @@ async fn cmd_list(repo: String, node: String) -> Result<()> {
Ok(())
}
async fn cmd_show(repo: String, id: String, node: String) -> Result<()> {
let (owner, name) = resolve_repo(&repo, &node).await?;
async fn cmd_show(repo: String, id: String, node: String, dir: Option<PathBuf>) -> Result<()> {
let (owner, name) = resolve_repo(&repo, &node, dir.as_deref()).await?;
let client = NodeClient::new(&node, None);
let client = signed_client(&node, dir.as_deref());
let id = resolve_cert_id(&client, &owner, &name, &id).await?;
// Fetch the certificate
let path = format!("/api/v1/repos/{owner}/{name}/certs/{id}");
let resp = client
.get(&path)
.get_authed(&path)
.await?
.error_for_status()
.context("certificate not found")?;
@@ -168,9 +186,9 @@ async fn resolve_cert_id(client: &NodeClient, owner: &str, name: &str, id: &str)
return Ok(id.to_string());
}
let path = format!("/api/v1/repos/{owner}/{name}/certs");
let path = format!("/api/v1/repos/{owner}/{name}/certs?prefix={id}");
let resp: Value = client
.get(&path)
.get_authed(&path)
.await?
.error_for_status()
.context("failed to list certificates")?
@@ -182,7 +200,6 @@ async fn resolve_cert_id(client: &NodeClient, owner: &str, name: &str, id: &str)
let matches: Vec<String> = certs
.iter()
.filter_map(|cert| cert["id"].as_str())
.filter(|cert_id| cert_id.starts_with(id))
.map(ToString::to_string)
.collect();
+38
View File
@@ -174,6 +174,19 @@ pub async fn run(args: DoctorArgs) -> Result<()> {
"node",
format!("{} — v{version} ({short_did}…)", args.node),
));
// Capability drift: a node newer than this CLI may require features
// (RFC 9421 signing, iCaptcha) the CLI doesn't speak.
let gl_ver = env!("CARGO_PKG_VERSION");
if version != "?" && is_newer(version, gl_ver) {
checks.push(Check::warn(
"gl version",
format!(
"node is v{version} but gl is v{gl_ver} — your CLI may be missing \
features (signing / iCaptcha) this node requires"
),
"upgrade gl: curl -sSf https://gitlawb.com/install.sh | sh",
));
}
}
Ok(resp) => {
checks.push(Check::fail(
@@ -191,6 +204,31 @@ pub async fn run(args: DoctorArgs) -> Result<()> {
}
}
// ── 4b. iCaptcha capability ───────────────────────────────────────────
// Gated writes (repo create / register / fork) auto-solve a challenge at the
// iCaptcha service; check it's reachable so the failure mode is obvious.
let icaptcha_url = std::env::var("GITLAWB_ICAPTCHA_URL")
.unwrap_or_else(|_| icaptcha_client::DEFAULT_URL.to_string());
match NodeClient::new(&icaptcha_url, None).get("/v1/pubkey").await {
Ok(resp) if resp.status().is_success() => {
checks.push(Check::pass("iCaptcha", format!("{icaptcha_url} reachable")));
}
Ok(resp) => {
checks.push(Check::warn(
"iCaptcha",
format!("{icaptcha_url} returned HTTP {}", resp.status()),
"gated writes (repo create / register) may fail until iCaptcha is reachable",
));
}
Err(e) => {
checks.push(Check::warn(
"iCaptcha",
format!("{icaptcha_url} unreachable: {e}"),
"set GITLAWB_ICAPTCHA_URL or check connectivity — repo create / register solve a challenge there",
));
}
}
// ── 5. git-remote-gitlawb helper ──────────────────────────────────────
// Use PATH lookup only — invoking the binary directly triggers git internals
// that error with "fatal: not a git repository" outside of a git repo.
+111 -38
View File
@@ -1,8 +1,19 @@
//! Signed HTTP client for gitlawb API calls (async).
//!
//! Writes are signed with RFC 9421 HTTP Signatures. When the node gates a write
//! behind iCaptcha (HTTP 403 `icaptcha_proof_required`, advertised via the
//! `x-icaptcha-url` / `x-icaptcha-level` headers), the client transparently
//! solves the challenge and retries the same signed request with the
//! `x-icaptcha-proof` header — see `crates/icaptcha-client`.
use anyhow::{Context, Result};
use gitlawb_core::http_sig::sign_request;
use gitlawb_core::identity::Keypair;
use icaptcha_client::IcaptchaCfg;
/// Max times we'll fetch a fresh proof and retry a 403-iCaptcha response
/// (absorbs proof expiry / first-seen replay).
const MAX_ICAPTCHA_RETRIES: usize = 2;
pub struct NodeClient {
inner: reqwest::Client,
@@ -34,6 +45,15 @@ impl NodeClient {
.with_context(|| format!("GET {url}"))
}
/// GET that signs when a keypair is available; falls back to unsigned for public repos.
pub async fn get_authed(&self, path: &str) -> Result<reqwest::Response> {
if self.keypair.is_some() {
self.get_signed(path).await
} else {
self.get(path).await
}
}
/// GET with RFC 9421 HTTP Signature auth, for owner-only read endpoints.
/// Signs over the empty body (same shape the node verifies for signed reads).
pub async fn get_signed(&self, path: &str) -> Result<reqwest::Response> {
@@ -52,63 +72,116 @@ impl NodeClient {
req.send().await.with_context(|| format!("GET {url}"))
}
/// POST with JSON body + RFC 9421 HTTP Signature auth.
/// POST with JSON body + RFC 9421 signing + transparent iCaptcha solve/retry.
pub async fn post(&self, path: &str, body: &[u8]) -> Result<reqwest::Response> {
let url = format!("{}{}", self.node_url, path);
let mut req = self
.inner
.post(&url)
.header("Content-Type", "application/json")
.body(body.to_vec());
if let Some(kp) = &self.keypair {
let signed = sign_request(kp, "POST", path, body);
req = req
.header("Content-Digest", signed.content_digest)
.header("Signature-Input", signed.signature_input)
.header("Signature", signed.signature);
}
req.send().await.with_context(|| format!("POST {url}"))
self.send_signed("POST", path, body).await
}
/// PUT with RFC 9421 HTTP Signature auth (idempotent write).
/// PUT with RFC 9421 signing + transparent iCaptcha solve/retry.
pub async fn put(&self, path: &str, body: &[u8]) -> Result<reqwest::Response> {
let url = format!("{}{}", self.node_url, path);
let mut req = self
.inner
.put(&url)
.header("Content-Type", "application/json")
.body(body.to_vec());
if let Some(kp) = &self.keypair {
let signed = sign_request(kp, "PUT", path, body);
req = req
.header("Content-Digest", signed.content_digest)
.header("Signature-Input", signed.signature_input)
.header("Signature", signed.signature);
}
req.send().await.with_context(|| format!("PUT {url}"))
self.send_signed("PUT", path, body).await
}
/// DELETE with RFC 9421 HTTP Signature auth.
/// DELETE with RFC 9421 signing + transparent iCaptcha solve/retry.
pub async fn delete(&self, path: &str, body: &[u8]) -> Result<reqwest::Response> {
self.send_signed("DELETE", path, body).await
}
/// Sign + send a write. On a 403 iCaptcha challenge (detected via the
/// `x-icaptcha-*` headers) solve it and retry the same signed request with
/// the proof header, up to [`MAX_ICAPTCHA_RETRIES`]. Emits an actionable
/// hint on a 401 "not an agent" (the old-CLI / unregistered failure mode).
async fn send_signed(
&self,
method: &str,
path: &str,
body: &[u8],
) -> Result<reqwest::Response> {
let mut proof: Option<String> = None;
let mut attempts = 0;
loop {
let resp = self.send_once(method, path, body, proof.as_deref()).await?;
let status = resp.status();
if status == reqwest::StatusCode::UNAUTHORIZED
&& resp
.headers()
.get("x-gitlawb-error")
.and_then(|v| v.to_str().ok())
== Some("human_detected")
{
eprintln!(
"note: this node requires signed requests (RFC 9421). If writes keep \
failing, your `gl` may be too old — upgrade it — or you're not registered: \
run `gl register`."
);
}
if status == reqwest::StatusCode::FORBIDDEN && attempts < MAX_ICAPTCHA_RETRIES {
if let Some(cfg) = self.icaptcha_cfg(resp.headers())? {
attempts += 1;
proof = Some(obtain_proof(cfg).await?);
continue;
}
}
return Ok(resp);
}
}
/// Build, sign, and send one request, optionally attaching a proof header.
async fn send_once(
&self,
method: &str,
path: &str,
body: &[u8],
proof: Option<&str>,
) -> Result<reqwest::Response> {
let url = format!("{}{}", self.node_url, path);
let mut req = self
.inner
.delete(&url)
.request(method.parse().expect("valid HTTP method"), &url)
.header("Content-Type", "application/json")
.body(body.to_vec());
if let Some(kp) = &self.keypair {
let signed = sign_request(kp, "DELETE", path, body);
let signed = sign_request(kp, method, path, body);
req = req
.header("Content-Digest", signed.content_digest)
.header("Signature-Input", signed.signature_input)
.header("Signature", signed.signature);
}
if let Some(p) = proof {
req = req.header(icaptcha_client::PROOF_HEADER, p);
}
req.send().await.with_context(|| format!("DELETE {url}"))
req.send().await.with_context(|| format!("{method} {url}"))
}
/// If `headers` describe an iCaptcha 403, build the solve config (binding the
/// proof's `sub` to our DID). Returns `None` for a non-iCaptcha 403.
fn icaptcha_cfg(&self, headers: &reqwest::header::HeaderMap) -> Result<Option<IcaptchaCfg>> {
let url = headers.get("x-icaptcha-url").and_then(|v| v.to_str().ok());
let level = headers
.get("x-icaptcha-level")
.and_then(|v| v.to_str().ok());
if url.is_none() && level.is_none() {
return Ok(None); // not an iCaptcha challenge
}
let kp = self
.keypair
.as_ref()
.context("iCaptcha challenge requires an identity keypair (run `gl identity new`)")?;
Ok(Some(IcaptchaCfg::new(
kp.did().to_string(),
url.map(str::to_string),
level.and_then(|l| l.parse().ok()),
)))
}
}
/// Run the (blocking) iCaptcha solve loop off the async runtime.
async fn obtain_proof(cfg: IcaptchaCfg) -> Result<String> {
tokio::task::spawn_blocking(move || icaptcha_client::obtain_proof(&cfg, None))
.await
.context("iCaptcha solver task panicked")?
}
+137 -9
View File
@@ -3,6 +3,8 @@
//! Communicates with the gitlawb node to list pinned CIDs and retrieve git
//! objects by their content-addressed CID.
use std::path::PathBuf;
use anyhow::{Context, Result};
use clap::{Args, Subcommand};
use serde_json::Value;
@@ -21,6 +23,9 @@ pub enum IpfsCmd {
List {
#[arg(long, default_value = "https://node.gitlawb.com", env = "GITLAWB_NODE")]
node: String,
/// Identity directory (default: ~/.gitlawb)
#[arg(long)]
dir: Option<PathBuf>,
},
/// Retrieve and display a git object from the node by its CIDv1
Get {
@@ -33,19 +38,24 @@ pub enum IpfsCmd {
pub async fn run(args: IpfsArgs) -> Result<()> {
match args.cmd {
IpfsCmd::List { node } => cmd_list(node).await,
IpfsCmd::List { node, dir } => cmd_list(node, dir).await,
IpfsCmd::Get { cid, node } => cmd_get(cid, node).await,
}
}
async fn cmd_list(node: String) -> Result<()> {
let client = NodeClient::new(&node, None);
let resp: Value = client
.get("/api/v1/ipfs/pins")
.await?
.json()
.await
.context("failed to parse pins response")?;
async fn cmd_list(node: String, dir: Option<PathBuf>) -> Result<()> {
// #134 gates /api/v1/ipfs/pins behind auth: sign the request with the
// caller's identity. On no identity, propagate load_keypair_from_dir's
// error (it already names `gl identity new`) rather than a bare 401.
let keypair = crate::identity::load_keypair_from_dir(dir.as_deref())?;
let client = NodeClient::new(&node, Some(keypair));
let resp = client.get_signed("/api/v1/ipfs/pins").await?;
if !resp.status().is_success() {
let status = resp.status();
let body = resp.text().await.unwrap_or_default();
anyhow::bail!("node returned {status} for pins listing: {body}");
}
let resp: Value = resp.json().await.context("failed to parse pins response")?;
let pins = resp["pins"].as_array().cloned().unwrap_or_default();
let count = resp["count"].as_u64().unwrap_or(pins.len() as u64);
@@ -108,3 +118,121 @@ async fn cmd_get(cid: String, node: String) -> Result<()> {
Ok(())
}
#[cfg(test)]
mod tests {
use super::*;
/// Seed a keypair into a temp dir the way `load_keypair_from_dir` expects,
/// then return the dir handle (keeps it alive for the test's duration).
fn seed_keystore() -> tempfile::TempDir {
let dir = tempfile::TempDir::new().unwrap();
let kp = gitlawb_core::identity::Keypair::generate();
std::fs::write(
dir.path().join("identity.pem"),
kp.to_pem().unwrap().as_bytes(),
)
.unwrap();
dir
}
#[tokio::test]
async fn test_cmd_list_signs_request_and_renders_pins() {
let mut server = mockito::Server::new_async().await;
let keystore = seed_keystore();
// Happy path: signed GET to /api/v1/ipfs/pins carrying the RFC 9421
// signature headers, node returns a populated pins body.
let m = server
.mock("GET", "/api/v1/ipfs/pins")
.match_header("signature", mockito::Matcher::Any)
.match_header("signature-input", mockito::Matcher::Any)
.match_header("content-digest", mockito::Matcher::Any)
.with_status(200)
.with_header("content-type", "application/json")
.with_body(
r#"{"pins":[{"cid":"bafyone","sha256_hex":"abc123","pinned_at":"2026-07-02T12:00:00.123456Z"}],"count":1}"#,
)
.create_async()
.await;
cmd_list(server.url(), Some(keystore.path().to_path_buf()))
.await
.unwrap();
m.assert_async().await;
}
#[tokio::test]
async fn test_cmd_list_empty_pins() {
let mut server = mockito::Server::new_async().await;
let keystore = seed_keystore();
let m = server
.mock("GET", "/api/v1/ipfs/pins")
.match_header("signature", mockito::Matcher::Any)
.with_status(200)
.with_header("content-type", "application/json")
.with_body(r#"{"pins":[],"count":0}"#)
.create_async()
.await;
cmd_list(server.url(), Some(keystore.path().to_path_buf()))
.await
.unwrap();
m.assert_async().await;
}
#[tokio::test]
async fn test_cmd_list_no_identity_errors_without_request() {
let mut server = mockito::Server::new_async().await;
// Empty keystore dir: no identity.pem present.
let empty = tempfile::TempDir::new().unwrap();
// The endpoint must never be hit when there is no identity.
let m = server
.mock("GET", "/api/v1/ipfs/pins")
.expect(0)
.create_async()
.await;
let err = cmd_list(server.url(), Some(empty.path().to_path_buf()))
.await
.expect_err("no identity should be an error");
assert!(
err.to_string().contains("gl identity new")
|| err.to_string().contains("no identity found"),
"error should name `gl identity new`, got: {err}"
);
m.assert_async().await;
}
#[tokio::test]
async fn test_cmd_list_non_success_status_is_error_not_empty() {
let mut server = mockito::Server::new_async().await;
let keystore = seed_keystore();
// A signed request the node rejects (401) must surface as an error,
// not be silently parsed into an empty pin list.
let m = server
.mock("GET", "/api/v1/ipfs/pins")
.match_header("signature", mockito::Matcher::Any)
.with_status(401)
.with_header("content-type", "application/json")
.with_body(r#"{"error":"unauthorized"}"#)
.create_async()
.await;
let err = cmd_list(server.url(), Some(keystore.path().to_path_buf()))
.await
.expect_err("non-2xx status should be an error");
assert!(
err.to_string().contains("401"),
"error should mention the status, got: {err}"
);
m.assert_async().await;
}
}
+10 -6
View File
@@ -10,6 +10,10 @@ use uuid::Uuid;
use crate::http::NodeClient;
use crate::identity::load_keypair_from_dir;
fn signed_client(node: &str, dir: Option<&std::path::Path>) -> NodeClient {
NodeClient::new(node, load_keypair_from_dir(dir).ok())
}
#[derive(Args)]
pub struct IssueArgs {
#[command(subcommand)]
@@ -215,10 +219,10 @@ async fn cmd_create(
async fn cmd_list(repo: String, node: String, dir: Option<PathBuf>) -> Result<()> {
let (owner, name) = resolve_repo(&repo, &node, dir.as_deref()).await?;
let client = NodeClient::new(&node, None);
let client = signed_client(&node, dir.as_deref());
let path = format!("/api/v1/repos/{owner}/{name}/issues");
let resp: Value = client
.get(&path)
.get_authed(&path)
.await?
.json()
.await
@@ -254,10 +258,10 @@ async fn cmd_list(repo: String, node: String, dir: Option<PathBuf>) -> Result<()
async fn cmd_show(repo: String, id: String, node: String, dir: Option<PathBuf>) -> Result<()> {
let (owner, name) = resolve_repo(&repo, &node, dir.as_deref()).await?;
let client = NodeClient::new(&node, None);
let client = signed_client(&node, dir.as_deref());
let path = format!("/api/v1/repos/{owner}/{name}/issues/{id}");
let resp = client
.get(&path)
.get_authed(&path)
.await
.context("failed to connect to node")?;
let status = resp.status();
@@ -347,10 +351,10 @@ async fn cmd_issue_comments(
dir: Option<PathBuf>,
) -> Result<()> {
let (owner, name) = resolve_repo(&repo, &node, dir.as_deref()).await?;
let client = NodeClient::new(&node, None);
let client = signed_client(&node, dir.as_deref());
let resp: Value = client
.get(&format!(
.get_authed(&format!(
"/api/v1/repos/{owner}/{name}/issues/{id}/comments"
))
.await?
+3 -3
View File
@@ -970,14 +970,14 @@ async fn call_tool(
}
u
};
let resp: Value = client.get(&url).await?.json().await?;
let resp: Value = client.get_authed(&url).await?.json().await?;
Ok(serde_json::to_string_pretty(&resp)?)
}
"bounty_show" => {
let id = args["id"].as_str().context("missing 'id'")?;
let resp: Value = client
.get(&format!("/api/v1/bounties/{id}"))
.get_authed(&format!("/api/v1/bounties/{id}"))
.await?
.json()
.await?;
@@ -1165,7 +1165,7 @@ async fn call_tool(
(owner, repo.to_string())
};
let resp: Value = client
.get(&format!("/api/v1/repos/{owner}/{name}/issues"))
.get_authed(&format!("/api/v1/repos/{owner}/{name}/issues"))
.await?
.json()
.await?;
+379 -12
View File
@@ -2,10 +2,12 @@
use anyhow::Result;
use clap::{Args, Subcommand};
use gitlawb_core::identity::Keypair;
use serde_json::Value;
use std::path::PathBuf;
use crate::http::NodeClient;
use crate::identity::load_keypair_from_dir;
use crate::node_stake;
#[derive(Args)]
@@ -20,6 +22,9 @@ pub enum NodeCmd {
Status {
#[arg(long, default_value = "https://node.gitlawb.com", env = "GITLAWB_NODE")]
node: String,
/// Identity directory (default: ~/.gitlawb)
#[arg(long)]
dir: Option<PathBuf>,
},
/// Check trust score for a DID
Trust {
@@ -121,7 +126,7 @@ pub enum NodeCmd {
pub async fn run(args: NodeArgs) -> Result<()> {
match args.cmd {
NodeCmd::Status { node } => cmd_status(node).await,
NodeCmd::Status { node, dir } => cmd_status(node, dir).await,
NodeCmd::Trust { did, node } => cmd_trust(did, node).await,
NodeCmd::Resolve { did, node } => cmd_resolve(did, node).await,
NodeCmd::Register {
@@ -177,7 +182,101 @@ async fn try_get_json(client: &NodeClient, path: &str) -> Option<Value> {
resp.json::<Value>().await.ok()
}
async fn cmd_status(node: String) -> Result<()> {
/// Outcome of fetching the IPFS pins panel for `gl node status`.
///
/// #134 gates `/api/v1/ipfs/pins` behind auth, so this panel signs its request
/// when an identity is available and otherwise reports that the caller must
/// sign in. A pins failure never aborts the dashboard.
#[derive(Debug)]
enum PinsPanel {
/// Signed read succeeded and returned pins (carries the resolved count).
Pins(u64),
/// Signed read succeeded but the node has no pins recorded.
Empty,
/// Signed read was rejected (401/other) or errored.
Unavailable,
/// No identity available; no request was issued.
NeedsIdentity,
/// An explicit `--dir` was given but held no usable identity; carries the dir
/// so the panel can name it instead of the misleading "sign in" prompt.
IdentityError(PathBuf),
}
/// Identity resolved for the signed pins read, decided before the request so the
/// dashboard can distinguish "no identity at all" from "an explicit `--dir` that
/// could not be loaded".
enum PinsAuth {
/// A usable keypair; the read will be signed.
Keyed(Keypair),
/// No identity requested and none in the default keystore.
Anonymous,
/// An explicit `--dir` was given but held no usable identity (carries the dir).
DirUnusable(PathBuf),
}
/// Resolve the pins-panel identity from an optional `--dir`. Mirrors `gl ipfs
/// list`'s identity selection: an explicit `--dir` that fails to load is reported
/// as `DirUnusable` (so the panel names the bad path) rather than collapsing to
/// the misleading "sign in to view"; a missing default keystore (no `--dir`)
/// degrades quietly to `Anonymous`.
fn resolve_pins_auth(dir: Option<&std::path::Path>) -> PinsAuth {
match load_keypair_from_dir(dir) {
Ok(kp) => PinsAuth::Keyed(kp),
Err(_) => match dir {
Some(d) => PinsAuth::DirUnusable(d.to_path_buf()),
None => PinsAuth::Anonymous,
},
}
}
/// Fetch the pins panel state. With a keypair, signs the `/api/v1/ipfs/pins`
/// read and maps the outcome; with no identity it returns `NeedsIdentity`, and
/// with an unusable explicit `--dir` it returns `IdentityError` — both without
/// issuing a request. Injectable (node URL + resolved auth) so tests drive it
/// with a mock server and never touch the default keystore.
async fn fetch_pins(node: &str, auth: PinsAuth) -> PinsPanel {
let kp = match auth {
PinsAuth::Keyed(kp) => kp,
PinsAuth::Anonymous => return PinsPanel::NeedsIdentity,
PinsAuth::DirUnusable(dir) => return PinsPanel::IdentityError(dir),
};
let client = NodeClient::new(node, Some(kp));
let resp = match client.get_signed("/api/v1/ipfs/pins").await {
Ok(r) => r,
Err(_) => return PinsPanel::Unavailable,
};
if !resp.status().is_success() {
return PinsPanel::Unavailable;
}
let Ok(body) = resp.json::<Value>().await else {
return PinsPanel::Unavailable;
};
let count = body["count"]
.as_u64()
.unwrap_or_else(|| body["pins"].as_array().map(|a| a.len() as u64).unwrap_or(0));
if count == 0 {
PinsPanel::Empty
} else {
PinsPanel::Pins(count)
}
}
/// Render the one-line pins-panel status for the `gl node status` dashboard.
fn pins_status_line(panel: &PinsPanel) -> String {
match panel {
PinsPanel::Pins(count) => format!(" Pinned CIDs: {count}"),
PinsPanel::Empty => " Pinned CIDs: 0".to_string(),
PinsPanel::Unavailable => " IPFS pins: unavailable".to_string(),
PinsPanel::NeedsIdentity => {
" IPFS pins: sign in to view (run `gl identity new`)".to_string()
}
PinsPanel::IdentityError(dir) => {
format!(" IPFS pins: no usable identity in {}", dir.display())
}
}
}
async fn cmd_status(node: String, dir: Option<PathBuf>) -> Result<()> {
let client = NodeClient::new(&node, None);
// ── Fetch node info (required — bail if unreachable) ──────────────────
@@ -194,13 +293,21 @@ async fn cmd_status(node: String) -> Result<()> {
let version = info["version"].as_str().unwrap_or("unknown");
let network = info["network"].as_str().unwrap_or("unknown");
// The pins panel signs its read (#134 gates /api/v1/ipfs/pins behind auth).
// `--dir` selects the same identity directory as `gl ipfs list` (#146); an
// explicit --dir that can't be loaded is surfaced in the panel rather than
// masquerading as "no identity", while a missing default keystore degrades
// quietly. A pins failure never aborts the dashboard.
let pins_auth = resolve_pins_auth(dir.as_deref());
// ── Fetch remaining endpoints in parallel ─────────────────────────────
let (peers_val, repos_val, p2p_val, events_val, pins_val) = tokio::join!(
// Peers/repos/p2p/events stay anonymous; only pins is signed.
let (peers_val, repos_val, p2p_val, events_val, pins_panel) = tokio::join!(
try_get_json(&client, "/api/v1/peers"),
try_get_json(&client, "/api/v1/repos"),
try_get_json(&client, "/api/v1/p2p/info"),
try_get_json(&client, "/api/v1/events/ref-updates?limit=5"),
try_get_json(&client, "/api/v1/ipfs/pins"),
fetch_pins(&node, pins_auth),
);
// ── Render dashboard ──────────────────────────────────────────────────
@@ -307,14 +414,7 @@ async fn cmd_status(node: String) -> Result<()> {
// Pins
println!("Pins");
if let Some(ref pins) = pins_val {
let count = pins["count"]
.as_u64()
.unwrap_or_else(|| pins["pins"].as_array().map(|a| a.len() as u64).unwrap_or(0));
println!(" Pinned CIDs: {count}");
} else {
println!(" IPFS not configured");
}
println!("{}", pins_status_line(&pins_panel));
println!();
Ok(())
@@ -409,3 +509,270 @@ async fn cmd_resolve(did: String, node: String) -> Result<()> {
Ok(())
}
#[cfg(test)]
mod tests {
use super::*;
use gitlawb_core::identity::Keypair;
#[tokio::test]
async fn test_fetch_pins_keyed_happy_signs_and_returns_pins() {
let mut server = mockito::Server::new_async().await;
let kp = Keypair::generate();
// A keyed fetch must sign the request (RFC 9421 headers) and, on a
// populated 200 body, land in the Pins state carrying the pins.
let m = server
.mock("GET", "/api/v1/ipfs/pins")
.match_header("signature", mockito::Matcher::Any)
.match_header("signature-input", mockito::Matcher::Any)
.match_header("content-digest", mockito::Matcher::Any)
.with_status(200)
.with_header("content-type", "application/json")
.with_body(
r#"{"pins":[{"cid":"bafyone","sha256_hex":"abc123","pinned_at":"2026-07-02T12:00:00Z"}],"count":1}"#,
)
.create_async()
.await;
let panel = fetch_pins(&server.url(), PinsAuth::Keyed(kp)).await;
match panel {
PinsPanel::Pins(count) => assert_eq!(count, 1),
other => panic!("expected Pins, got {other:?}"),
}
m.assert_async().await;
}
#[tokio::test]
async fn test_fetch_pins_keyed_empty_returns_empty() {
let mut server = mockito::Server::new_async().await;
let kp = Keypair::generate();
let m = server
.mock("GET", "/api/v1/ipfs/pins")
.match_header("signature", mockito::Matcher::Any)
.with_status(200)
.with_header("content-type", "application/json")
.with_body(r#"{"pins":[],"count":0}"#)
.create_async()
.await;
let panel = fetch_pins(&server.url(), PinsAuth::Keyed(kp)).await;
assert!(
matches!(panel, PinsPanel::Empty),
"expected Empty, got {panel:?}"
);
m.assert_async().await;
}
#[tokio::test]
async fn test_fetch_pins_keyed_rejected_returns_unavailable() {
let mut server = mockito::Server::new_async().await;
let kp = Keypair::generate();
// Node rejects the signed read (401): the panel must degrade to
// Unavailable without panicking, so cmd_status still completes.
let m = server
.mock("GET", "/api/v1/ipfs/pins")
.match_header("signature", mockito::Matcher::Any)
.with_status(401)
.with_header("content-type", "application/json")
.with_body(r#"{"error":"unauthorized"}"#)
.create_async()
.await;
let panel = fetch_pins(&server.url(), PinsAuth::Keyed(kp)).await;
assert!(
matches!(panel, PinsPanel::Unavailable),
"expected Unavailable, got {panel:?}"
);
m.assert_async().await;
}
#[tokio::test]
async fn test_fetch_pins_unkeyed_needs_identity_without_request() {
let mut server = mockito::Server::new_async().await;
// With no keypair the endpoint must never be hit.
let m = server
.mock("GET", "/api/v1/ipfs/pins")
.expect(0)
.create_async()
.await;
let panel = fetch_pins(&server.url(), PinsAuth::Anonymous).await;
assert!(
matches!(panel, PinsPanel::NeedsIdentity),
"expected NeedsIdentity, got {panel:?}"
);
m.assert_async().await;
}
#[tokio::test]
async fn test_fetch_pins_malformed_body_returns_unavailable() {
let mut server = mockito::Server::new_async().await;
let kp = Keypair::generate();
// 2xx but the body is not valid JSON: must degrade to Unavailable,
// never panic.
let m = server
.mock("GET", "/api/v1/ipfs/pins")
.match_header("signature", mockito::Matcher::Any)
.with_status(200)
.with_header("content-type", "application/json")
.with_body("not json{{{")
.create_async()
.await;
let panel = fetch_pins(&server.url(), PinsAuth::Keyed(kp)).await;
assert!(
matches!(panel, PinsPanel::Unavailable),
"malformed body -> Unavailable, got {panel:?}"
);
m.assert_async().await;
}
#[tokio::test]
async fn test_fetch_pins_transport_error_returns_unavailable() {
// Bind then drop to obtain a definitely-closed port -> connection
// refused -> get_signed Err -> Unavailable (no panic).
let port = {
let l = std::net::TcpListener::bind("127.0.0.1:0").unwrap();
l.local_addr().unwrap().port()
};
let kp = Keypair::generate();
let panel = fetch_pins(&format!("http://127.0.0.1:{port}"), PinsAuth::Keyed(kp)).await;
assert!(
matches!(panel, PinsPanel::Unavailable),
"transport error -> Unavailable, got {panel:?}"
);
}
#[test]
fn test_pins_status_line_renders_each_state() {
assert_eq!(pins_status_line(&PinsPanel::Pins(3)), " Pinned CIDs: 3");
assert_eq!(pins_status_line(&PinsPanel::Empty), " Pinned CIDs: 0");
assert_eq!(
pins_status_line(&PinsPanel::Unavailable),
" IPFS pins: unavailable"
);
assert_eq!(
pins_status_line(&PinsPanel::NeedsIdentity),
" IPFS pins: sign in to view (run `gl identity new`)"
);
assert_eq!(
pins_status_line(&PinsPanel::IdentityError(PathBuf::from("/tmp/id"))),
" IPFS pins: no usable identity in /tmp/id"
);
}
// Adversarial follow-up to jatmn #146 P2: an explicit `--dir` that holds no
// usable identity must resolve to DirUnusable (which names the path), NOT the
// Anonymous "sign in to view / run `gl identity new`" case — otherwise a user
// who did supply an identity dir is misdirected to create one they may have.
#[test]
fn resolve_pins_auth_reports_explicit_bad_dir_as_unusable() {
let bad = PathBuf::from("/nonexistent/gl-id-xyz");
match resolve_pins_auth(Some(&bad)) {
PinsAuth::DirUnusable(d) => assert_eq!(d, bad),
_ => panic!("explicit unusable --dir must be DirUnusable, not Anonymous"),
}
}
// Mirror of the above for the absent-`--dir` case: `None` must never resolve to
// DirUnusable — it degrades to Anonymous (no default keystore) or Keyed (one
// present). Env-independent: only the None->DirUnusable mapping is guarded, so it
// holds whether or not the runner has a default `~/.gitlawb` identity.
#[test]
fn resolve_pins_auth_none_is_never_dir_unusable() {
assert!(
!matches!(resolve_pins_auth(None), PinsAuth::DirUnusable(_)),
"absent --dir must degrade to Anonymous (or Keyed), never DirUnusable"
);
}
// The success wiring for #146: an explicit --dir with a valid identity must
// load that key (dir -> load_keypair_from_dir -> Keyed), so the pins read is
// signed with the selected identity rather than the default keystore.
#[test]
fn resolve_pins_auth_loads_keyed_identity_from_explicit_dir() {
let dir = tempfile::TempDir::new().unwrap();
let kp = Keypair::generate();
std::fs::write(
dir.path().join("identity.pem"),
kp.to_pem().unwrap().as_bytes(),
)
.unwrap();
let PinsAuth::Keyed(loaded) = resolve_pins_auth(Some(dir.path())) else {
panic!("an explicit --dir with a valid identity must resolve to Keyed");
};
// A bare `Keyed(_)` match would also pass if load fell through to the default
// keystore; assert the loaded identity is the one written to --dir so the pins
// read signs with the selected key, never the default.
assert_eq!(
loaded.did(),
kp.did(),
"must sign with the identity from --dir, not the default keystore"
);
}
#[tokio::test]
async fn test_fetch_pins_dir_unusable_reports_identity_error_without_request() {
let mut server = mockito::Server::new_async().await;
// An unusable explicit --dir must render the identity error and never hit
// the endpoint (no identity to sign with).
let m = server
.mock("GET", "/api/v1/ipfs/pins")
.expect(0)
.create_async()
.await;
let bad = PathBuf::from("/nonexistent/gl-id-xyz");
let panel = fetch_pins(&server.url(), PinsAuth::DirUnusable(bad.clone())).await;
match panel {
PinsPanel::IdentityError(d) => assert_eq!(d, bad),
other => panic!("expected IdentityError, got {other:?}"),
}
m.assert_async().await;
}
// jatmn #146 P2: `gl node status` must accept the same `--dir` identity
// selector as `gl ipfs list`, so a user who selected an identity via --dir can
// authenticate the status pins panel instead of seeing "sign in to view".
#[test]
fn status_accepts_dir_identity_selector() {
use clap::Parser;
#[derive(Parser)]
struct TestCli {
#[command(flatten)]
args: NodeArgs,
}
let cli = TestCli::try_parse_from([
"gl",
"status",
"--node",
"http://example",
"--dir",
"/tmp/id",
])
.expect("`node status` must accept --dir");
let NodeCmd::Status { dir, .. } = cli.args.cmd else {
panic!("expected the Status subcommand");
};
assert_eq!(dir, Some(PathBuf::from("/tmp/id")));
// Absent --dir must stay None so the default keystore path is unchanged.
let cli = TestCli::try_parse_from(["gl", "status"]).expect("status parses without --dir");
let NodeCmd::Status { dir, .. } = cli.args.cmd else {
panic!("expected the Status subcommand");
};
assert_eq!(dir, None);
}
}
+43 -32
View File
@@ -208,20 +208,16 @@ pub async fn run(args: RepoArgs) -> Result<()> {
}
/// Derive the short DID key segment from a keypair, or fall back to the node's DID.
async fn resolve_owner_did(node: &str, dir: Option<&std::path::Path>) -> Result<String> {
if let Ok(kp) = load_keypair_from_dir(dir) {
let did = kp.did().to_string();
return Ok(did.split(':').next_back().unwrap_or(&did).to_string());
}
let client = NodeClient::new(node, None);
let info: Value = client
.get("/")
.await?
.json()
.await
.context("failed to fetch node info")?;
let did = info["did"].as_str().context("node missing DID")?;
Ok(did.split(':').next_back().unwrap_or(did).to_string())
/// Resolve the owner short-DID for a bare repo name from the LOCAL identity.
/// Never falls back to the node's own DID (that produced bogus "owned by the
/// node" results for repos that don't exist) — if there's no local identity the
/// caller must pass an explicit `owner/name`.
async fn resolve_owner_did(_node: &str, dir: Option<&std::path::Path>) -> Result<String> {
let kp = load_keypair_from_dir(dir).context(
"no local identity to resolve the repo owner — pass `owner/name`, or run `gl identity new`",
)?;
let did = kp.did().to_string();
Ok(did.split(':').next_back().unwrap_or(&did).to_string())
}
async fn cmd_create(
@@ -306,25 +302,28 @@ async fn cmd_list(node: String, dir: Option<PathBuf>) -> Result<()> {
}
async fn cmd_clone(name: String, node: String, dir: Option<PathBuf>) -> Result<()> {
let did = if let Ok(kp) = load_keypair_from_dir(dir.as_deref()) {
kp.did().to_string()
// Owner is taken from an explicit `owner/name`, else the LOCAL identity —
// never the node's own DID. A missing repo then surfaces as the helper's
// clear 404 rather than a clone under an invented owner.
let (did, repo_name) = if let Some((owner, rest)) = name.split_once('/') {
(owner.to_string(), rest.to_string())
} else {
let client = NodeClient::new(&node, None);
let info: Value = client.get("/").await?.json().await?;
info["did"]
.as_str()
.context("node missing DID")?
.to_string()
// Bare name: derive the SHORT owner key via the same helper the other
// commands use, so the URL is `gitlawb://z6Mk.../name` — not the full
// `did:key:z6Mk...` form, whose colons in the authority break the helper.
(resolve_owner_did(&node, dir.as_deref()).await?, name)
};
let url = format!("gitlawb://{did}/{name}");
let url = format!("gitlawb://{did}/{repo_name}");
println!(" cloning {url}");
let status = std::process::Command::new("git")
.arg("clone")
.arg(&url)
// Point the remote helper at the same node the user selected.
.env("GITLAWB_NODE", &node)
.status()
.context("failed to run git clone — is git installed?")?;
if !status.success() {
anyhow::bail!("git clone failed");
anyhow::bail!("git clone failed — does the repo exist on this node?");
}
Ok(())
}
@@ -340,12 +339,24 @@ async fn cmd_info(repo: String, node: String, dir: Option<PathBuf>) -> Result<()
(short, repo)
};
let r: Value = client
.get(&format!("/api/v1/repos/{owner}/{name}"))
.await?
.json()
.await
.context("repo not found")?;
let resp = client.get(&format!("/api/v1/repos/{owner}/{name}")).await?;
// A non-existent (or unreadable/quarantined) repo is a real 404 from the
// node — surface it plainly instead of printing a stub card with `?` fields
// and a placeholder owner DID.
if !resp.status().is_success() {
if resp.status().as_u16() == 404 {
anyhow::bail!("repository '{owner}/{name}' not found");
}
let status = resp.status();
let msg = resp
.json::<Value>()
.await
.ok()
.and_then(|v| v["message"].as_str().map(String::from))
.unwrap_or_else(|| "request failed".to_string());
anyhow::bail!("repo info failed ({status}): {msg}");
}
let r: Value = resp.json().await.context("parse repo info")?;
let owner_did = r["owner_did"].as_str().unwrap_or(&owner);
let gitlawb_url = format!("gitlawb://{owner_did}/{name}");
@@ -647,10 +658,10 @@ async fn cmd_label_remove(
async fn cmd_label_list(repo: String, node: String, dir: Option<PathBuf>) -> Result<()> {
let (owner, name) = resolve_owner_repo_pair(&repo, &node, dir.as_deref()).await?;
let client = NodeClient::new(&node, None);
let client = NodeClient::new(&node, load_keypair_from_dir(dir.as_deref()).ok());
let resp: Value = client
.get(&format!("/api/v1/repos/{owner}/{name}/labels"))
.get_authed(&format!("/api/v1/repos/{owner}/{name}/labels"))
.await?
.json()
.await
+15 -9
View File
@@ -34,12 +34,14 @@ pub enum StarCmd {
#[arg(long)]
dir: Option<PathBuf>,
},
/// Show star count for a repository (no auth required)
/// Show star count for a repository
Count {
/// Repository in owner/repo format
repo: String,
#[arg(long, default_value = "https://node.gitlawb.com", env = "GITLAWB_NODE")]
node: String,
#[arg(long)]
dir: Option<PathBuf>,
},
}
@@ -47,7 +49,7 @@ pub async fn run(args: StarArgs) -> Result<()> {
match args.cmd {
StarCmd::Add { repo, node, dir } => cmd_add(repo, node, dir).await,
StarCmd::Remove { repo, node, dir } => cmd_remove(repo, node, dir).await,
StarCmd::Count { repo, node } => cmd_count(repo, node).await,
StarCmd::Count { repo, node, dir } => cmd_count(repo, node, dir).await,
}
}
@@ -110,15 +112,15 @@ async fn cmd_remove(repo: String, node: String, dir: Option<PathBuf>) -> Result<
Ok(())
}
async fn cmd_count(repo: String, node: String) -> Result<()> {
async fn cmd_count(repo: String, node: String, dir: Option<PathBuf>) -> Result<()> {
let (owner, name) = repo
.split_once('/')
.map(|(o, n)| (o.to_string(), n.to_string()))
.context("use owner/repo format for count (e.g. alice/myrepo)")?;
let client = NodeClient::new(&node, None);
let client = NodeClient::new(&node, load_keypair_from_dir(dir.as_deref()).ok());
let resp = client
.get(&format!("/api/v1/repos/{owner}/{name}/star"))
.get_authed(&format!("/api/v1/repos/{owner}/{name}/star"))
.await
.context("failed to connect to node")?;
@@ -310,16 +312,20 @@ mod tests {
.create_async()
.await;
cmd_count("alice/myrepo".to_string(), server.url())
cmd_count("alice/myrepo".to_string(), server.url(), None)
.await
.unwrap();
}
#[tokio::test]
async fn test_cmd_count_requires_slash() {
let err = cmd_count("noslash".to_string(), "http://127.0.0.1:1".to_string())
.await
.unwrap_err();
let err = cmd_count(
"noslash".to_string(),
"http://127.0.0.1:1".to_string(),
None,
)
.await
.unwrap_err();
assert!(err.to_string().contains("owner/repo format"));
}
+285 -10
View File
@@ -1,4 +1,4 @@
use anyhow::Result;
use anyhow::{Context, Result};
use clap::{Args, Subcommand};
use std::path::PathBuf;
@@ -30,15 +30,39 @@ pub enum SyncCmd {
pub async fn run(args: SyncArgs) -> Result<()> {
match args.cmd {
SyncCmd::Trigger => {
let keypair = load_keypair_from_dir(args.dir.as_deref()).ok();
let client = NodeClient::new(&args.node, keypair);
let resp: serde_json::Value = client
.post("/api/v1/sync/trigger", b"{}")
.await?
.json()
.await?;
let peers = resp["peers_reached"].as_u64().unwrap_or(0);
let enqueued = resp["repos_enqueued"].as_u64().unwrap_or(0);
// /api/v1/sync/trigger always requires a signature, so a missing or
// unreadable identity must fail here, locally, rather than sending an
// unsigned request that can only 401 remotely (matches the other
// signed CLI writes).
let keypair = load_keypair_from_dir(args.dir.as_deref())
.context("identity not found — run `gl identity new` first")?;
let client = NodeClient::new(&args.node, Some(keypair));
let resp = client.post("/api/v1/sync/trigger", b"{}").await?;
// The node now requires a signature on this route and rate-limits it,
// so a denial (401/429/…) is expected. Check the status BEFORE parsing:
// otherwise a JSON-ish error body deserializes into a zero-count struct
// and prints a fabricated "✓ sync triggered / 0 peers" success.
let status = resp.status();
if !status.is_success() {
// Bound the read: a hostile or broken node must not force an
// unbounded allocation just to surface a denial (INV-6, read half).
let raw = read_body_capped(resp, 8 * 1024).await;
let msg = serde_json::from_str::<serde_json::Value>(&raw)
.ok()
.and_then(|v| {
v.get("message")
.or_else(|| v.get("error"))
.and_then(|m| m.as_str())
.map(str::to_string)
})
.unwrap_or(raw);
anyhow::bail!(
"sync trigger failed ({status}): {}",
sanitize_node_msg(&msg)
);
}
let resp: serde_json::Value = resp.json().await?;
let (peers, enqueued) = trigger_counts(&resp);
println!("✓ sync triggered");
println!(" peers reached: {peers}");
println!(" repos enqueued: {enqueued}");
@@ -69,3 +93,254 @@ pub async fn run(args: SyncArgs) -> Result<()> {
}
Ok(())
}
/// Extract `(peers_reached, repos_enqueued)` from a successful sync-trigger
/// response. Split out so the extraction is unit-testable (missing or malformed
/// fields default to 0 rather than panicking).
fn trigger_counts(resp: &serde_json::Value) -> (u64, u64) {
(
resp["peers_reached"].as_u64().unwrap_or(0),
resp["repos_enqueued"].as_u64().unwrap_or(0),
)
}
/// Read at most `cap` bytes of a response body. Bounds the allocation from a
/// hostile or broken node returning a huge error body — the display is capped
/// separately, but the read itself must not be unbounded (INV-6, read half).
async fn read_body_capped(mut resp: reqwest::Response, cap: usize) -> String {
let mut buf: Vec<u8> = Vec::new();
while buf.len() < cap {
match resp.chunk().await {
Ok(Some(chunk)) => {
let take = (cap - buf.len()).min(chunk.len());
buf.extend_from_slice(&chunk[..take]);
if take < chunk.len() {
break; // hit the cap mid-chunk
}
}
_ => break, // end of body or read error — return what we have
}
}
String::from_utf8_lossy(&buf).into_owned()
}
/// Strip terminal-dangerous characters from (and cap the length of) a
/// node-supplied error string before surfacing it. The node a caller talks to
/// could be hostile and embed escape sequences in its error body; those must not
/// reach the terminal verbatim (INV-6). We drop the C0/C1 control bytes (which
/// defangs ANSI/OSC escapes) AND the Unicode bidi/format controls (which
/// `char::is_control` does not cover — they can reorder the displayed line).
fn sanitize_node_msg(s: &str) -> String {
s.chars()
.filter(|c| !c.is_control() && !is_bidi_format(*c))
.take(200)
.collect()
}
/// Unicode bidirectional and directional-isolate format characters (category
/// `Cf`). These are not `char::is_control()` (that is category `Cc` only), but a
/// right-to-left override or isolate can visually reorder a terminal line to
/// spoof the error text, so they are stripped alongside the control bytes.
fn is_bidi_format(c: char) -> bool {
matches!(c,
'\u{200E}' | '\u{200F}' | '\u{061C}' // LRM, RLM, ALM
| '\u{202A}'..='\u{202E}' // LRE, RLE, PDF, LRO, RLO
| '\u{2066}'..='\u{2069}' // LRI, RLI, FSI, PDI
)
}
#[cfg(test)]
mod tests {
use super::*;
fn trigger_args(node: String) -> (SyncArgs, tempfile::TempDir) {
// Seed a real identity so `run` gets past the mandatory-keypair check and
// reaches the status-handling path. The mocks below return a fixed status
// regardless of the signature, so these tests exercise the client's
// status-check-before-parse, not signature verification (that is proved
// server-side). Return the TempDir so the caller keeps it alive.
let dir = tempfile::TempDir::new().unwrap();
let kp = gitlawb_core::identity::Keypair::generate();
std::fs::write(
dir.path().join("identity.pem"),
kp.to_pem().unwrap().as_bytes(),
)
.unwrap();
let args = SyncArgs {
cmd: SyncCmd::Trigger,
node,
dir: Some(dir.path().to_path_buf()),
};
(args, dir)
}
#[tokio::test]
async fn trigger_requires_identity_fails_before_request() {
// Empty identity dir → no keypair. `sync trigger` must fail locally with
// a clear identity error BEFORE issuing any request. The node URL points
// at an unreachable port, so a request attempt would surface a different
// (connection) error; getting the identity error proves we never dialed.
let dir = tempfile::TempDir::new().unwrap();
let args = SyncArgs {
cmd: SyncCmd::Trigger,
node: "http://127.0.0.1:1".to_string(),
dir: Some(dir.path().to_path_buf()),
};
let err = run(args).await.unwrap_err();
assert!(
err.to_string().contains("identity not found"),
"expected a local identity error before any request, got: {err}"
);
}
#[tokio::test]
async fn trigger_surfaces_401_as_error_not_fake_success() {
let mut server = mockito::Server::new_async().await;
let _m = server
.mock("POST", "/api/v1/sync/trigger")
.with_status(401)
.with_header("content-type", "application/json")
// Valid JSON: the parse-without-status-check bug deserializes this
// into a zero-count success struct and prints "✓ sync triggered".
.with_body(r#"{"message":"unauthorized"}"#)
.create_async()
.await;
let (args, _dir) = trigger_args(server.url());
let err = run(args).await.unwrap_err();
assert!(
err.to_string().contains("401"),
"expected 401 surfaced, got: {err}"
);
}
#[tokio::test]
async fn trigger_surfaces_429_as_error() {
let mut server = mockito::Server::new_async().await;
let _m = server
.mock("POST", "/api/v1/sync/trigger")
.with_status(429)
.with_header("content-type", "application/json")
.with_body(r#"{"message":"slow down"}"#)
.create_async()
.await;
let (args, _dir) = trigger_args(server.url());
let err = run(args).await.unwrap_err();
assert!(
err.to_string().contains("429"),
"expected 429 surfaced, got: {err}"
);
}
#[tokio::test]
async fn trigger_sanitizes_control_chars_in_node_error() {
// A hostile node embeds an ANSI color escape (ESC) and a bell (BEL) in
// the JSON message field. The surfaced error must contain neither raw
// control byte, while keeping the printable text.
let mut server = mockito::Server::new_async().await;
let _m = server
.mock("POST", "/api/v1/sync/trigger")
.with_status(401)
.with_header("content-type", "application/json")
// Valid JSON whose message carries JSON-escaped ESC (\u001b) and
// BEL (\u0007); serde decodes them to real control bytes a naive
// client would print. (The status-check bug fake-successes here.)
.with_body("{\"message\":\"pwned\\u001b[31m\\u0007bad\"}")
.create_async()
.await;
let (args, _dir) = trigger_args(server.url());
let err = run(args).await.unwrap_err();
let s = err.to_string();
assert!(!s.contains('\u{1b}'), "ESC leaked to terminal: {s:?}");
assert!(!s.contains('\u{07}'), "BEL leaked to terminal: {s:?}");
assert!(
s.contains("pwned") && s.contains("bad"),
"message text dropped: {s:?}"
);
}
#[tokio::test]
async fn trigger_ok_prints_counts() {
let mut server = mockito::Server::new_async().await;
let _m = server
.mock("POST", "/api/v1/sync/trigger")
.with_status(200)
.with_header("content-type", "application/json")
.with_body(r#"{"peers_reached":2,"repos_enqueued":5}"#)
.create_async()
.await;
let (args, _dir) = trigger_args(server.url());
run(args).await.unwrap();
}
#[test]
fn sanitize_strips_controls_bidi_and_caps_length() {
// C0 (ESC/BEL) and the Cf bidi override (U+202E) are both removed; the
// printable text survives. (Note: a stripped ESC leaves any following
// "[31m" as inert literal text — that is the point, so the input here
// avoids that residue to keep the expectation unambiguous.)
let out = sanitize_node_msg("a\u{1b}\u{07}b\u{202e}c");
assert!(
!out.chars().any(|c| c.is_control()),
"control char leaked: {out:?}"
);
assert!(
!out.contains('\u{202e}'),
"RLO bidi override leaked: {out:?}"
);
assert_eq!(out, "abc");
// Length is capped at 200 chars regardless of input size.
let long = "x".repeat(250);
assert_eq!(sanitize_node_msg(&long).chars().count(), 200);
}
#[tokio::test]
async fn trigger_handles_oversized_error_body_without_unbounded_output() {
// A hostile/broken node returns a 2 MB error body. The command must still
// surface the denial with a bounded message, not hang or dump the body.
let mut server = mockito::Server::new_async().await;
let _m = server
.mock("POST", "/api/v1/sync/trigger")
.with_status(401)
.with_body("A".repeat(2_000_000))
.create_async()
.await;
let (args, _dir) = trigger_args(server.url());
let err = run(args).await.unwrap_err();
let s = err.to_string();
assert!(s.contains("401"), "denial not surfaced: {s:.80?}");
assert!(
s.len() < 500,
"error message not bounded: {} chars",
s.len()
);
}
#[tokio::test]
async fn read_body_capped_bounds_the_read() {
// The read must stop at the cap — a 2 MB body yields at most `cap` bytes,
// not the whole thing (which resp.text() would return).
let mut server = mockito::Server::new_async().await;
let _m = server
.mock("GET", "/big")
.with_status(200)
.with_body("A".repeat(2_000_000))
.create_async()
.await;
let resp = reqwest::get(format!("{}/big", server.url())).await.unwrap();
let out = read_body_capped(resp, 8192).await;
assert!(out.len() <= 8192, "read not bounded: {} bytes", out.len());
assert!(!out.is_empty(), "expected some body");
}
#[test]
fn trigger_counts_extracts_both_values() {
let v = serde_json::json!({"peers_reached": 2, "repos_enqueued": 5});
assert_eq!(trigger_counts(&v), (2, 5));
// Missing/malformed fields default to 0, never panic.
assert_eq!(trigger_counts(&serde_json::json!({})), (0, 0));
assert_eq!(
trigger_counts(&serde_json::json!({"peers_reached": "x"})),
(0, 0)
);
}
}
+17
View File
@@ -0,0 +1,17 @@
[package]
name = "icaptcha-client"
version = "0.4.0" # x-release-please-version
edition.workspace = true
rust-version.workspace = true
license.workspace = true
authors.workspace = true
repository.workspace = true
[dependencies]
gitlawb-core = { path = "../gitlawb-core" }
reqwest = { workspace = true }
serde = { workspace = true }
serde_json = { workspace = true }
anyhow = { workspace = true }
thiserror = { workspace = true }
tracing = { workspace = true }
+406
View File
@@ -0,0 +1,406 @@
//! Client for the iCaptcha proof-of-intelligence service.
//!
//! The node gates spam-prone writes (repo create / fork / register) behind an
//! iCaptcha proof. This crate implements the *sanctioned client flow*: on a
//! `403 icaptcha_proof_required`, request a challenge for the required level,
//! solve the deterministic computational types locally, obtain the signed
//! proof, and hand it back so the caller can retry the original signed request
//! with the `x-icaptcha-proof` header.
//!
//! `requesterId` is always the caller's DID, so the proof's `sub` claim matches
//! the authenticated signer (the node enforces `sub == authenticated DID`).
//!
//! Blocking HTTP (reqwest::blocking) so the git remote helper can use it
//! directly; `gl` (async) calls it via `tokio::task::spawn_blocking`.
use std::io::Read;
use anyhow::{anyhow, bail, Context, Result};
use serde::Deserialize;
use serde_json::json;
pub mod solvers;
/// Default iCaptcha service base URL (used when the node doesn't advertise one).
pub const DEFAULT_URL: &str = "https://icaptcha.gitlawb.com";
/// Default required level (the node's default floor).
pub const DEFAULT_LEVEL: u32 = 3;
/// Header the gated write must echo the proof back in.
pub const PROOF_HEADER: &str = "x-icaptcha-proof";
/// Cap on bytes read from an error response, and characters kept in the excerpt.
/// The iCaptcha origin is only as trusted as the node that advertised it (a MITM
/// on the default transport qualifies), so its error bodies are attacker-
/// influenceable bytes that reach the TTY — they must be bounded and stripped of
/// control characters before printing (CWE-150, same treatment as #137).
const MAX_ERR_BYTES: u64 = 4096;
const MAX_ERR_CHARS: usize = 500;
/// Read a bounded error body and return a control-char-stripped, length-capped
/// single-line excerpt safe to print to a terminal.
fn error_excerpt(resp: reqwest::blocking::Response) -> String {
let mut buf = Vec::new();
if resp.take(MAX_ERR_BYTES).read_to_end(&mut buf).is_err() {
return String::new();
}
sanitize_excerpt(&String::from_utf8_lossy(&buf))
}
/// Strip C0/C1 control characters, collapse whitespace runs to single spaces,
/// and cap the length — so a hostile response body cannot inject terminal
/// escapes or flood the TTY.
fn sanitize_excerpt(s: &str) -> String {
let mut out = String::new();
let mut pending_space = false;
let mut kept = 0;
for ch in s.trim().chars() {
if kept >= MAX_ERR_CHARS {
break;
}
if ch.is_control() {
if matches!(ch, '\n' | '\r' | '\t') {
pending_space = true;
}
continue;
}
if ch.is_whitespace() {
pending_space = true;
continue;
}
if pending_space && !out.is_empty() {
out.push(' ');
kept += 1;
if kept >= MAX_ERR_CHARS {
break;
}
}
out.push(ch);
kept += 1;
pending_space = false;
}
out
}
/// Whether `u` parses as an `https` URL.
fn is_https(u: &str) -> bool {
reqwest::Url::parse(u)
.map(|p| p.scheme() == "https")
.unwrap_or(false)
}
/// Lowercased host of a URL, if it parses and has one.
fn host_of(u: &str) -> Option<String> {
reqwest::Url::parse(u)
.ok()
.and_then(|p| p.host_str().map(|h| h.to_ascii_lowercase()))
}
/// Decide which iCaptcha origin to actually talk to, and whether it is trusted
/// enough to receive the operator's API key.
///
/// The node-advertised `x-icaptcha-url` is untrusted: a hostile node could point
/// it at an attacker origin (client-side SSRF) or capture a bearer token on the
/// first hop. So an advertised URL is honored only when it is `https` AND its
/// host is allowlisted — the public default host, or the operator's own
/// `GITLAWB_ICAPTCHA_URL`. Otherwise we fall back to the operator/default origin.
///
/// The API key is attached only when the resolved origin is the operator's
/// explicitly-configured host (never a node-discovered one), so a key is never
/// exfiltrated to an origin the operator did not choose.
fn resolve_solver_url(advertised: Option<&str>, operator: Option<&str>) -> (String, bool) {
let operator = operator.filter(|u| is_https(u));
let operator_host = operator.and_then(host_of);
let default_host = host_of(DEFAULT_URL);
let allowed = |host: &Option<String>| -> bool {
host.as_ref()
.map(|h| Some(h) == default_host.as_ref() || Some(h) == operator_host.as_ref())
.unwrap_or(false)
};
let fallback = || {
operator
.map(String::from)
.unwrap_or_else(|| DEFAULT_URL.to_string())
};
let chosen = match advertised {
Some(a) if is_https(a) && allowed(&host_of(a)) => a.to_string(),
Some(a) => {
tracing::warn!(
advertised = %a,
"ignoring iCaptcha URL advertised by node (must be https with an allowlisted host); using trusted origin"
);
fallback()
}
None => fallback(),
};
// Key goes only to the operator's own configured origin.
let key_trusted = operator_host.is_some() && host_of(&chosen) == operator_host;
(chosen, key_trusted)
}
/// Computational challenge types this client solves locally. Restricting the
/// request to these avoids dictionary (anagram/logic) and LLM (wordproblem/
/// riddle) types, which can't be auto-solved.
const SOLVABLE_TYPES: [&str; 3] = ["arithmetic", "algebra", "sequence"];
/// Bound on challenge/answer rounds (the service escalates difficulty on a miss;
/// correct solvers shouldn't escalate, but cap it regardless).
const MAX_ROUNDS: usize = 8;
/// Where + at what level to solve. `did` becomes the proof's `sub`.
#[derive(Debug, Clone)]
pub struct IcaptchaCfg {
pub url: String,
pub did: String,
pub level: u32,
/// Optional bearer token for an API-key-protected iCaptcha deployment.
pub api_key: Option<String>,
}
impl IcaptchaCfg {
/// Build config from the caller DID plus an optionally node-advertised
/// url/level (e.g. the `x-icaptcha-url` / `x-icaptcha-level` headers) and
/// optional level.
///
/// The advertised URL is validated and allowlisted (see [`resolve_solver_url`])
/// so a hostile node cannot redirect the solve to an attacker origin. The
/// `GITLAWB_ICAPTCHA_API_KEY` bearer token is attached only when the resolved
/// origin is the operator's own `GITLAWB_ICAPTCHA_URL`, never a
/// node-discovered one.
pub fn new(did: impl Into<String>, url: Option<String>, level: Option<u32>) -> Self {
let operator = std::env::var("GITLAWB_ICAPTCHA_URL")
.ok()
.filter(|s| !s.is_empty());
let (resolved_url, key_trusted) = resolve_solver_url(url.as_deref(), operator.as_deref());
let api_key = if key_trusted {
std::env::var("GITLAWB_ICAPTCHA_API_KEY")
.ok()
.filter(|s| !s.is_empty())
} else {
None
};
Self {
url: resolved_url,
did: did.into(),
level: level.unwrap_or(DEFAULT_LEVEL),
api_key,
}
}
}
/// A challenge handed back by the service (mirrors `icaptcha` `Challenge`).
#[derive(Debug, Clone, Deserialize)]
#[serde(rename_all = "camelCase")]
pub struct Challenge {
pub challenge_id: String,
#[serde(rename = "type")]
pub kind: String,
pub difficulty: u32,
pub prompt: String,
pub token: String,
}
#[derive(Debug, Deserialize)]
#[serde(tag = "status", rename_all = "lowercase")]
enum AnswerResult {
Passed { proof: String },
Continue { challenge: Challenge },
Failed { reason: String },
}
/// Solver callback for types this crate can't solve deterministically.
pub type Solver<'a> = dyn Fn(&Challenge) -> Option<String> + 'a;
/// Run the full challenge → solve → answer loop and return a fresh proof token.
///
/// `solver` is consulted for challenge types the built-in solvers don't handle
/// (anagram/logic/LLM); pass `None` to fall back to an interactive stdin prompt.
pub fn obtain_proof(cfg: &IcaptchaCfg, solver: Option<&Solver>) -> Result<String> {
let client = reqwest::blocking::Client::builder()
.timeout(std::time::Duration::from_secs(30))
.build()
.context("build iCaptcha http client")?;
let mut challenge = request_challenge(&client, cfg)?;
for _ in 0..MAX_ROUNDS {
let answer = solvers::solve(&challenge.kind, &challenge.prompt)
.or_else(|| solver.and_then(|s| s(&challenge)))
.or_else(|| interactive_prompt(&challenge))
.ok_or_else(|| {
anyhow!(
"cannot solve iCaptcha challenge type '{}' automatically; \
set GITLAWB_ICAPTCHA_API_KEY/solver or solve interactively",
challenge.kind
)
})?;
match submit_answer(&client, cfg, &challenge.token, &answer)? {
AnswerResult::Passed { proof } => return Ok(proof),
AnswerResult::Continue { challenge: next } => challenge = next,
AnswerResult::Failed { reason } => {
bail!("iCaptcha challenge failed: {}", sanitize_excerpt(&reason))
}
}
}
bail!("iCaptcha not solved within {MAX_ROUNDS} rounds")
}
fn request_challenge(client: &reqwest::blocking::Client, cfg: &IcaptchaCfg) -> Result<Challenge> {
let url = format!("{}/v1/challenge", cfg.url.trim_end_matches('/'));
let body = json!({
"requesterId": cfg.did,
"requiredLevel": cfg.level,
"types": SOLVABLE_TYPES,
});
let mut req = client.post(&url).json(&body);
if let Some(key) = &cfg.api_key {
req = req.bearer_auth(key);
}
let resp = req.send().with_context(|| format!("POST {url}"))?;
if !resp.status().is_success() {
let status = resp.status();
let text = error_excerpt(resp);
bail!("iCaptcha challenge request failed ({status}): {text}");
}
resp.json::<Challenge>().context("parse iCaptcha challenge")
}
fn submit_answer(
client: &reqwest::blocking::Client,
cfg: &IcaptchaCfg,
token: &str,
answer: &str,
) -> Result<AnswerResult> {
let url = format!("{}/v1/answer", cfg.url.trim_end_matches('/'));
let mut req = client
.post(&url)
.json(&json!({ "token": token, "answer": answer }));
if let Some(key) = &cfg.api_key {
req = req.bearer_auth(key);
}
let resp = req.send().with_context(|| format!("POST {url}"))?;
if !resp.status().is_success() {
let status = resp.status();
let text = error_excerpt(resp);
bail!("iCaptcha answer request failed ({status}): {text}");
}
resp.json::<AnswerResult>()
.context("parse iCaptcha answer result")
}
/// Last-resort fallback: show the prompt and read an answer from the terminal.
/// Returns `None` when stdin isn't a usable interactive source (e.g. an agent),
/// so the caller surfaces a clear "couldn't auto-solve" error instead.
fn interactive_prompt(challenge: &Challenge) -> Option<String> {
use std::io::{stderr, stdin, Write};
let mut err = stderr();
let _ = writeln!(
err,
"iCaptcha challenge ({}, level {}): {}\nAnswer: ",
challenge.kind, challenge.difficulty, challenge.prompt
);
let _ = err.flush();
let mut line = String::new();
match stdin().read_line(&mut line) {
Ok(0) | Err(_) => None,
Ok(_) => {
let a = line.trim().to_string();
if a.is_empty() {
None
} else {
Some(a)
}
}
}
}
#[cfg(test)]
mod tests {
use super::*;
// ── P1: hostile error bodies must not reach the terminal raw ──────────
#[test]
fn sanitize_strips_control_chars_and_escapes() {
// ESC-based terminal escape + NUL + bell must all be removed.
let hostile = "\x1b[2J\x1b[31mowned\x07\0 done";
let clean = sanitize_excerpt(hostile);
assert!(!clean.contains('\x1b'), "ESC survived: {clean:?}");
assert!(
!clean.chars().any(|c| c.is_control()),
"control survived: {clean:?}"
);
assert!(clean.contains("owned") && clean.contains("done"));
}
#[test]
fn sanitize_collapses_whitespace_and_caps_length() {
let collapsed = sanitize_excerpt("a\n\n\t b");
assert_eq!(collapsed, "a b");
let long = "x".repeat(MAX_ERR_CHARS + 500);
assert_eq!(sanitize_excerpt(&long).chars().count(), MAX_ERR_CHARS);
}
// ── P2: never talk to (or hand a key to) an untrusted node-chosen URL ──
#[test]
fn rejects_non_https_advertised_url() {
// A plaintext node-advertised URL is ignored; we fall back to default.
let (url, key_trusted) = resolve_solver_url(Some("http://evil.example/x"), None);
assert_eq!(url, DEFAULT_URL);
assert!(!key_trusted);
}
#[test]
fn rejects_https_url_with_non_allowlisted_host() {
let (url, key_trusted) = resolve_solver_url(Some("https://evil.example"), None);
assert_eq!(url, DEFAULT_URL, "attacker host must not be honored");
assert!(!key_trusted);
}
#[test]
fn honors_advertised_default_host_but_withholds_key_without_operator() {
// Default public host is allowlisted for talking, but with no operator
// origin configured the API key has no trusted destination.
let (url, key_trusted) = resolve_solver_url(Some("https://icaptcha.gitlawb.com/v2"), None);
assert_eq!(url, "https://icaptcha.gitlawb.com/v2");
assert!(!key_trusted);
}
#[test]
fn key_trusted_only_for_operator_configured_origin() {
let op = "https://icap.mynode.example";
// Node advertises the operator's own origin → talk there, key allowed.
let (url, key_trusted) =
resolve_solver_url(Some("https://icap.mynode.example/v1"), Some(op));
assert_eq!(url, "https://icap.mynode.example/v1");
assert!(key_trusted);
// Node advertises an attacker origin while an operator is configured →
// ignore the advert, fall back to the operator origin, key stays with it.
let (url, key_trusted) = resolve_solver_url(Some("https://evil.example"), Some(op));
assert_eq!(url, op);
assert!(key_trusted);
}
#[test]
fn no_advert_uses_operator_or_default() {
let op = "https://icap.mynode.example";
assert_eq!(resolve_solver_url(None, Some(op)), (op.to_string(), true));
assert_eq!(
resolve_solver_url(None, None),
(DEFAULT_URL.to_string(), false)
);
}
#[test]
fn non_https_operator_is_not_trusted() {
// A misconfigured plaintext operator URL is not a valid key destination.
let (url, key_trusted) = resolve_solver_url(None, Some("http://icap.mynode.example"));
assert_eq!(url, DEFAULT_URL);
assert!(!key_trusted);
}
}
+270
View File
@@ -0,0 +1,270 @@
//! Deterministic solvers for the computational iCaptcha challenge types.
//!
//! Prompt/answer formats mirror the iCaptcha service generators
//! (`icaptcha/src/generators/{arithmetic,algebra,sequence}.ts`). The service
//! grades numerics by value (`Number(a) === Number(b)`), so returning the plain
//! integer string is sufficient. Anagram/logic (need a dictionary/parser) and
//! the LLM types are intentionally NOT solved here — the client requests only
//! these three types and falls back to a hook/interactive prompt otherwise.
/// Solve a challenge of the given `type` from its prompt. Returns `None` for
/// types we don't solve locally or when the prompt can't be parsed.
pub fn solve(challenge_type: &str, prompt: &str) -> Option<String> {
match challenge_type {
"arithmetic" => solve_arithmetic(prompt).map(|n| n.to_string()),
"algebra" => solve_algebra(prompt).map(|n| n.to_string()),
"sequence" => solve_sequence(prompt).map(|n| n.to_string()),
_ => None,
}
}
/// `What is 12 + 7 - 3?` -> evaluate the additive chain left to right.
fn solve_arithmetic(prompt: &str) -> Option<i64> {
let expr = prompt
.trim()
.strip_prefix("What is ")?
.trim_end_matches('?')
.trim();
let mut tokens = expr.split_whitespace();
let mut acc: i64 = tokens.next()?.parse().ok()?;
while let Some(op) = tokens.next() {
let n: i64 = tokens.next()?.parse().ok()?;
match op {
"+" => acc += n,
"-" => acc -= n,
_ => return None,
}
}
Some(acc)
}
/// `Solve for x: 3x + 4 = 19` (and the `=`-both-sides and `a(x + b)` variants).
/// Parses each side into `coeff*x + const`, then x = (cR - cL) / (aL - aR).
fn solve_algebra(prompt: &str) -> Option<i64> {
let eq = prompt.trim().strip_prefix("Solve for x:")?.trim();
let (lhs, rhs) = eq.split_once('=')?;
let (al, cl) = parse_linear(lhs.trim())?;
let (ar, cr) = parse_linear(rhs.trim())?;
let denom = al - ar;
if denom == 0 {
return None;
}
let num = cr - cl;
if num % denom != 0 {
return None;
}
Some(num / denom)
}
/// Parse a linear expression in `x` into `(coeff_of_x, constant)`.
/// Handles `Nx`, `x`, integer constants, and a single `N(x ± M)` product,
/// with `+`/`-` separators (the formats the generator emits).
fn parse_linear(s: &str) -> Option<(i64, i64)> {
let s = s.trim();
// Parenthesized product: a(x ± b) -> coeff a, const a*(±b).
if let Some(open) = s.find('(') {
let a: i64 = s[..open].trim().parse().ok()?;
let close = s.find(')')?;
let inner = &s[open + 1..close]; // "x + 4" / "x - 4"
let mut it = inner.split_whitespace();
if it.next()? != "x" {
return None;
}
let (coeff_inner, const_inner) = match it.next() {
None => (1, 0),
Some(op) => {
let m: i64 = it.next()?.parse().ok()?;
match op {
"+" => (1, m),
"-" => (1, -m),
_ => return None,
}
}
};
return Some((a * coeff_inner, a * const_inner));
}
// Sum of `±`-separated terms.
let mut coeff = 0i64;
let mut konst = 0i64;
let mut sign = 1i64;
for tok in s.split_whitespace() {
match tok {
"+" => sign = 1,
"-" => sign = -1,
t => {
if let Some(cpart) = t.strip_suffix('x') {
let c: i64 = match cpart {
"" | "+" => 1,
"-" => -1,
_ => cpart.parse().ok()?,
};
coeff += sign * c;
} else {
let n: i64 = t.parse().ok()?;
konst += sign * n;
}
sign = 1;
}
}
}
Some((coeff, konst))
}
/// `What is the next number in this sequence? 2, 4, 6, 8, 10, ?`
fn solve_sequence(prompt: &str) -> Option<i64> {
let tail = prompt.split_once("sequence?")?.1;
let nums: Vec<i64> = tail
.split(',')
.filter_map(|t| t.trim().parse::<i64>().ok())
.collect();
if nums.len() < 3 {
return None;
}
next_in_sequence(&nums)
}
fn next_in_sequence(n: &[i64]) -> Option<i64> {
let last = *n.last()?;
// Arithmetic: constant first difference.
let d = n[1] - n[0];
if n.windows(2).all(|w| w[1] - w[0] == d) {
return Some(last + d);
}
// Geometric: constant integer ratio.
if n.iter().all(|&v| v != 0) && n[0] != 0 && n[1] % n[0] == 0 {
let r = n[1] / n[0];
if r != 0 && n.windows(2).all(|w| w[1] == w[0] * r) {
return Some(last * r);
}
}
// Fibonacci-like: each term is the sum of the two before it.
if n.len() >= 3 && (2..n.len()).all(|i| n[i] == n[i - 1] + n[i - 2]) {
return Some(n[n.len() - 1] + n[n.len() - 2]);
}
// Squares: all perfect squares with consecutive roots.
let roots: Option<Vec<i64>> = n.iter().map(|&v| isqrt_exact(v)).collect();
if let Some(roots) = roots {
if roots.windows(2).all(|w| w[1] == w[0] + 1) {
let nr = roots[roots.len() - 1] + 1;
return Some(nr * nr);
}
}
// Alternating sign over an arithmetic magnitude (generator starts positive).
let signs_alternate = n
.iter()
.enumerate()
.all(|(i, &v)| if i % 2 == 0 { v >= 0 } else { v < 0 });
let mags: Vec<i64> = n.iter().map(|v| v.abs()).collect();
let md = mags[1] - mags[0];
if signs_alternate && mags.windows(2).all(|w| w[1] - w[0] == md) {
let next_mag = mags[mags.len() - 1] + md;
let next_sign = if last >= 0 { -1 } else { 1 };
return Some(next_sign * next_mag);
}
None
}
/// Exact integer square root, or `None` if `v` isn't a perfect square.
fn isqrt_exact(v: i64) -> Option<i64> {
if v < 0 {
return None;
}
let r = (v as f64).sqrt().round() as i64;
[r - 1, r, r + 1]
.into_iter()
.find(|&cand| cand >= 0 && cand * cand == v)
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn arithmetic_chains() {
assert_eq!(
solve("arithmetic", "What is 12 + 7 - 3?").as_deref(),
Some("16")
);
assert_eq!(solve("arithmetic", "What is 5?").as_deref(), Some("5"));
assert_eq!(
solve("arithmetic", "What is 100 - 40 - 35 + 2?").as_deref(),
Some("27")
);
}
#[test]
fn algebra_linear() {
assert_eq!(
solve("algebra", "Solve for x: 3x + 4 = 19").as_deref(),
Some("5")
);
assert_eq!(
solve("algebra", "Solve for x: 5x - 8 = 12").as_deref(),
Some("4")
);
// both sides: 2x + 16 = 5x - 8 -> 3x = 24 -> x=8
assert_eq!(
solve("algebra", "Solve for x: 2x + 16 = 5x - 8").as_deref(),
Some("8")
);
// parenthesized: 3(x - 4) = 9 -> x-4=3 -> x=7
assert_eq!(
solve("algebra", "Solve for x: 3(x - 4) = 9").as_deref(),
Some("7")
);
// negative solution: 2x + 10 = 4 -> x=-3
assert_eq!(
solve("algebra", "Solve for x: 2x + 10 = 4").as_deref(),
Some("-3")
);
}
#[test]
fn sequence_patterns() {
assert_eq!(
solve(
"sequence",
"What is the next number in this sequence? 2, 4, 6, 8, 10, ?"
)
.as_deref(),
Some("12")
);
assert_eq!(
solve(
"sequence",
"What is the next number in this sequence? 3, 6, 12, 24, 48, ?"
)
.as_deref(),
Some("96")
);
assert_eq!(
solve(
"sequence",
"What is the next number in this sequence? 1, 4, 9, 16, 25, ?"
)
.as_deref(),
Some("36")
);
assert_eq!(
solve(
"sequence",
"What is the next number in this sequence? 1, 1, 2, 3, 5, ?"
)
.as_deref(),
Some("8")
);
}
#[test]
fn unsupported_types_return_none() {
assert_eq!(solve("anagram", "Unscramble: tca"), None);
assert_eq!(solve("riddle", "What has keys but no locks?"), None);
}
}
+1 -1
View File
@@ -120,7 +120,7 @@ Positive:
Risks:
- Many commands default to the public node; `git-remote-gitlawb` defaults to localhost. This split should be called out in README examples.
- `gl sync trigger` signs the request when a local identity is available and falls back to the legacy unsigned request for live compatibility.
- `gl sync trigger` requires a local identity and always sends a signed request; the `/api/v1/sync/trigger` route rejects unsigned calls, so the command fails locally when no identity is configured.
- Several CLI commands parse dynamic JSON responses permissively; good for compatibility, but error messages can hide response-shape regressions.
## CI and release readiness
+10
View File
@@ -58,6 +58,16 @@ services:
GITLAWB_P2P_PORT: "${p2p_port}"
GITLAWB_REPOS_DIR: /data/repos
GITLAWB_KEY: /data/keys/identity.pem
# Only trust X-Forwarded-For when the Caddy sidecar above is actually
# deployed (domain_name != ""). Caddy appends the real client as the
# rightmost hop, letting the push rate limiter key on the true client IP
# rather than Caddy's container address. Without Caddy, this header is
# fully client-controlled and must not be trusted.
GITLAWB_TRUSTED_PROXY: "${domain_name != "" ? "x-forwarded-for" : ""}"
# iCaptcha proof-of-intelligence gate (spam protection on create/fork/register)
ICAPTCHA_MODE: ${icaptcha_mode}
ICAPTCHA_PUBKEY: ${icaptcha_pubkey}
ICAPTCHA_REQUIRED_LEVEL: "${icaptcha_required_level}"
GITLAWB_PUBLIC_URL: $${GITLAWB_PUBLIC_URL}
GITLAWB_BOOTSTRAP_PEERS: $${GITLAWB_BOOTSTRAP_PEERS}
GITLAWB_AUTO_SYNC: $${GITLAWB_AUTO_SYNC}
+13 -10
View File
@@ -290,16 +290,19 @@ locals {
)
compose_yaml = templatefile("${path.module}/compose.yaml.tftpl", {
image_repo = local.image_repo
image_tag = var.image_tag
domain_name = var.domain_name
db_host = var.use_rds ? aws_db_instance.node[0].address : ""
gitlawb_port = var.gitlawb_port
p2p_port = var.gitlawb_p2p_port
metrics_port = var.metrics_port
expose_metrics = local.expose_metrics
pg_user = var.postgres_user
pg_db = var.postgres_db
image_repo = local.image_repo
image_tag = var.image_tag
domain_name = var.domain_name
db_host = var.use_rds ? aws_db_instance.node[0].address : ""
gitlawb_port = var.gitlawb_port
p2p_port = var.gitlawb_p2p_port
metrics_port = var.metrics_port
expose_metrics = local.expose_metrics
pg_user = var.postgres_user
pg_db = var.postgres_db
icaptcha_mode = var.icaptcha_mode
icaptcha_pubkey = var.icaptcha_pubkey
icaptcha_required_level = var.icaptcha_required_level
})
user_data = templatefile("${path.module}/user-data.sh.tftpl", {
+22
View File
@@ -81,6 +81,28 @@ variable "image_tag" {
default = "latest"
}
# ---------------------------------------------------------------------------
# iCaptcha proof-of-intelligence gate (spam protection on writes)
# ---------------------------------------------------------------------------
variable "icaptcha_mode" {
description = "iCaptcha gate mode: off | shadow | enforce. The live gitlawb network enforces."
type = string
default = "enforce"
}
variable "icaptcha_pubkey" {
description = "base64url Ed25519 public key of the iCaptcha service, pinned so startup doesn't depend on fetching it. This is the live icaptcha.gitlawb.com key."
type = string
default = "xjyPNqIbvc9U-kwXW6u9mDqRJ7E2UUMOaJdUWhpEXq8"
}
variable "icaptcha_required_level" {
description = "Minimum iCaptcha proof level required to pass the gate."
type = number
default = 3
}
# ---------------------------------------------------------------------------
# Networking / ingress
# ---------------------------------------------------------------------------
+14
View File
@@ -16,6 +16,10 @@ primary_region = "iad"
GITLAWB_BOOTSTRAP_PEERS = "https://node.gitlawb.com,https://node2.gitlawb.com,https://node3.gitlawb.com"
GITLAWB_AUTO_SYNC = "true"
GITLAWB_MAX_PACK_BYTES = "524288000"
# On Fly: the edge sets Fly-Client-IP (overwriting any client-supplied value),
# so the push rate limiter keys on the real client. Without this the limiter
# would fall back to the socket peer (Fly's proxy) and over-throttle.
GITLAWB_TRUSTED_PROXY = "fly"
[http_service]
internal_port = 7545
@@ -34,6 +38,16 @@ primary_region = "iad"
# slots (hard_limit) — root factor in the 2026-06-12 prod outage
idle_timeout = 120
[[http_service.checks]]
# Gate routing (and deploys) on real readiness: /ready probes the DB, so a
# node stuck retrying a dead or misconfigured database is never routed to,
# and a deploy with a bad DATABASE_URL fails visibly instead of going green.
interval = "15s"
timeout = "5s"
grace_period = "75s"
method = "GET"
path = "/ready"
[[services]]
protocol = "udp"
internal_port = 7546